As a reader, I love many types of "literary fiction," which is hard to define (but you know it when you see it). While the genius of some modern writers eludes me, I appreciate the casual brilliance of a Kurt Vonnegut and the simple elegance of a Margaret Atwood. But what about mysteries? How literary can they get, and is it even desireable in a genre-type work?
There are three levels of what I'll call literary-ness in genre fiction, at least according to my personal categorization. At the lowest level are formulaic novels, which have a predictable plot and stereotypical characters. Their claim to publishability seems to be a twist of some sort: the protag has an interesting job, a zany family, an intriguing character flaw. or the situation is somewhat unique: a serial killer who decorates his victims like wedding cakes or whatever. In short, they present something that amuses or thrills the reader. From the number of these novels that are published, they appeal to a large audience.
A level above that are the writers who have a good story and tell it well. If as a reader I'm pulled into an adventure, a plot, a scene, or a puzzle that I want to follow, I don't mind a bit if the writer's skill with words is less than brilliant. Telling a story with fast-moving exposition is okay with me, and with lots of other readers as well. Of course, the longer the author tries to drag out a series, the more formulaic it gets, but for certain moods, I like knowing that, say, Dirk Pitt or Hester Latterly will prevail. The characters' growth and/or the background material keeps me reading.
At the top are the writers who have the skills of a storyteller along with the soul of a poet. They turn a phrase neatly and make their characters unique with a brush of description or a sliver of dialogue. We feel what they feel and hurt when they hurt. Craig Johnston's Walt Longmire is a literary character in a genre novel, and I love that combination. Jan Burke, Laura Lippman, and Julia Spencer-Fleming all have the ability to make us care so deeply for their characters that we're sorry when we finish the book. Yes, the crime is solved, but I won't have those characters' company at breakfast tomorrow morning.
The next question is: Can a writer develop from one level to the next? I believe some can, but most can't. They find a comfortable level and stick with it, and for many, that works. Since readers come in the same three levels, it's fine that there are writers who soar, writers who entertain, and writers who meet the waves of a current trend.