1. The protags are bad folks, or good folks gone bad, or good folks trying to stay good in the face of much badness.
This one's the worst. For instance, a lot of vampire fiction gets classified as dark fiction, but I think it's cuz most of the story takes place in the dark. Ditto: werewolves.
2. The characters kill each other in especially bloody, horrifying ways
While horror and gore-fests can be dark, and a whole crapload of fun, in my experience, all that blood is generally balanced by a good dose of justification (often in the form of a terrible childhood. Yawn.) A good example is CARRIE, by the master. It works for me because while she draws sympathy, and we're rooting for her to get them back, she also is not particularly likeable. However, I don't find enough of these twists in character and emotion. So while there are dark happenings, I'd say often our hero in horror and gore get what's coming to them--vindication! And vindication for our hero, when it comes without a price, excludes a story from dark fiction to my sensibility.
3. Achieving the goal comes at a personal cost or The Grand Dilemma.
Now we're getting closer, but I often find these losses (in the form of a character who's introduced mid-book as I did in HINTERLND or a thousand faceless innocents in pick-your-thriller) don't take it far enough. When I want dark fiction, I want the loss to irrevokeably change the protag's life, not just linger on in the form of a bad guilt trip. For instance, Ryder in THE HUNDREDTH MAN has a love interest who is a drunk. She's trying to get sober. I'd like for it to complicate his murder case (so far he seems to be handling it fine), and not only that, I'd like for him to have to sacrifice helping her for the sake of solving his case. I want him to choose. Maybe he has to go to a crime scene and leave her at a particularly vulnerable time, during which she wanders off in a Maker's Mark fugue and drowns. He lives on an island; it could happen. I have no idea what will happen (nodonttellme!!), but I find myself rooting for the alcohol in this one.
4. The Anti-hero
I like me a good anti-hero--particularly the sort with moral and emotional complexity who is at odds with society around them. I tire of the anti-hero who embraces his/her own brand of morality wholeheartedly, or the sort who generates sympathy from run of the mill tragedy (war, family murdered, etc). Being an anti-hero should have a definite cost. (I'm writing two anti-heros at the moment: twins Aidan and Kaelin. It's tough because they're anti-heroes in radically different ways, but their actions have definite, real costs.) I also like when anti-heroism is their nature rather than from lack of nurture. I like to think this is how the character would be anyway, regardless of past difficulties. No Fonzies for me. I want to see them struggle with who they are on a daily basis--maybe even hate themselves a little.5. Failure means survival or success means death.
This is classic dark-themed fiction: The Tragedy. Romeo and Juliet. To Stop a War. This is the stuff that makes you really think, makes you feel helpless and sad. I enjoy tragedy, because to me, it's real. (See Shit that I've Done with This Fuck of a Gun on sidebar.) It's a powerful thing to evoke such sadness in a reader. In "To Stop a War," it's a tragedy because though Snipe thinks he's doing a heroic act--and it is brave--it's ultimately a failure. And Romeo and Juliet...of course we all knew what happened there. (Aside: my husband had the romantic idea of stenciling a quote from some Shakespearean love story around the top of our bedroom walls. It's a good idea, though I couldn't use R&J cuz look how that turned out.)
To be clear to those readers who are trying to sell me stories, I'm not fond of my heroes dying, especially those who tell their story in first person. Plausibility is something we think a lot about in SF/F so it's something to bear in mind. What works for literary may not work in genre because when you're taking your reader on a ride into the furthest reaches of space (or the furthest reaches of your imagination) a little verisimilitude serves as gravity.
Anything to add?