[Expanded from a comment on a friend’s Kindle-specific request. Most of these are also available in epub (iBooks) format as well.]
I picked up a bunch of Bujold and Lee/Miller books on the Webscription site for around $5 per omnibus. In fact, The Warrior’s Apprentice and The Mountains of Mourning are free now that they’re in the Free Library program. Many of Subterranean Press’s excellent books, which are out of print in their fancy collectible print editions, are $5 or $6 as Webscriptions (which are available in almost any ebook format). Elizabeth Bear’s New Amsterdam is one of my favorite works of hers. They have John Scalzi’s two blog collections, Your Hate Mail Will Be Graded and You’re Not Fooling Anyone When You Take Your Laptop to a Coffee Shop. There’s Kage Baker, Cherie Priest, Poppy Z. Brite… good stuff.
Amazon has several of Jenny Crusie’s books for cheap. Bet Me is $3, Strange Bedpersons under $2 (although I don’t think it’s one of her better ones), and most of the others right at $5. Anyone But You is one of my favorites — a flu/PMS/bad-day comfort read.
I just noticed that you can get Georgette Heyer’s The Black Moth for $1, although WTF were they thinking with that cover?
I must take the opportunity to shill for Feed as loudly as I can. Best book I’ve read all year.
Several Small Beer Press books are available free: Kelly Link’s Stranger Things Happen and Magic for Beginners, and Maureen McHugh’s Mothers and Other Monsters.
There are free samplers from several of John Joseph Adams’s anthologies, including By Blood We Live, The Living Dead 2, and The Improbable Adventures of Sherlock Holmes. (Kindle editions also courtesy of Webscription, that Swiss Army Knife of ebook retailers.)
Courtney Milan’s novella in The Heart of Christmas is excellent; well worth the price of the whole thing.
The excellent Shadow Unit web serial can be had in Kindle formats. There are bootlegs of the first two seasons (in the sense that a fan created them, not that the authors disapprove), and an official edition of episode 3×01. Alas, I haven’t seen any of the more recent episodes as ebooks, but of course you can read them online.