Interesting comments, but we're all vulnerable. Even if a particular fiction author doesn't experience heavy piracy, his publisher does and will be less likely to spend money on authors and promotion.
While we argue whether or not downloading something for free does or does not cost a sale, we forget a very important point.
If you go to B&N, stuff a paperback down the front of your pants and walk out without paying, you are a thief. In my mind, you are also a thief if you download a book without paying for it.
"Shrinkage" has caused businesses as big as Walmart and major grocery chains to shut down stores, and have driven smaller stores right out of businesses.
Sadly, we're not like musicians. A kid can illegally download a song, and sure, it may not have cost a sale. He may become a fan then shell out a few (or a lot) of bucks to see the band in concert. The band still benefits. But people don't pay to see authors, at least not often. Yes, if you illegally download my book, I may pick up a fan, but I've gained no financial benefit from it, unless you can prove that the kid later purchased a book.
But at the core my primary reason for opposing the downloading of materials (books, movies, music, etc) that cost money is simple. It is stealing.