It's one of those things that, in our culture, can earn you social rejection and stigma (at best), or a stint in a prison cell (at the absolute worst). And yet, it really is VERY common.
Here is why I think that is: a lot about human sexuality is demonized. The worst kinds of crimes that it is possible to commit (at least as they are perceived by our culture) are all related to sex … you would literally face a smaller criminal punishment for getting drunk and hacking someone's arm off with a sword than you would for having sex with that person while they're drunk, and them later deciding that their drunkenness meant that their consent was not "real" and deciding to make a fuss over it -- I'm not being pro-rape or anything with that statement, but ... I do make the comparison to demonstrate how we elevate the status of any sex-related violation to "ULTIMATE EVIL" territory; we really have no greater sin in our worldview than a sex-related violation of another person.
It's our culture, the culture of taboos. It doesn't necessarily turn OFF these little fantasies or life experiences of ours, because we're driven by our lust and shit still goes down ... it just means we all walk around feeling guilty over those fantasies, and often feeling like you're the "only one with them." It's kind of sad what we do to each other, really ... it's like we never grew out of that phase when we were all 12-15 years old, on the school playground, each of us accusing each other of jerking off like it was an insult, but all of us adamantly denying that WE did, too.
We have evil-ized sex itself. What's considered "acceptable sex" in our culture is all bound into narrow little categories, and anything that strays outside of it becomes weird and perverted. All things considered, I wish we'd take some of the magic power out of sex and approached it again (like our ancestors did) with a more level, less hysterical point of view.
The sooner we make the bonobo chimpanzees (google their sexual habits) our role models, the sooner we end all wars for our species! ;D