I think is mostly because in patriarchal society male qualities are overly honored to the point of exacerbation.
Men are supposed and encouraged to act in a way that encompasses what the society defines as 'masculine'. Any deviation from the norm is met with disgust.
Men are supposed to be strong, competitive, macho and sexually agressive (both in conduct and sexual role). Women are encouraged to be more 'passive'. So a gay man is seen as 'passive', 'woman-like' and other bunch of sense-less stuff, since there can be gay men that are ultra masculine and straight men who can act girly (yeah, the ones that mess up our gaydars)
There's also the belief that homophobia is the result of the inability to cope with one's own homosexual desires and inquiries, transfering the self hate to others. Keep in mind that heterosexual men are more constrained emotionally and sexually. They can't afford to come close emotionally to another male (even among straight friends) and he can only be 'sexually adventurous' as long as it doesn't defy the strict heterosexual macho role of penetration.
Examples: threesome fantasies must be woman-male-woman (for both straight males AND females -see the patriarchal vibe on that?), any form of penetration must be done by them (there's a reason why 'pegging' is seen more of a fetish than a normal sexual option).
So basically, they purge any type of same-sex curiosity unconciously. The fear of gay men 'seducing them' is but the manifestation of their repressed desire to try sex with another man. Look at society's stance toward females: they are allowed to experiment with members of their same gender, and some men will be turned on by a woman who admits even a tiny lesbian experience. If a man dares to say he had experimented or is curious about sex with another man he's met with glances of disapproval. Do women are afraid of lesbians assaulting them sexually? No. (Well, maybe the most backwater, repressed and ignorant straight females are) A woman is more freely to experiment their sexual curiosity without a big backlash from society. They are allowed to dispel any doubts about their sexual orientation, wether a male must always think of possible consequences.
Women are more tolerant towards homosexuality because they don't see gay men as sexual predators and we allow more emotional connections with them than our straight counterparts (and emotional connection is very treasured by females, of any sexual orientation), they don't fear lesbians because they don't have the same taboos or role expectations as straight men.
While I might (once in a blue moon) feel (very little) pity towards some homophobes, that doesn't condone heinous behaviors and acts toward LGBT people. Hurting others (physically, mentally or emotionally) because you cannot cope with your own issues is disgusting, no matter through what glass you see it.