It’s not transphobic. It’s just a thing. Squick, repulsion, whatever you want to call it – it doesn’t mean anything about the person in question.
MH
“You’re not [gender] because you have [genitalia design]” is transphobic.
“I don’t want to have sex with you because you have [genitalia design]” is perfectly reasonable.