Was it wrong to punch some guy! For being phobic?

I was out on saturday night and only had 3 beers but my friends brother just turned round to me as i was talking to some of my girly mates and said "why do you have to let everyone know your gay" and i got up and squared up to me and said "ooo did i hit a nerve" i replied only slightly! and then said "and in answer to your previous question its cause i am" ....Then hit him - Outcome was he was crying and phoned the police but chrges where dropped after i claimed it was a phobic attack. Was i wrong to give in to this abuse or right??