Is it wrong that I dont want to participate in straight culture in the office?

Today a woman, that I dont even know, brought her new born baby to the office, together with cakes and whatnot. All the straights ran to meet the baby, while I just kept working. Apparently they all gave her some money as a gift. After that, some coworkers asked me, why I didn't come by, to what I said I was just not a baby person.

I am happy for her, but while she can blatantly celebrate straight culture, I have to keep my life private in order to not get unnecessary attention from bigots.

Whats your opinion on this?