Having "weird" interets

For the past 2 years since i started to work, i NEVER met or added someone new to my life.

When we talked about this with my friends (2 people btw) they said my interests are too niche to start or keep any kind of conversation. And i should have more "mainstream" knowledge.

And to some point i agree, i'm a guy who plays not so known games, listens to not so known music, read / watch not so known series, drive a not so known car, and i've never dedicated time for anything "mainstream".

Now my question is, is it even fair for me to want to meet people with similar "niche" interests? And if you had a similar situation what did you do ?