Me [25/F] and my roommate [29/M] (who has a gf) have a relationship-like relationship. It's confusing me. Advice?

I have 2 flatmates, 1 girl 1 guy. All three of us get along well. We're all uni students. We live together since about 2 years. H. and I get along particularly well since a year. He has a 45y old girlfriend and 2 kids (5 and 6) with his divorced ex-wife.

We are emotionally very close, he spends 4x more time with me than with his gf, we spend evenings watching movies, mornings having breakfast together (nothing sexual, we never kissed or cuddle). We get along super well, talk about nearly anything that happens in our lives. We have been open a few months ago about feeling somewhat attracted to each other but it's not gonna work out for either of us because he doesn't want more kids and I do. (He also has a chronic illness that might make him quite ill in a few decades so I think that's a good idea.).

So, where's the problem? I feel confused because I catch myself being bothered when his girlfriend is around for 2 days (which is not so often). I think our relation has the emotional character of a relationship, just without the sexual part. I don't even want a relationship with him (I don't have any doubt about this.). Or maybe we're emotionally already in a kind of relationship and it "works" as long as his GF isn't around. I'm confused why I feel this way and what to do. Anyone have some advice?