Premature ejaculation from infrequent sex?

Hey all. Throwaway for obvious reasons. I’ve always had some issues with finishing quickly (24M), but never to an extent that creates issues. Over the last year, my relationship of 4 years has been struggling with some bedroom issues. My partner has a much lower sex drive than me for several reasons. It’s recently gotten to the point where we have sex maybe once a month.

When sex does occur, I feel like I’m finishing SO fast. The time between sessions and the anticipation of having sex makes it much harder. Sadly, this is lowering my self confidence since the infrequent sex feels related to the early ejacultion. Does this make sense? Can infrequent sex cause acutely PE or is this maybe more serious? I also don’t masturbate more than once or twice a week.