I can't maintain a full erection

So here’s the thing: I’ve (30) been in a friends-with-benefits situation with a woman (26) for about two months now. I find her very attractive and all that, we’ve already slept together six times, but I often have the problem that I lose my erection during sex, or rather, I just can’t keep it fully hard. At the beginning, when we’re still watching a movie together and cuddling a bit, I notice how my penis slowly gets hard and stays in this semi-erect state. When we start foreplay, I get an erection and everything’s fine, but as soon as we get to actual intercourse, I notice how the erection gradually fades. Everything works normally for the first few minutes, but after about 5 minutes, I’m back in that “semi-erect” state...
Every now and then, with a lot of effort, I manage to get my penis to about 80% erect, so we can keep having sex for a little while, but then it keeps fading again.
After like 20 minutes, I just find it hard to get a full erection again.
I don’t take any medication (except for the occasional vitamin supplement or something like omega-3 capsules, and sometimes magnesium) and I don’t have any pre-existing conditions or anything like that.

I know it’s probably all in my head, but no matter how hard I try to relax, it just doesn’t work. Especially since we’ve known each other for a while now, so it’s not like I’m still super nervous. I’ve even talked to her about it, but somehow I still feel this mental block.

Could it perhaps also be because I’m not very fit, and as soon as my thighs get really worked and tired, it affects my erection?

Does anyone have any other “more unconventional” tips for me?
For example, I was thinking about something like strength training, which might boost my testosterone a bit and make things work better.
Or maybe some kind of pelvic floor exercise.
I'm just at a loss, I find this whole thing really annoying.

Hope someone can help me a bit.