Performance issues when playing longer sessions

I guess it was asked already, but is there some kind of fix for it? The performance gradually gets worse by the hour, last night while I was doing The King is Dead, Long Live the King, my fps dropped into the 10s while being indoors and made cutscenes unbearable. I closed and restarted the game when I could save and it was back into the 100s.

I have a 5600 with a 3080, 32GB of memory and running off of an SSD, RT on, DLSS balanced at 1440p with high-ultra settings.