FPS drops but ONLY in menus

First off, me specs:
RTX4070
i7-12700k
64gb RAM
Win10 if that matters for some reason
Settings are all on full/extreme save for the 2 Ray Tracing options, which are each turned down one notch, and all Motion Blur settings are off or on their lowest setting. Game is running in 2k, matching desktop resolution.

Now that a couple patches have dropped, the stutters and drops while driving around have gone away, I can maintain a pretty stable 60 fps, EXCEPT the moment I go into the map or pause menus, then it drops to 30-40, unless I set EVERYTHING to low and then back to desired settings. Then menus can maintain 60fps no prob.

WTAF, and can anyone drop some advice so I don't have to reset everything every time I wanna pause the game?