Massive Performance Issues - Xenoblade Chronicles 3

I am having extreme performance issues with this game on Ryubing. It regularly drops to 1 - 2 fps, and then recovers again to 30. It is especially frustrating during cutscenes. I don't know what to do anymore. I have tried everything, but I'm struggling.

I have an AMD RX 6950 XT and a Ryzen 5 7600X, so I don't think I should have this much trouble with the game. Drivers are all updated.

I tried setting it to Vulkan or OpenGL, removed all mods except the Scaler Enabler, I play on Version 2.2.1. All the AMD Adrenalin Tools are turned off.

The Problem is, that even at the worst graphic, with the lowest settings, with no mods enabled, I still struggle to get a stable framerate. I tried it on different Emulators too and it seems to have the same issues there. I tried every solution I found in different old threads, but nothing helped.

When I run a benchmark tool, the PC isn't even at 50% usage, so I have no idea, why it is behaving this way. Is there anyone that might know a solution to this, or has something that I could try?