HDR video on Windows flickers when moving cursor

Some details: I have the latest version of Windows installed, as well as the latest drivers for my RX Vega 56 (also tried with older drivers). I don't have any HDR Calibration Profiles set up, and I don't have Auto HDR, Auto Colors Calibration, or any other color settings enabled in AMD Adrenaline. The display itself is a Samsung Viewfinity s8 - s80ud, 32 inch monitor, s32b800pxu, plugged in via DP 1.4 and with the newest firmware. Sleeping Mode, Energy Saving, Auto Source Switch etc. have all been disabled. This issue was not present on Windows 10.

When watching a video in Windows HDR, whenever I move my mouse, my entire screen flickers, and the video switches to another color profile that is slightly brighter. After a while, when the cursor and search bar has disappeared. The screen flickers again and the video switches to a darker color profile. How can I fix that? The same issue persist across MPC-HC and YouTube HDR. The flickering gets worse if I don't have Windows HDR enabled permanently in the background.

Also on a side note, whenever I restart my GPU drivers using Ctrl+Win+Alt+B while having HDR on, my display suddenly gets all snowy and laggy. I have to turn HDR off again in order for the snow to go away.

Been troubleshooting this issue for hours, and it's really annoying. Any help is appreciated.