NVDisplay.Container uses all available RAM

I noticed my computer which has Ryzen 9 7950X3D being laggy when typing. I looked into task manager and found out I am out of RAM and using swap.

The most memory was used by NVidia display container. I'd post a screenshot, but apparently that's not allowed on this subreddit, which is weird.

NVDisplay.Container.exe was using about 3GB of RAM out of ~30GB available. When I closed my browser, it expanded to 5.5GB to fill any RAM I just freed. I restarted the service NVIDIA Display Container LS. It did not help. The memory usage moves between 1.5 to 6 GB. But the total system usage remains near constant, so it seems the process is simply adjusting its usage depending on free memory. This would be OK if it didn't clearly overshoot and slow down other processes.

Any ideas why is this happening? I have NVME SSD system drive. I'd prefer this process used that and not RAM for whatever it is storing. I don't have any games running.