Primary monitor losing signal when launching game for the first time in the day.

To the point:
Got a new monitor, LG UltraGear 24” 24GS60F-B 180hz. I'm using the Display port that it came with it and everything is fine. Except when I boot up a game for the first time in the day.
Then it loses the signal and I need to turn the monitor off and back on to make it work.
After that it runs kinda smoothly, but still flicker off now and then (but rarely).

My second monitor, that is a Samsung (Don't remember the model, but its 60hz) used to be my first, and never showed this kind of problem. And even now, while the primary monitor loses the signal, the second one doesn't, and keeps working normally.
Even with my LG losing signal, the game that is running does not switch to my second monitor either.

Currently, I'm using a 3060ti and windows 11.

I already uninstalled the ghost monitors that remained after I changed my old second monitor as well, checked the GPU drivers and the PC drivers, and they are all up to date.

Anyone has some theory of why this might be happening?

I thought maybe a faulty cable, but I'm not sure since outside of gaming, it never gives me a problem.