Bad quality and lag unless I play at midnight

Last couple of weeks I've been getting terrible network issues on VR. I haven't changed anything, I'm just getting network spikes unless I play midnight. I can play with 500 mbps 90fps with no drops if I play at midnight but can hardly play with 200 during the day. I'm guessing it's due to my neighbors congesting but I wanted to ask if there is anything I can do about this. I'm using channel 40 on wifi 6 5ghz.