High ping only on CSGO?

Looked around for a solution, tried a few things and still can't figure out why. My ping is fine in every other game but CSGO. I join a community/MM/casual server, my ping starts off fine around 40 and after a minute it will rise to 250-300 and stay around that range and may occasionally drop back to normal for a minute before going back up again.

Any ideas?