While gaming with my brothers on the same network our ping temporarily goes crazy at random moments

In the past year I helped my 3 brothers ascend, I build a PC for 2 of my brothers and myself and the third bought a laptop. Me and 2 of my brothers really like counterstrike and we are about the same skill level so we play a lot together. The problem we are having is that while playing cs our ping goes sky high (over 300) for sometimes a few seconds, sometimes a minute. At first I thought it was the crappy router from our ISP we where all gaming on, but after replacing that with a Netgear N900 dualband router (we are all using wifi) that is supposed to be able to do up to 900 Mbps the problem keeps existing, only a little less. We have a 50 Mbit up 50 Mbit Down fiber connection and every one of our pc's gets those speeds with around 15 ping on speedtest.net

Maybe one thing to note is that this only happens in 1 out of 2 matches on average. So sometimes everything is fine and an hour later everything goes to shit. Also when someone is getting ping spikes at one point in the match they are probably going to come back later on in the match.

TLDR: 3 PC's playing counterstrike on the same network have ping spikes at random times during the match.