Have to Restart ATT Gateway Every other Week to get back to Speed (1000/1000). Is it them or me?

For some reason every other week I have to reset my Pace 5268AC Gateway to get my speed back to 1Gb from ATT. I've called and emailed ATT and I'm on my 4th gateway from them so I'm starting to thing it might be on my end.

What steps can I take to troubleshoot if its my pfSense box that is causing the slowdown?

If it happens again I'm going to remove the pfSense box and go directly to the Gateway just to rule out pfSense. I'm running pfSense as a VM on Server 2016 as a Hyper-V guest with 2GB RAM and 4 cores (so it can handle the 1Gb connection). When I do speedtest the VM doesn't get above 35% CPU usage [cpu usage during speed test].

When the speedtest is low this typically what I do to get back to speed.

1. Notice that Netflix is fuzzy for too long or Driver download is slow.
2. Check the pfSense box to make sure nothing else is using up the bandwidth.
3. Go to dslreports.com/speedsdtest and run a speed test and see that I'm at 50/90.
4. Login or try to login to the gateway to see if I can see anything wrong. (I don't know what it is but after a day or two the Gateway's admin page takes forever to load anything on it).
5. Give up looking at the gateway and depress the restart button on the back.
6. Restart the ATT Media Converter (fiber to ethernet)
7. Wait about 10 minutes (the gateway takes a few min to turn on and the status to go green).
8. Do a speed test to see that I'm back at 1000/1000 (930/980 rather).