Having trouble using this as a local-only Moonlight client, controllers disconnect constantly

My xFintiy router reboots every 15 minutes, so I am trying to enjoy the one medium I have that doesn't require internet, games over moonlight. Since my xFinity router reboots constantly, I am trying to run it with a switch to my PC.

But like this, my controllers (both Playstation and Xbox) disconnect every few minutes. Seems strange, but has anyone else seen controllers disconnecting when the shield is not connected to the internet.