Internet fully cutting out only while Ethernet connected

As said in the title, my internet fully cuts out (red blinking light and all) intermittently. This happens 1-2 times a day and is usually fixed within 20 minutes, whether by restarting or just waiting it out. What seems odd about it is that it only seems to happen while I have my router connected to my PC via Ethernet cable. I have tested this with multiple different cables including a brand new one, multiple different ports on the router, and nothing seems to prevent these cut outs besides leaving it disconnected entirely. Does anyone know what the problem might be, and how I could fix it?