How does upload/download speed affect direct play on local network?

When I play big 4K remux files directly on my local network with no transcoding I’m getting buffering. I’m using an Apple TV 4K with a dedicated laptop and HDDs for a server, all hardwired.

Am I just limited by my shitty spectrum 20Mbps upload speed that can’t handle uploading that much data at once?

I originally thought your internet speed was irrelevant on local but after some reading I’m realizing maybe you can only play as fast as your server can upload? Is there even then a difference between uploading for home use and uploading for remote use?

Feel free to ELI5, still trying to wrap my head around a few concepts here