Plex Direct Play / Stream: Cannot get plex to Direct Play or Direct Stream

Server version: 1.12.0.4829

OS: OS X 10.9.5

Clients, w/ versions, if applicable: Roku, Xbox One, Fire fox, all of them

What you did?

Files were all originally .mkv. I used the plex Optimizer feature to optimize for TV. All of my files are now in an mp4 container. All of them have video codecs of H264 and audio codecs of either AAC or AA3. Had user check that Direct Stream and Direct Play were enabled. Had user check that Bitrate was set to original (unlimited). My upload speed is supposed to be 20 mbps but a speed test shows 15.2 mbps. I have 15 mbsp set as internet upload speed on the plex server. No limit is set on bit rate upload speed.

What happened? When users remote in to the server (primarily using xbox one plex app, but also using roku and a couple of others) it shows that the server is transcoding video (Transcoding H264 to H264) and audio (Transcoding AAC to AAC). Using the same devices on my home network allows direct play.

What you expected? I expected plex to use direct play and not transcode.

I would like this to direct play. My server is not very powerful and cannot handle much transcoding without causing buffering on the users end.

Thanks in advance for any help.