Game feels unplayable if enemy is lagging

3 games in a row, 3 different players and all of them had the yellow or red clock which I believe indicates they had bad connection.

each game units had like a half second delay before doing the action I assigned them, and more often than not my melee units would walk right up the enemy and just freeze for like a full second or two before actually attacking

why does the enemy lagging cause my game to have such lag and stutter when I'm running at 160+ fps and ms is only 90-165 latency

Sometimes it slowed down the the point it was like playing on a slideshow during fights