Is it just me, or do enemies not attack invaders?

Yeah this is getting really annoying. Invaders run freely through my shit while I get attacked by enemies. Working as intended or nah?