Do the infected know about Holy island?

Surely after 28 years the Alpha / any infected would've clocked the island that only has a wooden gate to protect itself when the tide is down? Also - when Jamie and Spike first go on the mainland and they walk past all the cut down trees, surely the noise would've lured infected