Who were the regular residents of "pirate towns" in the golden age of piracy?

Who were the regular people, who didn't commit piracy, who lived in ports that pirates frequented? I imagine there were plenty of inns, shipwrights, taverns, brothels, etc. to provide services to pirates. But who else lived in these places/ Why did they decide to live there? Were many of these towns founded by pirates, or were they essentially taken over by pirates?

Thanks in advance for any answers