Are pirates only operational in the grand line?

The four blues seem to be safe from pirates because the four emperors only seem to be operational in the New World.