Why do you always get teleported away and have to go back?

Ubisoft!!! If I'm already on an island, let me finish it RIGHT THEN AND THERE—don't make me wait until later just because I have to travel back. What is the point of that?