Question about biome paths

Did a full run earlier today. After defeating Rhabdom in biome 3, I chose to take the right turn off the bridge and landed in Blighted Mars where I met Legion. I was unable to turn back to the bridge and go for the Desecrated Fortess, but was sent to Acolyte Haven directly unable to return.

Is there a way of doing this? Or you need to choose a path?