The Temple Isle quest marker is so insanely stupid

Every single time that I replay the game and try to get the cat armor, the quest marker is always messing up my brain and I can never find the entrance. Every single playthrough. It’s the most stupid thing ever, I thought it would be easy to find considering the amount of times I’ve played this game but no it’s always the same stupid marker that gets me all fucked up.