Religion: When isn't it worth it? [Civ5]

I love religion. I always try to aim for getting one in a game, and because of this civs like Ethiopia are my favorite due to their abilities pertaining of religion. I play at Emperor, so I generally get a religion if I get a starting shrine, pnatheon related to faith generation, or do some quests for a religious city state. Sometimes I can even regularly build Stonehedge guaranteeing me one. I intend to move up to immortal soon, and I know it's a lot more difficult to achieve a religion, or even a pantheon.

So is there a situation where I should go for it? Is there one where I know it's not worth it at all? I love the mechanic, and I'd hate not being able to use it as I advance up the difficulties.