Confused about why Mystic Mine is so hated

Everywhere I read people talk about how difficult/inconsistent it is to out this card with most decks and that if you don't draw MST (or equivalent) you lose.
But can't you just out it by... not playing monsters 1 turn? And then it self-destructs?
Like if you go second with Mystic Mine it's bad since your opponent already has their board of negates and you would rather play Evenly Matched or Lightning Storm.
If you go first with Mystic Mine (or in any board state in which the opponent doesn't have monsters), they can just pass the turn back to you and it's gone, leaving you in pretty much the same game state but with minus 1 card. I guess now you've unlocked the battle phase (if you were going first before), but you've also allowed your opponent to set backrow AND you've built your whole deck around it.
Why is nobody ever mentioning the self-destruct effect when talking about this card? I haven't played with/against it, so surely I must be missing something.