Why don't rockets tip over?

When a rocket launches, I understand it that its mainly a huge fuel tank with an engine at the end. The engine is applying thrust at the end of this long fuel tank, but doesn't that mean it would tip over at the slightest blow of wind? I understand that while the engines are off, the rocket is still connected to the launch structure, but it is disconnected after all engines are at 100%, isn't it?

I know that rocket engines are a little bit steerable, but I can't imagine they can move that fast, to keep the system stable.

I think I'm overlooking something simple in physics, so please help me :)