Firefox using over 1GB of RAM. What am I doing wrong?

When Firefox first starts up it uses ~200MB. After opening a couple tabs and doing some browsing for about 30 minutes (not necessarily streaming video or anything intense), it suddenly slows down. Pages don't scroll smoothly, there is delayed reaction from clicks, and typing is jumpy. I check task manager and firefox.exe32 is using <5% CPU but 1.6 GB RAM. The various plugin-container.exe32's are using less than 10MB each.

Running Windows 7 with dedicated graphics card and 6 GB RAM.

Since I have a large amount of RAM I would assume that maybe Firefox is simply reserving a good portion, not actually using it all. But, when Firefox is occupying 1.6 GB RAM my memory is only at about 60% usage total, so it shouldn't be this jumpy and slow, right?

While Firefox is freezing and sporadically ignoring input, I can switch to any other program, and it runs smoothly. Do I need to mod Firefox to use even more memory than it's using to fix performance? Why would it need this much memory anyway? I hardly ever have more than 5 tabs open, and this happens even without streaming youtube or netflix.

Specs:
Windows 7 x64
i7 CPU
6GB RAM
Firefox 8

Addons:
Adblock Plus
Colorzilla
Download Statusbar
Firebug
Add-on Compatibility Reporter
Fox to Phone
Google Translator
Greasemonkey
Moveable Firefox Button
Murdoch Block
Priv3
Reddit Enhancement Suite
Speed Dial
Stylish
Tor Button
User Agent Switcher