What is actually limiting the performance of the game ?

Well, as soon as you have much stuff the game begins to lag. I know lag can be expected.. but only 20 fps on normal speed ?

I was surprised to find out, that the game doesn't even use a quarter of my hardware ressources. Does anyone know what is actually limiting the game ?