What's the benefit of gems like the Fire Gem?

Don't they remove attribute bonus's and scaling? Why are they in the game?