Any reason to not use the fastest belts all the time?

Is there a reason I should use slower belts when I can just use the fastest belts available to me? I'm not really a math wiz so in my mind the faster the item moves from a to be the better right?