What was the purpose of mounted infantry, roundabouts 1066-1300 or so?

I can understand the value in moving infantry from place to place quickly...but once soldiers had dismounted to fight, wouldn't the horses be a liability? Limiting maneuver, requiring handlers and fodder, risk of capture/conversion to cavalry...so what was the advantage of using mounted infantry prior to firearms? Why not just use your mounted infantry as straight cavalry?