Anyone else's levies dropping constantly even when control and food isn't an issue?

I can't figure it out. I've gone from 311 as two provinces to 98 with like 10 provinces.