Dragon Quest III vocation change question

So everything I've read says we should ideally start changing vocations at level 20 when it unlocks. But mages and priests haven't yet learned all their spells, especially late game ones. So is it better to change them anyway for the stat boosts, or should I wait?