Starting Invisalign

Hey guys! I’m starting Invisalign in a couple of weeks but still have my wisdom teeth. I am 28 years old and my orthodontist wants me to get all 4 removed before doing Invisalig. I really don’t want to do this and he says that invisalign will straighten my teeth but even with the permanent retainer, my wisdom teeth will shift my teeth again. is this true and to what extent? I’m not looking for perfection and don’t want to spend the extra money on wisdom teeth surgery.