Can you get rid of Follower permanently?

Hi there,

was wondering if you could get rid of a Follower permanently outside of quest's like Midas, that Guy from the Abyss and opening the Doors in Post-Game (already done all of that).

I'm fine with the size of my cult and I'm having trouble naming the newer ones with good names. Of course I could just sacrifice them and go on, but then the Resurrection Ritual is clustered with all those weirdos.

Is there a way to permanently get rid of certain followers? Oh and I haven't started Woolhaven yet.