Nioh 3 - Is there a way to send an Acolyte home?

The blue NPC summon. Is there a way to send them home before their timer runs out? I’m at a mission that requires being solo and would like to avoid hitting the shrine.