[KCD2] Why do bandits always keep fighting after I let them go?

After they surrender, I usually just choose the option to let them go? Every single time they will walk off like 2 steps, and then turn around and continue fighting. Am I doing something wrong? Is it a bug? Or am I missing a mechanic?