Automations still show old phone name after find/replace in automations.yaml + reload: what am I missing?

I recently switched phones and went through automations.yaml, did a find/replace to swap the old phone’s name for the new one (mainly in notify.mobile_app_<name> calls), reloaded automations from Developer Tools, and even did a full restart. But the automations in the UI still show the old phone name.
If I open the automation and then do ‘edit with yaml’ I can change the old phone to the new phone name and it then works, but I have dozens of automations, I cannot do them one by one.
What am I missing?