Users cannot use AI summary component, what am I missing?

Today I was playing around with creating a custom prompt template for the AI Summary component on opp page layout, and while it works for me, my reps can't actually generate anything. They get an error "We couldn't summarize the interaction, try again later".

It's gotta be a permission thing, and I've assigned the permission sets "Einstein for service innovations" and "Prompt template user" to their accounts. I'm definitely missing something here and was hoping someone knows more about this than I do. Is there maybe an agentforce PS that needs to be assigned as well?

Thanks.