I made a GPT with a custom action, do you know if there is a way to grant permission for longer?

Hello, I created a custom GPT only for me (so it's not public). In this GPT I set it up to use a custom calculator that I run on my own server.

Sometimes when I ask the GPT my query, the GPT try to connect to my custom calculator to be more precise in his answer but the problem is that every time this happens, he ask for my permission to connect. Every time, even multiple times in the same chat.

This is starting to be a bit annoying, do you know if there is a way to grant a longer permission here?