Raid invitation sent button not working?

I've hosted a few raids at this point, and I fell like around 40% of the time, my "Invites sent" button simply do not work. I accept the friend requests, send the invites, push the "Raid invitation sent" button, and do the raid. Just to receive a notification saying I timed out. Now I'm even pressing it multiple times, but even then sometimes it just don't work and I don't get to answer the final questions nor get the host points. I even got a penalty and can't host raids for a time. Is this a bug? Am i doing something wrong (even if im following the instructions)?