Can you use longer casting time scrolls without a dedicated caster?

We were in a situation where we wanted to temporarily split our party and reconvene after doing some scouting. Our plan was to buy a Scroll of Sending and use that to update the other half of the party once we had the right info, but then we realised Trick Magic Item has an action cost of 1, and Sending an action cost of 3, so they can never be used together, even out of combat?

None of our party could cast it, but we had bought the scrolls thinking we'd be able to. We're high level (20th) so the gold cost and DCs are effectively trivial, but it seems strange it's impossible to use magic consumables that have a longer casting time than 2 actions, especially considering thats where all the fun utility lives.

Are we misunderstanding a rule or is this the intention?