Gym goers who ask to share without the intention to actually share

I'm an AF user who usually does 3-4 sets with about 2-3minutes rest in between sets for each equipment. There are times when some specific gym-goers ask to share the equipment but proceed to use the equipment for 10 minutes without breaks in between.

I'm thinking of not agreeing to share and just tell them how long more I need to use the equipment next time when those specific users ask to share.

Does that make me the selfish ass here? Feel free to leave suggestions how I can better deal with these.