AIO Rude Vet

So I have two cats who are the best pets I have ever had. They are the first pets I've had that I am 100% responsible for. I've had other pets, but they were family pets, so my parents took care of all the serious stuff, and I just got to play with them. Anyways my cats are 3 so I took them for their annual checkup. My boy cat has had some health issues so I had mentioned when I had made this appointment to be gentle with him as he is sensitive and his bladder issues will flair. So, I take my cats in and they are in the same carrier because they have never been separated in their lives and are more comfortable traveling together. So, I am speaking to the vet, and she is reaching into the bag and she scruffs my boy and throws a towel over him like he's going to attack her. I ask what she is doing and to please stop and I reached over and pulled the towel away from my cat. She was like just because he's your baby doesn't make him unpredictable. I get that I do but shouldn't she have waited to see if he was going to be aggressive before throwing things at my cat?!!! He didn't even hiss at her. I stopped the visit right then and asked for a different vet. She said she was the only one, so I walked out. When I told my husband he said that I shouldn't have walked out Am I overreacting