Bus driver tried to kick me off because I had an “attitude”

I (23f) was almost kicked off the bus last night.
For context the bus was 7 mins late and drove past the stop I was waiting at. It was around 10 pm and I was tired after working 8 hours.
When he stop not to far off and opened the door. He started to tell me to use the flashlight on my phone to flag him down and said it in a way that made it seem like it was my fault.
I was annoyed and walked to a seat. then he began to yell at me to get off for having an attitude.
As I was walking off, he began to rant on how he was just helping and I should be more respectful. After that he let me on.
When i pulled the string for my stop though. He literally stopped the bus far from the spot to make me walk.
Could I have been a little nicer, sure. But I don’t think he had a right to try and kick me off. Even after I paid.
I have also seen people do worse and still be aloud to be on the bus.
Am I in the wrong?