Angry over small things

I notice that I often get irrationaly angry over small things but I am a calm person when I am in argument with someone.

For example, when someone's late, I am so angry that I want to hurt myself and I rarely have that thoughts in my entire life. I always think "why is it so hard for these people to show up on time? it's literally so easy"

Another example, yesterday I was taking the elevator, There were just me and this one person, I pressed the button and turned out we're heading to the same floor. When I wanted to exit, I stepped forward and when the door was opened, this lady behind me insisted to go first and I was so annoyed by it, I even muttered curse word. She said sorry but like I was already annoyed because for me, it just doesn't make sense, why would she does that, it's so stupid, isn't it common sense to let me out first, like why can't she read the room.

However, if I'm arguing with someone, or dealing with someone I know is annoying, I am usually chill and let it go most of the time, I can sympathise with them. It's so weird

I already talked to my therapist about this and he said that I possibly created assumptions of people's intention. Like if someone's being late, I assume they dont respect me. in the second case maybe I assume that lady is a self centered bi*ch.

but to me, it looks like I'm offended by people's stupidity more than a well crafted intentional malicious actions. It's always "isn't it common sense?" that gets on my nerves.

Even knowing this, I still can't fix it. Do you guys have any advices to deal with this irrational anger.