Am I (26/M) being secretive or my boundaries I’m setting with my boyfriend (29/M) are valid?

So this question isn't really a current argument topic, I’m just curious if the boundaries I'm setting are correct or if I’m somewhat of an AH.

Me and my boyfriend are dating for a year now, and have very different backgrounds when it comes to romantic relationships. He had more shorter ones, and a bunch of them cheated on him, while I only had two boyfriends, both for 2+ yrs, and no cheating or anything bad, just grew apart.

My problem with my boyfriend is when I’m near him and texting a friend, he sometimes asks who is it and what are we talking about. (Not necessarily a red flag). Other times when he comes close to me, he just randomly starts reading my texts. Or when my phone is locked on the table, he touches the screen and checks if I have notifications, and when I have, he goes through them. When these incidents first happened, I used to just sarcastically ask him "are they texting you or me" or something like that. (I know, sounds like I’m a d!-k, but sarcasm is a big part of my personality, and I usually mean no harm.) To be fair, he does this phone checking with his parents and friends too, as I found out, and I don’t think that’s great either, but that’s not my business.

After the first few incidents, I tried to make him understand that while being curious and wanting to know about my life is a green flag, reading my personal messages and invading my privacy isn't something I’m happy about. He got offended, saying that "where are the us then in this relationship if there's my business and yours". And if I want privacy it means I have something to hide. So now everytime he asks, I answer, or checks my notifications I just try to ignore it, to avoid another argument.

I understand that this comes from the trauma of his exes being secretive and cheating, but I’m still not comfortable when he is in my phone. Just to be clear, I have no secrets, and we have each others passcodes, so it’s not like he's unable to go through my phone if something is up. He also grew up without learning what boundaries are. His childhood home has one bathroom with one toilet, so during him growing up, anytime someone was having a shower (no shower curtain either), other family members just walked in and took a piss or started brushing their teeth. He also regularly walked around nude while his parents were at home, up until he moved at the age of 28. I get that growing up this open is somewhat good, but living with no regard of the other persons personal boundaries is a huge red flag.