AITAH for being annoyed when a friend always messages me in the middle of the night?

So, I have a friend, not close, who insists on sending me messages at 2 or 3 am mostly. I've told him repeatedly not to do this. His excuse is that he's SO busy during the day with his business that's the only time he has, also these messages are just conversational, nothing of importance.

He says everyone puts their phone on silent, which I get. But the one time I put my phone on silent a few years ago, I missed a call from my 23 year old son who hung himself that night. So right or not, I have trouble doing that anymore. So am I'm being too sensitive here?