What do I (M19) say to my grandpa (M70) about his behavior? I worry it is too late to say anything

New to reddit but I hear a lot of stories from Youtube so I thought I might as well give it a shot. I was adopted by my grandparents when I was a kid and I do love and care for them. The problem is that I have so much built up resentment towards my grandpa for the way that he does stuff. I don't know the best way to describe but he can be a real asshole sometimes. He's an old man set in his way that never admits if he's wrong and couldn't be so hurtful even when sometimes I think he has good intentions. Most of the time when he does something really hurtful I block it out of my mind and I try to not let it affect me so I can't give an example for every time he's done something but these are just some of the things that I've had issues with.

I think it's important to point out that I feel like he is sometimes emotionally unavailable,  I don't know if it's the best way to put it but when I spoke up to my grandparents about a traumatic event that happened to me as a kid where I was taken advantage of by someone a lot older than me and did stuff with, All my grandpa said was “ I'm happy that you confessed your sin”. For context a couple of months before talking to my grandparents about it I talked to the priest of the church that I go to asking him to pray for me as I was really depressed and didn't know what to do and how to talk to my family about it, my grandpa was there and seen me talked to him but didn't hear what I was talking to the priest about. My grandfather never said he was sorry for what happened, that I'm here for you, or anything comforting. I was trying to be so vulnerable and I felt like he tore my heart out.

Another incident was when I first started dating a girl two years ago who is now my ex. My Grandma loved her immediately but my grandpa had an issue with her. It was perfectly normal and did nothing wrong to him but the thing he didn't like was her looks and he didn't like her because he wanted me to take a supermodel. He pointed to girls on the TV when we were watching a show saying stuff like “ that's the type of girl you should be with”,  when he had never said anything like that until I was dating someone. 

One time a couple years ago I was at a restaurant with him and he went to go to the bathroom and he left his phone on the table. I saw that he got a message and peeked over, not able to read the text as the phone was upside down but I saw a kissing emoji from the text. It was from Facebook from a girl that I've never seen before. I found out that he had been flirting with other girls. It's hard for me to go to restaurants with them now because at times that's all I think about. 

I know that this whole message is a mess but I honestly can't think of what to say because I am just so tired of being around him. I wanted to ask for advice on what to say to him because I'm leaving to go back to college soon and I want to leave him a letter talking about how I'm tired of how he treated us at times and how hurtful he can be at times. I have a lot of trouble remembering things that he does and can't form the words to say anything to him face to face. There have been times that I have tried to say something you would never listen to and just refute anything that I said. 

As crazy as it might sound I truly love him because he has been taking care of me my whole life and I wouldn't be in this spot that I am now without my grandparents and I'm truly grateful for the life that I have. There is a lot of time where I know that I should have said something about how I felt but I couldn't bring myself to say anything because I've just learned from my grandma to “let it go” not because she doesn't care but because we both have tolerated his behaviors a lot even when we know how bad it is. 

I know I have to work on my communication skills. I know that I probably left a lot out that I should talk about, but my brain has made me forget about stuff that he has done and I don't have the energy to type out everything that I remember him doing off the top of my head.

I'll take any advice in comments that anyone has for me on what I should do or what I need to do but I want to write in the letter that I leave for him after I go to college telling him that his behavior needs to change. That I'm tired of how he treats me and my grandma and how he needs to treat her better and needs to listen to us more and understand what he's doing wrong. I just worry that I have been holding it in for so long that he won't listen and its too late to say anything at his older years of his life.

Thank you for listening and I appreciate any advice that is given