AITA for not being carefull enough about my friend's consent ?

Disclaimer : English is not my first language so I'm sorry if I left mistakes here and there.

So me (F21) and my friend (M20) were hanging out in my room, listening to music and chilling. It was the middle of the night, we were starting to get tired, but my friend didn't have any ways to go back to his place (no buses, no subways nothing) so we decided to share my bed. At first we just hugged and kissed a bit, but at one moment he tried to have sex with me, I refused as I was too tired. Once we woke up from our little night, he asked me again to sleep with him, and this time, I said yes.
After that, he goes back to his house, and a few minutes later, he sent me a message saying that " He didn't wanted it to happen, that it was the first and last time, and that he hoped we could stay friends ". I responded to him "Alright, ok, no problems". I was a little bit bemused but moving on.

Maybe 2 weeks later, he comes back to my room, and we're trying 2-CB (a drug that can have aphrodisiac effects, he knew it and he was the one asking to try it) and then, once again we have sex. (Side note : I barely remember anything about this, probably because the day ater that I took a little bit too much of antipsychotics because I was feeling bad). After that, he ignores me during a month, a month and a half. I was trying to reach him (mainly cause I thought I was pregnant and I was afraid I will had to get an abortion by myself) I sent him messages, I asked to see him, but no answers.

A few days ago, I was really mad, so I told him "What's wrong with you, why are you ignoring me ?" and at that moment he told me that he didn't wanted to sleep with me last time, and that he had felt used, that he was uneasy and that was why he was ignoring me since a month, and still to this day, doesn't want to talk to me. When I read that I get mixed feeling, obviously I felt bad for him, but also I was really angry that it didn't cross his mind that me too, I felt used by him.

As I didn't remember it very well I asked him if he had told me "No" if he had pushed me back, and apparently he didn't say anything to me (from his own words, I have the messages and I will obviously not show them). I do remember asking him if everything was alright during "the action", but right now he's saying that I've never asked him anything. He also told me that I was always the one making the first step, and if it might be true for the second time we slept together, I know it's not true for the first time, the fact that he's sometimes lying to me doesn't help me to know what did I really do. From what I recall, he was active, told me that he was fine, so no, when we were having sex, it didn't seem like he didn't want to, and most important he didn't say anything, didn't do a single move to stop it !

I asked him "How was I supposed to guess you didn't want to, if you didn't told me anything" he told me that if he didn't say anything it was because he was high (as I was) and he justifies that by saying that with the old message ("He didn't wanted it to happen" etc.) I should have known he didn't wanted to have sex. Obviously I was high, the message was from 2 weeks ago, I totally had forgot about it.

Now, he doesn't want to talk to me, and even if I'm mad at him for ignoring me during weeks rather than talking to me like a grown ass adult, I also feel bad. He's my friend, and I don't wanted to hurt him. I'm not even sure I'm the one who started all that ! The worst is that I never really wanted to sleep with him I did all that because I thought it was what he wanted and I wanted to make him happy...

So tell me, Am I The Asshole for not being carefull enough about my friend's consent ?