Was I in the wrong for hitting a boy in my school for holding my wrist

Today during study hall I had a moment that honestly scared me because it felt so unlike me. Me and a friend were debating, and he started talking about how he could beat me in a fight because I’m smaller. And he started getting upset and annoyed that his other freand agreed i could win in a fight even with my size. Then he grabbed my wrist tightly and told me to try to pull away so he could “prove” how strong he was and how I wouldn’t be able to fight back if he pushed me down because weight difference also his main argument was about how this one time at the mall he side bumped me and i fell and dropped my pretzels also because sometimes while we are out, if i wander off he sometimes pulls me by my back pack or grabs my wrist to pull me. Anyway I couldn’t really pull away because of how he was holding me, and suddenly my body just reacted and I smacked him in the face and smacked his glasses off with my free hand without even thinking. It wasn’t hard, I apologized immediately, and we’re okay now him and his friend was even laughing and he said it was fine but that it didn't even hurt , but it still bothered me a lot because I genuinely hate violence and don’t want to be that kind of person. I just feal bad I donno.

Also I told him before that I didn't like getting tested on my strength. He just still grabbed onto me.