My really sweet male cat attacks my legs in a specific area of my apartment

I've owned my cat for about a year. He's a 5 year old fixed male. He's very affectionate and sweet, but ever since I adopted him whenever I go in a specific part of my apartment he will attack my legs. I live in a small studio and the spot he attacks me in is the area around my bathroom.

This isn't playful attacking, he bites hard and he is clearly very unhappy whenever he does it (angry meow, tail wagging hard, big eyes, ears back). Also, he only attacks me when I'm walking towards my bathroom, but whenever I go the other way he doesnt seem to care. My only guess is that maybe it is trauma related as his past owner ab*sed him, but I am curious why he only does it in this specific area.

In general he is a very sweet gentle boy, but I worry about this behavior, the stress that it clearly causes him whenever I go in this area, and the possibility of him getting accidentally stepped on by me or something like that with being under my legs and all.

Is there anything that might explain this, and what steps should I take to make this better for us?

Thanks