Does anyone understand why Men will stare at you without saying a word with a blank expression on their face and then get mad when it's pointed out? How do I deal with this?

Before I start, I want to point out that I'm not intending to put down any gender, I'm genuinely curious and am experiencing problems with what the post suggests and I can't understand the what behind the why that's happening.

In multiple situations Men who I'm either talking to or in close vicinity, will look at my face or in my direction with a completely straight expression for an extended period of time. It has nothing to do with romantic attraction or them being creepy, I can tell that much (for one, I'm uglier than heck, so maybe they're staring at my ugliness, I don't know.), but they'll straight up stare without a clues on their face as to what they're thinking about. I can be looking in the opposite direction and feel the eyes on me, when I go to check they're just staring. It'll last for 10 seconds or more and it's seriously confusing. I've pointed it out numerous times, bringing it to their attention with things like "Why are you looking at me?" In the kindest of tones I could have in the moment, only to get back a "I wasn't doing anything" when they were clearly looking at me, and for a very long time.

Is it just your guys' way of trying to analyze someone but don't realize what it may seem like to others? I can't tell if they're trying to read me but aren't being very incospicuous about it?

I don't know if this falls under social skills but I need somewhere to air this out and understand what's going on and what I may be interpreting wrong!

Thanks for any insight or help, again I'm not trying to put anyone down, just want to understand it better and how to deal with it! Thanks!! :)