How do people figure out which parts of a town/city are safe and which are dangerous?

My parents are afraid of everything including all black people they randomly see in parking lots, imaginary shootouts from drug dealers, public parks for some unknown reason and on and on and on. As a result, when I mention any town they tell me don't go there its not safe its a dangerous area. Therefore I have no reliable frame of reference for what an actual dangerous area is and how to recognize it. One time when I was a kid we saw a black man in the parking lot and they frantically rushed us to the car like we were about to be shot. One time I said I'd go for a walk after dinner and my dad said drug dealers would shoot me. Also if I use a reusable bag at the grocery store the employee will apparently tackle me and break my back lol.

In trying to unlearn the excessive anxiety, if I ever feel nervous somewhere I just rationalize that it's an incorrect reflex and everything is fine. But how do you know when something is actually not fine? Or if you go to a town and people say "just avoid the sketchy areas"-what are those? How do you know? I guess my question applies to regular big cities and small cities/large towns. I can't tell in either place and have been around both.

If its relevant I'm autistic so probably miss warning signs the same way I miss social clues