Memphis as a white tourist?

Hi

I'm coming to Memphis with my mother in a few months, as we've had the desire to visit Memphis for quite a few years now. We intend on seeing every tourist attraction of course, but I wanted to know if I need to actually be afraid of the crime rate in this city? We will be staying in Downtown, and do not intend on going out at night either, so I basically wanted to know if it's safe to venture around Memphis during daytime? This includes places such as Whitehaven and Orange Mound, if travelling by car or taxi.

Also, we will be arriving at 12 AM but we intend on taking a taxi to our our stay. Will this be dangerous if going by taxi as well? Sorry if I sound so insecure, but I just want to be sure..