To me one of the biggest flaws of the Fit is blind-spots due to the double A-pillar and rounded rear window. However one of the best parts of the Fit is the side mirrors have a wide usable range combined with a narrow body car.
I started noticing that I wasn't using my rear view mirror very often and when I did my view wasn't very good because of the rear window. I took off my center mirror and got a much more open view of the traffic in front of me and the interior felt bigger.
It's simple enough to try out and I'd say safety is improved in more situations than it's decreased. I'd say it depends on your driving situations though.
