Walking around downtown Houston

I'm coming into town next week for a few baseball games. The Jays are in town and we're coming from Canada. We go on a different road trip every year and have been all over the states. We don't always see the Jays, we just like baseball and are excited to come to Houston. However, if we're staying downtown are we ok to walk to the game and restaurants? We've rented a car for day trips to the Space Centre and a few other spots but how careful do we need to be, streets to avoid? Areas not to go to? Anything would be helpful.