Thinking of visiting for anniversary in Aug. Suggestions for stuff to do?

We are thinking of hitting up Your great city for our 2nd anniversary for 2 or 3 days. We love history, romantic stuff, ghost tours, and anything that's must see. What is there to do? What do we need to see? Where is great food (all price ranges welcome)? Where are good places to stay (lower high end to moderate)?

I'm not familier with the area so feel free to let me know everything!