What is the big deal about wearing hats inside buildings?

I really enjoyed almost everything i read in the recent etiquette thread. But i really never understood the hat rule. my grandfather would always yell at me about it. so when i went to his house i would always remove my hat. but it never made sense, and i never took it off anywhere else. I'm not trying to disrespect anyone who feels strongly about this idea, i just don't understand it. (http://www.reddit.com/r/AskReddit/comments/rbtwq/what_etiquette_rule_do_you_wish_more_people_were/)