Why does my system always take up so much of my memory? Picture inside

http://i.imgur.com/ajLxXyz.jpg

When my computer starts it doesn't take up hardly any. But as the day goes by it starts taking up 99-100%! I know what they say, unused memory is wasted memory.. But still.. 99%?

Anyone have any idea? It's been like this for awhile.