Student debt repayments

Bear with me, I'm not too sure of the lingo around these parts...and I was one of those stupid kids who went to college and took out loans to pay for everything without thinking twice.

I had five loans to pay for school. Now I'm down to two, and the other loan will be paid off in full in just a few sweet, sweet months.

However, I have one more "whale" of a loan. Right now, I am paying off a private loan for $43,945.11 locked in @ 3.18%. The monthly payment on this is somewhere around $580/mo.

With that interest rate, what is the benefit of paying it off sooner? I've been told that I should call and refinance for a smaller monthly payment (as long as the interest remains the same) and pay off as little as I can for as long as I can.

My problem is...I just can't see being in debt as a way to personal economic prosperity.

On the flip side of things, I also do not see (or fully understand) the benefits of paying off the loan quicker. It seems like I would have to be paying at least $1500/mo. to actually save money in the long term (but what would be the long term savings anyway?).

So...what should I do? Pay off the loan quicker by putting down more money a month (the most I could do is around $1k/mo) or ask to refinance to a lower monthly payment?

EDIT I should probably mention that I've been paying this loan off for almost 6 years now. Still barely put a dent in it.

EDIT 2 I should also mention that I have over $20k in savings right now. I did some budgeting and found that I could live 6 months on just over $8k (being incredibly frugal) and still pay rent and my loans on time. I have no investments to speak of.