Very Poor Credit from Student Loan Delinquency

I've just applied for a secured credit card to try and bump my credit score. It is very poor.

It is being caused by the only line of credit I have. My student loans. There was a point where I hadn't made a payment for near 5 months. I couldn't and they denied me deferment. I've been making payments on time for near 2 years now but it is still hurting me.

How do I go about fixing this? The loan needs to be paid in full? I'm already searching for a second part time job that I can just put directly into the loans to pay them off. But I also heard that paying off a loan can actually hurt your score. My score is pretty bad so lowering it even more isn't something I want to do.