Advice Needed: My cat is suddenly terrified of me. Please help!

I'll preface this story with the fact that I took care of a friend's cat yesterday.

My cat was fine last night (sitting with me on the couch, letting me pet him, etc.). He woke me up this morning at around 5:30 and I fed him, pet him, etc. and he was fine.

I went back to sleep and woke up around 10:30 and he is now running from me like I just murdered his entire family. He also went and slept in his litter box, which worried me a bit, but he let me pick him up out of the box and put him in my bed. I sat and pet him for a bit while he was purring, made some lunch in the kitchen, came back to check on him again and he bolted off the bed. He keeps letting me pet him randomly, but continues to run away at other times.

I gave him a can of food just a little while ago and he was eating and randomly stopped and ran into my room scared again.

What could be causing this? I thought it was the scent of the other cat, but again, he was perfectly fine last night. It's breaking my heart to see him so scared! Thanks in advance for any help.

He is a senior, FIV+ kitty that I took in around September.

Edit: I should also add that my boyfriend lives with me and the cat has been just fine toward him all day.