some employee isn't taking tasks from me

I'm a junior project manager. It's sort of my ghost title. My boss tells me that some little things I can give them to my coworker cause I'm paid alot more and he's not. But the thing is when I give him such work he usually says things like the boss didn't tell me about this and it drives me nuts. Like what do I say? I don't want to clarify much and it's an annoying pain in the... How do I communicate that I have the ability to give him tasks without sounding weird or involving my boss. I can involve him but it's such a stupid simple 30 minute task that I don't think is work this hassle. What would you do in my case..

I'm gonna tell him tomorrow that boss told me a week ago to let him do this sort of stuff. But if he argues me I'm gonna tell him that I really don't give s$$$ about his opinion, my job is just to tell him