Am I wrong for only reracking the plates I added?

Today at the gym, I used some equipment that already had plates on it when I arrived. I added the plates I needed for my set, and after I was done, I only put back the plates I added. I left the ones that were already there.

Later my trainer asked me to re-rack the plates. I explained that those plates were already there before I used the machine, so I only took care of what I added. He insisted that since I used the equipment, I should re-rack everything.

I told him I wouldn’t do that because I didn’t load those plates in the first place. Normally, I do re-rack plates in these situations, but this time I felt like I shouldn’t be responsible for someone else’s mess.