When to be a jerk

I know DB isn’t perfect but it’s kind of all I got to be able to play. But had a bad interaction today. Opponent had face up Jinzo and Sinister with one set back row. My hand is exile, megamorph and last will. I play last will and then exile his jinzo. He then chains book on jinzo and I go grab Stein. He thought exile didn’t work on face down monsters for some reason so I let him take his book back. Then I put stein back in deck and grabbed lily instead. He then called judge on me with no judges online. I just gave him the win. Was I in the wrong?