Left when I wasn’t supposed to

I’m genuinely confused here. I work at a restaurant and was scheduled til 8:30. At 5:15 ish my manager came up and said I was good to go, so I packed up at left around 5:30.

He messaged me at 4:46 asking where I was, that I was a double and wasn’t supposed to leave… however I was still there at 4:46.

I just seen his message at 8:52 and I don’t know what to do. I don’t want to get in trouble.