About sizes for baseball jersey

I'm buying my first baseball jersey and I'm not sure about the sizing.
I'm 5'6" (168 cm) tall and my waist is 42.5 inches (108 cm).

I’m torn between Large and X-Large. I don’t want it to look oversized or sloppy, but I also don’t want it super tight either. If I have to sacrifice one thing, I’d prefer it not to look like I'm wearing someone else’s jersey.

If anyone around my height has bought one, I’d love to know what size worked best for you.

Appreciate any help – thanks!