Heating W/O insulation?

I'm moving into a house soon that has a detached garage that I will be using for a workshop as well as my dirt bikes, etc. this will not be the primary Garage where the cars are kept. I'm trying to decide if it's worth insulating the garage even if I don't keep the space condition full-time.
There might be some weeks that I don't go in and some weeks that I go in a few times during the week working on a project. I guess my debate is do I just blast a big heater when I'm in there and call it a day or do I insulate it wondering if the insulation even helps if it's not heated or cooled full-time. In fact, someone told me that if you don't actually heat the space and insulated it, it might be worse? I'm in Maryland for what that's worth.

Thanks for any ideas? I'm excited to have this space free from cars for toys and tools.