Is it better to turn off your heat?

I live in an apartment in Pennsylvania, and during the winters my girlfriend and I have contests to see who can go the longest before begging the other to turn on the heat. Inevitably one of us breaks, and we turn the heat up to a balmy temperature (no specifics because our temperature controls don't have actualy thermometers built in and they are only very vaguely labeled dials starting at 50 degrees Farenheit and ending at 90. Most likely we let it hit about 85 before saying it's too hot and then turning it off, and beginning the game again. Now, we can sometimes go a day or two before we break down again, but we are usually miserable. Would it really cost us that much more to leave it on all the time? (we each probably only spend about 10 hours per day in the actual apartment).

TL;DR: how much of a difference does it make to turn off your heat for a few days at a time if in the end you turn it way up?