Its cold outside. Fermenting in my outdoor shed

I wrapped my barrel in a sleeping bag and a comforter since its getting down to the high 30s (0 C) at night now. I pitched my yeast last night at 90F (32 C) and let it sit overnight and it didnt budge in temp. I dont know if this will be a good or a bad thing. Im sure the temp will drop some over the next few days, but if it does somehow stay 85-90 will it mess with my flavor? Im worried if I take my insulation off my barrel itll get to cool at some point and I have no way to warm it back up