Years ago when we were running dirt oval outside in the cold we all had those cheap, soft coolers. We'd cut a hole for the wires and charge the batteries inside the cooler. The heat generated by the battery while being charged was retained within the cooler.
You can probably pick one up at Walmart for a few bucks. You don't need anything more than a 6-pack size. Something like this:
It might even help to throw some towels in to fill a bit more space and keep it in the house until you leave for the track so as much heat is retained in it as possible.