I don't always clean, but sometimes the car comes off the track with a diorty buildup on the tires, it almost looks "salt and peppery". When I see that I clean it up with the lighter fluid. Also I'll clean them if the tires come off the track feeling at all tacky or gummy.

Without the cleaning, the grip increases from one run to the next. If there is plenty of grip in the track already, that can slow your lap times down... if I feel I have plenty of grip, I clean. If I feel like a little more grip could help, I don't...