On one hand I would say that aerodynamics make a difference always, even at slow speeds and even if that differrence is not noticeable. It is there.
That however would not stop me from running a sports car body among the Saleens and McClarens or for that matter run a VTA shell on my USGT chassis. I don't think its enough of a difference for me to not run whatever I want for the sake of handling.