You are 90 correct, most race engines have a very aggressive cam profile and lift, which makes low rpms very rough. They either need a slightly hevier flywheel or a higher idle to keep running. given that racers had rotating mass, they will go for th lighter fly and rougher idle. races aren't run at idle.
If you ever go to a real track, this is why the divers are allways blipping the throttle at idle. To keep the car running.
Sands