Eric2004: The most important function of a dyno for personal use will always be to compare engine A to engine B to engine C and so on. I don't think many will buy a dyno just to brag to their friends that their motor puts out a particular HP. Thus, the motors should be broken in and tuned to race or operational settings before being put on the dyno -- just like a real car. Needle settings will differ from engine to engine, so to get the best and most usable results, the engines should be tuned and then dyno'd.
The throttle is opened mechanically at a constant rate across the length of the pull, keeping it consistent. You're correct that a hand-operated throttle would skew the results.
Can't help you on the fan, though I suspect that there is no cooling done to the engine during the test -- once again, a controlled environment is your best bet.
Check out the upcoming (and subsequent) issues of XRC for more information. A multi-engine test is coming soon.