It's more about speed than torque. The faster the servo is, the quicker it responds to your input. But of course, it requires a proper torque to maintain the speed...
So it takes a little time to get used to a fast servo, if you are used to a slow one. But usually, people learn it fast.
Personally I find that a servo cant be too fast. In general, I would say buy the best you can afford.
Schumacher Mi5 - Sanwa MT-4 / RX-462 / KO Propo PDS-2123 - Nosram 5600 X-treme Saddle Pack - LRP SPX Zero/X12 17,5 - Nosram Stealth Touch Evolution - MAAS SPS 9250