A faster servo requires a different driving (or handling of the controls) style. With a slow servo, you can get away with a "digital" style of driving (only 0 or 1 - full or nothing, so to speak) easier, since the slower servo will smooth out these hectic or jerky inputs, while a quick servo will react... well quicker to them, so you've got to make sure those inputs are smoother in the first place.