Take the front half of the servo cover off exposing the gears. Pull off the final gear (the one that exits the case) and flip it over. There is usually an insert in it that wears out and won't make tight contact with the pot shaft in the servo and this can cause your issue. Replacements are as close as your local hobby shop.
The other possibility while you have it apart is the nut (assuming your servo uses one) that holds the pot in place has come lose and is allowing the pot to walk a little when it rotates. If you see a recessed nut on the pot when you have the final gear off, make sure it's snug!
Either part only needs to allow a fraction of a millimeter play to yield great big results at the linkage. Good luck!
Matt