Thanks for the suggestions everyone.

I've set the 2 radios up the same where they have the same settings, neg steering expo, steering lock on the setup station etc.

Both cars have the same setup, motor, esc, tyres and body but different servos, but I've swapped those between cars and there was no difference.

My current servo can run off up to 7.4v, so I'm going to directly power it off the battery via an external BEC and see what happens.