to test the esc, try another motor.
if you have overheated the motor, its possible you have damaged a stack in the motor. To test that take the motor out and turn motor shaft by hand. you should be able to feel 6 distinct points of cog in one full revolution of the motor shaft.
If there are less than 6 or some feel a bit mushy/odd compared to the others then the stator is toast and will need replacement.