Make sure all the connections are good, including solder joints. Any weakness in connections and/or solder joints can contribute to issues like you describe.

At the risk of political incorrectness, this can be an occasional shortcoming of a non sensored brushless system. Especially when it’s rolling backwards and you try to go forward.