It can create undesirable noise, operation and the connector will get hot so if your battery terminals push in to the case with bullets and are loose, replace them asap or risk battery damage.
In a basic of essential theory the connector making and braking a connection causes a sort of switch bounce, inductive kick back is limited to the wire itself added to what current inrush there might be to satisfy the capacitance of the esc.

The only real damage is the loss of control or crashing of course if your better has an external connector.