That setup would only be bad if your motor were so current hungry that the ESC was the bottleneck. So long as your motor isn't capable of drawing more than your batteries or your ESC can provide you should be good, but with a big stack of batteries like you're describing they're not going to be the limiting factor...