If it restricted the power I dont think many people would be using them at all!
I have tried with and without on various motors and not "felt" any difference.
I would err on the safe side though,so if your esc instructions say use one then I would. Better safe than sorry....