17.5 motor is fine to start off with, even in 4wd. ROAR even ran a 17.5 4wd buggy class last summer in the SuperNationals.

Your problem, plain and simple, is that you're gearing the car as if you're going to be driving a low turn mod motor.

For a 17.5 motor, grab yourself a 72 spur and like a 32 pinion and check temps after a few minutes of running, but you should be fine, and will likely end up something more like 72/34.

Your Orion ESC & motor is good equipment; no need to change it out.

Good luck.