That's going to happen, but there are little things you can do about it. Running a higher voltage, lower amperage setup will give you a more efficient buggy, and running a receiver pack will let you power the servos and reciever, and transponder off of another battery. While it will be heavier, you could run a higher capacity battery. Either a larger 4s, or two high capacity 2s batteries in series.

Also, make sure you start the main with a fresh pack.

How long are the buggy mains?