If you are comparing batts with same C rating, there will be a diff between 3200 and 4800 lipos. If both are, for example, 20C rated, the 3200 will deliver 64amps where the 4800 will make 96amps continuously. The real question is whether this makes any diff to the motor you're running. The bigger lipo should also maintain higher voltage if they are both pulled to a signficant percentage of their current capacity and that will also make a diff if your motor pulls a lot of amps.
The way to offset this is to get a 3200 with a higher Crating - if you have a 3200 thats 30C rated, again for example, it would supply the same 96 amps as a 20C rated 4800. If the 4800 won't fit, I'd be looking for highest C rated 3200 I could find.