I'm thinking about getting into the hobby and I have a question about the numbers on lithium batteries. What does 4000mAh or 5000mAh mean? Does it mean that the 5000 will last longer and the motor will go faster?

Think of it as the amount of electricity stored in the battery.
The higher the number the longer the run time.
Dont count on a higher mAh rateing makeing your car faster.
Even though a higher rateing usually gives a little more punch.

The power difference will be so subtle that you probably won't even notice. As said earlier, the main difference would be the run-time.

as the others said, it's capacity. think of it as how far you can go or like how large a gas tank is in a real car. a car with a 20 gallon tank will be able to travel farther than a car with a 15 gallon tank, while a RC car with a 5000mah pack wil be able to go farther (or run longer) than a RC car with a 4000mah pack.

dont forget the c-rating. this is another thing people like high like 30c+, its the discharge rate which makes you fly off the jumps farther and stuff i guess.

Thanks for your help!

Many people think that, but it is wrong; the C rating is used as a reference to how many amps a battery can supply i.e 5000mah 20c can deliver about 100 amps or a 3800 35c can deliver 133 amps

So unless you are running a super high amp drawing car the C rating is just to bring in the money