Hi, I'm new to RC's and I have this 5000mah 80c 11.1v lipo I was wondering what kind of ESC I would need to run it, and from my calculations, it has 400amps of continuous power and I have a 45amp ESC

You need an ESC that can handle 3s LiPO to start. The amperage is about what the motor pulls, not what the battery gives.

Welcome to the hobby.

Basic battery stuff:
5000 mAh: this is equivalent to how big the tank of fuel is. More mAh (milliamp hours) = the longer the battery will run for.
3S = number of cells. More cells = more voltage = more power.
11.1v = nominal voltage. It can be charged to 4.2v/cell if it is a normal LiPo, so 3 cells x 4.2v = 12.6v.
C rating = basically marketing garbage. Higher C ratings mean the battery is capable of delivering more amps, but the ratings are not consistent enough and often inflated. In general, it is used as a measure of quality (more C = more better).

LiPo safety basics:

1. Never let it get below 3.0v / cell (3S = 9.0V), and store it at 3.8v / cell (3S = 11.4v).
2. Always balance charge! This means you keep each cell near the same voltage as the others while charging. Almost every charger balance charges by default, so just don't go cheap on a charger and don't bypass the balance charge setting. If you don't keep them balanced, one cell may go to low or too high and then the battery becomes a fire hazard.
3. Charge and store your batteries in a LiPo bag (fiberglass) away from combustibles at a minimum, get a Batt-Safe or set up a concrete block/ammo can for charging and storage.
4. Do not charge them unattended.
5. Do not store them in the vehicle (hard to tell when they swell or puff and become dangerous).
6. Do not leave them plugged in when not in use (slow drain can lead to instability and fire).

Now on to your question:
Which ESC you need will depend on the size of the vehicle and how it is used. A lightweight 1/10th scale 2WD buggy can use a much different ESC than a monster truck. The motor size and weight of the vehicle will determine how many amps the motor will pull, and the ESC needs to be able to handle at least that amperage.

wouldn't it be best to get numbers from a fully charged battery rather than a battery needing a full charge
a 3 cell motor will work as long as you keep in the manufactures recommended then fact check what's said..
3200kv or less motor

I don't follow what you're trying to say. The motor really isn't rated for a certain voltage, they'll take the voltage that's thrown at them and spin accordingly. kV is RPM/volt, so a 3200kV motor will spin 3,200 RPM @ 1V, or 25,600 RPM at 8V, or 35,200 RPM @ 11V.

okay so I have a Surpass Hobby 4370Kv 45a Brushless combo and it supposedly can handle 3s but my motor stutters and heats up when put under load why does it do that and can I fix it and how

When does it stutter? If you're talking from a standing start then yes that is what happens with unsensored brushless setups.

How hot exactly is it getting?

What is it installed in?

What us the gearing setup?

If it does it right away (within a few seconds), something is bound up or seized. What vehicle is it? What conditions are you running it in? We need more info.

Tell that to Castle who told me my warranty claim was invalid because I exceeded the voltage rating. The 7700KV motor was rated for 100,000 RPM. With a 3S pack, that's 97,000 RPM. But they only rate that motor for 2S operation.

4370 is not a 3 cell motor 2 cell max thats why its stutters

it only stutters right at the beginning and the track turns freely by hand and is a conversion kit for RC snowmobile and I have a 20tooth pinion and a 56tooth spur gear here's where I got the conversion kit RC creations

rc creations

I thought as long as it did not exceed its max rpm which is 50,000

No, it stutters because it's unsensored. That motor is rated for 12V.

I have always liked hydraulic comparisons for electrical circuits. I agree, that 5000 mAh is equivalent to how big the tank of fuel is. Excellent analogy. I also agree that C rating is typically garbage, but as far as the hydraulic analogy, C rating is how big the fuel line is. The higher C rating is like having a larger diameter fuel line that will allow you to drain the fuel tank (battery) faster.
EDIT: Actually to be more accurate, larger gage wires would be the equivalent to a larger fuel line. However, it's similar. I guess since the higher the C rating, the lower the battery's internal resistance and this allows you to empty the tank faster. So I would think a more accurate analogy would be the higher the C rate is equivalent to a fuel that has lower viscosity and therefore will flow out of the tank faster.