I had a similar issue w/ our oval racing a couple years ago...seeming to regularly have about 12-14 entries in one class and running 10 car "A" mains.
I started doing the 'bump up' format this way.
TOP 8 qualify - locked into the "A", unless there are only 10, then ALL go directly to "A".
If MORE than 10 (say 11) those 3 cars would do a "B" with 2 of the 3 getting to transfer. That give the ODD MAN OUT one last shot at getting in.
We normally take a break after qualifying for guys to work on their cars for 20-30 minutes, but we run our "B" mains before that break and immediately after the last qualifier.
If we have 3 classes - Say STOCK 19t & MODIFIED, after the LAST STOCK Qualifier, but before the last round of 19t, I print out the qualifying order for stock and announce WHO the "B" main drivers will be. This usually gives them time to be ready.
After the last "B" main we take our break and that gives everyone a chance to freshen up the cars and be ready for the mains...plus we open the track briefly for some practice so guys can shake their cars down if they broke anything during the day.
Most of the guys have liked the format, probably the ones who don't are the ones who regularly qualified in the 9th or 10 position and now have to work EXTRA hard to get into the main.