I have this charger- http://www.hobbypartz.com/tht6smbach16.html

It says it can charge up to 6 cell lipos. Can I hook up two, 2 cell batteries to it with a y-harness and charge them at the same time?

I have a 10 amp power supply and I'll be charging at 4 amps so I dont think it would be a problem, but I wanted to make sure. Thanks a lot!

Thats a 50watt output charger. You can do that, Y'ed together series that will be 16.8v and the charger will limit current to stay under its 50watt output limit. 50watts/16.8v = 2.9amps. Charger will only be able to charge at about 3 amps and stay under the 50watt limit.

Given the hassle of balancing them together, I think you'd do it quicker to charge them separately where you can charge at 4amps, but it may be a toss up time wise, Y'ed together versus doing the separate.

What do you plan to do for balancing them?

The charger balances also so i figured i would put it on after for 1 min to balance the cells. I'll try both ways and see which one is faster. Thanks

If you use a parallel Y harness there are virtually no qualifications to charging them.
If you use a series Y harness then there is one qualification: they have to require the same amount of charge.
If you will balance series packs during the charge there is one more: they should be the same size (capacity) cells.

I hope I'm not forgetting any. There are others, but nothing out of the ordinary that doesn't also apply to any lipo charge.

Here's what I made-
When I charge I just put it in 4s mode right?
there both 4000 20C packs.

That looks perfect, set for 2Sx8000

You made a parallel connector - neg to neg and pos to pos. That keep voltage same, but doubles mah.

Set for 2S and 8000msh (2 x 4000). SInce your charger is limited to 5amps, it really won't make a diff. You would have been lmited at 4S (if you had made a series connector) as well.