If you have a decent charger, it will not take any longer to balance charge than to just charge. The only way the charger can 'see' individual lipo cell voltages is with the balancer connected. If there is a potentail prob, then the charger will react sooner and prevent it from becoming a real prob. Without the balancer connected, the charger see only total voltage and it may be too late by the time the charger reacts based on total voltage to prevent a real prob from occuring.
I always balance charge, its the safest way.