Spacers prevent the inner races from being compressed together by the clamping wheel hex. Much more important than if the outer races are allowed to move ever so slightly.
Ideally the bearings should be able to float a little bit but the inner races should be clamped together so you can tighten the wheel nuts and not have to worry about crushing bearings.