Add shims anywhere you think is appropriate. It's easy to tell if the extra movement is necessary or not. In that location, the extra movement obviously isn't necessary. However, I don't think it will cause any problems either; I have the same movement on the spur shaft in my TB04R, and it's fine because there is no sideways load placed on that shaft.