As a ground rule yes, any heat build up more then the operating temp will hurt my advise is you use a heat gun and the bearing extractor or installer to assit the in the job without causing damage.
Or you can just get a new set and do a new install, I had a bad install once and took the chance of reheating to extract and reinstall properly , the engine leaked and got damaged.