It's is perfectly fine to leave it in the remote. I personally take mine out but that's just to remind myself to charge it every few months.