Changing your prod LLM to a new model

How do you test/evaluate different models before deciding to change a model in production. We have quite a few users and I want to update the model but im afraid of it performing worse or breaking something.