We caught 1 in 50 AI responses hallucinating in production and users had no idea

We run a B2B SaaS with AI features. Users started reporting that summaries were "making things up." Maybe 1 in 50 calls, but enough to erode trust fast.

The scary part: our monitoring showed nothing wrong. Status 200, latency normal, tokens normal. A hallucinated response looks identical to a good one in every standard dashboard.

We added automated evaluation on every response. A lightweight check that flags when the model states something not in the input context. Every request now gets a quality score alongside cost and latency.

Two surprises. Most hallucinations came from one feature where context was too long. Model lost track of relevant info. Shortened it, hallucinations dropped to near zero. Second, when we moved classification to a smaller model, hallucinations on that task went down. Smaller models overthink less on simple tasks.

If you are only monitoring latency and error rates, you are missing the quality layer. A response can be fast, cheap, and completely wrong.