What AI scribes still get wrong — and a 20-minute test consult that finds it
The errors people complain about are the harmless ones. A mangled breed name is annoying and you fix it in three seconds. The errors worth worrying about are the ones that read perfectly well and are wrong — because those are the ones that get signed.
Here's what I'd test for before committing to any scribe, roughly in order of how much trouble it can cause.
1. Differentials that turn into diagnoses
You say "main concern would be Addison's, but the electrolytes don't fit, so I'm setting that aside." The note records Addison's under Assessment.
This is the one to test first. Thinking out loud is how most of us work a case, and a system that flattens hedged reasoning into a flat statement is actively dangerous. That note follows the patient, and it follows you into any complaint.
2. Dropped negatives
"No vomiting, no diarrhoea, no change in thirst" comes back as a history of vomiting and diarrhoea.
Negation is genuinely hard for these systems, and a lost "no" doesn't look like an error — it looks like a finding. Pertinent negatives are also the part of the record that defends you, so losing them costs twice.
3. Hallucinated normals
The note says "abdomen soft and non-painful, no organomegaly" on a cat whose abdomen you never touched because he was trying to remove your hand.
The model knows what a physical exam section usually contains, and fills it in. A gap in a record is visible. A plausible fabrication isn't. Ask whether the system will leave a section empty — then check that it actually does.
4. Client report recorded as your finding
Owner says the dog's been limping on the left hind since Tuesday. The note puts it in Objective as observed lameness.
Subjective is what you were told. Objective is what you found. Those carry very different weight if that record is ever read by someone other than you, and scribes blur the line constantly.
5. Numbers, units and decimals
0.2 mg/kg versus 2 mg/kg. 5 mL versus 50.
This is the one error class where being wrong isn't recoverable, and it deserves its own separate check rather than a general impression that "the notes seem accurate."
6. Multi-patient appointments
Two cats from the same household, one consult. Which findings land in which record, and does the system notice when you switched animals? Most handle one patient per session cleanly and degrade from there.
7. Language switching mid-consult
You speak to the scribe in English and to the client in Spanish. Most systems assume one language per recording.
Related and worth knowing: a language being understood and a language being written are two different capabilities. A scribe may transcribe a Polish consult perfectly and still only produce the note in English. Vendors often quote one number for both. Ask specifically which languages the note can be written in.
The test consult
Twenty minutes, one recording, on any trial. Run it as a real appointment, or with a colleague playing the owner. Say all eight of these on purpose:
- Voice a differential, then reject it. "Main concern would be Addison's, but the electrolytes don't fit." → Does the assessment show it as considered and excluded, or as a diagnosis?
- Three explicit negatives in a row. "No vomiting, no diarrhoea, no change in thirst." → All three present, all three still negative?
- Skip part of the exam and say why. "Not palpating the abdomen today, he's too painful." → Empty section, annotated section, or invented findings?
- Attribute something to the owner. "Owner says he's been limping on the left hind since Tuesday." → Subjective and attributed, or promoted to your exam findings?
- Say a dose with a decimal. "Meloxicam, 0.2 mg per kg, once daily." → Check the decimal, the unit and the frequency.
- Introduce a second animal. "While you're here, let's look at Luna's ear." → Where did Luna's findings go?
- Correct yourself mid-sentence. "Left ear — sorry, right ear." → Which side made it into the note?
- If it's relevant to you, say two sentences in another language. → Transcribed at all? And what language is the note written in?
Score it out of eight. Personally I'd treat a failure on 1, 2, 3 or 5 as disqualifying regardless of how good the prose reads — those are record-integrity failures, not quality niggles. Items 4, 6 and 7 are workflow annoyances you can adapt around.
Run it twice. These systems aren't deterministic. One clean pass isn't evidence.
Two things to get in writing
Whether the note is ever generated from anything other than what was said in the room. And what happens to your audio — how long it's kept, who can access it, and what happens to it if you cancel.
Posted Sep 21, 2026, 6:13 PM UTC