I tested DeepSeek V4 Pro, Gemini 3.7 Flash, and Sonnet 5 on the same large private codebase.

This is not a standardized benchmark, and the results should not be generalized to every programming task. It is simply a comparison of how these models performed on one large, complex repository.

Each model received the same audit prompt, containing approximately 7,500 words of requirements. The prompt asked them to inspect the repository and produce an evidence-backed QA report covering:

• Confirmed bugs
• Missing or partially implemented systems
• Unwired data pipelines
• Performance and resource-management concerns
• Reliability issues
• Potential false positives
• Reproduction conditions
• Existing test coverage
• Suggested fixes
• An evidence ledger of files and commands used

The prompt specifically required the models to verify every claim against the current source, attempt to disprove their own findings, and avoid reporting plausible but unverified bugs.

Completion time

Model	Time
Gemini 3.7 Flash	7 minutes
DeepSeek V4 Pro	12 minutes
Sonnet 5	28 minutes

How the reports were scored

I used two separate models as judges:

• GPT-5.6 Sol Extra High
• Opus 5 Max

The judges reviewed the submitted reports, checked their important claims against the repository, and then cross-examined each other’s scoring.

This cross-check materially changed the results. Opus initially scored Sonnet at 96/100, but later found that it had accepted one of Sonnet’s conclusions without checking an alternative implementation elsewhere in the repository. It revised Sonnet to 81/100.

Opus also discovered that it had incorrectly dismissed part of Gemini’s statistics as fabricated. The underlying category totals were reproducible, although Gemini’s headline totals and several citations were still wrong.

Final corrected scores

Model	GPT-5.6 Sol	Opus 5 Max	Average
DeepSeek V4 Pro	83	85	84.0
Sonnet 5	79	81	80.0
Gemini 3.7 Flash	43	38	40.5

Overall comparison

Model	Speed	Investigation depth	Evidence quality	False-positive control	Final result
DeepSeek V4 Pro	Fast	Good	Excellent	Excellent	1st
Sonnet 5	Slowest	Excellent	Very good	Good	2nd
Gemini 3.7 Flash	Fastest	Good	Weak	Weak	3rd

DeepSeek V4 Pro

DeepSeek produced the most consistently trustworthy report.

Its strongest qualities were:

• Best falsification discipline
• Strong evidence for each reported finding
• Clear separation between confirmed bugs and unverified ideas
• Very few unsupported conclusions
• Good explanations of why existing tests did not detect the problems
• Honest handling of areas it could not measure

DeepSeek’s main weakness was breadth. It investigated fewer areas and missed the deepest architectural issue found by Sonnet.

One of its minor findings also had incomplete measurement: it showed only two relevant logging sites even though a broader search returned more. The specific two hot-path sites it identified were real, but the accompanying search output was incomplete.

My summary: the safest report to act on directly.

Sonnet 5

Sonnet conducted the broadest and deepest investigation.

It found the strongest unique issue in the entire comparison: a major state pipeline was decoded correctly but never consumed by the runtime system. Sonnet traced the problem across several layers and demonstrated why the existing test suite did not cover it.

Its strongest qualities were:

• Best architectural reasoning
• Widest repository exploration
• Best unique discovery
• Strong end-to-end tracing
• Detailed explanations and fix plans
• Good disclosure of runtime checks it could not complete

However, Sonnet also produced one material false positive.

It correctly found an unused pipeline, but then concluded that the corresponding user-facing functionality never worked anywhere. An alternative implementation elsewhere in the repository already provided that functionality. Sonnet’s falsification section claimed it had searched for an alternate path, but it missed the real one and introduced an incorrect protocol statement while doing so.

Its strongest finding was also rated HIGH without a demonstrated runtime reproduction. The underlying issue was real, but the evidence supported MEDIUM until the concrete runtime trigger was confirmed.

My summary: the best model for discovering deep problems, but its final verdicts still require review.

Gemini 3.7 Flash

Gemini was dramatically faster than the other two and produced the most polished-looking report.

It identified several real missing features. However, those findings already appeared in an existing backlog document inside the repository. Gemini presented them as independently discovered findings and added citations that did not match the actual source locations.

Its report also contained:

• Incorrect file and line references
• Two impossible headline statistics
• Incorrect descriptions of existing algorithms
• Stale findings for issues that had already been fixed
• Unsupported performance claims
• Unsupported percentage-based subsystem scores
• Claims of perfect resource behavior without measurements

One correction is important: Gemini’s detailed category breakdown was reproducible. One evaluator initially called the entire table fabricated, but running the repository’s own diagnostic tool produced the same category totals. Only Gemini’s two headline totals were impossible.

My summary: useful as a fast source of leads, but not safe to use without independently checking every claim.

Final verdict

• DeepSeek V4 Pro: best overall audit and highest trust per claim
• Sonnet 5: best deep investigator and strongest unique discovery
• Gemini 3.7 Flash: fastest result, but required the most manual verification

This remains one test on one private repository, so your results may vary.