This eval is based on real SWE work: agents compete head-to-head on real tasks (each in their native harness), and we track whose code actually gets merged.

Ratings come from a Bradley-Terry model fit over 399 total runs. gpt-5.4 only has 14 direct runs so far, which is enough for an early directional read, but error bars are still large.

**TL;DR: gpt-5.4 already looks top-tier in our coding workflow and as strong or stronger than 5.3-codex.**

The heatmap shows pairwise win probabilities. Each cell is the probability that the row agent beats the column agent.

We found that against the prior gpt-5.3 variants, gpt-5.4 is already directionally ahead:

* gpt-5-4 beats gpt-5-3-codex 77.1%
* gpt-5-4-high beats gpt-5-3-codex-high 60.9%
* gpt-5-4-xhigh beats gpt-5-3-codex-xhigh 57.3%

Also note, within gpt-5.4, high's edge over xhigh is only 51.7%, so the exact top ordering is still unsettled.

Will be interesting to see what resolves as we're able to work with these agents more.

Caveats:

* This is enough for a directional read, but not enough to treat the exact top ordering as settled.
* Ratings reflect our day-to-day dev work. These 14 runs were mostly Python data-pipeline rework plus Swift UX/reliability work. YMMV.

If you're curious about the full leaderboard and methodology: [https://voratiq.com/leaderboard/](https://voratiq.com/leaderboard/)

Early gpt-5.4 (in Codex) results: as strong or stronger than 5.3-codex so far