We have been comparing not only how well model-generated game players perform, but also how much they cost to generate. One result surprised me.

Claude Fable 5 XHigh currently scores 73.7 at an estimated mean cost of $1.58 per game-specific player. GPT-5.6 Sol XHigh scores 72.4, only 1.3 points lower, but costs an estimated $10.06, which is 6.4× more.

The pattern also appears within the GPT-5.6 family. Moving from Medium to XHigh improves the score by 14.0 to 25.7 points across Sol, Terra and Luna, but raises estimated generation cost by roughly 22× to 88×. Recorded output-token use rises even more sharply.

This is consistent with XHigh allocating much more reasoning than this particular

We have been comparing not only how well model-generated game players perform, but also how much they cost to generate. One result surprised me.

Claude Fable 5 XHigh currently scores 73.7 at an estimated mean cost of $1.58 per game-specific player. GPT-5.6 Sol XHigh scores 72.4, only 1.3 points lower, but costs an estimated $10.06, which is 6.4× more.

The pattern also appears within the GPT-5.6 family. Moving from Medium to XHigh improves the score by 14.0 to 25.7 points across Sol, Terra and Luna, but raises estimated generation cost by roughly 22× to 88×. Recorded output-token use rises even more sharply.

This is consistent with XHigh allocating much more reasoning than this particular

After a lot of building, validating, and running tournaments, we’re excited to launch DuelLab GameBench 2.

GameBench 2 evaluates AI models by asking them to generate game-playing programs. We compile the code and run the resulting players head-to-head on a hidden, growing suite of strategy games.

The rankings come from executable behavior in competition and not from how plausible the generated code looks or from an LLM judge.

The current public leaderboard includes 51 models across three reasoning levels, with overall rankings, charts, detailed views, and a public methodology. Scoring accounts for uncertainty, and the standings are deliberately live rather than frozen: they can evol

We made an update that improves on some little things and makes the game more stable. Also changes a bit the login mechanism, moving you from google to play games account connection as the former will be removed by google in 2028. Hopefully it will not cause any problems and you should retrain all your progress, if not, please report it.

We recently benchmarked ChatGPT 5.5 on DuelLab.

The interesting result was not that GPT-5.5 topped the whole leaderboard. It did not. It was #3 overall in the current results.

But on the medium setting, it ranked #1.

Scores:

• none: 60.3
• medium: 75.0
• highest: 62.9

So in this benchmark, giving the model the highest setting did not produce its strongest game-playing programs. The best result came from the middle setting.

That seems useful because medium-style settings are often closer to what people actually use in products: enough thinking to improve quality, but not always the slowest or most expensive mode.

Full writeup:
[https://duellab.org/blog/gpt-5-5-wins-the-middle.html](h

Kimi K2.6 takes #1 in mixed play, where high-, medium-, and no-reasoning entries can all face each other:
https://duellab.org/blog/kimi-k2-6-mixed-play

1. Kimi K2.6 is #6 overall on the official DuelLab leaderboard, but its highest-effort version is #1 on the mixed-play page.

2. The mixed-play win is not a clean sweep across every setting. It comes mainly from one especially strong highest-effort row.

3. K2.6 may still end up looking weaker than K2.5 at the medium setting, but that part of the picture is not settled yet because K2.6 has fewer completed rows in the current results.

We ran Claude Opus 4.7 through our LLM benchmark. At the top of the leaderboard, 4.7 looks like a small step up over 4.6. If you stop there, there is nothing out of the ordinary.
If you look at the reasoning effort, it gets more interesting:

¹ Gemini's none number comes from a single game (small sample), so I'd take it with a grain of salt.

What stands out

1. 4.7 > 4.6 at the highest (max) and none reasoning effort, 4.7 < 4.6 at medium. Opus 4.7 picks up ~8 points at none a

New DuelLab result for GLM-5.1: 62.5 overall, 1574.3 raw Elo, and 65.0 / 72.5 / 49.9 across Highest / Medium / No reasoning.

It doesn’t look like a straightforward across-the-board upgrade over GLM-5, which has a bit higher 63.7 overall score. But a reasoning profile optimization:

• Highest reasoning: 48.6 → 65.0
• Medium reasoning: 74.3 → 72.5
• No reasoning: 68.2 → 49.9

On the current Highest board, 65.0 would put GLM-5.1 above GPT-5.4 Nano, Qwen3.6 Plus Preview, and Claude Sonnet 4.6, while still below GPT-5.3 Codex and GPT-5.2.

We pushed a benchmark site update today.

The rankings pages now surface summary views more clearly, the overall leaderboard table includes Raw Elo alongside the published score, and the Advanced charts page adds multi-model selection, reasoning-specific views, head-to-head comparisons, and per-game heatmaps.

This update also adds Grok 4.20, Gemma 4 31B, and Gemma 4 26B A4B to the benchmark.

Current site: https://benchmarks.duellab.org/

As always, feedback on both the benchmark and the presentation is welcome.

DuelLab benchmarks AI models by having them write game-playing code, compiling it, and evaluating the resulting players through actual match play.

One thing that surprised us in the current results is how capable GPT-5.4 Nano looks. Overall, it scores 60.3, ahead of DeepSeek V3.2, GPT-5 Mini, Gemini 3 Flash Preview, and even GPT-5.4 Mini. On Highest reasoning, however, it scores 60.5, just behind GPT-5.4 Mini’s 61.9 and ahead of Claude Sonnet 4.6’s 55.2. On no reasoning, it scores 64.6, beating GPT-5.2’s 61.8 and Kimi K2.5’s 63.3.

Hey everyone, welcome to r/DuelLab.

This subreddit is the home for discussion around DuelLab, AI evaluation through games, model-generated game-playing code, benchmark design, and the broader idea of testing AI by what its outputs actually do after compilation and play.

What is DuelLab?

DuelLab started from a very open-ended abstract strategy game engine built to discover genuinely new, unique games.

Along the way, we found that the engine's games were novel enough to be used to benchmark AI models on things they likely haven’t already seen.

The current benchmark asks models to write game-playing code. That code is then compiled into players and evaluated throu