I benchmarked GPT-5.6 Sol/Luna/Terra by role. Sol high won the main-session slot
I ran a small role-based benchmark: three strategic decision memos, one repository-grounded execution brief, and two planted-bug repairs with direct tests.
Same prompt or fixture within each comparison. Strategy scores used a fixed rubric covering judgment, grounding, risk, boundaries, actionability, and efficiency; model identities were visible to the evaluator. These are local workflow scores, not general intelligence scores.
Strategic task results:
| Task | Best result | Other useful results |
|---|---|---|
| Multi-layer productization decision | Fable 5 reference 95 | Sol max 94, xhigh 90, high 87, GPT-5.5 high 81 |
| Decision-method / wrong-object review | Fable 5 reference 92 | Sol max 91, xhigh 90, Opus-class model 87 |
| Bounded opportunity comparison | Fable 5 reference 95 | Sol high 94, Sol max 90, Luna max 88, Terra max 88 |
Sol high scored within one point of the reference at 81.5s and 369 reasoning tokens. Sol max took 216.9s and 5,178 reasoning tokens without changing the decision.
For a repository-grounded execution brief:
| Route | Local score | Observed time (n=1) | Reasoning tokens |
|---|---|---|---|
| Sol high | 93 | 80.73s | 1,818 |
| Sol medium | 91 | 70.66s | 779 |
Medium was about 12.5% faster and used 57% fewer reasoning tokens. High retained the edge on dependency reasoning in this test, so I kept high for integration and medium as an operational profile.
The top three implementation routes then ran a second planted-bug repair. All three fixed the code and passed on the first agent attempt, with no supervisor correction.
| Bounded implementation route | Fixtures | Direct tests | Observed time | Result |
|---|---|---|---|---|
| Sol low | 2 | 10/10 | ~37.2s avg | strict; probationary implementer |
| GPT-5.4 mini low | 2 | 10/10 | ~37.8s avg | lower input use; strong tiny-task route |
| GPT-5.4 medium | 2 | 10/10 | ~43.7s avg | established rollback |
| Luna high | 1 | 4/4 | 52.59s | correct, no distinct win |
| Terra max | 1 | 4/4 | 102.20s | correct but slow |
| Luna max | 1 | 4/4 | 121.34s | heavily over-reasoned |
One useful counterexample: in a separate frozen role-specific suite, Terra max ranked first in the primary-output role at 4.86/5, while Sol xhigh led the quality-control and adversarial-review roles at 4.70 and 4.93. I found no general Terra role; that does not mean it has no specialized one.
Luna ended up without a standing slot. High and max did not beat the existing routes here.
My resulting Codex routing:
main strategic session: Sol high main operational session: Sol medium consequential review: Sol xhigh manual deep review: Sol max (rare) read/map/cleanup: GPT-5.4 mini low tight tested implementation: Sol low implementation rollback: GPT-5.4 medium
Caveats: small n; one evaluator who knew model identities; local latency includes CLI overhead; subscription consumption is not API token cost; raw API, CLI, and multi-agent workflows are different surfaces. I treated one-point differences as noise. A blinded holdout is the next useful check.
Posted Jul 11, 2026, 1:41 PM UTC