Made this for myself trying to figure out which open model to actually run after April's release cadence. Sharing in case it's useful to anyone else.

What surprised me building it:

\- Qwen3.6-27B (dense, 17 GB on disk) beats Qwen3.5-397B-A17B (807 GB) on SWE-bench Verified, SWE-bench Pro, and Terminal-Bench 2.0. Same lab. Same benches. \~47x smaller on disk.  
\- DeepSeek V4-Pro's Codeforces ELO is 3206. GPT-5.4 sits at 3168. First open model I've seen lead a major frontier benchmark outright.  
\- gpt-oss-120B with tools hits 97.9% on AIME 2025 — edges GPT-5.5.  
\- GLM-5.1 held SWE-bench Pro #1 for 9 days in April before Opus 4.7 reclaimed it. Trained on 100k Huawei Ascend chips, zero NVIDIA hardware.  
\- Qwen3.6-35B-A3B runs at 17 t/s on a GTX 1060 + 24GB RAM with --n-cpu-moe. An 8-year-old 6GB GPU.

Sources: vellum open-LLM leaderboard, llm-stats, marc0.dev SWE-bench tracker, Artificial Analysis, lmarena, vendor model cards (DeepSeek V4, Qwen 3.6, Kimi K2, GLM-5.1, Gemma 4, MiniMax M2.7), Anthropic's Opus 4.7 system card, OpenAI's GPT-5.5 release notes.

Caveats I want to call out before anyone else does:  
1. Vendor-reported scores are systematically optimistic. Independent re-runs typically lose 3-5 points. Treat anything 80%+ as best case.  
2. Scaffold/harness differences alone create 3-10 point deltas on SWE-bench Pro and Terminal-Bench. Anthropic measured Opus 4.6 at 53.4% under their harness vs 57.3% under Z.ai's harness. Same model.  
3. NIST CAISI's May 2026 evaluation put DeepSeek V4-Pro \~8 months behind US frontier on aggregate capability. Single-benchmark wins are real but don't tell the whole story.  
4. "Best open model" depends entirely on axis. Coding -> DeepSeek V4-Pro. Agents -> Kimi K2.6 or MiniMax M2.7. Local single-GPU -> Qwen3.6-27B. Math -> gpt-oss-120B.  
5. This is a May 7-11 snapshot. It'll be partly wrong by June.

Curious what people are actually running daily right now and whether your real-world experience matches these numbers. Especially interested in Kimi K2.6 vs Qwen3.6-27B for agentic coding; the benchmark gap is small but I keep seeing conflicting takes on the sub.

Qwen3.6-27B beats the 397B Qwen3.5 flagship on coding - 15x smaller, same lab [data]