Why no larger MoE models?
Aside from perhaps the niche nature of the size class, are there any good reasons that anyone can think of for why we have no recent mid-sized MoE models?
I’m thinking that a Qwen3.6 80b a12b would be an interesting model for those with unified memory (Mac/Strix Halo) who can’t handle dense models like 27b with reasonable quantization and performance.
Posted Jun 8, 2026, 10:08 PM UTC
2431