r/LocalLLM•
DawgBawb

Honey, i shrunk Qwen3. 8-Flash-Next

Research

Apologies for the brief and AI-sloppy write-up, but I'm on my phone and just trying to get this out in case someone finds this useful.

This is a follow-up to my qwen 3.6 prune. I wanted to see if the same thing worked well here. It doesn't, but there are other levers to pull.

I noticed Qwen3.8-Flash-Next spends 51B of its 176B params on a big n-gram hash table, and it turns out you can delete most of it without hurting the model. I kept 2 of its 16 hash heads and copied everything else byte-for-byte out of unsloth's Q3_K_XL quant, which took the file from 90GB down to 64.8GB with no training, no llama.cpp patch, and no measurable loss on tool calling, GSM8K, or MMLU (wikitext perplexity goes from 2.4 to 4.7 though, so heavy verbatim recall might feel it). Whole thing cost me $8 in cloud time. Weights are at https://huggingface.co/Cyronius/Qwen3.8-Flash-Next-131B-A6B-GGUF and the writeup plus the surgery script are at https://github.com/Cyronius/qwen-prune-heal-pipeline if you want to poke at it. Needs a llama.cpp build from Aug 27 or newer.

If i get a chance I'll try to shave a few more gigs of of it for those folks trying to run this on 64gb. Welcome input, feedback, and if you got it, cloud time.

Posted Aug 29, 2026, 1:32 PM UTC

2132
Loading comments…