Qwen3.8-flash-next - all Unsloth quants benchmarked on 0 to 4 x 5060 Ti 16GB
Hope this is of interest to somebody. Host is Threadripper Pro 3955WX with 8 channel DDR4-3200.
For the CPU-only case, it's interesting that the larger quants seem to run faster than the smaller ones. It must be the cost of quantization algorithms. The weights far exceed the 128GB RAM, yet speed was still improved.
No MTP, no sensor split, in any of these runs. Small server context size and fixed small prompt, so this is best case, but I thought the relative numbers would still be helpful.
These numbers are all much, much worse than the best I achieved with Qwen3.8-27B on the same system. This was llama.cpp. I tried vLLM and SGLang and they fell flat, so far.
Posted Aug 28, 2026, 7:34 AM UTC
612