Qwen3.5-397B-A17B local Llama-bench results
Well, I mean it ran...but it took a LONG time. Running the Q4_K_M unsloth on the latest llama-bench I could pull about an hour ago.
Rig:
EPYC 7402p with 256GB DDR4-2666
2x3090Ti
Ran ngl at 10 and cpu-moe at 51 for the total 61 layers of the model.
Any recommendations for bumping the numbers up a bit? This is just for testing and seeing how much I can push the AI system while power is cheap after 7pm CST.
Posted Feb 17, 2026, 3:13 AM UTC
1635