r/LocalLLaMA•
knob-0u812

vLLM Recipe for Qwen38 Flash Next NVFP4 TP=2 for RTX Pro 5000 72g

Other

I couldn't find a recipe for this model on my hardware, so I used Hermes and Unsloth's 4-bit quant of the same model to cook up a vLLM recipe for the NVFP4 quant with PLE offloading.

I've been running the model for about a week and it's taken everything I've thrown at it. Very happy with how it's performing.

Here's the Git repo

Criticism welcome.

Posted Sep 25, 2026, 10:53 PM UTC

10
Loading comments…