r/LocalLLM•
former_farmer

What are your configuration tricks to increase prefill and decode speed?

Question

I'm not an expert on this but this is what I do. I have an old macbook with 32gb of unified ram so my definition of usable is 10 to 15 t/s. I know some of you want more than 50 t/s+ but many of us are not in that territory yet :)

  1. Play around with quants. Although we have to know we are using a dumber version of the model, so be careful.

  2. Trying MTP, MLX (if mac), Unsloth models, and other speculative decoding methods. Can't wait to try DSpark when it's available btw.

  3. Disabling reasoning.

  4. Using KV cache quantization (set to Q8).

  5. Not setting up a big context if not needed.

What else are you doing? I read you! thanks.

Posted Jul 5, 2026, 8:54 PM UTC

22
Loading comments…