The "best local model" doesn't exist—it depends completely on your hardware setup

"What's the best local model?" is unanswerable without knowing your VRAM.

Every thread asking it collects forty replies, and none of them are wrong. They're just answers to a different machine's question.

A model that's excellent on 24GB is a different product on 8GB. Same weights, same benchmark scores, completely different day-to-day experience.

So before I look at any leaderboard I check two things: does it fit in VRAM at a quant I can live with, and does it still fit at the context length I actually work at. Speed falls out of those two. The moment layers spill to CPU, tokens/sec drops off a cliff and the model is technically running but practically unusable.

A big model at 1–2 t/s loses to a smaller one at 30 for anything interactive. The big one only wins when I'm batching overnight and not watching.

Curious where other people land. When your model doesn't fit, what do you sacrifice first: quantization quality or context length?