Really interested in how you all respond to this. I'm a big LLM fan - I use Claude daily for research and code review - but I've noticed a pretty consistent gap between what people in the local AI community proclaim and what it actually feels like for an end user on consumer hardware.

The crux of my argument here is that local consumer-grade LLMs are currently pretty bad compared to frontier cloud models, and most of the "local is winning" crowd is running hardware that the average person simply doesn't have. But the benchmark gap is closing so fast - 17.5 points in 2023 down to roughly 6 points today - that I think we're one or two hardware generations away from local models having genuine use cases where they outperform cloud and become the daily driver for the average user.

What I'm curious about from this community specifically: where are you actually seeing local models hold up today, and where do they fall apart for you? And do you think the hybrid approach - local for routine/private workloads, cloud for complex reasoning - is a reasonable interim position? Or alternately am I just plain wrong - and are you already full local?

I'd also like push back if anyone has a strong case that I'm being too pessimistic about the current state. Happy to be wrong on this!

Local Consumer LLMs Are Kind of Terrible Right Now. They're Going to Kill Cloud AI Anyway.