So....who here is actually running 70b at a speed that doesn't make you want to throw the computer out the window

Then I'll make first. 3090 24gb. llama 3.1 70b q4. sitting at around 8 tokens per second on a good day.

is it usable. technically yes. is it the experience i was promised when everyone was hyping up local AI last year. absolutely not. feels like driving a ferrari in a school zone, constantly.

i've done the math on dual 3090s and the pcie bandwidth thing is a real problem that nobody talks about enough. you don't just double your speed, it's more complicated than that and the results in practice are all over the place depending on what you're running.

the mac studio m4 ultra thing is real but i'm not spending four thousand dollars and also being locked into apple's entire ecosystem just to run inference. hard pass.

so what's the actual answer here in 2026. because from where i'm sitting the options are still:

1. underpowered and fast enough to use

2. powerful enough and too slow to use

3. actually good and requires a second mortgage

feels like there should be a fourth option by now and i'm either missing it or it just doesn't exist yet