Model Registry (RTX 4090)

I like testing local models so I’m making a registry just for people who might be interested or care to discuss what works for them.

You can see everything at Aether (your agent can browse it too)

RTX 4090 Local Model Registry 24GB

GPU: RTX 4090 - 24,564 MiB VRAM
Runtime: llama.cpp / llama-swap
KV: q8_0
FA: ON
Parallel: 1

MTP = speculative draft-MTP
Vis = vision/mmproj

## Current Registry

```text
MODEL CTX MTP VIS DECODE VRAM FREE
──────────────────────────────────────────────────────────────────────────────
Qwen3.8-Distill 256K Y Y 165.9 t/s 22,346 2,218
Qwen3.8-Distill-Reasoning 256K Y Y 121-123 22,346 2,218
Qwen3.8-27B-Stock 256K Y Y 63.5 23,928 636
TielCoder 128K Y - 257.0 21,050 3,514
CyberTiel 256K Y Y 234-245 23,624 940
Qwen3.6-Coder-Pruned 192K Y Y 141-162 23,336 1,228
Nail-Qwen3.6 128K Y Y 153.9 23,742 822
Qwen3.8-GAIN-V1.1 128K Y Y 74.8 22,684 1,880
Occult-Nail 128K N Y - - -
Qwen3-Coder-Abliterated 128K N - 147.6 23,050 1,514
Qwen3-Coder-Stock 128K N - 218.0 20,380 4,184
Devstral-Small-2 128K N - 63.1 18,016 6,548
Nemotron-Cascade-2 256K N - 123.4 - -
Nemotron-Cascade-2-Reasoning 256K N - 226.0 - -
Nemotron-Lightning-3.5 256K Y - 363.0 21,939 2,625
Nemotron-Lightning-3.5-Reasoning 256K Y - - - -
Qwen3.6-Stock 256K Y - 95.1 23,300 1,264
Qwen3.6-APEX 256K Y - 108.9 22,500 2,064
Qwen3.6-Abliterated 256K Y - 95.4 23,100 1,464
Qwen3.8-TwinTurbo-Tools 128K Y Y 89.1 23,552 1,012
Qwen3.8-TwinTurbo-NeoMax 128K Y Y 78.4 23,654 910
Qwen3.8-TwinTurbo-NeoCoder 128K Y Y 81.7 23,757 807
Qwen3.6-TwinTurbo-UnHeretic 128K Y Y 81.7 23,757 807
Gemma-4-26B 128K Y* - 210.0 - -
Qwen3.8-Whittle-MoE 110K N - 53.9 21,542 3,022
Qwen3.5-9B-Aggressive 256K N - - - -
Qwen3.5-9B-Defiant-Fable 256K N - - - -
nomic-embed-text-v1.5 2K N - - 150 24,414
```

* Gemma uses an external MTP head.

TOP MEASURED DECODE

363 t/s Nemotron-Lightning-3.5
257 t/s TielCoder
245 t/s CyberTiel
226 t/s Nemotron-Cascade-2-Reasoning
218 t/s Qwen3-Coder-Stock
210 t/s Gemma-4-26B
165.9 t/s Qwen3.8-Distill

VRAM / CONTEXT CLIFFS

MODEL GOOD CLIFF
----------------------------------------------------------------
Qwen3.8-Whittle-MoE 110K -> 53.9 t/s 128K -> 9.8
Qwen3.6-Coder-Pruned 192K -> 141-162 256K -> 24.1
TwinTurbo family 128K -> 78-89 256K -> 12-16
Qwen3-Coder-Abliterated 128K -> 147.6 192K -> 59.6