Hey everyone! It's been a while.

I've been posting updates on X, but Reddit is where I got started with local AI, and I've missed posting here. Sorry for the long gap! (I'm still reading every day.)

I have a new dev release for you:  
[https://github.com/jundot/omlx/releases/tag/v0.7.0.dev2](https://github.com/jundot/omlx/releases/tag/v0.7.0.dev2)

**DeepSeek V4.1 Flash is now supported**, including DSpark MTP and Engram SSD offload. Here are some numbers from my M3 Ultra 512 GB with oQ4e and Engram in RAM:

|Context|Prefill, MTP off → on|Generation, MTP off → on|
|:-|:-|:-|
|4K|458.0 → 452.2 tok/s|20.2 → 32.1 tok/s|
|16K|459.1 → 454.8 tok/s|20.0 → 34.7 tok/s|
|32K|452.2 → 447.7 tok/s|19.8 → 31.5 tok/s|
|64K|439.4 → 435.6 tok/s|19.7 → 39.7 tok/s|

Python code prompts, temperature 1.0, 128 generated tokens, no prefix cache. One measured run per configuration after warm-up.

I've uploaded DeepSeek V4.1 oQ4e and oQ3e checkpoints. oQ3e is designed to fit 256 GB machines with Engram SSD offload, with context depending on remaining memory.

There's also **experimental MoE Expert SSD Offload**, thanks to beatakouchnir and fxd0h. It keeps some experts in memory and reads the rest from SSD when needed. Supported models include DeepSeek V4.1, Qwen3.8-Flash-Next, Gemma 4 MoE, and OLMoE. You can combine it with Engram or PLE offload, but MTP and DFlash need to be off.

It's slower 🥲 But I started with Llama 70B at 2-3 tok/s, so I understand wanting to try a model even when your machine isn't quite big enough.

For M5 users, PowerSpy's **INT8-activation prefill kernels took Qwen3.8-27B from 615.2 to 826.7 tok/s at 32K on my M5 Max (+34.4%)**. It's off by default because outputs can change. I think it's worth trying if prefill is your bottleneck; the release notes have the settings and quality comparisons.

This is still a dev release, so please tell me what works and what breaks. Thank you for continuing to test, send PRs, and help each other while I've been quiet here. I'm always grateful 🙏

oMLX 0.7.0.dev2: DeepSeek V4.1 Flash, SSD expert offload, and an M5 prefill speedup