Hi all,

I wanted to share my recent experience (and save others some hours of troubleshooting!) trying to run the new GPT-OSS-20B F16/MXFP4/MOE GGUF models locally via llama.cpp and llama-cpp-python — and to confirm that as of August 7, 2025, this is NOT yet supported, regardless of what you try.

What I did:

1. Built an isolated Python virtual environment Using Windows 11, Python 3.11, latest pip, etc.
2. Compiled llama-cpp-python from source
   • Cloned abetlen/llama-cpp-python with --recursive
   • Explicitly updated the vendor/llama.cpp submodule:
     • Switched to upstream origin: git remote set-url origin https://github.com/ggerganov/llama.cpp.git
     • Checked out latest master, did git pull origin master
     • Confirmed commit:yamlCopyEditcommit 5fd160bbd9d70b94b5b11b0001fd7f477005e4a0 (HEAD -> master, tag: b6106, origin/master, origin/HEAD) Date: Wed Aug 6 15:14:40 2025 -0700
   • Compiled with FORCE_CMAKE=1, CPU only
3. Downloaded the official Unsloth GPT-OSS-20B F16 GGUF
   • 13.4 GB
   • Downloaded directly from HuggingFace, verified SHA256, file size matches exactly.
4. Tested file integrity with a custom Python script:
   • Confirmed GGUF header, no corruption, full SHA256 check.
5. Tried loading the model with llama_cpp.Llama (chat_format="gpt-oss")
   • Also tested with the latest compiled main.exe from llama.cpp directly.
   • Tried both with F16 and Q0_0 versions.

The error (every single time):

pgsqlCopyEditgguf_init_from_file_impl: tensor 'blk.0.ffn_down_exps.weight' has invalid ggml type 39 (NONE)
gguf_init_from_file_impl: failed to read tensor info
llama_model_load: error loading model: llama_model_loader: failed to load model from xxx.gguf
llama_model_load_from_file_impl: failed to load model
[ERRO] Failed to load model from file: xxx.gguf

What this means:

• As of the most recent commit (b6106, Aug 6, 2025) on llama.cpp and the latest source build of llama-cpp-python, there is still NO support for the new MXFP4 tensor type (ggml type 39) required by GPT-OSS F16/MXFP4/MOE models.
• This is not an issue with your build, Python, environment, or file.
• The GGUF files themselves are valid and pass header/hash checks.
• No one can run these models locally via vanilla llama.cpp at this time**.** (I even tried other quantizations; only the latest MXFP4/F16 fail like this.)

What to do?

• Wait for an official update / PR / patch in llama.cpp that adds MXFP4 and GPT-OSS F16/MOE support.
• Track issues on ggerganov/llama.cpp and the HuggingFace repo for progress.
• When that happens, just update and recompile — no extra hacks should be needed.

Conclusion:

If you’re seeing
gguf_init_from_file_impl: tensor 'blk.0.ffn_down_exps.weight' has invalid ggml type 39 (NONE)
trying to load GPT-OSS-20B F16/MXFP4, it’s not you — it’s the code!

We’re all waiting for upstream support.