Project write-up with numbers rather than claims. stuntd records typed decisions (choice / score / yes-no) an app asks an LLM for and trains, per decision site, the head Laya ships (type embedding, two transformer layers, a scorer over option markers) on a frozen 400M ModernBERT encoder. Temperature is fitted on a holdout, the operating threshold is picked from the coverage curve at a target agreement (0.99 by default), and serving falls back to the teacher below it. A monitor demotes a site whose agreement with the teacher drops.

Three things worth knowing:

- Caching the encoder output once per site makes epochs nearly free: 24 epochs over 3000 rows went from 730 s to 231 s on a laptop RTX 5060, with metrics inside run-to-run variance (seeds 0/1/2 on one site: 0.973-0.985 agreement).
- The frozen encoder learns what the text states, not arithmetic over fields: a risk rule over amount/hour/country reached 0.42 agreement; the same rule restated in words reached 0.94. Snake: an ASCII board trained to 0.73, four relational lines to 0.957.
- After temperature fitting the heads land at ECE 0.02-0.08; the coverage curve is what decides how much traffic the head takes at a given agreement (support triage: 90% at 0.99).

Teachers in the demos are rules and a BFS oracle because the hosted model is early access; the loop is identical with a real provider. Code, demos and every number: [https://github.com/bladedevoff/stuntd](https://github.com/bladedevoff/stuntd) (Apache-2.0).


Distilling a decision API into a head on a frozen encoder with a calibrated fallback loop: numbers, the cache trick, and where it fails