Excited to share Arch-Router, our research and model for LLM routing. Routing to the right LLM is still an elusive problem, riddled with nuance and blindspots. For example:

“Embedding-based” (or simple intent-classifier) routers sound good on paper—label each prompt via embeddings as “support,” “SQL,” “math,” then hand it to the matching model—but real chats don’t stay in their lanes. Users bounce between topics, task boundaries blur, and any new feature means retraining the classifier. The result is brittle routing that can’t keep up with multi-turn conversations or fast-moving product scopes.

Performance-based routers swing the other way, picking models by benchmark or cost curves. They rack up points on MMLU or MT-Bench yet miss the human tests that matter in production: “Will Legal accept this clause?” “Does our support tone still feel right?” Because these decisions are subjective and domain-specific, benchmark-driven black-box routers often send the wrong model when it counts.

**Arch-Router skips both pitfalls by routing on** ***preferences you write in plain language***\*\*.\*\* Drop rules like “contract clauses → GPT-4o” or “quick travel tips → Gemini-Flash,” and our 1.5B auto-regressive router model maps prompt along with the context to your routing policies—no retraining, no sprawling rules that are encoded in if/else statements. Co-designed with Twilio and Atlassian, it adapts to intent drift, lets you swap in new models with a one-liner, and keeps routing logic in sync with the way you actually judge quality.

**Specs**

* **Tiny footprint** – 1.5 B params → runs on one modern GPU (or CPU while you play).
* **Plug-n-play** – points at any mix of LLM endpoints; adding models needs *zero* retraining.
* **SOTA query-to-policy matching** – beats bigger closed models on conversational datasets.
* **Cost / latency smart** – push heavy stuff to premium models, everyday queries to the fast ones.

Exclusively available in Arch (the AI-native proxy for agents): [https://github.com/katanemo/archgw](https://github.com/katanemo/archgw)  
🔗 Model + code: [https://huggingface.co/katanemo/Arch-Router-1.5B](https://huggingface.co/katanemo/Arch-Router-1.5B)  
📄 Paper / longer read: [https://arxiv.org/abs/2506.16655](https://arxiv.org/abs/2506.16655)

Aligning LLM routing to (subjective) usage preferences via Arch-Router