How to choose the inference orchestration solution? AIBrix or Kthena or Dynamo?
Workload Orchestration Projects
- llm-d - Dual LWS architecture for P/D
- Kthena - Volcano-based Serving Group
- AIBrix - StormService for P/D
- Dynamo - NVIDIA inference platform
- RBG - LWS-inspired batch scheduler
| Pattern | llm-d | Kthena | AIBrix | Dynamo | RBG |
|---|---|---|---|---|---|
| LWS-based | ✓ (dual) | ✗ | ✗ | ✓ (option) | ✓ (inspired) |
| P/D disaggregation | ✓ | ✓ | ✓ | ✓ | ✓ |
| Intelligent routing | ✓ | ✓ | ✓ | ✓ | ✓ |
| KV cache management | LMCache | Native | Distributed | Native | Native |
Posted Dec 3, 2025, 1:46 AM UTC
21