Miles + FSDP2 = Megatron-Level Performance with More Flexibility

Miles training framework now supports FSDP2 integration, delivering Megatron-level performance with basically zero vendor lock-in.

SGLang team just shipped this and experiments show numerical alignment with Megatron while supporting advanced features like Context Parallelism out of the box.

FSDP2 gives you a flexible, high-performance distributed training backend. Works alongside existing Miles features and scales efficiently for next-gen model training.

Perfect if you're:

• Training custom models at scale
• Looking for Megatron performance without the complexity
• Building on SGLang's serving stack and want end-to-end integration

Docs: https://lmsys.org/blog/2025-12-03-miles-fsdp/

X: https://x.com/lmsysorg/status/1997768901648871925