What's missing from the open-source AI infrastructure ecosystem?

Models are improving rapidly.

Deployment, routing, failover, and cost optimization still feel fragmented.

What infrastructure layer needs the most attention from open-source contributors?