I'm a Technical Lead at a mid-sized global company and have been doing a personal research exercise around whether the latest frontier open-weight models (particularly Kimi K3) fundamentally change the economics of enterprise AI.

This isn't an active company initiative, I'm trying to understand whether my assumptions are realistic before even bringing up the idea internally.

Some context:

• ~500–700 employees
• ~100 software engineers
• AI is used across engineering and non-technical teams
• Annual AI spend is in the mid six figures
• Heavy use of Cursor, Claude API, Claude Chat/Codex, GPT API, and a few specialized AI tools
• Significant use of autonomous agents and internal AI-powered applications
• Teams are distributed across multiple time zones, but primarily based in Americas. So at least some demand is spread throughout the day

My current thinking is something like this:

• Keep the existing UX where possible (Cursor, Claude Code, internal tooling).
• Replace the inference layer with an internal OpenAI-compatible gateway.
• Run Kimi K3 for complex coding/reasoning tasks.
• Route simpler requests (autocomplete, summarization, classification, embeddings, etc.) to smaller open models.
• Retain a limited number of commercial model subscriptions for cases where they still clearly outperform.

Preliminary numbers

Very rough assumptions:

• Initial deployment: one 8-GPU Blackwell server (roughly US$400k–500k)
• Expand horizontally if utilization justifies it
• If frontier open models can realistically replace 70–90% of inference, the infrastructure could potentially pay for itself within 1–3 years.

I realize those assumptions could be completely wrong, that's exactly why I'm posting.

Questions for people who've actually built or operated something similar

1. Are my hardware assumptions reasonable, or am I significantly underestimating what production deployment requires?
2. What level of concurrency are you seeing with modern MoE models?
3. How much engineering effort is required to operate an internal inference platform reliably?
4. How noticeable was the quality difference compared to Claude/OpenAI for developers?
5. Were there hidden operational costs (MLOps, networking, storage, staffing, maintenance) that materially changed the ROI?
6. If you were making this decision today, with models like Kimi K3 available, would you still pursue self-hosting?

I'm much more interested in hearing from people with production experience than benchmark discussions. If you've gone through this evaluation (even if you ultimately decided not to self-host), I'd really appreciate hearing what drove that decision. Thanks!