Local-first + hosted fallback: looking for feedback on an OpenAI-compatible gateway
Hi everyone,
Disclosure: I built Codjz Gateway, so this is self-promo. I want to be upfront about that.
I know many people here prefer local-first LLM workflows. I’m interested in the same idea, but I kept running into one practical case: sometimes the local model is good enough, and sometimes I need a hosted fallback for testing, image routes, long context, or app workflows.
So I built a small OpenAI-compatible API gateway:
- OpenAI SDK compatible /v1 endpoint
- chat and image routes
- transparent route pricing before usage
- usage logs
- small test credit for new accounts
- works with Dify, Open WebUI, n8n, and custom apps
The goal is not to replace local models. It is more for hybrid workflows:
local model first, hosted route only when needed.
I’d love feedback from local LLM users:
-
Would this be useful as a fallback endpoint?
-
What integrations would matter most: LiteLLM, Open WebUI, Dify, oobabooga, KoboldAI?
-
What would make you trust or distrust an API gateway?
-
Do you care more about price, latency, privacy, model choice, or billing control?
Site: https://codjz.com
New users get $3 free API credit for testing real requests, but I’m mainly looking for product feedback from people who actually build with LLMs.
Posted Jun 27, 2026, 10:33 AM UTC