I’ve been building a project called OMNI CORE

It started as a fairly normal local LLM setup: one good model, a Web UI, some tools, maybe a coding model on the side. Eventually I realized that running the models was actually the easy part.

The interesting problem was making multiple models, tools, users, memory systems, long-running projects and GPUs behave like one coherent AI operating system.

The user only interacts with one intelligence: OMNI MAIN.

MAIN decides whether to answer directly, browse the web, use the terminal, inspect files, work with GitHub, retrieve memory, delegate engineering work to CODER, or invoke FLUX for image generation.

Current hardware:

4× RTX 2080 Ti 22GB  
\~88GB total VRAM  
129GB RAM

Current stack:

USER  
↓  
OMNI Web UI / Telegram  
↓  
Hermes orchestration  
↓  
OMNI MAIN — Qwen3.6-abliterated 27B Q8  
↓  
├── Web / Browser  
├── Terminal / Files  
├── GitHub  
├── User + Project Memory  
├── CODER — Qwen3-Coder 30B A3B (aliberated)  
└── IMAGE — FLUX.2-klein-9B BF16

https://preview.redd.it/rng5mtbffvmh1.png?width=1484&format=png&auto=webp&s=1445da5c8403c98e26ac596b3a71669ec9bed9aa

The part I find more interesting is everything underneath the models.

OMNI has its own Resource Manager / Scheduler.

The target GPU topology is:

GPU0 + GPU1 → MAIN  
GPU2 → CODER  
GPU3 → FLUX

MAIN has the highest availability priority. Specialist work is subordinate to it.

The scheduler tracks much more than “GPU busy”.

A job is correlated with its:

user  
conversation  
request  
project  
specialist  
resource pool  
GPU allocation  
runtime  
lease

Resource state and job state are deliberately separate.

A specialist can be:

READY  
BUSY  
LOADING  
DEGRADED  
OFFLINE  
FAILED

while an individual job moves through an actual lifecycle:

DECISION\_REQUIRED  
→ QUEUED  
→ CLAIMED / STARTING  
→ RUNNING  
→ COMPLETED / FAILED / CANCELLED

Jobs are durable and owner-bound.

There are leases and heartbeats, so if a worker disappears, the system can distinguish a stale owner from a legitimate running task and recover the resource.

A PID file by itself is not proof that something is alive.

Likewise:

process exists ≠ correct process  
model resident ≠ valid job running  
GPU allocated ≠ specialist healthy  
worker disappeared ≠ job completed

If CODER is busy, OMNI also doesn’t silently queue the task or silently pretend MAIN is CODER.

The user can choose:

WAIT FOR SPECIALIST

or

LET OMNI MAIN DO IT NOW

WAIT creates a real durable specialist job.

MAIN fallback creates no fake queue entry, reservation or ghost lease.

The scheduler is designed around owner-bound queues, fairness between users, FIFO within equivalent-priority work, cancellation, queue position, leases, heartbeat and stale recovery.

The other major part of OMNI is something I didn’t expect to build when I started:

a governance and acceptance system for AI-driven development.

I became extremely skeptical of this workflow:

prompt  
→ agent writes code  
→ agent says “done”

So OMNI has permanent canonical project state.

The main sources of truth are:

ARCHITECTURE — what the system is supposed to be and which rules cannot be violated

ROADMAP — development order and phase dependencies

STATE — the authoritative current checkpoint

ACCEPTANCE — exactly what evidence is required before something is considered complete

LEGACY — deprecated components and paths that must not silently re-enter the architecture

Every substantial change follows roughly:

ARCHITECTURE  
↓  
CURRENT STATE  
↓  
PHASE SPEC  
↓  
PREFLIGHT  
↓  
CHECKPOINT / BACKUP  
↓  
BOUNDED IMPLEMENTATION  
↓  
FUNCTIONAL VERIFICATION  
↓  
END-TO-END TEST  
↓  
WHOLE-SYSTEM REGRESSION  
↓  
INDEPENDENT RE-AUDIT  
↓  
ACCEPTANCE  
↓  
STATE UPDATE  
↓  
NEXT GATE

And every capability moves through:

IMPLEMENTED  
→ VERIFIED  
→ REGRESSION SAFE  
→ COMPLETE

Those words have specific meanings.

IMPLEMENTED means the code exists.

VERIFIED means the real behavior was exercised.

REGRESSION SAFE means previously accepted functionality was tested again after the change.

COMPLETE means the acceptance gate passed.

Several things are explicitly not accepted as proof:

“The process is running.”

“The port returned HTTP 200.”

“The model exists.”

“The file exists.”

“The build succeeded.”

“The configuration looks correct.”

“The child agent said done.”

For example, when CODER completes an engineering task, its success message isn’t trusted.

The actual files are checked, expected changes are verified, unrelated files must remain untouched, tests/builds are run where relevant, and MAIN/system health is checked afterward.

If the gate fails, the phase does not advance.

The previous accepted state remains authoritative and a rollback checkpoint exists.

Another important part of the OS is continuity for long-running projects.

OMNI doesn’t treat every interaction as a disposable request that ends when the assistant stops generating.

An ongoing project can have a continuation plan attached to its authoritative state:

current phase  
accepted checkpoint  
remaining objectives  
next allowed action  
blocking conditions  
planned continuation trigger

So a long-running project can behave more like:

WORK  
→ VALIDATE  
→ ACCEPT / CHECKPOINT  
→ SAVE AUTHORITATIVE STATE  
→ SCHEDULE NEXT CONTINUATION  
→ RESUME

The important part is that continuation is governed.

When a planned trigger fires, the system is expected to reload the project’s authoritative state, architecture, current phase and acceptance requirements before doing more work.

It should resume from something like:

“Phase 8.3 is the next accepted gate. Here is the checkpoint. Here is what may change.”

not:

“I vaguely remember we were fixing recovery.”

A continuation cannot legitimately skip a failed acceptance gate just because an earlier agent claimed success.

It has to re-enter through the accepted project state.

That gives long-running projects a form of persistent, scheduled continuity without turning them into uncontrolled autonomous loops.

Memory follows a similar philosophy.

OMNI separates:

conversation memory  
long-term user memory  
project memory  
ephemeral tool state

I specifically did not want every conversation dumped into one giant vector database and sprayed back into unrelated future prompts.

Long-term memory is curated and owner-bound, with provenance, correction, deletion and conflict handling.

Project memory is separate again and tracks things like:

architecture  
current phase  
decisions  
accepted changes  
regressions  
open issues  
next step

OMNI is also currently a 3-user system.

Users have isolated conversations, projects, files, memory and specialist ownership.

The Web UI has functional areas for:

Conversations  
Projects  
Files  
Memory  
Models / GPUs  
Specialists  
Activity  
Integrations  
GitHub  
System  
Settings

Streaming, stop generation, regeneration, file/image attachments, drag/drop, pasted images, file/image analysis, coding workflows and image workflows have all gone through the acceptance system.

Phase 6, the operational Web UI, is globally accepted.

Phase 7, the advanced interaction/workflow layer, is globally accepted.

I’m currently in Phase 8: operations, observability and recovery.

Health modelling and logging are accepted. The next work is deeper stale-process identity/recovery behaviour.

There is also an important limitation.

The application architecture supports multiple users, but true parallel MAIN inference is still not where I want it.

The current Qwen/Ollama runner launches MAIN with `-np 1`.

So:

enough VRAM for multiple conversations ≠ truly parallel token generation

The product/session architecture is multi-user.

The inference layer is still more serialized than I want.

That is one of the problems I’m actively working on rather than hiding behind the word “concurrency”.

At some point this stopped feeling like a local LLM setup.

The question became:

How close can I get to building a private local AI operating system using consumer GPUs and open models?

Because there’s a big difference between:

“I can run a 30B model.”

and:

“Multiple users can use one persistent AI OS while it manages isolated memory and projects, schedules GPUs, delegates coding work, generates images, uses real tools, continues long-running projects from accepted checkpoints, recovers failed jobs, and proves after every major change that the rest of the system still works.”

OMNI CORE is private and I currently do not plan to open-source the complete system.

Almost everything underneath it exists because of open-source models and software, but I want to keep the complete orchestration/product layer private for now.

What I am considering, if people are interested, is publishing much more of the engineering process:

architecture diagrams  
real phase / acceptance reports  
GPU experiments  
scheduler design  
benchmarks  
failure postmortems  
memory architecture  
multi-user experiments  
continuity / long-running project design  
things that looked like they worked but failed independent acceptance  
detailed build logs

Potentially through Patreon if there is enough interest to justify documenting the project properly.

Not:

“pay me and I’ll give you the source.”

More like following the development of a private local AI OS from the engineering side.

If people want a technical follow-up, I can go deeper into one of these:

1. Resource Manager / Scheduler — GPUs, MAIN residency, CODER/FLUX, queues, leases, recovery and multiple users
2. Governance / Acceptance — phase specs, checkpoints, regression testing and independent re-audits
3. Project Continuity — how planned triggers resume long-running work from authoritative state instead of relying on chat memory
4. Memory Architecture — conversation vs user vs project memory
5. Multi-user inference — what actually works today, what doesn’t, and why VRAM capacity and true inference concurrency turned out to be two different problems

https://preview.redd.it/u6yzpd0fo3nh1.png?width=2048&format=png&auto=webp&s=11517f7c4a4d3a4670e6b397c37a44045fd5a9db



I built a private AI operating system on 4× RTX 2080 Ti GPUs