After a very surprising [\#1  post of the week](https://www.reddit.com/r/hermesagent/comments/1uevy29/sharing_my_totally_nutty_hermes_setup/) on this sub, I'm releasing the OSS version of Joshu today!

[https://github.com/db-aeon/joshu-oss](https://github.com/db-aeon/joshu-oss)

Joshu is a rethinking of how to package and deploy AI agents. The design principle is that the agent should work through all the modalities of a human assistant. You and the agent should be able to cowork together as needed, the agent should be available 24/7 through voice, text, or a shared desktop.  The repo contains p

**1 – Introduction (\~1m 44s)**

Quick context on what Joshu is, why I built it, and what the OSS release means. A few important notes:

* The license is **AGPL**. You can use it for your own projects. I just ask that you don't build something that directly competes with the commercial version of Joshu.
* It's just me. One person. There will be rough edges, slop, and things that aren't polished. Please be patient — we're all moving fast.
* The two areas that got the most attention in comments were the **voice system** and the **LGUI (Language Graphical User Interface)** — how they work together is what the demo video covers in depth.

**2 – OSS Setup (\~2m 10s)**

A walkthrough of getting Joshu running from scratch. Two paths:

1. **Local install** — clone the repo, run the install script, build on your machine.
2. **VPS quickstart** — uses a Docker image. You log in, install Git, clone Joshu, set your hostname, and run the bootstrap script. It installs Docker if it's missing, sets up keys, and you're ready to go.

Once it's running, you'll land on a desktop with all the default apps. Hermes isn't configured yet out of the box — you run the **Welcome program** to set that up. It asks for your OpenRouter API key and your Google Gemini API key, and it handles the rest: installs Hermes, wires everything up. That's it.  It pulls down pinned versions of Hermes, Hindsight, Gbrain, and more.

**3 – SDK Demo / Voice / LGUI Walkthrough (\~9m)**

This is the one most of you were asking about. It gets into what "agent-first application development" actually looks like with the Joshu SDK.

The core idea is **LGUI — Language Graphical User Interface**. Instead of an agent taking screenshots and trying to figure out what's on screen, the application itself injects its current state directly into the Hermes context. The agent always knows exactly what's in the UI.

Here's what the demo covers:

* **JMAIL** — a simple email client that ships with Joshu. Your agent has its own mailbox. You have yours. JMAIL keeps track of who's who.
* Every app in Joshu has an embedded **chat**. You can ask the agent "what do you see here?" and it responds with context pulled directly from the application state — not from a screenshot. In the demo, it correctly identifies the inbox, the account, and summarizes 40 visible threads.
* I show this in **Langfuse** so you can see exactly what's being injected: identity (soul.md), skills, and then a GUI snapshot — the panel, the inbox state, subject lines, etc. The application state is baked into the system context.
* Apps also register **GUI actions** in a manifest file. These are shortcuts with voice pathways — keywords like "new mail," "compose," "write a mail" — that route directly from Gemini Voice to the application *without* going through the Hermes gateway. This is why the compose window opens almost instantly.
* For more complex requests ("draft an introduction email to Joshu"), it *does* go through Hermes and the LLM — but the result is injected back into the app directly, not into the chat thread. The compose window fills in with a subject and body. No copy-paste.
* As new apps are developed, Hermes does **auto-discovery** — it finds the app, registers its actions, and they become available to the agent. Calendar, task manager, whatever you build — it just works.

The architecture basically means your agent isn't just answering questions in a chat window, it's actually operating inside your applications, with full context and the ability to take actions.

This concept is built on several open standards and tools, including AG-UI and Copilotkit:

[https://github.com/ag-ui-protocol/ag-ui](https://github.com/ag-ui-protocol/ag-ui)  
[https://github.com/copilotkit/copilotkit](https://github.com/copilotkit/copilotkit)

Interested in further development of Joshu?  Building LGUI native apps?   Join me on the Joshu subreddit: [https://www.reddit.com/r/joshu\_backrooms/](https://www.reddit.com/r/joshu_backrooms/)



🇺🇸 Joshu Cloud Desktop for Hermes is now Open Source!