# OpenMuse: "compatible with any agent harness" is one ternary branch

> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/openmuse
> date: 2026-09-22
> tags: agents, architecture, security, tooling, explainer
[OpenMuse](https://github.com/CopilotKit/OpenMuse) is a self-hostable personal agent: a React Native app for iOS, Android and web, a Hono server, a Playwright browser worker, and an optional Docker Linux box the agent can type into. It is MIT, it is seven days old as I write this, and its README leads with a portability claim.

> A personal agent with a browser, terminal, files, and work that keeps going. **Compatible with any agent harness.**

Three commits ago — earlier today — that sentence ran *"…and work that keeps going and works with any agent harness."* Someone split it and softened the verb, which is usually a sign that someone checked it.

I spent a day checking it too, because this is a claim with a shape the corpus has learned to test. An abstraction over harnesses is worth exactly its adapter list. So: how many adapters are there, what does each one do, and — the question that actually matters for a product — what is still yours once the agent behind the abstraction is somebody else's?

| | |
|---|---|
| Project | [CopilotKit/OpenMuse](https://github.com/CopilotKit/OpenMuse) · MIT · TypeScript monorepo |
| Version read | `0.1.0-alpha`, commit `c79178f` (2026-09-22), 28 commits, 81 stars |
| Size | 21,952 lines of TS/TSX — 5,522 server · 8,603 client · 1,161 browser worker · 4,646 tests |
| Protocol | `@ag-ui/core` + `@ag-ui/client` 0.0.59 · `@copilotkit/runtime` 1.70.1 |
| Harness adapters | 1 (`packages/backends/src/openbot.ts`), disabled, zero non-test call sites |
| Agent backends | 3 (`sample`, `model`, `agui`) selected by one env var |
| Tools | 29 distinct names, 40 registrations, 8 with a bespoke UI card |
| Threat model | Yes — `SECURITY.md`, four sections, states its own non-guarantees |

<Callout type="note">
Everything below is read from the repository at that commit, plus the published `@ag-ui/core` and `@ag-ui/client` 0.0.59 tarballs. I did not stand up a live deployment with a model key, so where I say "this is what the code does" I mean the code, and I say so each time it matters. The project's own [verification log](https://github.com/CopilotKit/OpenMuse/blob/main/docs/VERIFICATION.md) is unusually honest about the same distinction, and is worth reading before any claim about what OpenMuse has been observed doing.
</Callout>

## Counting the adapters

`packages/backends/` contains one directory, `src/`, containing one file, `openbot.ts`, 343 lines. It has no `package.json`. It is not a workspace member — `pnpm-workspace.yaml` lists `apps/mobile` and `apps/worker` and nothing else. Nothing under `apps/server` imports it. `grep -rn OpenBotAdapter` over the whole tree returns the class itself, `tests/openbot.test.ts` — which imports it by relative path and exercises it 13 times against a fake transport — and one link in `docs/OPENBOT-INTEGRATION.md`. That is every reference.

The adapter itself is careful work. It takes an injected `OpenBotTransport` rather than calling global `fetch`, so it cannot carry a shared administrator token; it Zod-parses every response; it marks ambiguous mutations `outcomeUnknown` and refuses to retry them; and it freezes a list of the capabilities it does *not* cover:

```ts
// packages/backends/src/openbot.ts
readonly unsupportedCapabilities = Object.freeze([
  "gmail",
  "calendar",
  "pdf",
  "approval_persistence",
] as const)
```

It is also inert. `probe()` opens with `if (this.options.enabled !== true) return { state: "disabled" }`, and `enabled` defaults to undefined. The roadmap says the same thing in plain English: *"OpenBot user/session bridge, routines, and computer backend. The disabled HTTP adapter is contract-tested; it is not a live connection."*

So the adapter count for "any agent harness" is **one, disabled, and connected to nothing**. That is not the interesting finding, because the adapter is not where the portability comes from.

## The seam is five lines

The real answer is an environment variable. `AGENT_BACKEND` takes three values — `sample`, `model`, `agui` — and `apps/server/src/agent.ts`, all 64 lines of it, turns that into a choice of constructor:

```ts
// apps/server/src/agent.ts
const agents: AgentsFactory = async ({ request }) => ({
  default:
    config.agentBackend === "sample"
      ? new ConversationAgent(config, service, await auth.owner(/* … */))
      : config.agentBackend === "agui"
        ? new HttpAgent({
            url: config.agentUrl ?? "http://127.0.0.1:1/unconfigured",
            headers: config.agentToken ? { Authorization: `Bearer ${config.agentToken}` } : {},
          })
        : new ConversationAgent(config, service, await auth.owner(/* … */)),
})
```

That is the whole of it. The `agui` arm is five lines. Everything else the feature needs is a two-line reachability test in `agentConfigured()`, a three-value union and its validation in `config.ts`, and two commented lines in `.env.example` — a dozen, generously counted, across three files. The unconfigured fallback points at port 1, which is not a routable service: fail-closed, deliberately.

The three constructors are interchangeable because they all extend AG-UI's `AbstractAgent`, whose contract is one method: `run(input: RunAgentInput): Observable<BaseEvent>`. Everything else about an agent is that agent's own business.

<HarnessSeam />

This is a better design than a per-harness adapter table, and it is a weaker claim at the same time — the two facts are the same fact. OpenMuse does not integrate with harnesses. It speaks a protocol, and any harness that also speaks it can be dropped into the slot. There is no LangGraph adapter, no CrewAI adapter, no Claude Code adapter, because there is nothing for such an adapter to *do*. `.env.example` is honest about the consequence in a comment the README does not repeat:

```sh
# An optional external raw AG-UI agent replaces conversational routing only.
# AGENT_BACKEND=agui
# AGENT_URL=https://your-agent.example/run
```

"Replaces conversational routing only." Hold onto that sentence.

## What AG-UI actually specifies

Since the portability rests entirely on the protocol, it is worth pinning down what the protocol is.

AG-UI is a schema for the conversation between a front end and an agent. The client side of it, in the version OpenMuse pins, is about as small as a protocol client gets — this is the whole of `HttpAgent.requestInit` from the published `@ag-ui/client@0.0.59` bundle, with the minifier's one-letter names expanded:

```ts
requestInit(input) {
  return {
    method: "POST",
    headers: { ...this.headers, "Content-Type": "application/json", Accept: "text/event-stream" },
    body: JSON.stringify(dropNullSubagentRunIds(input)), // one normalisation pass
    signal: this.abortController.signal,
  }
}
```

`run()` is then `this.fetch(this.url, this.requestInit(input))` piped into an event parser. That is the client.

One POST. The body is a `RunAgentInput`: `threadId`, `runId`, optional `parentRunId`, `state`, a `messages` array discriminated on role, `tools`, `context`, `forwardedProps`. The response is a stream of typed events, and the agent's job is to emit them in a well-formed order — a run opens with `RUN_STARTED` and closes with `RUN_FINISHED` or `RUN_ERROR`; a message is a `TEXT_MESSAGE_START`, one or more `TEXT_MESSAGE_CONTENT` deltas, then `TEXT_MESSAGE_END`; a tool call is the same shape with `TOOL_CALL_START` / `TOOL_CALL_ARGS` / `TOOL_CALL_END`, and whoever executed the tool returns a `TOOL_CALL_RESULT`.

Two details are worth stating precisely, because "AG-UI compatible" gets used loosely.

First, the event vocabulary is bigger than the documentation says. [The spec page](https://docs.ag-ui.com/concepts/architecture) describes "16 standard event types". The `EventType` enum shipped in `@ag-ui/core@0.0.59` has **36**, including `THINKING_*`, `REASONING_*`, `ACTIVITY_SNAPSHOT`/`DELTA`, `SUBAGENT_STARTED`/`FINISHED`/`ERROR`, and `REASONING_ENCRYPTED_VALUE`. The protocol has been growing toward the things frontier harnesses actually stream. A harness that emits only the documented 16 is compatible; a client that assumes only 16 will arrive is not.

Second, and more to the point here: AG-UI is deliberately transport- and behaviour-agnostic. The docs say so — *"AG-UI doesn't mandate how events are delivered"*, and *"Events don't need to match AG-UI's format exactly — they just need to be AG-UI-compatible."* It standardises the shape of the conversation. It says nothing whatsoever about what tools exist, who may call them, what a tool result means, or who is allowed to approve an action.

<Callout type="tip">
This is the load-bearing asymmetry. A protocol that specifies the *conversation* makes agents swappable at zero cost. It also means nothing about the agent's *authority* travels over it. Every question that a personal assistant has to answer — which tools, whose credentials, what needs a human — lives outside the protocol by construction, which is to say it lives in the shell.
</Callout>

## What the swap takes with it

Here is the part that makes "replaces conversational routing only" concrete.

OpenMuse's tools are not registered on the CopilotKit runtime. They are arguments to a `BuiltInAgent` that `ConversationAgent.run()` constructs on every request, inside the branch that `AGENT_BACKEND=agui` replaces. Twenty-nine distinct tool names, forty registrations across two agents:

<ToolLedger />

Point `AGENT_URL` at your own harness and every one of those names goes away. Your harness brings its own tools, which is the entire point of a harness — but it brings them to *its* machine, not to OpenMuse's mailbox, browser profile, Docker container or PDF store. The `RunAgentInput` it receives carries messages and context. It does not carry `run_computer_command`.

The client makes the same cut from the other side, and this is where it stops being an abstract point about layering. `apps/mobile/src/chat.tsx` calls CopilotKit's `useRenderTool()` eight times, each keyed on an exact server tool name:

```tsx
// apps/mobile/src/chat.tsx
useRenderTool({
  name: "browse_web",
  description: "Follow the agent as it reads a webpage",
  parameters: displayParameters,
  render: ({ args, result, status }) => (
    <BrowserToolCard url={args.url} result={result} loading={status !== "complete"} />
  ),
})
```

`search_mail` and `read_mail_thread` draw the email card. `browse_web` draws the browser card with its live screenshot and the **Take control** button. `delegate_task`, `agent_status`, `create_goal`, `watch_page` and `remember_fact` each draw a labelled server card that deep-links into the right screen. Match by string, and no fallback that inspects an unknown tool.

So the interface — which is most of what a personal assistant *is* — is a lookup table on OpenMuse's own tool names. Here is the product being the interface, for forty-two seconds:

<Video
  src="/articles/openmuse/web-demo"
  poster="/articles/openmuse/web-demo-poster.jpg"
  alt="A screen recording of the OpenMuse desktop web app. The user types a request about a school-trip email; an inline card appears showing a search of the mailbox, then a card for a message from Lincoln Middle School about permission slips, which expands into a full email viewer. The user then asks for research on Monterey Bay Aquarium; a browser card appears inline with a live screenshot of the aquarium website, followed by three exhibit excerpts and their source URL. The user presses Take control and the same browser session opens full-width, scrolling through the site's Exhibits page, then returns to the chat."
  caption="The 42-second desktop demo committed to the repository, re-encoded and silent. Worth knowing what it is: per docs/DEMO.md the model replies come from CopilotKit AI Mock, not a live model, while search_mail, read_mail_thread and browse_web are the real tools running against a real Chromium worker and a fictional local mailbox; the capture is cut and sped up in places. Every card in it is one of the eight name-matched renderers (OpenMuse, assets/demos/2026-09-16/web.mp4, MIT)."
/>

One smaller thing in the same seam, stated because I checked it rather than because it bites. `ConversationAgent` passes the client's declared tools through a filter before handing them to the model:

```ts
agent.run({ ...input, tools: input.tools.filter((t) => t.name === "open_workspace") })
```

`open_workspace` is not defined anywhere in the repository — that string appears exactly once, in the line above. So the filter's allow-list is empty in practice and every client-declared tool is stripped. Fail-closed, and the kind of vestige that is fine until someone reads it as an interface.

## Computer use, plainly

"Browser, terminal and files" on a machine you own is the security surface, so it deserves a flat description of where the trust boundary sits. OpenMuse publishes a `SECURITY.md` that does most of this work itself, including the parts that are not flattering, and I will credit it as I go rather than pretend I found it.

<Figure
  src="/articles/openmuse/fig1.png"
  alt="An iPhone screenshot of OpenMuse's Agent computer sheet. The header reads 'Agent computer — A browser and files that stay with your agent.' Below it a blue panel says 'Computer connected · Persistent Chromium sessions · shared with your agent', then two pill tabs labelled Browser and Files, a 'Website address' field prefilled with https://example.com, an 'Open a browser session' button, and a Browser card showing a thumbnail of the Example Domain page."
  caption="The computer surface as the user sees it: a browser and a file tree, both belonging to the agent rather than to the host. (OpenMuse, assets/computer.png, MIT.)"
/>

**The terminal is a container, not a shell.** `run_computer_command` is disabled by default (`COMPUTER_ENABLED=false`) and requires you to build a local image. When enabled, the only host process the server will ever spawn is `docker`, with an argv array and `shell: false`, and the source says why in a comment that reads like a scar: *"User input is an argv element or stdin, never a host shell program. Do not add a shell fallback here."* The command itself runs as:

```
docker exec --user 1000:1000 --workdir <cwd> <container>
  /usr/bin/timeout --signal=TERM --kill-after=2s 30s
  /bin/bash --noprofile --norc -c "<command>"
```

There is no allowlist and no sandbox beyond the container — arbitrary bash, up to 16,000 characters, and the isolation is entirely in how the container was created: `--user 1000:1000`, `--read-only`, `--cap-drop ALL`, `--security-opt no-new-privileges`, `--network none`, `--ipc private`, `--memory 512m` with swap pinned equal, `--cpus 1`, `--pids-limit 128`, one `tmpfs` at `/tmp` mounted `noexec`, a single named volume at `/workspace`, `--pull never`, and an entrypoint of `sleep infinity`. No host directory is mounted, no Docker socket, no provider key, no Google token.

The part I did not expect is that the server does not trust its own `docker create`. Before attaching to any existing container it runs `docker container inspect` and checks the result against **38 conjoined predicates** — every flag above, plus the image reference, the label set, the entrypoint and command arrays, an environment allowlist, the mount list, and the network map — failing closed on the first mismatch:

```ts
// apps/server/src/computer.ts
if (!safe)
  throw new AppError(
    "Computer ownership or isolation does not match this deployment; refusing to attach",
    409,
  )
```

That is the difference between "we start it with the right flags" and "we will not talk to it unless it still has them," and it is the single most defensive thing in the repository.

**"Files" does not mean your files.** There are two file surfaces and neither is your home directory. The container tools operate inside `/workspace` through a path guard that rejects `..`, null bytes, symlinks and non-regular files, capped at 256 KB per text read or write. The app's Files are PDFs written to `DATA_DIR/files/<uuid>.pdf` with mode `0600` and addressed by generated id. No tool in the tree takes a host path.

**The browser is a separate service with its own egress policy**, and — unlike [BrowserSkill, which hands the agent the Chrome you are already signed into](/articles/browserskill) — it is never your browser. The Playwright worker runs in its own container with its own profiles, authenticates the server with a shared `WORKER_TOKEN`, and exposes no JavaScript evaluation endpoint. Destinations are checked against a hand-written public-unicast test that rejects loopback, link-local, CGNAT, documentation and reserved ranges in both families, and — the good bit — Chromium is pointed at an internal loopback proxy that resolves the hostname once and then connects to *that IP*, so a second DNS answer cannot rebind the socket somewhere private. `CONNECT` tunnels are restricted to port 443; QUIC and non-proxied WebRTC are disabled. The project states the limit of this itself: it is application-enforced egress, not a kernel firewall, and Playwright's default launch disables Chromium's own sandbox.

**What leaves the machine.** The provider key stays server-side, but the data does not stay local, and that is inherent rather than a flaw: a model that reads your mail has to be sent your mail. Concretely, in the default `model` backend, a provider request can contain up to 20 message summaries with 240-character snippets, a full thread of up to 20 messages at 12,000 characters each, up to 30,000 characters of page text per `browse_web`, and up to 128 KB of combined command output per terminal run. For delegated tasks the system prompt is built by interpolating every stored memory, the task's prior state, its evidence list and its artifact ids as JSON — so all saved personal context goes with every run of that task. Google credentials are encrypted at rest with AES-256-GCM under a versioned envelope, the authorization flow uses PKCE with an S256 challenge and a ten-minute state row, and the scopes requested on connect are read-only (`gmail.readonly`, `calendar.events.readonly`, `calendar.calendarlist.readonly`) with `gmail.send` and `calendar.events` added only when you separately grant write access.

<Callout type="warning">
The non-guarantees are the project's own words and belong beside the rest: one owner per deployment, a shared access key rather than account authentication (the owner is the literal string `local-user`), Docker sharing the host kernel and providing no hostile-tenant boundary, persistent browser profiles holding real logins, and signed URLs that are credentials until they expire. Every one of those is in `SECURITY.md`. A project that ships its own non-guarantees at week one is doing something most do not.
</Callout>

## The residue

So: the model is swappable, the tools are OpenMuse's, the interface is a lookup on OpenMuse's tool names. What is actually left in the shell, once you subtract the part any harness could supply?

Three things, and they are the product.

**First, the work that never reaches a model.** A delegated task has a `kind` — `agent`, `document`, `monitor`, `finance`, `plan` — and three of the five are decided in deterministic TypeScript with no inference anywhere: `document` runs attachment → PDF → requested field values → filled copy → prepared reply; `monitor` runs a page check with exponential failure backoff and an automatic pause after five consecutive failures; `finance` parses a CSV in exact cents. Only `agent` and `plan` fall through to `executeModelTask`. This is the split [the three-tier posts](/articles/three-tiers) argue for, shipped without ceremony: if the job has a shape, code owns it, and the model is the fallback for jobs that do not.

<Figure
  src="/articles/openmuse/fig2.png"
  alt="An iPhone screenshot of OpenMuse's PDF viewer showing a document titled 'Community visit — filled.pdf', subtitled '2 pages · 9 KB · Filled from Community visit.pdf'. Page controls read Previous, 1 / 2, Next, Zoom out, Zoom in. The rendered page is a permission form whose Participant name field contains 'Sam Rivera' and whose Parent or guardian name field contains 'Alex Rivera'."
  caption="The output of the deterministic path: the document workflow produces a new filled PDF and a prepared reply, and never asks a model to decide anything. (OpenMuse, assets/document.png, MIT.)"
/>

**Second, the durable task machine.** Tasks live in SQL with plans, checkpoints, attempt counts, leases and scheduled wake-ups. Two workers can race for the same task and one gets the lease; an expired lease is recovered; `pause`, `resume`, `cancel` and `retry` are real state transitions rather than UI affordances. Tool executions inside a model run are serialised through a promise queue, because a provider can request parallel tool calls and durable checkpoints must stay ordered. Each computer command carries an `operationId` that is hashed into an idempotency key, and re-issuing the same id with a different command is a 409 rather than a second run.

**Third — and this is the one no protocol could carry — the approval gate.**

<ProposalGate />

The model's most powerful tools are `prepare_email` and `prepare_event`, and neither sends anything. They build a proposal, flip the task to `waiting_approval`, and stop. The prompt states the rule in the imperative — *"External writes require prepare_email/prepare_event; there is no tool to approve them"* — and the code makes it structural: the only caller of `ActionService.decide` with `"approve"` is the handler for `POST /api/actions/:id/decide`, an authenticated request from the app. The two calls to `decide` inside the task engine both pass `"deny"`.

The gate is not a model-only safeguard, which is the part that convinced me it is structural rather than decorative. The `document` workflow never invokes a model at all — its reply body is a hardcoded three-line template unless you supply one — and it still ends at `prepare()` and `waiting_approval` like everything else. Nothing in this system sends mail without a stored approval, including the parts with no model in them.

What is gated is not just "did a human click yes". The proposal is hashed over its input, its connection, its target and the target's version, and approving requires presenting that hash back, so an approval is bound to the exact thing that was shown. The connection id and the Google account are re-checked at decision time, so reconnecting a different account invalidates everything pending. And the real interlock is one `UPDATE … WHERE` that re-tests status, expiry and the parent task's state in the same statement, so two racing approvals both pass the TypeScript and exactly one gets a row back.

<Callout type="note">
One small dead end I noticed while reading the state machine: `ActionProposal["status"]` declares eight values, and `"cancelled"` is never assigned anywhere in the tree. Cancelling a task denies its pending proposals instead. Harmless, but it means the type is one state wider than the machine.
</Callout>

## What this says about harnesses

The corpus has spent a lot of words on what belongs *in* a harness — [the harness as the thing that actually determines behaviour](/articles/agent-harness), [deterministic code versus decision model versus reasoning model](/articles/three-tiers). OpenMuse is the consumer-facing end of the same argument, and reading it settles something.

The harness-agnostic framing suggests a layering where the harness is the substantial part and the app is a skin over it. The code says the opposite. `HttpAgent` is a POST and a stream reader. `ConversationAgent` is 283 lines that mostly declare tools. The parts with real content are the twenty-nine tool definitions and their caps, the isolation policy and its 38-predicate re-verification, the egress proxy that pins an IP, the task machine with its leases and idempotency keys, the proposal gate with its hash binding and its `outcome_unknown` state, and the eight-entry table that turns tool names into cards a person can act on.

None of that is portable, and none of it is the model's. The swappable layer is the thin one. That is not a criticism of AG-UI — it is what a well-scoped protocol looks like. It is a correction to how a sentence like "compatible with any agent harness" tends to be read. It is true, it costs five lines, and those five lines are not where the assistant lives.

The honest version of the README's promise, which the `.env.example` comment already gets right: you can replace who does the talking. You cannot replace who holds the keys.

<ChangeMyMind>

<Falsifier claim="There is exactly one harness adapter, and it is connected to nothing.">
`packages/backends/src/openbot.ts` is the only file under `packages/backends`, has no `package.json`, is absent from `pnpm-workspace.yaml`, and is imported by exactly one file — its own test. A single `grep -rn OpenBotAdapter` over `apps/` returning a hit would overturn this. So would a second adapter appearing; the roadmap's "OpenBot user/session bridge, routines, and computer backend" line says one is intended.
</Falsifier>

<Falsifier claim="AGENT_BACKEND=agui gives an external harness zero of OpenMuse's 29 tools.">
Read from construction order: the tools are arguments to `BuiltInAgent` inside `ConversationAgent.run()`, and the `agui` branch returns `HttpAgent` instead of `ConversationAgent` entirely. I did not run it — the check is twenty minutes with a trivial SSE echo server on `AGENT_URL` that logs the `RunAgentInput` it receives. If `tools` arrives non-empty, I am wrong about the cut, and the interesting follow-up is what a harness could do with those definitions given that executing them requires the OpenMuse process.
</Falsifier>

<Falsifier claim="The interface is a string match on OpenMuse's own tool names, with no fallback.">
Eight `useRenderTool()` calls in `apps/mobile/src/chat.tsx`, each with a literal `name`. If CopilotKit's `useRenderToolCall()` has a wildcard or catch-all renderer that I missed in `@copilotkit/react-native@1.70.1`, an unknown tool from a swapped-in harness would still draw something structured, and "you get a chat pane" becomes "you get a generic card". That is a library question, settled by reading one package.
</Falsifier>

<Falsifier claim="The Linux computer's isolation is re-verified, not just requested.">
38 conjuncts in one `safe` expression in `apps/server/src/computer.ts`, with a 409 on failure. The cheap test: create a container with the expected name and labels but `--network bridge`, then start the service and confirm it refuses to attach. If it attaches, the inspection is decorative and this paragraph is wrong. The repository's own `pnpm test:computer` smoke test claims to cover disabled networking against a real daemon, so a disagreement would be visible there first.
</Falsifier>

<Falsifier claim="No tool can dispatch an external write.">
`prepare_email` and `prepare_event` set `waiting_approval` and return; `decide(..., "approve")` has one call site, a route handler. This breaks the moment any new tool calls `actions.decide` or a provider adapter directly, which is a plausible thing to add for a "send without asking me" preference. The standing check is one grep, and it is the invariant I would put a test around if I forked this.
</Falsifier>

<Falsifier claim="Three of the five task kinds never reach a model.">
`service.execute()` branches on `document`, `monitor` and `finance` before falling through to `executeModelTask`, which `agent` and `plan` share. If a future kind routes deterministic work through the model — or if the document workflow starts calling a model for field inference, which the roadmap's "more form types" line hints at — the three-tier reading of this codebase weakens accordingly. Worth re-checking on any release that adds a workflow.
</Falsifier>

</ChangeMyMind>