# Xiryn: every agent gets its own Linux desktop, and a vault it fills without reading
> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/xiryn
> date: 2026-10-13
> tags: agents, computer-use, security, privacy, multi-agent
The post that sent me here makes a large claim in one line. Xiryn is "the open-source personal agent that does everything Muse, Grok Bot and Dots do — on your own computer, without ever getting in your way." Sourav Bera posted it on 12 October with a minute of screen recording, a [GitHub repository](https://github.com/SouravBeraAkaSpeed/xiryn) and a [site](https://xiryn.com/).
Claims like that usually come attached to a thin wrapper: a chat box, a system prompt, and a call to somebody else's computer-use API. So the first thing I did was count. The repository at `ae9ad97` has about 18,000 lines of TypeScript, TSX and shell outside `package-lock.json`, of which 1,767 are stock shadcn UI primitives. There is an agent runtime, a tool belt of 22 tools, a Docker image the project builds for its agents, a Telegram bridge, and an encrypted vault with its own approval flow. It is MIT. It is not a wrapper.
What it is, mechanically, is simple to state. Each agent you create gets a Linux desktop of its own in a Docker container on your machine. The model looks at that desktop through screenshots, moves the mouse and types through `xdotool`, runs commands in a shell, and remembers things in a SQLite table. When a website wants your password or your card, the agent asks Xiryn to type it, and Xiryn types it without the model ever seeing the value. That last trick is the most interesting thing in the repository, and it is also where the README's wording runs ahead of the code.
## Two days of history
A repository's history tells you how much of it has been lived in, so I fetched all of it. There are 110 commits, every one by Sourav Bera, and 101 of them land on 11 October. The first 39, from `chore: initialize Next.js 16 project` to `chore(release): 0.1.0`, are stamped within the same minute, 18:32. Somebody finished the project first and then committed it as a series of logical steps. It is tidy, and it means the log cannot show how the code was developed.
The later commits are real iteration, and some of it is large. Version 0.1.0 ran each agent's machine on Daytona, a hosted sandbox service; 0.2.0, the same evening, replaced that with local Docker and says so in the changelog. The vault, payment approvals and the phone take-over page arrived in 0.3.0 and 0.4.0, also on the 11th. The project was called Oryn until commit `771cfd9` on the 12th renamed it.
There are no tests. CI runs `typecheck`, `lint`, `format:check` and `build`. For an app that types card numbers into web pages, that is the gap I would close first.
## One process, one file
Xiryn is a Next.js 16 server bound to `127.0.0.1:6796` and a SQLite database at `data/xiryn.db`. There is no queue and no worker. When you send a message, `POST /api/threads/:id/turn` calls `startTurn` in `lib/agent/runtime.ts`, which saves your message, assembles a system prompt, builds the tool belt, and calls the Vercel AI SDK's `streamText`.
The interesting choice is where the turn lives. Its output is pushed into an in-memory `Run` object that buffers every chunk and fans them out to however many browser tabs are listening. Close the tab and the turn keeps going; reopen it and `GET` on the same route replays the buffer and follows it live. The README calls these durable turns. They are durable against the tab, and the code comment says what they are not durable against: "Runs are kept in memory: they survive the tab, not a server restart."
The tool loop is `streamText` with four ways to stop:
```ts
// lib/agent/runtime.ts:537-542
stopWhen: [
isStepCount(maxSteps),
hasToolCall("request_takeover"),
hasToolCall("request_payment_approval"),
hasToolCall("request_secret"),
],
```
`maxSteps` defaults to 40 (`XIRYN_MAX_STEPS`). The three `request_*` tools end the turn the moment the model calls them, which is how Xiryn implements "ask the human": the agent does not wait inside a tool for an answer; it stops, and your answer arrives later as an ordinary user message that starts a new turn. On the 40th step, `prepareStep` switches tools off with `toolChoice: "none"` and appends an instruction to report what it found and what is left, so a long task never ends mid-click with nothing said.
Models come through the AI SDK's provider packages: OpenAI, Anthropic, Google, xAI, Groq, Mistral, DeepSeek, OpenRouter, and OpenAI-compatible endpoints for Ollama, LM Studio and anything else. Those are the README's "11 providers". Each agent has its own model, chosen as `provider/model`. Nothing here is a computer-use API from any vendor; every tool is a plain function tool with a Zod schema, so any model that can call tools can drive Xiryn, and any model that can also read images can drive the desktop.
## The computer is a container
Most of the engineering is here. `lib/workstation.ts` creates one container per agent, named `xiryn-ws-`, from an image the project builds on your machine out of `workstation/`. The image is Debian trixie with Xvfb, the xfwm4 window manager and an XFCE panel, Chromium, x11vnc and noVNC, `xdotool`, `scrot`, Node 24, Python, and `cloudflared`. The container is created like this:
```text
# lib/workstation.ts:131-164, abridged
docker create --name xiryn-ws- --hostname
--env XIRYN_VNC_PASSWORD=<8 random chars> --env XIRYN_IDLE_MINUTES=15
--volume xiryn-home-:/home/xiryn
--publish 127.0.0.1::6080/tcp
--shm-size 1g --cpus 2 --memory 4096m
```
The home folder is a named volume, so files and browser sign-ins survive restarts and image updates. The noVNC port is published on loopback with a random host port, and your browser connects to it directly with the container's password, which only the Xiryn server hands out. A watchdog inside the container (`xiryn-desktop`) stops it after 15 idle minutes, counting a running command or anyone watching the screen as activity, and it does this even if Xiryn itself is closed. That last detail is the kind of thing you only write after a laptop fan has told you about it.
The model sees the desktop the hard way. `screen_capture` runs `scrot` and returns a JPEG of the 1280×800 display, at quality 70 with the pointer drawn in, as base64. Every action tool, `pointer_click`, `pointer_move`, `pointer_drag`, `pointer_scroll`, `key_type` and `key_press`, runs one `xdotool` command and then waits and takes another screenshot, so the model sees the result without spending a step to look:
```ts
// lib/agent/tools.ts:139-147
async function thenLook(agentId: string, done: T, settleMs: number) {
await new Promise((resolve) => setTimeout(resolve, settleMs))
try {
return { ...done, ...(await capture(agentId)) }
} catch {
// The action happened; only the look afterwards failed
return { ...done, image: null, mediaType: "image/jpeg", width: null, height: null }
}
}
```
The waits are fixed: 900 ms after a click, 400 ms after a move, 500 ms after typing, and 1,500 ms after Enter, because Enter usually submits a form. There is no accessibility tree, no DOM and no element ids. The model reads pixels and answers with coordinates. Pixels-and-coordinates is the most portable way to do computer use and the slowest, and the [arc-driver piece](/articles/arc-cua) showed how much of a desktop agent's wall time hides in exactly these fixed sleeps. A screenshot after every action also means every step carries an image, which brings us to the part of the runtime I liked best.
## Keeping forty screenshots out of the prompt
A tool loop resends the whole conversation on every step. If each step adds a screenshot, step 40 resends 39 of them, and the total input over the turn grows with the square of the step count. Xiryn's answer runs in `prepareStep`, before every model call:
```ts
// lib/agent/runtime.ts:273-275
const IMAGES_KEPT = 2
const OUTPUTS_KEPT = 6
const STUB_CHARS = 600
```
Walking the history from the newest end, the two most recent screenshots stay as images; every older one is replaced by its text caption and the words `[earlier screenshot omitted]`. The six most recent tool outputs stay whole; older text outputs are cut to their first 600 characters with a note of how much was trimmed. Saved history gets a similar pass on the way to disk, keeping only the screenshots the agent chose to look at or show plus the latest action's, so the database does not grow by megabytes a task.
The architecture doc makes a specific claim about the effect: "A long desktop session therefore costs roughly the same per step at step 40 as at step 4." It is close, and the widget shows where it bends. The rules are Xiryn's; the token costs are my assumptions and you can move them. With a 4,000-token fixed prefix (my estimate for a system prompt of about 5,700 characters plus 22 tool schemas), 1,365 tokens per screenshot (Anthropic's published width × height / 750 rule at 1280×800), and 60 tokens of call and caption per step, step 4 resends 6,910 tokens and step 40 resends 9,070. Without the pass, step 40 would resend 59,575. Over the whole 40-step turn that is 311,905 input tokens against 1,271,500, about a quarter.
So "roughly the same" is fair, with one honest caveat: the per-step cost still rises, by about 30% from step 4 to step 40 under these numbers, because every call and caption stays in the history forever. Compaction turns a steep line into a shallow one. It also changes what the model can know. After two more actions, the screenshot it just reasoned about is gone, and if it needs to compare a price on page one with a price on page three, it has to have written the first one down in its own reply. The instructions push it to do exactly that ("write what you found into your message — every option with its details"), which is a prompt doing the job of memory within a turn.
The 132k input tokens in the flight-search screenshot above is the same arithmetic summed over a real run. I can't tell the step count from the picture, but at roughly 7,000 to 9,000 tokens a step it is consistent with a run of fifteen to twenty steps.
## Memory is a list in the system prompt
Long-term memory is a table of notes per agent. `memory_write` inserts one; `memory_forget` deletes one by id. On every turn, `buildInstructions` pulls the agent's notes newest-first until a 12,000-character budget runs out and pastes them into the system prompt under `# Memory`, each with its id, and says how many older notes were dropped.
There is no retrieval, no embedding, no summarisation and no decay. A flat list is defensible for a personal agent whose memory is a few dozen facts about one person, and it fails in an obvious way once it is not: the 12,001st character pushes the oldest fact out, whatever it was.
When someone asked on X whether they would have to re-teach yet another agent, the author replied that Xiryn "learns as you work" and saves what is worth keeping "on its own". Mechanically, that is the model deciding to call `memory_write`, prompted by a paragraph in the instructions: "Save a note in the same turn you learn something worth knowing later … Saying you will remember is not the same as saving." It works as well as the model follows that paragraph. The [TencentDB memory piece](/articles/tencentdb-agent-memory) is a good contrast for what a memory system looks like when it is designed around what may enter a cached prompt; Xiryn rebuilds the whole prompt every turn, memory included, which is simple and cache-unfriendly in equal measure.
## A squad is a relay
You can put agents in a squad and address any of them. If the addressee decides another member fits better, it calls `delegate` with that member's id and a brief. The tool flips a `current` variable in the runtime, and on the next step `prepareStep` swaps both the system prompt and the model for the new agent's. Every tool resolves "the agent holding the turn" at call time, so after the hand-off the shell, the desktop and the memory are the new agent's.
It is a relay, one agent at a time, inside one turn. Nothing runs in parallel, and a request cannot go back to anyone who already held it: `held.includes(target.id)` throws "already had this request. Handle it yourself." That one rule prevents the ping-pong you would otherwise get from two polite models each insisting the other is better suited. It is small and it is right.
## The vault types what the agent never reads
Now the part that justifies the comparison with hosted agents that shop for you. A personal agent that books flights needs your login and your card. The question is who gets to hold them.
Xiryn keeps logins, cards, UPI IDs, personal details and one-time codes in a `vault_items` table. Each item is sealed with AES-256-GCM under a 32-byte key, with the item's id and kind bound in as associated data, so a ciphertext cannot be moved onto another row. The key is not in the data folder. It comes from the operating system: `security` and the Keychain on macOS, DPAPI through PowerShell on Windows, `secret-tool` and the Secret Service on Linux, or a `0600` file in `~/.xiryn` when none of those exist. If items exist and the key is missing, Xiryn refuses to make a new one rather than silently orphaning them.
The agent gets four tools. `vault_list` returns labels and masked hints ("Visa ···· 4242 · 12/29"). `request_secret` ends the turn with a form in the chat; what you type goes straight into the vault and the agent gets back an item id. `request_payment_approval` ends the turn with a card showing merchant, amount and site, which you answer with a saved card, by taking over the screen yourself, or with a decline, in the app or with buttons on Telegram. And `vault_fill` does the typing:
```ts
// lib/workstation.ts:469-478
// Types a vault value. It goes in on stdin, so it never shows up in a
// process list, a log or a tool result.
export function typeSecret(agentId: string, value: string) {
return execOrThrow(
agentId,
["xdotool", "type", "--delay", "12", "--clearmodifiers", "--file", "-"],
{ input: value, timeoutMs: 60_000 },
)
}
```
After typing, `xiryn-mask` connects to Chromium's debugging port inside the container and sets `-webkit-text-security: disc` on the focused field in every page and frame, so the card number shows as dots in the next screenshot and to anyone watching the live desktop. The tool returns `{ filled, item, site }` and no image. The value goes from the vault to `docker exec`'s stdin to the X server, and the model's context only ever holds a hint.
Before it types, `vault_fill` checks where the browser is. `currentPage` asks Chromium on `127.0.0.1:9222` for its open tabs and asks `xdotool` for the focused window's title, and picks the tab whose title matches. Then:
```ts
// lib/agent/tools.ts:392-409
if (item.kind === "login" && !hostMatches(where?.host ?? null, item.site)) {
throw new Error(`"${item.label}" is a login for ${item.site}, but the browser is on …`)
}
if (item.kind === "card" || item.kind === "upi") {
const approval = activeApproval(env.threadId, item.id)
if (!approval) throw new Error("… Call request_payment_approval first.")
if (approval.host && !hostMatches(where?.host ?? null, approval.host)) {
throw new Error(`The payment was approved on ${approval.host}, but the browser is now on …`)
}
}
```
`hostMatches` accepts the same host or a subdomain of it, on a dot boundary, after stripping `www.`. As a phishing check it is sound: `swiggy.com.login-help.example` does not pass for `swiggy.com`. I expected the payment side to be built the same way, as a one-shot token for one amount. Reading `activeApproval` and the fill path, it is a looser thing. Step through the scenarios:
Three behaviours fall out of the code, and none of them is visible from the README.
An approval is not spent by a fill. `activeApproval` returns any approved row for this thread and this card whose `expiresAt` is in the future, and nothing marks it used. For 20 minutes (`APPROVAL_MS`) the agent can fill that card on that site as often as it likes. I suspect it is on purpose, because a payment form that fails validation needs a second attempt. It still means you approved a card on a site for 20 minutes rather than one payment.
The amount is never checked. The agent writes the amount into `request_payment_approval` and you see it; nothing compares it with the page at fill time. The approval is honest if the agent was, and the README's phrase "every payment needs your yes" is true of the first payment in each 20-minute window.
An approval recorded with no site is not site-locked. The host check is guarded by `approval.host &&`. If `currentPage()` returned null when the agent asked for approval, because the debugging port timed out at 4 seconds or no web page was focused, the approval is stored with `host: null`, the card on Telegram shows no site, and the later fill skips the host check entirely. I reasoned this from the code; I did not run it. It is a one-line fix: refuse to create an approval without a host, or treat a null host as a mismatch.
There is a softer fourth. `currentPage` matches the focused window's title against the tab list and falls back to `pages[0]`, the first tab, when nothing matches. With several tabs open and a title that changed between the two reads, the host it checks may not be the page that will receive the keystrokes.
And then there is the limit that no check addresses, because it is architectural. The fill lands in a form field of a browser that the same agent controls through a shell in the same container, and Chromium listens on `127.0.0.1:9222` there; `xiryn-mask` uses that very port. An agent that has been talked into it by a malicious page can read the field back with one `Runtime.evaluate`. To the project's credit, `SECURITY.md` says this in so many words:
> A value typed into a page is in that page, and the agent controls the browser it's in. The vault keeps secrets out of the model's context and out of Xiryn's records, and makes their use deliberate and approved; it doesn't make a compromised agent harmless.
It is the right description of what the vault is. The README's "Agents never see the values" is the marketing statement of the same thing, and the difference matters exactly when the agent has been browsing pages you did not choose.
## Open by design
The container is where Xiryn's trade-off is most visible. The agent works as user `xiryn` with `NOPASSWD:ALL` in `/etc/sudoers.d/xiryn`, so it is root inside its box. The container has the default bridge network and full internet access, no `--cap-drop`, no `--read-only`, no `--pids-limit`, no seccomp profile beyond Docker's default, and Chromium runs with `--no-sandbox` because Docker withholds the kernel features its own sandbox wants. `SECURITY.md` describes the workstation user as "unprivileged". With passwordless sudo in the image, that word is wrong; the accurate claim is that the agent is root in a container that has no host mounts and no Docker socket.
I do not think the openness is a mistake. An agent that books flights and installs a package to finish a task has to reach the internet and has to be able to `apt install`. The cost is that the boundary is Docker's default isolation and nothing else, and the blast radius of a prompt injection is "anything a root shell with your home IP address can do, plus whatever is in that browser's cookie jar". The README's best line on the matter is accurate: sites see "an ordinary home connection rather than a data center". It is why the agent gets through on sites that block cloud IPs, and it is also whose IP any abuse comes from.
The [OpenMuse teardown](/articles/openmuse) is the useful mirror. OpenMuse runs its terminal in a container with `--network none`, `--read-only`, `--cap-drop ALL`, `no-new-privileges`, 512 MB, one CPU and 128 processes, re-checks 38 predicates on that container before every attach, and keeps the browser in a separate worker with its own egress filter. It also binds an approval to a hash of the exact action and spends it once. Xiryn chooses one container that can do everything, which is why it can do things OpenMuse cannot, and why its approval needs to be tighter than it is.
The phone take-over adds one more exposed surface. When an agent hands off at a login or a payment and Telegram is paired, `xiryn-tunnel` starts a Cloudflare quick tunnel to the container's noVNC for 30 minutes and sends you the link, with the desktop's password after the `#` so it stays out of the tunnel's logs. That puts a live desktop, already signed in to whatever the agent was doing, on a public `trycloudflare.com` address protected by an 8-character VNC password. VNC's classic authentication cannot use more than 8 characters anyway. The feature is on by default; `XIRYN_PHONE_TAKEOVER=off` removes it, and I would turn it off unless I needed it.
The Xiryn server itself has no accounts. `proxy.ts` refuses any request whose `Host` or, for writes, `Origin` header is not loopback, which defeats DNS rebinding from a page in your own browser. Telegram pairing uses an 8-character code from a 32-symbol alphabet that rotates after five wrong guesses, and the bot answers exactly one account. One-time codes you reply with on Telegram are deleted from the chat and stored once in the vault. These are careful.
## The claims, one by one
"Does everything Muse, Grok Bot and Dots do." I can't check what those hosted products do from here, and the README's comparison table is about where things run, not what they can do. The README's own roadmap is the honest answer: scheduled and recurring tasks, MCP connectors for email, calendar and docs, more remote channels, and voice out are listed as not shipped. Xiryn today does chat, a desktop, a shell, web research through Firecrawl or its own browser, memory, squads, and Telegram. Without scheduling and without mail or calendar access, it is not yet a personal assistant in the sense of one that acts while you sleep.
"On your own computer, without ever getting in your way." The tools execute only in containers, and nothing from your machine is mounted into them. That holds. Each agent's box defaults to 2 CPUs and 4 GB, and two agents working at once is 4 CPUs and 8 GB of your laptop, which the FAQ admits. Running Xiryn itself under `docker compose` mounts `/var/run/docker.sock` into its container, which the compose file correctly calls root-equivalent.
"Your data stays on your disk." Conversations, memory, settings and the vault do. Everything the model reads goes to the model provider you configured, screenshots included, unless you point it at Ollama or LM Studio with a model that can both call tools and read images.
"Agents never see the values." By default, in the model's context, yes, and the stdin path plus the mask is well built. Against an agent that decides to look, no, and `SECURITY.md` agrees.
"Every payment needs your yes." Every first payment in a 20-minute window on one site does. Subsequent fills, different amounts and approvals with no recorded site do not ask again.
## Where it sits among the agents on this site
The agents I have read for this site fall on one axis: whose computer the agent is using.
| | where tools run | browser | approval to pay or send |
|---|---|---|---|
| Xiryn | one Docker desktop per agent, root, full network | its own Chromium in that box | card + site + thread, 20 minutes, reusable |
| [OpenMuse](/articles/openmuse) | sealed container, no network; separate browser worker | its own Playwright profiles | hash of the exact action, 30 minutes, spent once |
| [BrowserSkill](/articles/browserskill) | your machine, through any harness | your real, signed-in Chrome | consent switch kept in browser storage the agent cannot reach |
| OpenClaw, [Hermes Agent](/articles/openhuman) | your machine | your apps and files | per the harness |
Xiryn's position is the middle one the README draws: not your desktop, but a whole desktop. It is the right shape for the one thing it does best, which is a visible, interruptible agent working real websites from your own IP while you watch and take the wheel. The take-over and hand-back flow is the most finished part of the product: `request_takeover` ends the turn, the UI opens the desktop with input enabled, and handing back sends a message flagged `handback` that tells the agent to take a fresh screenshot before it trusts anything it remembers about the screen.
Would I run it? On a spare machine, for research and for clicking through sites I would otherwise click through myself, yes. The runtime is clean, the compaction pass is a good idea worth copying into any screenshot-driven loop, and the vault's design, values out of the model's context and typed from stdin, is better than most agent projects bother with. I would not save a card in it until an approval is spent on use and refused without a site, and I would switch phone links off. Those are small changes against a codebase that is two days old, and the author ships fast.
## How I checked
I shallow-cloned `SouravBeraAkaSpeed/xiryn`, then fetched the full history, and read it at `ae9ad97` (12 October 2026, version 0.4.1): the runtime, the tool belt, the instruction builder, the workstation and Docker modules, the image's Dockerfile and its scripts under `workstation/rootfs`, the vault and its server actions, the Telegram bridge's pairing and code paths, `proxy.ts`, `SECURITY.md`, the README, the architecture doc and the changelog. Line counts are `wc -l` over `.ts`, `.tsx`, `.mjs`, `.sh` and `.html` files; commit counts and timestamps are from `git log`. File and line references in the code blocks are to that commit.
I did not run Xiryn, did not build its image, and did not execute any of its code. The behaviours of the vault in the widget are read from the code paths named in each row; the null-host and title-fallback cases in particular are reasoned, not observed. I also did not test whether a workstation can reach the Xiryn server on the host. On Docker Desktop that route would be `host.docker.internal`, and `proxy.ts` checks a `Host` header that any client can set, so that is the first thing I would probe. Token costs in the compaction widget are my assumptions, stated in the widget; the 132k and 221k figures come from the project's own screenshots. The four screenshots are the README's, resized; the video is the post's, trimmed to 60 seconds and re-encoded without sound. I could not check what Muse, Grok Bot or Dots do.