# Qwen-CUA: a computer-use agent that only ever sees pixels
> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/qwen-cua
> date: 2026-08-03
> tags: agents, computer-use, reinforcement-learning, moe, systems
Qwen Team and XLang Lab published [Qwen-CUA](https://github.com/xlang-ai/Qwen-CUA) on 2026-08-02: a computer-use agent that never sees anything but a screenshot and never acts through anything but keyboard and mouse events. No DOM tree, no accessibility metadata, no task-specific API. The backbone is a 397B-A17B Qwen mixture-of-experts model, and a scaled variant, Qwen-CUA-Max, pushes past one trillion total parameters. The headline number is 86.2 on OSWorld-Verified. That is a real result, but it is one of eight benchmarks, and it is not even the most interesting fact in the paper.
The part worth taking apart is the context-management scheme that makes long-horizon screenshot-only control workable at all: fold the visual history in blocks of 10, not one screenshot at a time. It sounds like a minor implementation detail. It is actually the difference between a rollout fleet that reuses its KV-cache and one that recomputes a fresh prompt prefix on every single turn.
## A narrow interface on purpose
Most production computer-use systems cheat a little. They read the DOM, they call an accessibility API, they get coordinates for free. Qwen-CUA's interface is deliberately narrower than that — the model observes a screenshot and emits one action from a fixed keyboard-and-mouse vocabulary (Appendix A):
| Category | Actions |
|---|---|
| Keyboard | `key`, `key down` / `key up`, `type` |
| Mouse | `move`, `left`/`right`/`middle click`, `double`/`triple click`, `left click drag`, `left mouse down`/`up`, `scroll`/`hscroll` |
| Control | `screenshot`, `wait`, `terminate` (success or failure), `call user` |
That last one, `call user`, is the tell that this is meant to run unattended: when the task cannot proceed autonomously — a login wall, a genuinely ambiguous instruction — the agent is allowed to stop and ask, rather than guess and keep going.
This is close to the opposite design point from a coding harness. [Agent harnesses](/articles/agent-harness) walks Lilian Weng's tool taxonomy for coding agents — `bash`, `edit`, `grep`, `git_status` — and her argument for why: the tools are "deliberately simple and generic" because the model has already seen a million shell sessions in training. Qwen-CUA leans on the same instinct pointed at a different substrate. It does not give the model `bash` or a DOM query; it gives it exactly what a person gets — a screen and two input devices — on the bet that native computer use is "a sufficiently general interface for interacting with almost any software accessible to a person." The tradeoff is real: a shell command can rename 400 files in one call, while Qwen-CUA has to click, drag, and type its way through the same job one primitive at a time. The payoff is that the interface never goes stale — it works on software that has no API at all, which is most software.
## The mechanism: folding the visual prefix in blocks of 10
Screenshot-only control has an obvious cost: every turn adds an image to the context, and a long task can run for a hundred turns. Two bad options present themselves. Keep every screenshot, and the context blows past any practical budget. Use a sliding window and drop the oldest ones, and the agent forgets what it did five minutes ago — the exact state that explains why the screen looks the way it does now.
Qwen-CUA's answer is in Figure 3 of the paper: scale the *active* visual history to 20 screenshots (up from 1 in Qwen2.5, 5 in Qwen3, 10 in Qwen3.5 — successive Qwen generations have simply been raising this number), and once the active window would exceed 20, fold the oldest 10 screenshots at once into a fixed textual placeholder. The reasoning and actions tied to those folded screenshots stay in the conversation; only the pixels get replaced.
The "at once" is the whole design. Fold one screenshot per turn — the obvious way to enforce a 20-image budget — and the folded-prefix boundary moves on every turn, so the text before the newest screenshot is different from what it was a moment ago. Fold 10 at a time instead, and the boundary only moves every 10 turns: steps 21 through 30 all extend the exact same prefix. Step through it below.
Training uses the identical operator. Reinforcement-learning episodes are sliced into context-bounded chunks by advancing the same fold boundary, each slice inherits the full terminal reward, and only the model's own generated tokens count toward the loss. Train and inference see the same folding rule, which is the detail that keeps this from being an inference-time hack layered on top of training that never saw it.
## Why prefix stability is a rollout-economics problem
Here is the part the paper is explicit about and worth spelling out: a stable prefix is not a memory nicety, it is a **KV-cache reuse story**. An inference server that serves the same prompt prefix repeatedly can cache the attention keys and values for that prefix once and reuse them for every subsequent request that shares it — skipping the prefill compute for everything except the new tokens at the end. A prefix that changes on every turn gets none of that: every request looks new to the cache, so every request pays full prefill cost. The paper names this directly, citing Anthropic's cache-aware batched-pruning guidance for computer use as the precedent for the design.
Multiply that by scale. Qwen-CUA's training infrastructure is a cloud rollout fleet with close to 100,000 vCPUs and tens of thousands of concurrent environments, generating roughly 40,000 verifiable tasks' worth of trajectories. At that volume, the difference between "the prefix changes every turn" and "the prefix is stable for 9 turns out of 10" is not a rounding error in the compute bill — it is close to an order-of-magnitude difference in how much of the prefill work has to be redone per rollout step. Folding 10 at a time instead of 1 at a time is, in effect, a decision about how much of a 100,000-vCPU cluster's time goes to recomputing text it has already computed.
It is the same underlying instinct as the file-backed context strategy in [Agent harnesses](/articles/agent-harness) — treat context as a bounded, managed resource instead of an ever-growing transcript — aimed at a different bottleneck. Weng's harness spills durable state to files so the model's context stays flat. Qwen-CUA can't spill screenshots to a filesystem the model can `grep`; there is no text index over pixels. So it does the analogous thing structurally: collapse old state into a fixed, cheap textual stand-in, and do it in a way that happens to also keep the serving engine's cache warm. Same principle — bound what has to be reprocessed — solved with the tool available to a vision-and-text model instead of a coding agent.
## Training: verifiable rewards at rollout-fleet scale
The RL recipe is RLVR — reinforcement learning with verifiable rewards — using **Soft Adaptive Policy Optimization (SAPO)**, a smooth, temperature-gated alternative to PPO-style hard clipping (Gao et al., 2025, not original to this paper). The gate temperature is asymmetric: `τ_pos = 1.0`, `τ_neg = 1.05`, so tokens on non-positive-advantage trajectories decay faster than tokens on positive ones — called out as important specifically for long multimodal trajectories on an MoE backbone. Task-pool calibration runs 8 trial rollouts per candidate task and keeps only the ones with a mix of successes and failures, discarding tasks that are already saturated or unreachable.
| Config | Value |
|---|---|
| Group size (valid trajectories/task) | 16 |
| Oversampling before filtering | 20 candidates |
| Outer batch size | 128 prompts (up to 2,048 valid trajectories/update) |
| Optimizer | AdamW, LR `1e-6` constant, no warmup |
| SAPO `τ_pos` / `τ_neg` | 1.0 / 1.05 |
| Total updates | 1,000 |
| Max turns/episode | 100 |
| Max context (after slicing) | 144K tokens |
| Slice interval | every 10 turn-pairs |
The distributed setup is 512 H200 GPUs across 64 nodes, split disaggregated-style (32 training, 32 rollout, `verl`-style), with SGLang serving the rollout side. A full 1,000-update run takes about 5 days, roughly 61,440 H200 GPU-hours, holding upwards of 2,000 environments active concurrently at better than 75% average utilization. Across the training curve, the cross-domain validation score climbs from about 0.734 before RL to a peak of 0.770 at checkpoint 40 — the checkpoint the paper actually ships — before drifting slightly to 0.762 by the final checkpoint 50. That's a real, disclosed detail: the best model on the training curve is not the last one.
Data comes from three sources layered together: environment-interaction tasks built off a feature taxonomy, user-interactive tasks with a simulated user holding back task-specific knowledge (the OSWorld 2.0 setting), long-horizon tasks chained through verifiable phase states, and personalized workflows collected from human trajectories in everyday and professional software — CAD tools and Blender included — with reasoning reconstructed via model-assisted chain-of-thought from the raw (task, screenshot, action, resulting state) tuples.
## Where it actually lands: eight benchmarks, not one
The 86.2 on OSWorld-Verified is real, and it is the best score in the set on that particular benchmark. It is also the exception. Across the other seven benchmarks the picture is more mixed — Qwen-CUA leads outright on two of eight, is close behind on several, and loses outright on the rest, most notably safety. Pick a benchmark:
Scaling the same recipe to Qwen-CUA-Max (over 1 trillion total parameters) moves OSWorld-Verified from 86.2 to 87.6, and helps more on partial-credit long-horizon completion:
1T)", value: 87.6, highlight: true },
]}
/>
On the safety benchmark, RedTeamCUA, Qwen-CUA is a clear improvement over its own predecessor and a clear loss against Claude Opus 4.8. RedTeamCUA runs indirect prompt injection through ownCloud, Rocket.Chat, and Reddit environments and jointly reports benign task success and attack success rate (ASR — how often the injected instruction actually hijacks the agent):
A 20.2-point reduction in attack success versus the previous Qwen generation is a genuine gain. It is also more than 20 times Opus 4.8's ASR. The paper states its own limits plainly here: "RedTeamCUA therefore shows improved resistance to indirect prompt injection, not a deployment-safety guarantee." Worth repeating rather than softening.
## Efficiency: the gain is not longer reasoning
One honest, checkable claim in the paper: Qwen-CUA's OSWorld-Verified score does not come from generating more tokens per task. It reaches 86.2 at 3,605.8 output tokens per task; Claude Opus 4.8 needs a similar budget to reach 80.0 and roughly 21,800 tokens to reach 83.3.
The second panel is where the paper pre-empts its own obvious gotcha. On OSWorld 2.0, Qwen-CUA averages 218.9 turns per task against 83.5 for GPT-5.5 and 105.7 for Opus 4.8 — a turn count that looks far worse. But GPT-5.5 and Opus 4.8 can batch several actions into one turn; Qwen-CUA emits exactly one native action per turn by construction. The turn-count gap is mostly an artifact of how each interface packages low-level actions, not evidence that Qwen-CUA needs more attempts to do the same work. The paper says as much itself rather than leaving a reader to work it out. A related experiment adds a Bash tool alongside native computer use on MyPCBench: trajectories get shorter for every model tested, but task completion drops too, for Qwen-CUA and Qwen-3.7 specifically — the paper frames this as an unresolved "capability-efficiency frontier," not a win.
## Grading your own exam
Here is the fact that belongs next to the 86.2, not three pages after it: **XLang Lab built OSWorld and OSWorld-Verified, and XLang Lab co-authored this paper.** The lab that defines what counts as a passing score on the headline benchmark is also a lab reporting how well its own model does on that benchmark. The paper does not flag this anywhere as a conflict of interest — it is simply true of the author list and the benchmark's provenance, stated here as a fact about who is grading whom, not as an accusation of anything specific.
The eval protocol has a second, quieter honesty issue: baselines are not run under matched inference budgets. Per the paper's own settings, Qwen-3.7 is evaluated in non-thinking mode, GPT-5.5 runs with `xhigh` reasoning effort, and Claude Opus 4.8 runs at its max inference setting. "Most scores for comparison models are taken from official reports released by the corresponding benchmark or model providers" — for the ones the authors reproduced themselves, the settings differ by model, and the paper does not report what a matched-budget comparison would look like. The Gym-Anything table is the one place a second Opus 4.8 setting (medium) appears alongside max, and the two settings score 43.7 versus 47.3 — a 3.6-point swing from inference budget alone, which gives some sense of how much slack "differing settings" can hide.
There is a third thing worth naming that I could not find explained anywhere in the paper. Figure 1's legend lists six systems, not the four in Table 1 and everywhere else in the text — it adds **Muse-Spark-1.1**, scoring 80.8 on OSWorld-Verified and 47.3 on Gym-Anything. Searching the full extracted paper text, that name appears exactly once: in the Figure 1 legend. It is not in Table 1, not in the eval-settings section, not in the references, not identified anywhere else in 24 pages. I don't know what it is or why it only appears in one chart.
Two more disclosed-but-real caveats round this out. MacAgentBench's "clock" domain scored 0.0% for every model across all 12 tasks; the paper reports manually inspecting the trajectories, finding they looked like correct completions, and keeping the official 0.0% score anyway rather than quietly correcting it — which means the reported 69.2 aggregate is very likely a slight undercount, in Qwen-CUA's favor by omission, and the paper says so. And Gym-Anything's headline 46.3 runs on 97 of 197 possible environments; the other 100 were excluded because their Windows, Android, or Linux setups didn't work, not because they were held out for any principled reason. Both caveats are in the paper. Neither is in the abstract.
Finally: several contributors are marked in the author list as having departed the Qwen Team by the time this was published, including researchers who worked on the original OSWorld and OpenCUA lines. The paper doesn't explain the departures, and neither can I — it's listed here because it's a real, checkable detail about who built this and who was still there to see it ship.
## The take
Native computer use is not new — UI-TARS, OpenCUA, Aguvis, and AutoGLM already established that a single model can ground pixels to actions without a separate grounding stage. What Qwen-CUA adds is mostly an engineering answer to what happens when you actually try to run that idea at rollout-fleet scale: fold visual history in blocks, not one screenshot at a time, so a 100,000-vCPU cluster spends its time on new work instead of recomputing prefixes it already has. The paper's own framing for where this goes next is worth keeping: "we view native computer use not as the only action interface, but as the universal grounding and fallback layer of a hybrid agent" — paired eventually with something more like the coding-harness tool table in [Agent harnesses](/articles/agent-harness), not replacing it. The honest scorecard is two benchmark wins out of eight, a real safety improvement that still trails the safest competitor by more than 20x on attack success, and a headline number graded in part by the lab that wrote the exam. All three of those things can be true about a genuinely useful piece of systems engineering at the same time.
---
*Built on Qwen Team & XLang Lab's [Qwen-CUA: Native Computer Use for (almost) Everything](https://github.com/xlang-ai/Qwen-CUA) (2026-08-02). Figures 1, 3, and 6 are reproduced from the paper for commentary, flattened onto white and cropped from the original PDF; the interactive fold timeline and benchmark explorer are my own illustrations of the mechanism and Table 1's data, not measured traces. Benchmark numbers are as reported in the paper.*