~/satyajit

Qwen-Planner-Agent: a phone agent that plans in tool calls, not taps

mdjsonmcp

2026-09-26 · 21 min · agents · qwen · benchmarks · evaluation · tool-calling · computer-use

On 2026-09-23 the Qwen account announced Qwen Intelligence, "bringing personal intelligence within everyone's reach," with three agents. I read the planner's 48-page technical report (arXiv 2609.29892), the Qwen-UI-Agent report (arXiv 2607.28227) and the MobilePA-Bench paper (arXiv 2608.23035), cloned the repositories, and parsed the replays embedded in the planner's project page.

The planner is the interesting part, and not for its score. It is a phone agent whose action space contains no tap. It plans in typed tool calls and delegates the screen to someone else. That decision explains most of what follows, including its benchmark.

Every number below is labelled. Reported means it comes from Alibaba's documents. Measured means I counted or recomputed it from files they published. Reasoned means it is my inference.

Three agents, no weights

Agent (launch post)What the post claimsPublic artifactWeights
Mobile Planner Agent"#1 on MobilePA-Bench"Qwen-Planner-Agent reportnone
Mobile-Use Agent"MobileWorld 82.1, MobileWorld-Real 92.2, AndroidDaily 97.2"Qwen-UI-Agent reportnone
Mobile Creative Agent"Image generated in 3s, about 2x faster than leading peers"none foundnone

The Hugging Face organisation Tongyi-MAI lists four models (Z-Image, Z-Image-Turbo, MAI-UI-2B, MAI-UI-8B) and one dataset, MobileWorld. ModelScope's Tongyi-MAI organisation lists the same four models. Neither has a Qwen-Planner-Agent or a Qwen-UI-Agent checkpoint (measured, 2026-09-26). The planner's README says so outright: "It is not the release repository for model weights, training code, or the agent implementation." Its repository is 17 files, one of them a 5.5 MB index.html:

Tongyi-MAI/Qwen-Planner-Agent@070038b · snapshot 2026-09-26
tracked files
17
branch
HEAD
tests
none found
source
5.5 MB
commit date
2026-09-26
source by language
HTML5.5 MB(1)

by size of tracked source at this commit, file counts in brackets; docs, data and vendored trees excluded

local clone, 2026-09-26 at 070038b — branch, commit, commitDate, fileCount, hasTests, languages

The Mobile-Use Agent's numbers are Qwen-UI-Agent's, except AndroidDaily: 97.5 in the report, 97.2 in the post. I treat Qwen-UI-Agent as the model behind the product (reasoned). Its page links Tongyi-MAI/MAI-UI, whose Qwen-UI-Agent/ folder holds a README, the PDF and images; the code there is the earlier MAI-UI. The post's arXiv link, 2608.16887, is not the planner paper: it is "An Empirical Study of Training Pixel-Space Text-to-Image Diffusion Models" (arXiv comment: "Z-Image-Pixel"), presumably the Creative Agent's research. I found no method behind "3s" or "2x", and no benchmark called "MobilePA-Bench Business" in any document.

Of the four benchmarks the post says it is "opening up," one is open. MobileWorld ships Apache-2.0 code and a Hugging Face dataset. MobilePA-Bench has a project page and a private evaluation portal. MobileWorld-Real has a description page. MobileWorld-Safety is not among the eight public Tongyi-MAI repositories.

The split: a planner with no screen in its loop

The report calls the system a Planner Model plus a Harness, which sits between the model and the phone: it assembles context, routes actions and relays feedback.

Architecture diagram. The Planner Model sends structured actions to the Harness, which contains Action Routing and Tool Dispatch, a Scenario Adapter (task guidance, routing notes, domain skills), a Memory Manager (indexing, hybrid retrieval, AI-assisted consolidation) and an Active Context. The Harness dispatches to Real-Device Execution (Executor, Environment, Verifier), which returns execution feedback. Interaction trajectories feed AI-assisted diagnosis and Harness refinement, producing version k+1 of the Harness for next-round rollouts.
The runtime: the Planner Model emits structured actions; the Harness routes them to an Executor and folds memory, Skill guidance and execution feedback back into the next context. Offline, traces feed Harness refinement (Qwen-Planner-Agent technical report, Figure 5).

Formally, at step tt the Harness builds the model's context and the policy picks from the actions currently exposed:

ct=Hη(x,mt,Kt,o≤t,a<t),at∼πθ(⋅∣ct,At)c_t = H_\eta(x, m_t, \mathcal{K}_t, o_{\le t}, a_{<t}), \qquad a_t \sim \pi_\theta(\cdot \mid c_t, \mathcal{A}_t)

Here xx is the user request, mtm_t the retrieved memory, Kt\mathcal{K}_t the loaded Skills, o≤to_{\le t} and a<ta_{<t} the observation and action history, η\eta the Harness's editable instructions, and At\mathcal{A}_t the structured action set. The report lists what that set contains: "typed tool calls, memory access and updates, skill selection and loading, clarification or refusal, and task-completion declarations." And then the sentence that matters: "The agent primarily acts through structured tools rather than pixel-coordinate GUI actions, although tools may return visual observations when needed."

So the answer to "screenshot or accessibility tree?" is neither. The planner's observations are tool returns, observable state changes and execution errors. When a task needs a screen, the planner calls control_gui_agent with an app name and a natural-language instruction, and a GUI sub-agent does the looking. The report's related-work section says it plainly: the planner "focuses on this planning problem rather than introducing a new screen-grounding architecture."

That is a different cut from the usual planner/grounder split, which MobileWorld's own code shows well. In src/mobile_world/agents/implementations/planner_executor.py, PlannerExecutorAgentMCP.predict() sends the planner LLM the three most recent screenshots (older turns are replaced with the text "(Previous turn, screen not shown)"). The planner answers with a thought and a JSON action whose target describes an element in words. For click, long_press, double_tap and drag, the nested get_executor_action() hands that description to a UI-Ins-7B grounder (agents/grounding/uiins.py), which returns coordinates; a drag costs two grounding calls. Meanwhile AndroidEnvClient.get_observation() in runtime/client.py raises ValueError for anything but screenshots: "Accessibility tree is not supported yet."

MobileWorld planner_executorQwen-UI-AgentQwen-Planner-Agent
Planner readslast 3 screenshotsscreenshot, CLI output, API responsestool returns, memory, Skill text
Planner emitsaction + element descriptioncoordinates, cli_command, api_calltool call, Skill load, memory op, sub-agent instruction
Who grounds pixelsUI-Ins-7Bthe same modela GUI sub-agent

Qwen-Planner-Agent moves the boundary up a level. Its planner does not even pick the element to click; it picks the app and writes the paragraph.

Two recorded traces

The project page embeds six recorded replays as JSON, with every tool call, argument and return. Two show the split best. Step through them below; each step lights the component that acts.

recorded replay · record MTH0073, report Figure 12

user Query today's weight, convert it from kilograms to pounds, and text the result to my coach.

Planner Model
Harness
Executor / tools
GUI sub-agent
query_health_datastep 1 / 12 · call
{"dataType": "weight", "date": "today"}

Three dependent steps, so it waits for each result before the next call.

planner acts on: the request, earlier returns, Harness context

Error recovery (record MTH0073, the report's Figure 12). Asked to read today's weight, convert it to pounds and text the coach, the planner calls query_health_data, gets 70 kilograms, then calls convert_unit with "kilograms" and "pounds". The tool returns code 400: "fromUnit or toUnit does not belong to the supported units of the specified category." It retries once with "kg" and "lbs" and gets the same 400. Its next reasoning quotes a Harness notice: "Repeated empty/error results in this user task: convert_unit (2). Stop guessing variations for those unresolved requests." The notice is not in the published message log, only in the planner's quotation of it. The planner switches to calculate_expression with "70 * 2.2046226218", gets 154.32, and sends the message to 138-1111-2222, a number saved earlier in the conversation. Five tool rounds, two of them failures.

Sub-agent handoff (record 396, the report's Figure 13). Asked to download three files from a WhatsApp group and file them into subject folders, the planner calls control_gui_agent with action: "start", an app name and a paragraph of instructions, then action: "execute" on the returned session. Only after WhatsApp reports completion does it start a File Manager session, carrying the three filenames and the folder mapping into the new instruction. The planner's entire view of two apps is four JSON returns.

One detail matters before reading any sub-agent score. Each GUI session returns three sub-steps (launch_app, operate_ui, verify_result), all carrying the same timestamp to the microsecond: 10:38:15.933623 for WhatsApp, 10:38:18.205295 for File Manager (measured). Two app sessions, 2.3 seconds apart, each finishing instantly: in this replay the GUI sub-agent is a stub returning a canned record, and the page labels its phone graphics "Illustrative device UI." That matches MobilePA-Bench, where Sub-agent success "measures delegation quality rather than downstream policy execution." The planner is graded on the handoff, not on whether anything tapped the right button.

Skills and memory live outside the weights

A Skill is loaded by a tool call. In the shutdown replay (Figure 10), one planner turn issues three calls: skill__ai_content_personal_assistant, skill__power_performance_battery and skill__conflict_order_arbiter. Each returns a loaded_tools list and a Markdown block of "Tool Families" and "Rules." The three lists hold 24, 9 and 0 tools (measured); the arbiter is pure procedure ("Power/screen blocker: shutdown, reboot, lock, or screen-off can block..."). This is MobilePA-Bench's action-space expansion, At+1=At∪G(s)\mathcal{A}_{t+1} = \mathcal{A}_t \cup \mathcal{G}(s), where loading Skill ss exposes its tool schemas G(s)\mathcal{G}(s). The Harness's Scenario Adapter decides which guidance applies. Offline it compiles a tool-to-Skill map; at request time it keeps only rules whose tools are exposed. It "neither discovers additional tools nor executes actions."

Memory is four stores with different trust levels: a compact user model, episodic records, curated long-term memory and prospective memory ("future intentions with explicit activation conditions, scope, expiry, and completion status"). Entries move through capture, validation, indexing, retrieval, consolidation and revision, and a new fact explicitly supersedes the old one. Retrieval mixes semantic and keyword search. Retrieved text is "supplied to the Planner Model as evidence rather than as an executable instruction," which is the right boundary if you have read MemHarness on what verbatim memory replay does to agents.

The long-history results say where the gains come from. With the Harness, the 27B planner goes from 21.99 to 67.24 on BEAM-10M (reported). But the untrained Qwen Baseline 27B with the same Harness averages 79.94 across the six memory benchmarks, against 80.01 for Qwen-Planner-Agent 27B, and it posts the best BEAM-1M score of all, 73.97 (reported). On long memory, the Harness does the work; planner training adds almost nothing (reasoned).

Recovery is also a training decision

A failed tool call is an observation, not an exception, so recovery is something the policy can learn. The cold start is supervised fine-tuning on curated trajectories, with a mask over turns the curation pipeline flagged as erroneous:

LSFT(θ)=−1Z∑i,j1{j∉Ei}log⁡πθ(yi,j∣hi,j)\mathcal{L}_{\text{SFT}}(\theta) = -\frac{1}{Z} \sum_{i,j} \mathbb{1}\{ j \notin \mathcal{E}_i \} \log \pi_\theta(\mathbf{y}_{i,j} \mid h_{i,j})

Ei\mathcal{E}_i is the set of erroneous turns in trajectory ii, yi,j\mathbf{y}_{i,j} the tokens of turn jj, hi,jh_{i,j} everything before it, and ZZ the count of unmasked tokens. The report adds: "Execution feedback is retained in the context, while verified recovery turns remain supervised." The model sees its mistake and the error it produced, and is trained only on what it did next. That is the recovery in the trace above, as a loss function.

Training: a flywheel, three backends and CARE

Data comes from an agentic flywheel. Task-construction agents write executable specifications (goal, resources, initial conditions, completion criteria) with no single reference trajectory. Rollouts run on three backends: programmatic sandboxes over structured app databases, LLM-simulated environments for the long tail, and "selected real-device sessions." Verified trajectories feed the cold start, resettable tasks feed online RL, and failures are kept for diagnosis. Non-mobile data (MCP tool use, coding, reasoning) is mixed in, and humans approve each data release. The stack is Roll, Megatron-Core, vLLM and Ray, with ROCK managing environments.

The report gives no counts of tasks, trajectories, GPUs or RL steps, never mentions MobileWorld, and does not say whether its sandbox shares tools with MobilePA-Bench's 212-tool sandbox. The same team built both, and the benchmark paper pitches its sandbox as "an interactive foundation for agentic reinforcement learning" (reasoned: an overlap worth asking about).

The RL stage's contribution is CARE, Competence-Aware Reward-and-Advantage Engineering. Sample a group of GG trajectories per task, score each si∈{0,1}s_i \in \{0,1\}, and let the group success rate ss pick the reward:

Ri=si+{λprogRprog,is<plow0plow≤s<phigh−λeffeis≥phighR_i = s_i + \begin{cases} \lambda_{\text{prog}} R_{\text{prog},i} & s < p_{\text{low}} \\ 0 & p_{\text{low}} \le s < p_{\text{high}} \\ -\lambda_{\text{eff}} e_i & s \ge p_{\text{high}} \end{cases}

Groups that rarely succeed get credit for verified progress; groups that mostly succeed pay a normalised execution-cost penalty eie_i. The catch is GRPO's normalisation. Divide by the group's reward standard deviation, and a fully successful group, whose rewards differ only by λeffei\lambda_{\text{eff}} e_i, comes back at unit scale whatever λeff\lambda_{\text{eff}} is: the coefficient cancels. CARE puts a floor under the denominator in that regime only:

A^i=Ri−Rˉmax⁡(σR,σanchor)+ϵ,σanchor=phigh(1−phigh)\hat{A}_i = \frac{R_i - \bar{R}}{\max(\sigma_R, \sigma_{\text{anchor}}) + \epsilon}, \qquad \sigma_{\text{anchor}} = \sqrt{p_{\text{high}}(1 - p_{\text{high}})}

The widget runs both on one toy group. The equations are the report's; the group size, thresholds and costs are mine, because the report publishes none of GG, plowp_{\text{low}}, phighp_{\text{high}} or λprog\lambda_{\text{prog}}, and gives λeff=0.1\lambda_{\text{eff}} = 0.1 only as an example. An LLM controller re-sets those four values every NN policy updates from training statistics and development-set scores.

CARE on one toy group of 8 rolloutsequations: report; numbers: illustrative
regime: efficiency refinementreward std 0.019 · anchor 0.433floor active
standard group normalisationspread (std) 1.000
CARE: success-derived floorspread (std) 0.043
okokokokokokokok

With all 8 rollouts successful, every reward is 1 minus a small cost penalty. Standard normalisation divides by that tiny spread, so the advantages come back at unit scale no matter how small the efficiency weight is: shaving tokens gets pushed as hard as finishing the task. CARE divides by at least the spread a 0.75 success rate would have, so the push shrinks with the weight. Drop to 4 successes and both columns match: success, not cost, is what separates the rollouts.

With all 8 toy rollouts successful, standard normalisation returns advantages with a spread of 1.000 at every efficiency weight; with the floor, the spread tracks the weight. Trimming tokens stops competing with finishing the task.

Two line charts over 60 training steps. Left: average accuracy for Vanilla RL and CARE both rise from about 0.85 to about 0.93, while CARE without calibration stays near 0.85. Right: average output tokens stay near 650 to 750 for Vanilla RL, fall from about 750 to about 460 for CARE, and fall fastest for CARE without calibration; a bracket marks a 32.5% reduction between Vanilla RL and CARE at step 60.
CARE against Vanilla RL from the same 27B baseline: similar accuracy, 32.5% fewer output tokens at the last plotted step. Without calibration, outputs shrink further but accuracy stalls near where it started. Smoothed means over four training environments (Qwen-Planner-Agent technical report, Figure 6).

The 32.5% is read off smoothed curves at step 60 of a 60-step plot (reported). Short, but the ablation is the right one: removing only the calibration isolates the failure mode the math predicts, compression bought with accuracy.

Last, model and Harness train in alternation: RL under fixed Harness instructions, evaluation on a held-out "evolve set," then an LLM editor revises the instructions. On the internal MobilePA-Internal, Overall goes from 82.67 (model only) to 84.23 (with Harness) to 88.50 after four rounds; on MCPMark from 38.00 to 42.26 to 46.98 after three (reported). The evolve set that steers the revisions is a development set, not an independent test.

MobilePA-Bench: the planner's home benchmark

The planner's team built the benchmark the planner is #1 on: all eleven MobilePA-Bench authors are on the planner report's author list (measured). It puts a central planner in a stateful sandbox: 1,705 tasks, 212 tools in 13 domains, a top-15 tool recall, and a 15-step budget.

Benchmark overview. A Mobile Planner Agent box contains four capability groups: Basic Tool Use (call, calendar, camera, message), Memory Usage (profile, preference, history, context), Sub-Agent Collaboration (GUI agent, AI image agent, search agent, other agent) and Skill Usage (book flight, payment, order food, meeting). Below, a Tool Executor pipeline (tool call, parameter validation, tool execution, result return) and Environment Feedback (state update, observation, event or trigger, execution status) connect to a Mobile Environment with executable tools, domain databases, mutable state and runtime logs, which feeds three checkers: tool-call, state-change and agent-behaviour.
MobilePA-Bench's evaluation loop: the planner calls tools, the sandbox executes them against live databases, and one of three checkers judges each task (MobilePA-Bench paper, Figure 2).

Tasks split 1,040 Tool Use, 376 Memory, 200 Skills and 89 Sub-agent. Each is judged by one of three checkers: an exact tool-call match, a database-delta match, or a rubric over observable behaviour. Memory and Skills tasks add gates: the gold memory IDs must have been retrieved, and the gold Skill loaded. Overall is fixed at 0.50 Tool Use + 0.20 Memory + 0.20 Skills + 0.10 Sub-agent.

The paper says its authors "fully open-source our complete infrastructure—including all 1,705 benchmark tasks." The repository, at commit 84fb7b2, contains the project page and nothing else. The page says "hidden evaluation tasks and ground truth remain private," and offers private evaluation of an HTTPS endpoint, one request per account every seven days.

The baselines were re-run, not copied. The benchmark paper (v2, 25 August) and the planner report (24 September) score the same models on the same benchmark, and the numbers disagree:

ModelOverall, benchmark paperOverall, planner reportMemory, paper → report
Claude Opus 575.5275.7158.51 → 71.81
Claude Fable 575.3174.5362.50 → 76.33
Kimi K373.0169.6463.56 → 71.01
Gemini 3.1 Pro71.1868.1648.67 → 65.69
Qwen 3.8 Max72.5171.7764.63 → 73.14

The shift is systematic across all ten models the two documents share: Memory rose for every one, by 6.91 to 19.15 points, and Tool Use fell for every one, by 2.31 to 6.64 (measured, from the two tables). Something in the benchmark or its setup changed between August and September. The report says neither what nor how many runs each row is.

The lead is 0.21 points. Qwen-Planner-Agent 27B scores 77.05; GPT 6 Astra 76.84 (reported). The benchmark paper ran Qwen3.6-27B three times under identical settings and got an Overall standard deviation of 0.22 and a range of 0.41, from 57.22 to 57.63 (reported). By its own benchmark's measured noise, the #1 is a tie (reasoned).

The lead also depends on the weights. The four capability columns recompute to the published Overall within 0.01 for every row (measured), so reweighting is legitimate arithmetic. The planner stays first under the official weights, equal weights and weights proportional to task count. Move a little over two points of weight from Tool Use to Skills, where GPT 6 Astra scores 93.25, and first place changes hands; give Sub-agent a large share and Claude Fable 5, at 68.54, takes it (reasoned, computed from reported scores).

MobilePA-Bench · report Table 1 · reweightedscores reported; weights yours
1Qwen-Planner-Agent 27B77.05
2GPT 6 Astra76.84
3Claude Opus 575.72
4Claude Fable 574.53
5GLM 5.373.88
6Qwen-Planner-Model 27B71.90
7Qwen 3.8 Max71.77
8Qwen-Planner-Agent 35B-A3B69.91

bars span 50 to 90 · top 8 of 17 rows

Leader under these weights: Qwen-Planner-Agent 27B, by 0.21 points. The official weighting puts Qwen-Planner-Agent 27B first by 0.21. Move a little over two points of weight from Tool Use to Skills and GPT 6 Astra takes first place; give Sub-agent a large share and Claude Fable 5 does. The purple rows are the Qwen checkpoints, with and without the Harness.

Two more things about that table. First, the #1 row is a system: the planner plus its Harness. The same checkpoint without the Harness, Qwen-Planner-Model 27B, scores 71.90, behind four of the frontier rows. The Harness is worth 5.15 points (reported). The caption defines the Harness only for the Qwen rows; it does not say the frontier models ran with it, and the Qwen Baseline 27B with the Harness is not in the table. Second, cost. The report prices the planner at $2.41 per 1,000 tasks against $3.06 to $67.76 for the others, counting output tokens including thinking, at a rate it does not list for a model it does not sell (reported). The benchmark's own page counts "visible output tokens only," excluding hidden reasoning, and lists Claude Opus 5 at $6.54; the chart data behind the report's Figure 1, shipped in the project page, lists it at $19.74. Same benchmark, same team, two cost definitions.

Three panels. (a) Overall bar chart: Qwen-Planner-Agent 77.1, Qwen 3.8 Max 71.8, Seed 2.1 Pro 64.9, Gemini 3.6 Flash 69.6, Kimi K3 69.6, GLM 5.3 73.9, Claude Fable 5 74.5, Claude Opus 5 75.7, GPT 6 Astra 76.8, on an axis from 60 to 80. (b) A radial chart of Tool Use, Memory, Skills and Sub-agent scores per model on independent truncated axes. (c) Performance against output cost in US dollars per 1,000 tasks, with Qwen-Planner-Agent at the far left near 2 dollars and GPT 6 Astra at the far right beyond a broken axis near 68 dollars.
The headline chart. The Overall axis starts at 60, the capability panels use truncated axes, and the cost axis is broken between 22 and 60 dollars (Qwen-Planner-Agent technical report, Figure 1).

MobileWorld: the Mobile-Use agent's leaderboard

MobileWorld is a Tongyi Lab benchmark (Kong et al., ACL 2026): 201 tasks over 20 apps, running in Docker with self-hosted app backends so results are checked against real databases. It has three categories: 117 GUI-Only, 44 User-Interaction (the agent must ask a simulated user) and 40 MCP (the agent must mix GUI actions with tool calls). The step budget is 50.

Tongyi-MAI/MobileWorld@e41d147 · snapshot 2026-09-26
tracked files
2,464
license
Apache-2.0
branch
HEAD
tests
3 files
source
2.1 MB
commit date
2026-09-22
source by language
Python1.9 MB(302)HTML159.6 kB(4)CSS41.0 kB(2)Shell29.5 kB(11)Makefile3.3 kB(1)Dockerfile3.0 kB(1)

by size of tracked source at this commit, file counts in brackets; docs, data and vendored trees excluded

local clone, 2026-09-26 at e41d147 — branch, commit, commitDate, fileCount, hasTests, languages, license, licenseFile, shallow, testFileCount

shallow clone: counts describe the pinned tree, not the history

The leaderboard is site/leaderboard.json in that repository. It has 41 rows; 19 publish trajectory bundles, 28 report a User-Interaction score and 12 report an MCP score (measured). The default view shows the General and Specialized types on GUI-Only tasks, with agentic frameworks toggled off. Submission means running mw eval, bundling the logs and emailing them; other rows are the MobileWorld team's own runs (several notes say "we"). Nothing is re-run by a third party, and nine of MobileWorld's thirteen authors are on the Qwen-UI-Agent report's author line (measured).

Qwen-UI-Agent's row reads 82.1 on GUI-Only, with no User-Interaction and no MCP score. Its bundle is public, so I recounted it: 117 tasks, 96 scored 1.0 by the evaluator, 82.1% (measured); GPT-5.6-Sol's bundle gives 82 of 117, its 70.1. The arithmetic is honest. The coverage is the problem: the post sells the Mobile-Use Agent as "API-first with GUI fallback," and MCP is the category that tests mixing tool calls with GUI actions. There is no MCP number. The report's table is "the GUI-only subset of MobileWorld"; at a 100-step budget the 27B reaches 85.5. The leaderboard also carries Kimi-K3 at 74.4 on the same subset, a row the report's table does not include, while the table's Seed 2.1 Pro (73.2) and Claude Opus 4.8 (67.5) are absent from the leaderboard.

The other two Mobile-Use numbers come from real phones. MobileWorld-Real is the team's own: 409 human-written tasks over 104 apps, held out from training, judged by AutoJudge, five VLM judges voting on the full action-and-screenshot trace. Runs judged env_error (CAPTCHAs, expired logins, network failures) are "excluded from the success-rate denominator." AutoJudge matched expert labels on 92.8% of 666 trajectories (reported). AndroidDaily is external (Sui et al., arXiv 2605.27761), and the 97.5 against Seed 2.1 Pro's 95.2 is the report's claim.

So "SOTA" means three different things. For the planner: highest Overall under a fixed 50/20/20/10 weighting, among models its team chose and ran, on a benchmark its team wrote, by 0.21. For the Mobile-Use agent on MobileWorld: top of the GUI-Only subset. On MobileWorld-Real: top on a held-out set its team wrote and its judge scores.

What Qwen-UI-Agent adds

Qwen-UI-Agent is the half of the system that does look at the screen. At each step it reads up to three channels: "the current screenshot," command output and API responses. Its action table has eight GUI actions (click, drag, type, open and the like, positions given as coordinates), plus cli_command, api_call, ask_user and terminate, and one model turn may emit a batch of actions (more than 30% of computer-use outputs are batched, reported). The 27B is the primary variant, with 35B-A3B and 4B siblings. Training ran on more than 100 physical phones and over 150 apps, with online RL on trajectories "exceeding 100 turns" and about 10,000 concurrent environments.

The published MobileWorld bundle shows what the loop looks like. The model emits normalised coordinates; [500, 410] became a tap at (540, 984) on the 1080 by 2400 screen. The actions across 117 tasks are mostly click (1,613) and drag (501). Episodes average 25.1 steps, median 20, and the longest hit the 50-step cap (measured). This is the screenshot-only, coordinate-emitting design Qwen-CUA takes on the desktop, plus a shell and an API escape hatch.

Neither report says that control_gui_agent is Qwen-UI-Agent. The product pairing makes it likely (reasoned). No document evaluates the two together: MobilePA-Bench stubs the GUI side, and MobileWorld never sees this planner. That is the question I would most like answered.

What holds

If you build agents, the transferable part is where the line is drawn. Five Jev harnesses put a model that cannot write inside a runtime that does the rest; Qwen-Planner-Agent draws the line higher, a model that writes arguments and instructions but never touches a pixel. Its own ablations say the runtime carries a large share of the score, the harness effect again, so ask for any planner result with and without its Harness before comparing it with a model. Agent harnesses covers that loop in general, and Qwen3.8-Max, a baseline in these tables, is the same lab's frontier model.

Cite this article

For attribution, please use the following reference or BibTeX:

Satyajit Ghana, "Qwen-Planner-Agent: a phone agent that plans in tool calls, not taps", ai.thesatyajit.com, September 2026.

bibtex
@misc{ghana2026qwenintelligenceplanneragent,
  author = {Satyajit Ghana},
  title  = {Qwen-Planner-Agent: a phone agent that plans in tool calls, not taps},
  url    = {https://ai.thesatyajit.com/articles/qwen-intelligence-planner-agent},
  year   = {2026}
}
share