~/satyajit

Qwen3.8-Max: 16 days, 265 commits, zero humans in the loop

mdjsonmcp

2026-08-03 · 22 min · qwen · agents · llm · benchmarks · moe

Alibaba announced Qwen3.8-Max today: 2.4 trillion parameters, 95B active, built on the architectural foundation of Qwen 3.5. It is also, by Alibaba's own framing, the first Qwen-Max-class model getting open weights at all — those weights are announced, not released; they ship "next week." Right now the only way to use Qwen3.8-Max is the API, through QwenCloud.

A 2.4T/95B split puts activation sparsity at about 25× (2,400 / 95). That is close to Kimi K3's roughly 27× (2.78T total, 104.2B active) — two labs, released weeks apart, converging on almost the same ratio of total-to-active parameters at the very top of the open-weight-adjacent scale. Where K3 backs that ratio with a 47-page technical report anyone can audit against a released config.json, Qwen3.8-Max's architecture claims are, for now, a paragraph in a blog post. The open weights next week will be the point where the second half of that comparison becomes checkable.

A grid of twenty benchmark bar charts spanning coding, work, and multimodal evaluations — SWE-Pro, TerminalBench-2.1, PaperBench, FrontierSWE, QwenReactBench, CoWorkBench, JobBench, Agents' Last Exam, BabyVision, CharXiv, ERQA, PerceptionBench, LVBench, Vision2Web, MobileWorld, OSWorld-Verified, and others — each comparing Qwen 3.8 Max, Qwen 3.7 Max, Qwen 3.7 Plus, Opus4.8, Fable5, Gemini3.1-Pro, and GPT5.6 Sol (max). Qwen 3.8 Max, the leftmost bar in each panel, leads or is close to the leader in most of them.
Qwen3.8-Max's own performance snapshot across twenty benchmarks — the full comparison lives in the tables further down (Alibaba/Qwen, 2026).

But the benchmark grid is not the interesting part of this release. The interesting part is what Alibaba says the model did completely unsupervised, for days at a time.

The case studies are the real headline

Every frontier lab now publishes agentic benchmark numbers. Fewer publish concrete, checkable claims about what their model actually built when nobody was watching it. Qwen3.8-Max's release includes five of those, spanning a 24-hour coding contest to a 365-simulated-day economy, and they are, collectively, some of the most specific long-horizon autonomy claims I have seen from any lab this year — specific enough that at least one of them (the coding harness) has a public commit history you can go read yourself.

five autonomous, no-human-in-the-loop runs24 hours → 365 sim-days

oh-my-cli — a self-evolving harness

~16 days
fully autonomous, as of Jul 30 2026
commits
265
PRs merged
127
issues
151

An issue state machine moves work through ready → leased → active; agents claim issues, run E2E/unit/build/desktop-lifecycle tests after every change, and route failures back to the same issue for repair — continuously evolving its own /goal, /resume, and session-replay features from community and user feedback.

public trace on GitHub

Different units, same shape: pick a goal, run for a long time with no one checking in, and produce a result that would normally need a team. The chip-design run is the one with a public, inspectable turn-by-turn trajectory beyond the GitHub trace above — worth its own chart, just below.

16 days, no one watching: oh-my-cli

Alibaba tasked Qwen3.8-Max with building oh-my-cli — a CLI tool — from an empty repository, and kept it running. The loop it built for itself: an issue state machine moves work through ready → leased → active; an agent claims a task, implements it, and triggers Build, Unit Test, E2E, and Desktop Lifecycle validation; failures route back to the originating issue for another pass; passing PRs merge. Community feedback and the model's own test results both feed back in as new issues, so the harness is quite literally evolving its own capabilities (/goal, /resume, Dynamic Workflow, Session Replay, Desktop) as it runs.

As of July 30, 2026 — about 16 days in — the repository held 265 commits, 127 PRs, and 151 issues, all without a human merging, reviewing, or filing anything. What makes this claim unusually checkable is that the trace is public: github.com/qwen-code-dev-bot/oh-my-cli. Most "our agent ran autonomously for weeks" claims ask you to take the vendor's word for it. This one, you can go read commit-by-commit.

Reproduce a paper, then beat it

Handed only a citation — arXiv 2605.22389, "Unified Data Selection for LLM Reasoning" — and a set of GPUs, Qwen3.8-Max had to write the entire pipeline from nothing: no starter code, no scaffold. The paper's claim is that when you have more training data than compute to use it on, the examples worth keeping are the ones full of "hard decision points" — places in a worked solution where the model was genuinely torn between next steps.

Over roughly 125 hours (about five days) of continuous, unattended work, Qwen3.8-Max wrote about 7,600 lines of code, took over 1,100 actions, and ran 33 rounds of GPU training. The first ~37 hours went into rebuilding the paper's pipeline from zero and reproducing all six of its findings — including the headline result, that the paper's selection method beats random selection by +7.7% on AIME24 after fine-tuning Qwen3-8B on the selected data.

Then it kept going. The next ~88 hours ran a self-improving loop — form a hypothesis, write the code, run it on GPUs, analyze the result, try again — across four rounds and 18 self-generated ideas, each round's diagnosis shaping the next round's hypothesis:

RoundBest idea that roundAIME24Gain vs. baseline
Paper's method, reproduced (baseline)49.58%
1Split the data by difficulty before selecting50.42%+0.84
2Weight examples by an entropy–score gap51.67%+2.09
3Tune the selection width51.25%+1.67
4Count the hard decision points ("nhighgate")52.29%+2.71

The final method — round 4's "nhighgate" idea — is not a tweak Alibaba fed it. It is something the model proposed, tested, and kept because it worked better than the paper it was asked to reproduce.

Beat 526 human teams in 24 hours

Alibaba entered Qwen3.8-Max into a live competition on its own Tianchi platform — the WWW2025 Multimodal Dialogue Intent Recognition Challenge — against 526 human teams, under a strict 24-hour clock. The task: read customer-service conversations, text and screenshots both, and classify what the customer actually wants.

Qwen3.8-Max read the rules and built a solution in code: fine-tuned and ensembled BERT, MacBERT, and RoBERTa for the text side; fine-tuned Qwen2.5-VL-7B, backed by Chinese-CLIP for uncertain cases, for the screenshots; then fused everything through a weighted-voting system, calibrated by cross-validation and re-weighted after every submission. Across 45 submissions in 24 hours, accuracy climbed from 0.60 to 0.853, finishing ahead of 458 of the 526 human teams — 87% of the field.

Autonomous chip design: a staircase that keeps stepping down

The most technically specific of the five case studies is silicon. Qwen3.8-Max was given a stub RTL workspace, an empty-module task description, and an evaluation script — no golden reference design, no human in the loop — and told to design a GCD/RSA cryptographic hardware accelerator that stays bit-exact across 4-, 6-, 8-, and 16-bit configurations under a randomized cocotb verification framework, while minimizing synthesized gate count.

Over one continuous run of about 500 turns and 71 evaluations across 13 milestones, working entirely in a sandbox wired to Iverilog (simulation), Yosys (synthesis), and OpenROAD (physical layout), the model's first functionally correct design measured 8,298 gates. It drove that down to 678 gates — a 91.8% reduction — and the milestones along the way show real architectural rewrites happening deep into the run, not just early low-hanging fruit:

chip design · gate count over ~500 turns8,298 → 678 gates (91.8%)
7001,0002,0004,0008,0000100200300400500turngates (log)
turn (drag)

2,010 gates Modulo divider → shift-subtract

Replaced the 16-bit hardware modulo divider in modular_multiplier with an iterative shift-subtract architecture — the single largest step, −6,288 gates, over 80% of the total reduction.

die area
106×106 46×46 µm² (−81%)
wirelength
33,369 4,187 µm
timing slack
−4.46 ns +0.66 ns at 500 MHz

Drag the turn marker or jump between milestones. The trajectory only ever goes down, and the biggest single cut — replacing a hardware modulo divider with iterative shift-subtract at turn 22 — happened early, not late, and still accounts for most of the total reduction. The later milestones (turns 60–500) show the model finding real, if smaller, structural wins hundreds of turns in, rather than plateauing after the first big rewrite. Physical layout (OpenROAD, Nangate45) confirms the front-end optimization actually routes: the die shrinks 81% and timing closes at 500 MHz with positive slack.

The single biggest step, by far, was recognizing that the 16-bit hardware modulo divider inside modular_multiplier could be replaced with an iterative shift-subtract architecture — one change, at turn 22, cutting 6,288 gates in a single move, over 80% of the entire reduction. Everything after that is smaller, later, and arguably harder: register and FSM pruning at turns 60–113, module fusion at turns 170–252, and gate-level refinement all the way out to turn 500. A model that only found the big early win and then plateaued would be a much less interesting story than one that kept finding real (if progressively smaller) structural improvements for 400 more turns.

Alibaba then re-ran the final RTL through a physical place-and-route flow (OpenROAD, Nangate45) to check whether the front-end gate-count win actually routes. It does: the die shrank from 106×106 to 46×46 µm² (−81%), wirelength dropped from 33,369 to 4,187 µm, and the design closed timing at 500 MHz with positive slack (+0.66 ns), up from a failing −4.46 ns at the start. Optimizing gate count without checking place-and-route is a common way to produce a design that looks good on paper and doesn't actually work in silicon; Alibaba closed that loop.

365 simulated days of running a business

The last of the five case studies is not a coding task at all. E-Commerce Bench simulates a full year of operating online stores against desensitized real Taobao/Tmall transaction data — 12 store types, 60 product categories, nearly 600 suppliers, 7,000 products — starting from ¥100,000 in capital. The model has to choose products, negotiate with suppliers, manage inventory, price dynamically, and handle returns, all while surviving seasonal demand swings, sudden supply shocks (typhoons, material shortages), and a settlement system with real cash-flow pressure. Buried in the supplier matrix: 152 fraudulent merchants, running classic scams — membership-fee traps, low-price bait, goods not as described.

Two things stand out in how Qwen3.8-Max played this. First, supplier negotiation is modeled with distinct personalities and concession strategies per supplier, and Qwen3.8-Max's negotiation efficiency measurably improved over the year — the same products from the same suppliers got progressively cheaper, and that experience generalized to similar products, where Alibaba says other models' negotiation efficiency plateaued mid-year. Second, it front-loaded capital early to establish position rather than playing conservatively, then converted the resulting inventory and operating gains back to cash before the simulation ended — timing that matters because unconverted assets left on the books at year-end hurt the final score.

The result: a final balance of ¥416,252 — a 4.16× return — 38% ahead of second-place GLM 5.2 and 152% ahead of Qwen3.8-Max's own predecessor, Qwen3.7-Max. Alibaba frames this as evidence of "adaptive learning from transactional feedback" across more than 2,000 rounds of interaction, rather than a model that locked in a strategy early and rode it out. That framing is plausible given the negotiation-efficiency detail above, but it is worth remembering this is Alibaba's own simulation, built on Alibaba's own marketplace data, scored by Alibaba.

Scaling real-world work

Underneath all five case studies is an infrastructure bet Alibaba is explicit about: jointly scaling RL environments and compute lifts "general working competence" across several harnesses at once (QwenWork, Claude Code, Codex, OpenClaw, Hermes), and doing that required three things to scale together rather than one at a time — environments along independent axes (task, workspace, harness) so growth compounds instead of requiring bespoke integration per new environment; a universal reward system unifying execution-based checks, rubric-conditioned judging over text and rendered visual output, and agentic inspection, so there is one reward mechanism instead of a pile of task-specific verifiers; and an online data balancer that keeps every training batch balanced across task, difficulty, workspace, and harness, which is what keeps gradient variance from blowing up RL training at scale.

A line chart titled 'Score Index across 10+ Benchmarks vs. RL Training Envs.' An aggregate score index climbs from an SFT baseline of 0.474 at zero training environments to a peak of 0.725 at 4,000 environments, marked as the best checkpoint, then drifts down slightly to 0.689 by 5,000 environments.
Aggregate score across a suite of in-house and public working benchmarks, tracked against how many RL training environments were scaled in (Alibaba/Qwen, 2026).

That chart is worth reading carefully rather than just squinting at the upward trend: the curve peaks at 4,000 environments (0.725) and is already down to 0.689 by 5,000 — the shipped checkpoint is not the last one on the curve, which is the kind of disclosed detail that makes the rest of the curve more credible, not less.

Three grouped bar charts titled 'Cross-Harness Generalization Performance,' for CoWorkBench, WorkspaceBench, and JobBench. Fable5, Opus4.8, and Qwen3.7-Max are each shown in a single native harness (OpenClaw or OpenCode); Qwen3.8-Max is shown across five to six different harnesses per chart (QwenWork, Claude Code, Codex, OpenClaw, Hermes, and for JobBench, OpenCode), boxed together as a 'Cross-Harness Group,' with scores clustered closely together across all of them.
Qwen3.8-Max evaluated across five-plus different agent harnesses lands in a tight band on all three benchmarks shown (Alibaba/Qwen, 2026).

That second chart is the practical payoff of training against a harness-agnostic reward system: Qwen3.8-Max does not have one harness it happens to be tuned for. Point it at QwenWork, Claude Code, Codex, OpenClaw, or Hermes and the CoWorkBench score moves in a band of about 73–76; Fable5 and Opus4.8, each shown in only their own native harness, land in a similar range without ever being tested for harness portability the same way. This is directly the concern The harness effect raises from the other side — that orchestration, not the model, is what actually determines an agent's cost and reliability on a task — and Qwen3.8-Max's answer is to train the reward system to not care which harness is wrapped around it, rather than picking one harness and optimizing hard for it.

The same Dynamic Workflows capability that lets it self-orchestrate shows up in a quant-research vignette Alibaba includes alongside the five headline case studies: given a one-line task description, Qwen3.8-Max built a complete ETF-rotation strategy over several hours, pruning overfit factors when it noticed design-period and validation-period metrics diverging, and separately parallelized factor mining from six short descriptions into 50 research directions each, dispatching roughly 330 sub-agents through about 6,000 backtests to find factors with excess Sharpe ratios of 0.64–1.48. Whether that generalizes past a demo is unverifiable from a blog post, but the mechanism described — noticing an overfitting signal and automatically triggering pruning, mid-run, without being told to — is the same "acting on evidence instead of a fixed script" pattern that shows up in the chip-design and paper-reproduction case studies above.

Multimodal and hybrid agents

Qwen3.8-Max's visual pipeline gets a similar "watch itself work" framing: while executing a task, the model inspects its own intermediate results — page layout, object orientation, spatial relationships, animation quality — and revises when something looks wrong (a television facing backward, a misaligned interface). Alibaba's phrase for this is a "native feedback loop across planning, execution, verification, and iteration," which is a reasonable description if the examples given hold up, though none of them are independently reproducible from the blog post alone.

The concrete new benchmark here is RecreationBench: the model observes a real running application as a black box — no source code, no network access — across five platforms (Ubuntu, macOS, Windows, Android, web), and has to rebuild the whole thing from scratch through interaction alone. Alibaba frames Qwen3.8-Max's showing here as "frontier-level Hybrid Agent capability" — the pairing of writing code (does the heavy lifting) with operating a GUI directly (reaches whatever a human can see and click, and reports back what a live system actually does).

That second half — driving a computer through screenshots and input events alone — is exactly Qwen-CUA's whole premise, published by a different Qwen team one day earlier. It is worth putting the two numbers next to each other: Qwen3.8-Max reports 86.1 on OSWorld-Verified; Qwen-CUA, a dedicated 397B-A17B computer-use specialist trained specifically for this, reports 86.2. A general-purpose 2.4T model and a purpose-built computer-use agent land within a tenth of a point of each other on the benchmark that agent was built for — which either means Qwen3.8-Max's general agentic training has genuinely absorbed computer-use skill, or that OSWorld-Verified has a ceiling both are bumping into. Both readings are consistent with the data; the blog post doesn't say which.

The benchmark tables — and where they don't hold up

Here is the full picture, reproduced from Alibaba's own release. The pattern is not "Qwen3.8-Max wins everything" — it wins some things outright, loses some things clearly, and several of its best numbers come with an asterisk worth reading before you trust them.

Coding Agent

BenchmarkOpus4.8Fable5GPT5.6 Sol (max)Qwen3.7-MaxQwen3.8-Max
Terminal Bench 2.184.684.688.874.586.6
SWE-bench Pro69.280.064.660.667.7
DeepSWE 1.159.070.073.021.656.6
NL2Repo-Bench69.4----47.255.9
FrontierSWE70.088.8--40.773.5
MLS-Bench-Lite42.849.946.231.741.0
PaperBench80.388.890.564.893.0
AndroidBench69.884.574.056.575.1
QwenSWEBench84.086.373.563.480.7
QwenQoderBench62.763.153.836.858.4
QwenReactBench16941770156415381724
QwenSVGBench16481690175814991713

General Agent

BenchmarkOpus4.8Fable5GPT5.6 Sol (max)Qwen3.7-MaxQwen3.8-Max
CoWorkBench72.375.971.564.674.8
WorkSpaceBench66.868.765.661.467.7
JobBench48.457.445.431.353.4
SkillsBench65.170.973.561.270.2
Agents' Last Exam (Pass / Score)27.0 / 45.1-- / --30.6 / 53.611.8 / 31.127.0 / 52.4
Automation-Bench (Pass@1)27.229.129.714.227.3
Toolathlon Verified (Pass@1)76.277.974.949.772.5
WideSearch72.981.2--75.281.9
HLE w/ tools57.964.558.053.556.2

General Capabilities

BenchmarkOpus4.8Fable5GPT5.6 Sol (max)Qwen3.7-MaxQwen3.8-Max
GPQA Diamond92.092.694.192.492.6
HLE45.753.347.241.443.6
IFBench62.263.572.779.182.8
$OneMillion-Bench (expert score)41.855.953.844.452.5
HealthBench52.4--55.354.560.2
PLawBench69.670.272.358.973.2
PRBench-Legal52.757.657.648.557.6
PRBench-Finance51.955.855.546.858.3
MRCR v2 256K (8-needle)83.2--93.886.792.9
LongBench v269.1--67.165.366.3

Explore the same numbers benchmark-by-benchmark, with the eval-setup note attached to whichever row has one:

full benchmark tables · wins and lossesharness caveat shown per row
Opus4.8
84.6
Fable5
84.6
GPT5.6 Sol
88.8
Qwen3.7-Max
74.5
Qwen3.8-Max
86.6

Qwen3.8-Max trails the leader (GPT5.6 Sol) on Terminal Bench 2.1 by 2.2 points — 86.6 vs 88.8.

Eval setup: Qwen3.8-Max: Claude Code, avg@10, 5h timeout, max_tokens 131,072. All other models: best published score across harnesses (Artificial Analysis for Opus/Fable5, OpenAI's own post for GPT5.6 Sol) — best-of-published vs. avg@10 is not the same comparison.

Every number above is copied from the source's own two tables — nothing here is estimated. Switch categories and benchmarks to see the honest spread: outright wins (PaperBench, IFBench, PLawBench, Parametric CAD Bench), close losses (Terminal Bench 2.1, WebArena-Verified), and clear losses (DeepSWE 1.1, HLE, MLS-Bench-Lite — not shown here, see the full table). Rows with an eval-setup note are the ones where the comparison is not strictly like-for-like.

Multimodal (selected rows)

The full multimodal table runs to roughly fifty rows across six categories; here are the ones that matter most for the agentic and visual-agent story above, using the table's own column set (Gemini3.1-Pro and Qwen3.7-Plus replace GPT5.6 Sol/Qwen3.7-Max from the tables above — Alibaba compares against a different baseline set for multimodal).

BenchmarkOpus4.8Fable5Gemini3.1-ProGPT5.6-SolQwen3.7-PlusQwen3.8-Max
MMMU-Pro75.681.280.583.079.082.3
LogicVista76.785.782.689.784.391.9
HiPhO69.378.685.486.884.190.0
OSWorld-Verified83.485.076.283.273.386.1
OSWorld 2.0 (binary / partial)20.6 / 54.8-- / 66.17.8 / 30.6-- / 62.62.8 / 21.519.4 / 46.7
WebArena-Verified67.971.364.369.755.366.8
Parametric CAD Bench85.187.573.586.273.891.5
VLMsAreBiased43.861.274.159.836.688.3
Dense20020.831.169.755.360.787.0

The recurring problem across both tables is one The harness effect names directly: orchestration changes the number as much as the model does, so a table that scores different models on different harnesses is measuring two things at once and reporting only one. Agent harnesses makes the complementary point about what a harness actually is — tools, context management, control flow, an evaluator — which is exactly the layer these footnotes are quietly holding constant for some models and not others. None of that makes the underlying capability claims false. It does mean the honest reading of "Qwen3.8-Max leads Terminal Bench 2.1" is "leads it, evaluated differently than the models it's compared against" — a real result, with an asterisk that Alibaba, to its credit, discloses rather than hides.

Getting it (or not)

Right now, Qwen3.8-Max is API-only, via QwenCloud. The API exposes a reasoning_effort parameter with three levels — xhigh (default, for demanding tasks), medium, and low — and preserve_thinking is on by default. The notable integration detail: QwenCloud's API is compatible with both the OpenAI and Anthropic protocols, so pointing Claude Code at Qwen3.8-Max is a matter of setting ANTHROPIC_BASE_URL and an auth token, no separate client needed. It also plugs into Codex, Qoder CLI, Qwen Code, and OpenClaw with similarly small config changes.

The open weights are, again, announced for "next week" — not this release. Until they land on Hugging Face and ModelScope, every claim in this article about what a 2.4T/95B model is rests on Alibaba's blog post and API behavior, not an inspectable checkpoint. That is a materially weaker evidentiary position than Kimi K3's, where the weights, a technical report, and a config.json all shipped together. Worth remembering the next time "first Qwen-Max-class open-weight model" gets repeated as though the weights were already out.

The take

Strip away the marketing framing and what is left is genuinely interesting: a model that, by its maker's account, ran a coding project unsupervised for 16 days with a public commit trail, rebuilt and then beat a research paper's method from a bare citation, out-negotiated a game-theoretic supplier matrix for a simulated year, and found real architectural wins in a chip design 400 turns after the obvious ones were gone. If even most of that holds up, it is a meaningfully more concrete set of long-horizon-autonomy claims than "our model scored X on benchmark Y."

But the benchmark tables sitting next to those case studies are graded on a harness-by-harness, judge-by-competitor-model, in-house-benchmark basis that Alibaba discloses in footnotes rather than in the headline number — Terminal Bench 2.1 at avg@10 against everyone else's best-of-published, PaperBench judged by Opus 4.6, PLawBench judged by Gemini. That is not disqualifying. It is the same asymmetry every major lab's self-reported benchmark table has, and Alibaba's footnotes are, if anything, more forthcoming than most about exactly where the comparison stops being apples-to-apples. Read the case studies for what the model can apparently do unsupervised. Read the tables — and their footnotes — for how much weight the number itself can actually carry. They are not the same kind of evidence, and this release is unusually clear about which is which.


Built from Alibaba/Qwen's Qwen3.8-Max release post (2026-08-03). Figures 1–3 are the post's own images — the overall performance grid, the RL-scaling curve, and the cross-harness generalization chart — flattened onto white for dark-mode compatibility and capped near 1600px; not reassembled or relabeled. The gate-count staircase and case-study switcher are original interactive reconstructions of the source's own numbers, not independently measured. Benchmark tables are Alibaba's; footnote caveats quoted or closely paraphrased from the source's own numbered notes. Weights and technical report were not available at publication time — every architectural and infrastructure claim here is Alibaba's, unverified against a released checkpoint.

Cite this article

For attribution, please use the following reference or BibTeX:

Satyajit Ghana, "Qwen3.8-Max: 16 days, 265 commits, zero humans in the loop", ai.thesatyajit.com, August 2026.

bibtex
@misc{ghana2026qwen38max,
  author = {Satyajit Ghana},
  title  = {Qwen3.8-Max: 16 days, 265 commits, zero humans in the loop},
  url    = {https://ai.thesatyajit.com/articles/qwen3-8-max},
  year   = {2026}
}
share