2026-08-14 · 8 min · llm · open-weights · agents · rl · multimodal · long-horizon · explainer
The dots team — Xiaohongshu's AI lab — open-sourced dots3-note Preview: 280B total parameters, 16B active, a 512K context window, and multimodal understanding across text, vision and speech, under Apache 2.0. Weights on Hugging Face, code on GitHub, architecture submitted to Transformers as PR #47844. A technical report is promised within a week.
"note" is the lightest of three planned models; jazz and aria follow.
The benchmark table is respectable and not the reason to read this. The reason is a training method for tasks that take longer than a working day.
The problem TEMPO solves
Reinforcement learning on genuinely long-horizon agent tasks runs into two walls at once, and the dots team states both plainly: a single rollout "can take more than ten hours, making training prohibitively inefficient, while sparse rewards hinder effective credit assignment."
Actor-critic methods like PPO exist to fix the second problem. But:
a critic estimates value through a fixed-compute forward pass. Unlike an actor, it cannot reason, reflect, or use tools to analyze the current state, making accurate value estimation difficult on complex problems.
That is the observation the whole method turns on. If a task is hard enough that acting well requires ten hours of tool use and reasoning, then judging whether it is going well is also hard — and a single forward pass through a value head is not going to manage it.
The insight is narrow and load-bearing. A PPO-style critic is a fixed-compute forward pass, so on a task hard enough to need ten hours of acting, it cannot think hard enough to say whether things are going well. TEMPO makes the critic the same model wearing a different hat, and lets it spend real inference on the question. That turns “how am I doing?” from a cheap regression into a reasoning task — and the dots team’s claim is that this direction is the easy one: evaluation is easier than generation.
TEMPO — Test-time-scaled Value Estimation with Macro-step Policy Optimization — cuts the task into macro-steps, each several rounds of interaction. At the end of each, the same agent switches from actor to critic and uses test-time-scaled reasoning to estimate expected remaining return. The policy can then be updated mid-task rather than after ten hours.
Reported result: +31.5% average score over the base checkpoint and +20.6% over GRPO on ARC-AGI-3, reaching the same level in fewer steps.
Evaluation is easier than generation
The claim underneath TEMPO is the interesting one, and dots found it during training rather than assuming it:
Even when the agent cannot yet solve a problem, it can act as a critic to distinguish between two superficially similar states, identify the one that represents a genuine breakthrough in understanding the environment's rules, and assign clearly different value estimates.
Their worked example is a "place knights" puzzle where two training branches both ran 64 rounds without clearing a level — identical by environment score. Branch B had misidentified the objective and was searching under a wrong assumption; branch A had found the real conflict rule and was close to a feasible layout.
A scalar reward cannot tell those apart. A model that reads both trajectories can. That is the entire argument for making the critic a reasoning model, and it is why the dots team frames self-evaluation as the direction they intend to keep pushing: real-world tasks "lack verifiable reward signals, while relying on human experts to evaluate model outputs may not scale."
The IMO result belongs here
666 contestants from 117 countries — records for both — and seven of them scored 42. The page counts are the part of this worth staring at: these are proofs a human panel read and certified, not answers a checker compared against a key. Ten pages for P2 and three for P6 is a distribution of effort, and it does not match the usual difficulty ordering of the competition.
At IMO 2026 in Shanghai, dots built "an internal harness around a branch of dots3-note Preview" that generated proofs recursively and used tools to evaluate and improve them. The committee's own graders awarded 7/7 on all six problems — 42/42, a score seven of 666 contestants from 117 countries matched.
Two things are worth being precise about, because the result is easy to over-read.
It was not this model. It was a branch of it inside a purpose-built harness, and the IMO write-up is a separate page from the model release. Nothing you can download reproduces it.
And it used no formal language. The model read the organizers' original LaTeX and worked in natural language plus Python — no Lean, no proof checker. The dots team is explicit about why: formalization "requires a person to translate a problem into a formal language," and most real problems resist that. So the only thing standing between a plausible-looking proof and a wrong one was the model's own critique loop, and then a human panel that reads for holes.
Which makes the IMO run an inference-time instance of the same bet TEMPO makes at training time. The proof lengths are the one signal that varies — 3, 10, 6, 5, 4 and 3 pages, and P6, traditionally the hardest slot, took one of the two shortest.
Where it actually lands
Sixteen billion active parameters against 21B, 39B and 104B, and the record splits cleanly: ahead of Hy3, GLM 5.2 and Seed 2.1 turbo, behind Opus 4.8, GPT-5.5, DeepSeek-v4-flash and Kimi K3. The asterisks matter more than the totals, though. On ARC-AGI-2, dots3-note’s 81.4 is starred — their own run on the official public set — while Opus 4.8’s 72.1 is unstarred, meaning an official leaderboard score on the private set. Those are different measurements, and on ARC-AGI the gap between them is not small.
Head-to-head across the 23 reasoning and agentic benchmarks in the appendix: ahead of Hy3 (18–6), GLM 5.2 (16–9) and Seed 2.1 turbo (12–5); behind DeepSeek-v4-flash (8–14), GPT-5.5 (8–17), Opus 4.8 (8–18) and Kimi K3 (2–7 on the nine rows they share). For a model with 16B active parameters against 21B, 39B and 104B, the first half of that sentence is the notable one.
Two rows stand out, both on the benchmark this release is built around:
- ARC-AGI-3 (arcagi3 harness): 6.9 against Opus 4.8's 1.5 and GPT-5.5's 0.4. More than four times the next best. This is the benchmark ARC Prize designed for autonomous learning in unfamiliar environments, where complex tasks need thousands of interactions over 40–50 hours.
- ARC-AGI-2: 81.4, above Opus 4.8's 72.1 and below GPT-5.5's 85.0.
That second one needs its asterisk read. dots' note says results marked * are their own testing, and specifically for ARC-AGI-2: "We evaluated models on the official public evaluation set; unmarked results are official leaderboard scores from the private set." dots3-note's 81.4 is starred. Opus 4.8's 72.1 is not. So a self-run public-set score is sitting in the same column as an official private-set score, and on ARC-AGI that difference is not cosmetic. The ARC-AGI-3 general-harness row has the same shape — dots' number is starred, and so are most of the competitors'.
To their credit, the harness details are unusually complete: Terminus-2 with a 10-hour timeout for Terminal-Bench, OpenClaw 2026.6.1 with a GPT-5.4 judge for WildClawBench, live-swe-agent for the SWE suite, Hugging Face access blocked during agentic search to prevent leakage. That is more methodology than most releases publish, and it is what makes the asterisk asymmetry visible in the first place.
The two benchmarks they released
Both are open-sourced alongside the model, and both target the gap dots says it cares about — tasks where the user does not state what they want up front:
- VibeSearchBench: 200 tasks across 20 domains. Each starts with an ambiguous request, and a persona-driven simulator reveals constraints over multiple turns. The agent's predicted knowledge graph is matched against ground truth by nodes and triplets, scored by Triplet F1.
- VibeLifeBench: 20 tasks across 10 domains, each spanning 20–30 stages on a simulated timeline, with 1,247 atomic checks on cross-stage state consistency, tool execution and final deliverables. Their example is a family trip that has to be re-planned as aircraft type, weather and flight status change underneath it.
Nobody scores well on either. On VibeLifeBench the whole field sits between 21.1 and 30.1, with dots3-note at 28.1; on VibeSearchBench, between 22.4 and 33.8, with dots3-note at 25.7. A benchmark where the best model in the world manages 30% is either badly designed or pointed at something genuinely unsolved, and the 1,247-check structure suggests the latter.
What they say is wrong with it
The Limitations section is short and unusually direct:
dots3-note Preview is an interim preview release. Reinforcement learning is not yet complete, and the model still has limitations in hallucination mitigation, the balance between text and multimodal capabilities, and overall stability.
And on the real-life results specifically: those tasks run in simulated environments, and turning them into real experiences needs "robust harnesses, connectors, data sources, safety and permission mechanisms, and product design."
That is the right caveat and it is load-bearing. The persona-driven simulator that plays the user is itself a model, so a system trained and measured against it may be learning to satisfy a simulator rather than a person. dots built the environments, the benchmarks, and the model being evaluated on them.
What I'd take from it
Ignore the parameter count and the leaderboard position. The transferable idea is that a critic should be allowed to think.
Every value-based RL setup assumes evaluation is cheap enough to do in one forward pass — an assumption that holds fine when the task is short and breaks silently when the task is ten hours long. TEMPO's answer is to spend inference on the value estimate, and the evidence for it is a picture of a model correctly separating two trajectories that the environment scored identically.
If "evaluation is easier than generation" holds up in the technical report, it is the more useful half of this release than any benchmark row in it.