Adapt-1 Machina: no critic, because the simulator is the critic

Here are two executions of the same layout, by the same system, before and after one stage of learning. Nothing else differs — same seed, same starting observation, same goal, seven commands each.

That is what Adapt-1 Machina is claiming to have learned, and the framing on the post is strong enough to be worth testing:

Adapt-1 Machina explores learning continuous control sequences from coarse outcome feedback, without demonstrations or a critic.

Critic-free reinforcement learning on continuous control with only sparse outcome signal is the kind of sentence that usually hides a critic somewhere — a learned baseline, a value head, a reward model, an evolutionary population doing the same job under a different name. So: what actually replaces it, what is the action space, and what is it demonstrated on.

The post is 22 minutes long, publishes per-case outcome records inline for all 384 evaluation layouts, and grades its own evidence in about a dozen places. It is the most self-critical thing I have read this month. The answers are all in it; they are just spread across five experiments.

The action space is a matrix, and the whole attempt is one decision

A YAM proposal is not a policy sampled step by step. It is a single object, committed before execution begins:

Twenty commands at most, seven continuous values each — three for position, three for orientation, one for the gripper. Fixed inverse kinematics turns each requested tool pose into joint targets. And critically: "For YAM, the client sends the starting context once for the full attempt." The system looks at the scene, writes down 140 numbers, and then watches what happens. It is open-loop by construction.

That single design choice explains most of what follows. There is no per-timestep credit assignment to do, because there are no per-timestep decisions — there is one decision with 140 dimensions, executed, and one grade at the end. Which makes this less a policy-gradient problem than a search problem over a 20×7 matrix, and the post is straightforward that this is what it does.

DOOM is the one task that breaks the pattern, and the post treats it as a separate question: it re-observes between windows of at most eight control steps, so retake 18 runs 8 + 8 + 8 + 6 steps across four proposals. Everything else in the post commits the whole sequence up front.

What replaces the critic

Nothing learned. The simulator.

A critic exists to estimate the value of an action without paying to execute it. Take it out and the estimate has to come from somewhere, and here it comes from running the thing and grading the outcome with hand-written stage detectors — for YAM: "contact, sustained lift, transfer above the receptacle and stable released placement", with completion requiring the object to stay put for one second untouched.

So the budget is the mechanism, and the post publishes it.

7,680 acquisition attempts produce 62 completed placements. The first lands on attempt 1,394. That is 0.81% of attempts, and the post prints both numbers without softening either. Acquisition alone is followed by selection, structural revision and refinement for a total of 18,400 learning episodes, inside a verification record of 19,808 physical episodes and 251,186 executed commands.

The AIM runs are more lopsided still: 1,024 acquisition attempts, then 18,432 to 49,152 executions in contextual refinement, because each refinement update evaluates eight candidates across 64 training contexts — 512 executions per update, and seeds 1 and 2 ran 96 updates each.

That is what removing the critic costs. It is not a criticism; it is the trade. In simulation, where an execution is cheap and perfectly repeatable, spending 50,000 of them to avoid learning a value function is a reasonable thing to do. On hardware it is not, and the post makes no claim that it is.

The four stages are a search, and the post says so

Read the acquisition log's own breakdown: 262 proposals are labelled initial trajectory explorations and 7,418 are revisions of earlier attempts. So 97% of acquisition is editing something already recorded. The edit is structured and tracked:

Parent links identify the earlier attempt used for a revision. Indexed edits locate each changed control value, and preserved-prefix metadata identifies the beginning retained by the proposal.

The worked example in the post is episode 19 revising parent episode 12: commands 0–2 preserved, coordinate 5 changed at command positions 18 and 19. Two numbers out of 140, on a recorded parent, with the lineage stored. Then deletion tries removing contiguous command spans, ordering tries moving when neighbouring coordinate changes happen, and selection re-runs candidates on further training layouts.

Parent links, indexed mutations, preserved prefixes, deletion of spans, reordering, and selection on held-out contexts — that is a population search with a hand-written fitness function. Calling it critic-free is accurate. Calling it reinforcement learning is a stretch the post itself does not make: the word appears nowhere in it, and the phrase it uses is "trajectory learning".

The evidence that the correction is real

Here is the part that deserves to travel, and almost certainly will not.

The obvious deflation of "contextual policy" is that it learned a constant offset that happens to help on average. That is easy to test and hard to think of: give the policy somebody else's observation and see whether it degrades. Rei Labs ran it, on three independently trained policies, over 256 configurations each.

The shuffled arm is worse than no correction at all — 62 of 768 against 85. A constant offset cannot produce that. A function of the observation can, and does, the moment you lie to it. The gap from 85 (base sequence, no correction) to 667 (its own observation) is the learned part, and the drop from 85 to 62 is the proof that it is a function rather than a bias.

The counterweight is in the same chart and the post puts it there itself. Under identical conditions the three seeds hit 255, 214 and 198 of 256, with mean miss distances of 4.39, 12.09 and 14.58 cm against a 22 cm target radius. Same protocol, same budget shape, a 3.3× spread. That is what run-to-run variance looks like when somebody bothers to run three seeds — and it is the number missing from the YAM result, which is one run, and which the post flags: "Variation across independent training runs remains unmeasured."

The YAM headline, and its interval

The number that will be quoted is 54.17% → 60.42%. It is a paired frozen evaluation on 384 unseen layouts, learning disabled, identical starting observations on both sides, and the post embeds every one of the 384 per-layout outcomes inline. So it can be recounted rather than believed:

# the per-case records embedded in the post, tallied
gain 70 · loss 46 · both 162 · neither 106        (384 layouts)
before = both + loss = 208 / 384 = 54.17%
after  = both + gain = 232 / 384 = 60.42%

Four for four. And the shape of that result is the part the percentages hide: the refinement broke 46 of the 208 placements it already had — 22% of them — to win 70 of the 176 it did not. Net +24, and 106 layouts fail under both.

The post publishes an uncertainty interval, which is rare enough to be worth naming: "The paired-layout bootstrap gives a 95% interval of +0.78 to +11.72 percentage points." That reproduces from the discordant counts under a normal approximation —

— which is within three hundredths of a point of the published bootstrap at both ends. McNemar on the same counts gives χ² = 4.97, p = 0.026. So the improvement is real at the conventional threshold and not comfortably so, on one training run, and the interval's lower bound is three placements out of 384.

The post's own framing is "a net increase of 24 placements or 6.25 percentage points", stated with the gains and losses beside it. That is the right way to report it and I would still lead with the interval.

Where it works much better

Rail is the task where the method looks strongest, and it is worth asking why.

On reserved 64-case panels: Rail A goes 22/64 → 44/64 and Rail B goes 19/64 → 63/64, both with zero regressions. Compare that with YAM's 70 gains against 46 losses on 384 cases and the difference is stark.

The reason is dimensionality. Rail's final policies retain six half-second commands in A and eight in B — a handful of scalars, with a smooth, nearly linear map from the starting condition to the right correction. YAM is 20 × 7 = 140 continuous values with contact dynamics, occlusion and a colour-based perception pipeline in the loop. The same method, applied to a problem two orders of magnitude smaller in action dimension, goes from a marginal 6-point gain to 63/64.

Harbor C1 is the other end of the same observation: 127 of 128 courses, across three policy variants that "produce identical controls and physical traces on each course". Identical traces across variants means the deletion and ordering stages changed nothing and every final contextual query returned zero offsets — the search converged and the learned part contributed literally nothing. The post says both.

expand

expand

What it is demonstrated on, and what it is not

Every task in the post is simulated. YAM and AIM are MuJoCo; Rail is a velocity servo with a pendulum model; Harbor is a cylinder-bodied character in a course; DOOM is DOOM. The post closes by saying so — "The control studies presented here are simulations" — and there is no arm, no bench, no sim-to-real transfer claim anywhere in it.

That is not a gotcha; it is the scope, stated. But it interacts with the sample budget in a way worth being explicit about. Nothing in this method survives contact with hardware at these numbers. 7,680 attempts to find the first 62 placements, at maybe fifteen seconds an attempt with a reset, is a month of continuous arm time before the first thing works — and that is before the 49,152 executions an AIM seed spends on refinement. The two plausible routes out are sim-to-real on the acquired sequence, or using contextual refinement alone on hardware to correct a sequence acquired in simulation: 128 training executions took the focused-alley Harbor panel from 12/36 to 36/36, and 128 real executions is an afternoon. The post gestures at neither.

There is a related thread on this site worth connecting, because it is the same architecture arriving from the other direction. The middle tier is not in the middle read an xArm7 control harness that drives a real arm by asking a decision model for X, Y, Z and the gripper as four separate questions, because that tier cannot see across its own options — so deterministic code builds the relation and hands each coordinate down independently. Machina's YAM proposal is the same decomposition at a different scale: a 20×7 matrix of independent continuous coordinates, edited one indexed cell at a time, with the relations between them supplied by fixed inverse kinematics and a hand-written stage evaluator rather than by the learner. Two systems, two years of robotics tooling apart, both routing around the same limitation.

The thing I would copy from this post

Not a method. The disclosure habit.

Counted across the piece, Rei Labs volunteers at least nine limitations that nobody would have found without them:

• "Those simultaneous changes prevent attribution of the score difference to sample efficiency."
• "The operation responsible for the added command remains unresolved in this audit."
• "The evidence bundle lacks the separate intervention trial journals, so these figures remain unverified against individual executions."
• "Verification of the original traces at every simulation tick remains unavailable."
• "The experiment lacks an equal-budget comparison with a single learner controlling the complete course."
• "Variation across independent training runs remains unmeasured."
• "The design leaves their individual effects unresolved."
• "Server-restart behavior remains untested."
• "The initial cold-start budget is unavailable."

Several of those are the exact questions a reviewer would ask. A post that pre-empts them reads as more credible than one that does not, and the reason is not politeness — it is that each admission is a precise statement about what the evidence does and does not cover, which is what lets a reader do arithmetic on it instead of guessing. Every reproduction in this article exists because the post published the granular records rather than the summary.

The gap it leaves is the one it cannot close by writing: no code, no weights, no environments. Adapt-1 is a hosted API. So the 384 per-layout outcomes can be recounted and the bootstrap can be re-derived, and not one execution can be rerun.

What I would actually take from this

• Open-loop sequence proposal plus a learned contextual correction is an underrated shape for low-dimensional control. Rail B at 19/64 → 63/64 with zero regressions, on eight half-second commands, from 1,620 executions. If your control problem is a handful of scalars with delayed dynamics, this is worth trying before anything with a replay buffer.
• Run the shuffled-observation ablation. It is the cheapest way to establish that a context-conditioned policy is conditioned on the context, it costs one extra evaluation pass, and it is more convincing than any learning curve. Aim for the result Machina got: shuffled worse than none.
• Publish per-case records, not aggregate percentages. 70 / 46 / 162 / 106 is a completely different sentence from "54.17% to 60.42%", and only one of them tells you the method breaks a fifth of what already worked.
• Report the interval with the point estimate. +6.25 points reads like a result; +0.78 to +11.72 reads like what it is. The post does this and the summaries of it will not.
• Do not read the critic-free framing as sample efficiency. Removing the critic does not make the problem cheaper, it moves the cost to the simulator — 18,400 executions for a 6-point gain on a pick-and-place. In simulation that is a fine trade. It is the whole reason none of this has touched hardware yet.

Nothing here was executed, and nothing here could be: Adapt-1 is a hosted API with no public code, weights or environments. Every figure is Reported, read off reilabs.org/blog/adapt-1-machina, published 21 September 2026, with API documentation at docs.reilabs.org/docs/machina. What I did rather than report: tally the 384 per-layout outcome records the post embeds, and re-derive its published bootstrap interval from the discordant counts. Both clips and both figures are Rei Labs' own, reproduced under academic use — the two YAM recordings placed side by side and the Rail clip rescaled, otherwise unaltered. Companion piece: The middle tier is not in the middle, for the xArm7 harness that decomposes a robot pose into independent per-coordinate questions for the same reason.