# Adapt-1 Machina: no critic, because the simulator is the critic
> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/adapt-1-machina
> date: 2026-09-22
> tags: explainer, robotics, reinforcement-learning, agents, benchmarks
Here are two executions of the same layout, by the same system, before and after
one stage of learning. Nothing else differs — same seed, same starting
observation, same goal, seven commands each.
That is what [Adapt-1 Machina](https://reilabs.org/blog/adapt-1-machina) is
claiming to have learned, and the framing on the post is strong enough to be
worth testing:
> Adapt-1 Machina explores learning continuous control sequences from coarse
> outcome feedback, without demonstrations or a critic.
Critic-free reinforcement learning on continuous control with only sparse outcome
signal is the kind of sentence that usually hides a critic somewhere — a learned
baseline, a value head, a reward model, an evolutionary population doing the same
job under a different name. So: what actually replaces it, what is the action
space, and what is it demonstrated on.
The post is 22 minutes long, publishes per-case outcome records inline for all
384 evaluation layouts, and grades its own evidence in about a dozen places. It
is the most self-critical thing I have read this month. The answers are all in
it; they are just spread across five experiments.
Everything below is Reported: read off reilabs.org on 22 September 2026. There is
no code, no weights and no repository — Adapt-1 is a hosted API with
[documentation](https://docs.reilabs.org/docs/machina) — so nothing here is
reproducible by me or by you. What I could do is check the arithmetic against the
per-case records the post embeds, which turns out to be quite a lot of it.
## The action space is a matrix, and the whole attempt is one decision
A YAM proposal is not a policy sampled step by step. It is a single object,
committed before execution begins:
$$
U = [a_0, \dots, a_{H-1}],\quad H \le 20,\qquad
a_k = (x, y, z, \alpha, \beta, \gamma, g)_k
$$
Twenty commands at most, seven continuous values each — three for position, three
for orientation, one for the gripper. Fixed inverse kinematics turns each
requested tool pose into joint targets. And critically: *"For YAM, the client
sends the starting context once for the full attempt."* The system looks at the
scene, writes down 140 numbers, and then watches what happens. It is **open-loop**
by construction.
That single design choice explains most of what follows. There is no per-timestep
credit assignment to do, because there are no per-timestep decisions — there is
one decision with 140 dimensions, executed, and one grade at the end. Which makes
this less a policy-gradient problem than a search problem over a 20×7 matrix, and
the post is straightforward that this is what it does.
DOOM is the one task that breaks the pattern, and the post treats it as a separate
question: it re-observes between windows of at most eight control steps, so retake
18 runs 8 + 8 + 8 + 6 steps across four proposals. Everything else in the post
commits the whole sequence up front.
## What replaces the critic
Nothing learned. The simulator.
A critic exists to estimate the value of an action *without paying to execute it*.
Take it out and the estimate has to come from somewhere, and here it comes from
running the thing and grading the outcome with hand-written stage detectors —
for YAM: *"contact, sustained lift, transfer above the receptacle and stable
released placement"*, with completion requiring the object to stay put for one
second untouched.
So the budget is the mechanism, and the post publishes it.
**7,680 acquisition attempts produce 62 completed placements.** The first lands
on attempt 1,394. That is 0.81% of attempts, and the post prints both numbers
without softening either. Acquisition alone is followed by selection, structural
revision and refinement for a total of **18,400 learning episodes**, inside a
verification record of 19,808 physical episodes and 251,186 executed commands.
The AIM runs are more lopsided still: 1,024 acquisition attempts, then 18,432 to
49,152 executions in contextual refinement, because each refinement update
evaluates eight candidates across 64 training contexts — 512 executions per
update, and seeds 1 and 2 ran 96 updates each.
That is what removing the critic costs. It is not a criticism; it is the trade.
In simulation, where an execution is cheap and perfectly repeatable, spending
50,000 of them to avoid learning a value function is a reasonable thing to do. On
hardware it is not, and the post makes no claim that it is.
### The four stages are a search, and the post says so
Read the acquisition log's own breakdown: **262 proposals are labelled initial
trajectory explorations and 7,418 are revisions of earlier attempts.** So 97% of
acquisition is editing something already recorded. The edit is structured and
tracked:
> Parent links identify the earlier attempt used for a revision. Indexed edits
> locate each changed control value, and preserved-prefix metadata identifies the
> beginning retained by the proposal.
The worked example in the post is episode 19 revising parent episode 12: commands
0–2 preserved, coordinate 5 changed at command positions 18 and 19. Two numbers
out of 140, on a recorded parent, with the lineage stored. Then **deletion** tries
removing contiguous command spans, **ordering** tries moving when neighbouring
coordinate changes happen, and **selection** re-runs candidates on further
training layouts.
Parent links, indexed mutations, preserved prefixes, deletion of spans, reordering,
and selection on held-out contexts — that is a population search with a
hand-written fitness function. Calling it critic-free is accurate. Calling it
reinforcement learning is a stretch the post itself does not make: the word
appears nowhere in it, and the phrase it uses is *"trajectory learning"*.
The one genuinely learned continuous component is **contextual refinement**, and
it is small by design: *"starts with zero correction weights and tests bounded
offsets to the resulting sequence."* For AIM the post is more specific — the model
uses standardised observations with quadratic features, and *"the decoder adds
twice the proposed offset to each base command and clips the result to the allowed
command range."* A quadratic-feature regression from the start observation to a
clipped, bounded correction on a fixed base sequence. That is the whole learned
policy, and everything else is search over what it corrects.
## The evidence that the correction is real
Here is the part that deserves to travel, and almost certainly will not.
The obvious deflation of "contextual policy" is that it learned a *constant*
offset that happens to help on average. That is easy to test and hard to think
of: give the policy somebody else's observation and see whether it degrades.
Rei Labs ran it, on three independently trained policies, over 256 configurations
each.
**The shuffled arm is worse than no correction at all** — 62 of 768 against 85.
A constant offset cannot produce that. A function of the observation can, and
does, the moment you lie to it. The gap from 85 (base sequence, no correction) to
667 (its own observation) is the learned part, and the drop from 85 to 62 is the
proof that it is a function rather than a bias.
The counterweight is in the same chart and the post puts it there itself. Under
identical conditions the three seeds hit 255, 214 and 198 of 256, with mean miss
distances of 4.39, 12.09 and 14.58 cm against a 22 cm target radius. Same
protocol, same budget shape, a 3.3× spread. That is what run-to-run variance looks
like when somebody bothers to run three seeds — and it is the number missing from
the YAM result, which is one run, and which the post flags: *"Variation across
independent training runs remains unmeasured."*
## The YAM headline, and its interval
The number that will be quoted is 54.17% → 60.42%. It is a paired frozen
evaluation on 384 unseen layouts, learning disabled, identical starting
observations on both sides, and the post embeds every one of the 384 per-layout
outcomes inline. So it can be recounted rather than believed:
```bash
# the per-case records embedded in the post, tallied
gain 70 · loss 46 · both 162 · neither 106 (384 layouts)
before = both + loss = 208 / 384 = 54.17%
after = both + gain = 232 / 384 = 60.42%
```
Four for four. And the shape of that result is the part the percentages hide:
**the refinement broke 46 of the 208 placements it already had** — 22% of them —
to win 70 of the 176 it did not. Net +24, and 106 layouts fail under both.
The post publishes an uncertainty interval, which is rare enough to be worth
naming: *"The paired-layout bootstrap gives a 95% interval of +0.78 to +11.72
percentage points."* That reproduces from the discordant counts under a normal
approximation —
$$
\frac{70-46}{384} = 6.25\%,\qquad
\mathrm{SE} = \frac{\sqrt{70+46}}{384} = 2.81\%,\qquad
6.25 \pm 1.96\times2.81 = +0.75,\ +11.75
$$
— which is within three hundredths of a point of the published bootstrap at both
ends. McNemar on the same counts gives χ² = 4.97, p = 0.026. So the improvement is
real at the conventional threshold and not comfortably so, on one training run,
and the interval's lower bound is three placements out of 384.
The post's own framing is *"a net increase of 24 placements or 6.25 percentage
points"*, stated with the gains and losses beside it. That is the right way to
report it and I would still lead with the interval.
## Where it works much better
Rail is the task where the method looks strongest, and it is worth asking why.
On reserved 64-case panels: **Rail A goes 22/64 → 44/64 and Rail B goes 19/64 →
63/64**, both with **zero regressions**. Compare that with YAM's 70 gains against
46 losses on 384 cases and the difference is stark.
The reason is dimensionality. Rail's final policies retain six half-second
commands in A and eight in B — a handful of scalars, with a smooth, nearly linear
map from the starting condition to the right correction. YAM is 20 × 7 = 140
continuous values with contact dynamics, occlusion and a colour-based perception
pipeline in the loop. The same method, applied to a problem two orders of
magnitude smaller in action dimension, goes from a marginal 6-point gain to
63/64.
Harbor C1 is the other end of the same observation: 127 of 128 courses, across
three policy variants that *"produce identical controls and physical traces on
each course"*. Identical traces across variants means the deletion and ordering
stages changed nothing and every final contextual query returned zero offsets —
the search converged and the learned part contributed literally nothing. The post
says both.
## What it is demonstrated on, and what it is not
Every task in the post is **simulated**. YAM and AIM are MuJoCo; Rail is a
velocity servo with a pendulum model; Harbor is a cylinder-bodied character in a
course; DOOM is DOOM. The post closes by saying so — *"The control studies
presented here are simulations"* — and there is no arm, no bench, no sim-to-real
transfer claim anywhere in it.
That is not a gotcha; it is the scope, stated. But it interacts with the sample
budget in a way worth being explicit about. **Nothing in this method survives
contact with hardware at these numbers.** 7,680 attempts to find the first 62
placements, at maybe fifteen seconds an attempt with a reset, is a month of
continuous arm time before the first thing works — and that is before the 49,152
executions an AIM seed spends on refinement. The two plausible routes out are
sim-to-real on the acquired sequence, or using contextual refinement alone on
hardware to correct a sequence acquired in simulation: 128 training executions
took the focused-alley Harbor panel from 12/36 to 36/36, and 128 real executions
is an afternoon. The post gestures at neither.
There is a related thread on this site worth connecting, because it is the same
architecture arriving from the other direction. [The middle tier is not in the
middle](/articles/three-tiers) read an
[xArm7 control harness](https://github.com/openroboto-ai/jev-robot-control) that
drives a real arm by asking a decision model for **X, Y, Z and the gripper as four
separate questions**, because that tier cannot see across its own options — so
deterministic code builds the relation and hands each coordinate down
independently. Machina's YAM proposal is the same decomposition at a different
scale: a 20×7 matrix of independent continuous coordinates, edited one indexed
cell at a time, with the relations between them supplied by fixed inverse
kinematics and a hand-written stage evaluator rather than by the learner. Two
systems, two years of robotics tooling apart, both routing around the same
limitation.
## The thing I would copy from this post
Not a method. The disclosure habit.
Counted across the piece, Rei Labs volunteers at least nine limitations that
nobody would have found without them:
- *"Those simultaneous changes prevent attribution of the score difference to sample efficiency."*
- *"The operation responsible for the added command remains unresolved in this audit."*
- *"The evidence bundle lacks the separate intervention trial journals, so these figures remain unverified against individual executions."*
- *"Verification of the original traces at every simulation tick remains unavailable."*
- *"The experiment lacks an equal-budget comparison with a single learner controlling the complete course."*
- *"Variation across independent training runs remains unmeasured."*
- *"The design leaves their individual effects unresolved."*
- *"Server-restart behavior remains untested."*
- *"The initial cold-start budget is unavailable."*
Several of those are the exact questions a reviewer would ask. A post that
pre-empts them reads as more credible than one that does not, and the reason is
not politeness — it is that each admission is a precise statement about what the
evidence does and does not cover, which is what lets a reader do arithmetic on it
instead of guessing. Every reproduction in this article exists because the post
published the granular records rather than the summary.
The gap it leaves is the one it cannot close by writing: **no code, no weights, no
environments.** Adapt-1 is a hosted API. So the 384 per-layout outcomes can be
recounted and the bootstrap can be re-derived, and not one execution can be rerun.
## What I would actually take from this
- **Open-loop sequence proposal plus a learned contextual correction is an
underrated shape for low-dimensional control.** Rail B at 19/64 → 63/64 with
zero regressions, on eight half-second commands, from 1,620 executions. If your
control problem is a handful of scalars with delayed dynamics, this is worth
trying before anything with a replay buffer.
- **Run the shuffled-observation ablation.** It is the cheapest way to establish
that a context-conditioned policy is conditioned on the context, it costs one
extra evaluation pass, and it is more convincing than any learning curve. Aim
for the result Machina got: shuffled *worse* than none.
- **Publish per-case records, not aggregate percentages.** 70 / 46 / 162 / 106 is
a completely different sentence from "54.17% to 60.42%", and only one of them
tells you the method breaks a fifth of what already worked.
- **Report the interval with the point estimate.** +6.25 points reads like a
result; +0.78 to +11.72 reads like what it is. The post does this and the
summaries of it will not.
- **Do not read the critic-free framing as sample efficiency.** Removing the
critic does not make the problem cheaper, it moves the cost to the simulator —
18,400 executions for a 6-point gain on a pick-and-place. In simulation that is
a fine trade. It is the whole reason none of this has touched hardware yet.
The post embeds all 384 per-layout outcomes inline as `604000: both`, `604001: neither` and so on. Tally them: I get both 162, neither 106, gain 70, loss 46, and 162+46 = 208 with 162+70 = 232. If the embedded list is regenerated and disagrees with the prose, one of them is stale. The load-bearing part is the 46 losses — if those are an artefact of how a layout is scored rather than real regressions, "breaks a fifth of what already worked" is wrong.
(70 − 46)/384 = 6.25%, SE = sqrt(116)/384 = 2.81%, and 6.25 ± 1.96 × 2.81 is +0.75 to +11.75 — within 0.03 points of the published bootstrap at both ends. If a resample over layouts gives a materially different interval than the normal approximation on discordant pairs, my reconstruction is coincidental rather than correct; the way to tell is to publish the bootstrap draws.
The post's own acquisition log labels 262 proposals as initial explorations and 7,418 as revisions of earlier attempts, with parent links, indexed coordinate edits and preserved prefixes; the stages are selection, deletion, ordering and refinement, with only refinement carrying learned weights and those starting at zero. If the Adapt-1 backend runs a value estimate, an advantage baseline or a learned surrogate that scores candidates before executing them, the framing here is wrong and the sample budget would look very different. The API docs or a released implementation would settle it; neither is public.
Three independently trained policies on 256 cases each: 667/768 with the case's own observation, 85/768 with the acquired sequence and no correction, 62/768 with another case's observation. A constant offset is scored identically under matching and shuffled observations by construction, so a shuffled arm below the no-correction arm rules it out. If the shuffling also perturbed something else — the target, the initial joint state, the scoring — then the arm is not clean and the inference does not hold.
Rail B retains eight half-second commands and goes 19/64 to 63/64 with zero regressions; YAM proposes up to 20 commands of 7 values and goes 208/384 to 232/384 with 46 regressions. These are different tasks with different physics, different perception and different budgets, so dimension is confounded with everything else — the comparison is suggestive, not controlled. The experiment that would settle it is YAM at a reduced command ceiling, say H ≤ 8, on the same layouts and the same budget.
YAM and AIM are MuJoCo, Rail is a described velocity servo with a pendulum model, Harbor is a simulated cylinder body, DOOM is a game — and the post closes with "The control studies presented here are simulations." If a physical run exists and I missed it, or if the YAM camera pipeline is reading real cameras rather than simulated ones, the sample-budget objection in this article softens a great deal, because it would mean somebody already paid that cost.
---
*Nothing here was executed, and nothing here could be: Adapt-1 is a hosted API with no public code, weights or environments. Every figure is Reported, read off [reilabs.org/blog/adapt-1-machina](https://reilabs.org/blog/adapt-1-machina), published 21 September 2026, with API documentation at [docs.reilabs.org/docs/machina](https://docs.reilabs.org/docs/machina). What I did rather than report: tally the 384 per-layout outcome records the post embeds, and re-derive its published bootstrap interval from the discordant counts. Both clips and both figures are Rei Labs' own, reproduced under academic use — the two YAM recordings placed side by side and the Rail clip rescaled, otherwise unaltered. Companion piece: [The middle tier is not in the middle](/articles/three-tiers), for the xArm7 harness that decomposes a robot pose into independent per-coordinate questions for the same reason.*