Natura-1: a whole robot episode as one token sequence

2026-10-13 · 18 min · robotics · world-models · flow-matching · mixture-of-experts · scaling-laws

Why read this

Pulls Natura-1's raw chart data: the 67.7% MSE drop reproduces, but it is teacher-forced error along one run, and the benchmark is hands vs grippers.

Original analysis
A new technique
Concrete numbers to act on

Robotics & embodiedNothing to runProprietaryPractitioner model

How this was scored

Is it new?: 2 of 3: A real new idea, method or capability
Can I trust it?: 2 of 3: Measures key facts from files, code or configs
Can I run it?: 0 of 3: Closed, nothing to run
Will I understand it?: 2 of 3: Mechanism from first principles with figures
Can I act on it?: 2 of 3: A concrete recipe, numbers or comparison
Will it last?: 1 of 3: Relevant for months
Does it affect many?: 1 of 3: A specialist community
Only here?: 2 of 3: A teardown or measurement few others did

Score 56 of 100, ranked 346 of 513 rated articles. Each question is answered 0–3 by hand, and a 3 is rare. How articles are scored

On 2026-10-13 a lab called Nature Will announced Natura-1 as "our first general foundation model for the physical world". The thread makes three claims worth an engineer's time. It is a 50-billion-parameter mixture-of-experts transformer pretrained on one million hours of egocentric human experience. It drives 20-DoF dexterous hands after post-training on "a limited amount of robot data". And its novelty is not a new block but "the arrangement of multimodal tokens within a single sequence and the causal structure of attention".

The long-form source is a technical blog post. There is no paper. I searched arXiv for the name and the lab, the Hugging Face model API (/api/models?search=natura and ?author=naturewill) and the web for a repository. I found nothing: no paper, no code, no weights, no model card. Everything below is therefore reported unless I say otherwise. The useful exception is that the blog's charts are HTML, not pictures, and each data point carries its raw value in a data-value attribute. I read those, and that is where the measured numbers in this piece come from.

Why dexterous hands are the hard case

A parallel gripper has one degree of freedom. A 20-DoF hand has twenty, and the blog's framing is that "the action space grows exponentially with the number of hand joints". The more practical problem is data. Robot demonstrations are collected by teleoperating a robot, so they only exist where a robot already works. The blog puts it as a loop: "a robot can only gather data in settings where it already works, and it can only work in settings where data already exists".

Humans do not have that problem. People already use their hands in nearly every environment, and head-mounted cameras record it cheaply. So the bet is the one several labs are now making: pretrain on first-person human video, then adapt to a robot hand with a small amount of robot data. The open lineage here is long. Physical Intelligence's π0 and π0.5 put a flow-matching action expert beside a vision-language backbone. NVIDIA's GR00T N1 mixes real robot data, human video and synthetic data. World-action models such as FLUX 3 Action and Flex-π denoise future video and actions together so that video pretraining becomes motor knowledge. Natura-1 sits closest to the last group, at a larger scale and with far more human data.

The mechanism: one episode, one sequence

The blog's model section is short, so I quote it where it matters. Natura-1 "models an entire episode autoregressively as one sequence of language, image, video, and action tokens". The backbone is "a standard MoE transformer", and "modality-specific inductive bias lives only in the sequence layout and the attention pattern".

Concretely, that means no separate vision tower talking to a separate action head through cross-attention, which is how many VLAs are wired. There is one stream. An instruction comes first, then the starting image, then the episode unrolls as runs of video tokens, each followed by action tokens. A transformer reads it left to right.

Nature Will's diagram: an upper plane plots video frames of a robot arm over time with a black state curve, a lower red plane plots actions, a vertical boundary splits past (kept as memory) from future (the world-modeling target), and a fan of futures on the right is labelled Language: another instruction, another future, and Flow matching: joint denoising. Along the bottom, one sequence of language, image, video and action tokens feeds a mixture-of-experts layer. — Nature Will's drawing of the model. Video and action are two projections of one state; the past is memory, the future is predicted, and language picks which future. The bottom strip is the single sequence routed through the MoE layers (Natura-1 technical blog, model figure).

Three design choices follow from that layout.

Every step is supervised in one pass. Because the whole episode is one sequence under a causal mask, a single forward and backward pass trains a prediction at every timestep. The blog contrasts this with "a single sampled observation–action window", which is how most robot policies are trained: sample one moment, predict one action chunk. With teacher forcing, every timestep's prediction is conditioned on the true past, so an episode of $T$ steps yields $T$ supervised predictions for the price of one sequence. This is the language-model trick applied to robot episodes, and it is why the blog can say the design lets physical learning "approach the training efficiency of frontier language and video models". That last part is a claim, not a measurement.

Video and action are predicted together. "At any moment, what the camera sees and what the hand does are two projections of the same physical state." Natura-1 keeps past video as memory and learns to predict future video "as part of its world model". Video and action are trained "together using flow matching and joint denoising, with a separate loss formulation for each modality". In flow matching the model learns a velocity field that carries noise to data. Here, noised future video tokens and noised future action tokens sit in the same sequence and are denoised together, so an action can be conditioned on the frames it will cause. The payoff is supervision density: action labels are scarce, video is abundant, and video "dominates the tokens of every trajectory". A model graded on predicting the video gets far more gradient per episode than one graded on actions alone.

Language has to change the future. The blog's "Prediction Is Not Understanding" paragraph names a real failure mode: in most clips the scene alone predicts what happens next, so a model can learn to ignore the instruction. Natura-1 is "shaped so that a difference in stated intent yields a different predicted future, in video and action alike", using fine-grained captions from the data pipeline. How it is shaped (a contrastive loss, paired captions, instruction dropout) is not described.

one episode = one sequence · layout and causal mask from the blog; token counts are schematic

now3 past · 3 future

mask

Stated: attention is causal across the episode. A timestep reads the instruction, the first image and everything before it, and nothing after it. The hatched diagonal is the part the blog leaves out: whether the video and action tokens of one timestep see each other while they are denoised together.

Paler rows are future timesteps (after now): noised inputs, predicted with separate video and action losses. In training, teacher forcing supervises every timestep of the episode in one pass.

Layout after the blog’s “one sequence” figure: language, image, then runs of video tokens each followed by an action. Counts per step are illustrative; Nature Will publishes none.

The widget lays out one episode as the blog draws it. Move now and the timesteps before it become clean memory while those after it become noised targets. The mask shows what is stated and what is not. Stated: attention is causal across the episode, so a timestep sees the instruction, the first image and every earlier timestep, never a later one. Not stated: what attention looks like inside one timestep. Joint denoising of a step's video and actions suggests those tokens see each other (block-causal), which is what FLUX 3 Action does. But the blog never says so, so the default view leaves those cells hatched. Token counts per frame, frames per step and the action chunk size are not published either; the counts in the widget are schematic.

What the sequence does not tell you

The model is a 50B MoE, and that is all. The blog gives neither the number of experts, the routing scheme, nor the active parameter count. For an MoE the active count sets the per-token compute and therefore the latency. A 50B MoE could activate 3B parameters or 20B, and the serving story changes by an order of magnitude between those. Readers who want the trade-off spelled out can see the mixture-of-experts architecture page. The tokenizers for video and actions, the context length and the number of denoising steps at inference are also not given.

The data: a million hours, and how they got there

The data section is the most concrete part of the blog. The corpus is "1 million hours of egocentric human data, consisting of 61,702,471 episodes, 317,842 tasks, and 14,286 scenes" (reported). It is "primarily built through controlled data acquisition, complemented by large-scale internet ego videos", with monocular, stereo, multi-camera and wrist-mounted views.

Dividing one by the other gives an average episode of about 58 seconds (reasoned: one million hours × 3,600 s ÷ 61,702,471 episodes = 58.3 s). That is a sensible unit for "one sequence": long enough to hold a multi-step task, short enough to fit in a transformer context once video is tokenized. The blog does not say how long the trained context actually is.

The pipeline has three stages that matter for robot transfer:

Cleaning. Deduplication by visual features across vendors and internet sources, sensor validation, and for internet video, shot segmentation, hand-visibility checks and VLM scoring.
Motion reconstruction. An in-house labeller recovers dense hand and body motion from monocular and stereo video. The blog reports "a 41% reduction in MPJPE compared with existing open-source approaches on an internal benchmark with motion capture ground truth" (reported). The benchmark, the baselines and the absolute error are not named, so the 41% cannot be placed. This stage is where human video becomes action labels: the reconstructed hand pose is the "action" in a human episode.
Language annotation. Hierarchical captions, goal to subtask to step, written by an agentic annotator that shares context across an episode.

The system "processes 350,000 hours of ego data per week" (reported), so the whole corpus is about three weeks of pipeline throughput (reasoned: one million ÷ 350,000 = 2.9). The X thread's "44K → 1M hours" refers to the scaling study below, not to collection history.

Two things are missing that would decide how much of this transfers. First, the retargeting from a human hand to the robot hand. The blog says only that "anthropomorphic dexterous hands offer the closest correspondence available today". Second, the robot data: "a limited amount", with no hours, episode count or task count. Without those I cannot tell whether "limited" means tens of hours or thousands. If you work on reconstruction from video, the TrackEverything piece covers the dense-3D-tracking side of the same problem; the Ego2Act and EgoTools article covers how egocentric understanding is being benchmarked.

Infrastructure: the numbers with no hardware attached

The blog makes four infrastructure claims, all reported:

Claim	Value	What is missing
Training MFU	"exceeds 50%", 98% GPU utilization	GPU model, count ("hundreds"), precision
Attention kernel throughput	≈1,200 TFLOP/s, "over 6×" a tuned FlexAttention	GPU, precision, sequence shapes
End-to-end speedups	1.7× training, 2× inference	baseline system
Serving latency	"within 100 milliseconds" end to end	GPU, how many, active params, chunk size

The kernel number says something about the hardware even though the blog does not. An H100 SXM's dense BF16 peak on NVIDIA's spec sheet is 989 TFLOP/s. An attention kernel averaging 1,200 TFLOP/s over forward and backward cannot be BF16 on an H100, so it is running on a newer part, at lower precision, or both (reasoned). That matters for the latency claim. Under 100 ms per observation is plausible for a 50B MoE with a few billion active parameters on a datacenter GPU or a small multi-GPU node. The infrastructure figure shows hybrid tensor, context and expert parallelism ("TP · CP · EP") in the inference workers, which suggests the serving path spans several GPUs. It is not something anyone will put on a robot. Treat "real time" here as "real time from a rack".

Two-panel infrastructure diagram. Left, Scalable training with 50%+ MFU and 98% GPU utilization: configuration, data pipeline, model recipe, training loop and distributed execution with FSDP, HSDP, context parallel, precision and compilation. Right, Robot serving within 100 ms end to end: robot clients, a session gateway, preprocessing and scheduler, data and control planes, and multi-GPU inference workers with CUDA graph replay, GPU state caches and hybrid TP, CP, EP parallelism. A shared GPU kernel layer of attention, fused tensor, matrix and encoder kernels and MegaMoE sits under both. — Training and serving share one kernel layer. Note the serving side's hybrid TP, CP and EP parallelism across inference workers; the GPU model is never named (Natura-1 technical blog, infrastructure figure).

Checking the scaling claim

The thread's headline is: "With egocentric training data scaled from 44K to 1M hours ... Prediction MSE error on real-robot data falls by 67.7%, without fine-tuning on robot data."

Natura-1 scaling validation: four log-log panels of teacher-forced action error against training data hours from about 44 thousand to 1 million. Human validation MSE falls from 0.0320 to 0.0168 and MAE from 0.0916 to 0.0574; robot validation MSE falls from 0.0272 to 0.0088 and MAE from 0.1034 to 0.0525. Each panel shows eight measured checkpoints and a dashed power-law fit. — Teacher-forced action error against training hours, eight checkpoints per panel, with power-law fits. Lower is better (Natura-1 technical blog, scaling figure).

I took the eight raw values per panel from the chart's HTML and recomputed everything (measured):

Validation set	Metric	First checkpoint	Last checkpoint	Change	Blog says
Human, held out	MSE	0.032041	0.016766	−47.7%	47.7%
Human, held out	MAE	0.091569	0.057439	−37.3%	37.3%
Robot, never trained on	MSE	0.027169	0.008764	−67.7%	67.7%
Robot, never trained on	MAE	0.103448	0.052470	−49.3%	49.3%

A least-squares fit of log error on log hours gives back the blog's printed power laws to four digits: robot MSE $= 1.7193 \cdot D^{-0.3893}$ with $R^2 = 0.948$ in log space, and human MSE $= 0.3569 \cdot D^{-0.2235}$ with $R^2 = 0.981$ (measured; the blog prints 0.3570 for the human coefficient). The arithmetic is honest. What it measures is narrower than the thread suggests.

eight checkpoints from Nature Will’s chart · values reported, ratios mine

checkpoint8 of 8 · ≈ 1,000K h · 22.90× first

teacher-forced errorfirstthis onechange

robot MSE0.02720.0088−67.7%

robot MAE0.10340.0525−49.3%

human MSE0.03200.0168−47.7%

human MAE0.09160.0574−37.3%

At checkpoint 8 the changes are the blog’s headline figures (−67.7% robot MSE). The first four checkpoints sit at exactly 1×, 2×, 3× and 4× of 43,660 hours, which fits the blog’s own wording (“as training advances”): points along one run, most likely, not separate models trained on separate datasets. None of this is a task success rate.

It is teacher-forced prediction error, not task success. The blog says so: "Both evaluations use teacher forcing, conditioning each prediction on the past observation." The model sees the true past of a recorded robot episode and is scored on how close its predicted actions come to what the teleoperator did. That is a useful signal of transfer, since the robot set was never trained on. But a policy can match a demonstrator's average motion closely and still fail at the moment of contact, which is where dexterous tasks are won or lost. Nature Will reports no curve of real-robot success against pretraining hours.

The x-axis is training progress, most likely within one run. The checkpoint positions in the chart's data are 43,660, 87,321, 130,981 and 174,641 hours, then 314,679, 578,264, 736,415 and one million (measured). The first four are exact multiples of the first: 1×, 2×, 3× and 4×. With the blog's own wording ("as training advances from approximately 44,000 to one million hours"), the natural reading is checkpoints along one run, labelled by hours of data seen so far. If so, data and compute grow together on this axis, and the curve cannot separate "more data" from "more steps". A data-scaling claim proper would train separate models on 44K-hour and 1M-hour corpora at matched compute. The blog may have done that; the chart does not show it (reasoned).

The tail is flattening. Over the last three checkpoints, from 578K to one million hours, robot MSE drops 8.6% and human MSE 7.4% (measured). The power-law fit for robot MSE has the lowest $R^2$ of the four panels, 0.948, and the last points sit above the fitted line. That is not a broken curve, just a reason not to extrapolate it to ten million hours.

Checking the benchmark

The thread's video speaks for itself, and the blog's six long-horizon workflows (wontons, a shipping box, tea, calligraphy, a screen protector, a yogurt bowl) are worth watching. Note the dagger: those six "use additional tuning on top of Natura-1". The zero-shot claims rest on a separate benchmark.

Natura-1 benchmark results. Overall score in percent: Natura-1 Agentic 59.1 and Natura-1 Base 53.4 on dexterous hands; GPT-6-Astra and pi 0.5 show no result on dexterous hands; on the gripper platform GPT-6-Astra scores 16.7 and pi 0.5 13.5. Average execution time: Natura-1 Base 63.16 s, Natura-1 Agentic 139.31 s, pi 0.5 on grippers 289.11 s, GPT-6-Astra on grippers 2,734.56 s. — Overall score grouped by the robot each model ran on, and average execution time. The baselines' bars are on a different robot (Natura-1 technical blog, benchmark figure).

The headline is Natura-1 at 53.4 (Base) and 59.1 (Agentic) against π0.5 at 13.5 and GPT-6-Astra at 16.7 (reported). Four things limit what that comparison says.

It is not the same robot. Natura-1 runs on a dual-arm ROKAE AR5 with Wuji 20-DoF hands. On that platform "neither baseline produces a scored result": π0.5 fails even after fine-tuning on Nature Will's dexterous-hand data, and GPT-6-Astra, used "directly as a coding-based policy", does not score within the 60-minute per-task limit. The 13.5 and 16.7 come from an ARX AC One dual-arm gripper platform, with π0.5 fine-tuned on gripper demonstrations of "exactly the same general tasks". To its credit, the blog states that "a strictly like-for-like baseline" is unavailable. The chart still puts the bars side by side. A gripper scoring 0.0% on bimanual coordination and "Not scored" on in-hand manipulation is measuring the robot as much as the policy.

The benchmark is in-house and unpublished. Twelve capability dimensions in three groups (dexterous control, instruction following, long-horizon execution). No task list, trial count, scoring rubric or success criterion is published. The four example tasks shown come with their "closest demonstrations retrieved from the training set", which is a good practice I wish more labs copied. It is still four tasks chosen by the authors.

The overall score is not the mean of its parts. I averaged the twelve per-dimension scores from the blog's chart (measured). Natura-1 Base averages 56.0 against an overall of 53.4. Agentic averages 66.2 against 59.1. On grippers, over the eleven dimensions that were scored, GPT-6-Astra averages 30.3 against 16.7 and π0.5 15.9 against 13.5. Averaging the three group means instead does not match either. The overall is presumably weighted by tasks or trials, which the blog does not publish, so a reader cannot reproduce 53.4 from what is shown.

Scores by dimension in three groups. Dexterous control: pose control and contact, bimanual coordination, in-hand manipulation, tool use. Instruction following: goal-directed reasoning, spatial and scene generalization, tool and object generalization, new-action generalization, compositional generalization. Long-horizon execution: planning, memory, recovery. Natura-1 Agentic and Base lead every row; both baselines show no result on dexterous hands; in-hand manipulation is not scored for grippers. — Per-dimension scores. Several sit at round fractions such as 16.7%, 66.7%, 83.3% and 100.0%, which is what small trial counts produce (Natura-1 technical blog, benchmark figure).

The trial counts look small. In-hand manipulation is 16.7% for both Natura-1 variants. Tool and object generalization is 100.0% for both. Agentic's recovery score is 83.3% and its tool use 66.7%. Those are one sixth, one, five sixths and two thirds, which is what a handful of trials per dimension looks like (reasoned; the counts are not published, and other cells such as 35.4% are not small fractions). A two-point gap between variants on such a dimension can be one trial.

Two smaller points. "Natura-1 Agentic" is never defined in the text; it scores higher but takes 139.31 s per task against Base's 63.16 s (reported), which is consistent with a planning-and-retry wrapper around the same policy, though that is my inference. And execution-time ratios such as "43.3× faster than GPT-6-Astra" also cross platforms; I get 43.3 and 4.6 from the reported seconds too (reasoned), but a gripper and a hand do not run the same motions.

What I'd take from it

The architecture is the interesting part, and it is a clean bet. Put the whole episode in one sequence, keep the transformer standard, predict future video and actions together, and let the abundant modality (video) carry most of the gradient. Teacher forcing over a whole episode is a real efficiency win over window sampling, for the same reason it is in language modelling. The data pipeline is described with more specificity than most labs offer: 61,702,471 episodes, 317,842 tasks, 14,286 scenes, a 41% MPJPE claim, a weekly throughput.

What the release does not support is a quantitative comparison with anything. The robot-transfer number is a prediction error along what looks like one training run. The benchmark compares hands with grippers on a private suite whose aggregate cannot be rebuilt from its parts. The 50B is total parameters with the active count withheld, and the 100 ms loop has no hardware attached. None of that makes the demos less impressive; a nearly 30-minute single take of a robot following spoken instructions is hard to fake. It means the evidence today is the video, not the numbers.

If you want an open model in the same family to read and run, FLUX 3 Action publishes weights and code for joint video-action denoising, Flex-π shows the same shared-latent idea with switchable outputs, and Cosmos 3 puts reasoning and generation in one sequence. For a flow-matching expert bolted onto a VLM rather than a single stream, see Qwen-Drive-1.0. Natura-1 is the biggest bet yet on human video as the pretraining corpus for hands. Whether it wins will show when someone outside the lab can measure it.