Which half of the scaffolding gets eaten

The authors of a 3D coding harness went back to their own system with a stronger base model and published what they found:

That is a claim against their own contribution, which is the kind of claim worth reading carefully. It also cuts against this site. Thirty-two of the articles here name a harness, a decision layer, a scaffold or an enumerated option set in their own title or description. If scaffolding is a bridge you build only until the base model can cross on its own, most of that is a depreciating asset and I have been writing it up as though it were a fixed one.

So: read the paper, work out what it measured, work out what the revisit adds, and then test the one distinction that decides how much of the warning applies. The distinction is between scaffolding that substitutes for a capability the model lacks and scaffolding that constrains what the model is allowed to do. I did not want to assert it. I wanted to take this harness apart and see whether its parts actually sort that way.

They mostly do. There is a third bucket the binary hides, and the constraint half is in worse shape than the split implies.

The paper

Thinking in Blender: Staged Executable Inverse Graphics with Vision-Language Models (He, Luo, Ma, Averbuch-Elor — Cornell, arXiv:2606.02580, 1 June 2026) asks whether a pretrained VLM can reconstruct a scene from a single image as an editable Blender program — no differentiable rendering, no multi-view supervision, and deliberately no specialist 2D or 3D foundation models. The output is Python that builds a scene, so the reconstruction can be relit, rearranged and dropped into a physics engine afterwards.

Their observation is that a VLM asked to do all of that at once fails, and their fix is decomposition. SEIG splits reconstruction into the order a human artist works in — geometry, then material, then composition, then lighting — and closes each stage with its own generator–verifier loop before the next one starts.

expand

The mechanics worth keeping in mind for later:

• Initialization samples four independent scene graphs and coarse scaffolds, and a rollout selector keeps the one with the most complete object coverage.
• Every object gets a stable Blender name at initialization and keeps it through every later stage, so refinement can be addressed at a node.
• Each stage has a round budget — five for geometry, three each for material and composition, two for lighting — because "refinement effectiveness" degrades as context accumulates.
• The verifier returns a checklist, not a critique. Free-form feedback was "noisy across attempts, giving the generator inconsistent targets"; an explicit approval checklist is injected into the generator's next context.
• Stages are scoped by the tools they get. Material refinement runs through "a material-only tool that permits only material-related edits." Composition "is not allowed to edit object geometry or materials." Lighting optimises illumination "while keeping object shape, appearance, layout, and camera fixed."

One implementation detail does more work than anything else in this article: every reported result uses Claude Opus 4.7, as both generator and verifier, at every stage, with no fine-tuning. The paper says why — "any observed difference in reconstruction quality between methods can be attributed to harness design rather than the underlying model." That is a good experimental decision and it has a consequence: there is no model-by-model table in this paper. Not a bar chart, not a row. One backbone, start to finish.

There is no bar

"None of the models we tested could really hit the bar" implies a threshold. The paper does not define one. It reports six continuous similarity scores between the reconstructed render and the reference image — PSNR and SSIM at the pixel level, LPIPS and DreamSim as learned perceptual distances, and cosine similarities from a DINOv2 ViT-L/14 [CLS] feature and a CLIP ViT-B/32 embedding. Where reference meshes exist, each reconstruction is first registered to the reference with NDP and ICP (whichever gives the smaller Chamfer distance) and re-rendered from the reference camera, so the camera estimate does not contaminate the score.

The evaluation set is two collections: five renders from each of seven NeRF synthetic scenes — the materials scene with its specular metal spheres is excluded, leaving 35 images — plus 13 object-centric scenes from Edit3D.

Nothing in those twelve columns is a pass mark. And the absolute level is worth saying out loud, because it supports the authors rather than undercutting them: a PSNR of 13.58 dB between a render and its reference is a poor pixel match by any standard. These numbers are not directly comparable to novel-view-synthesis figures — different task, post-registration, single input view — but at this level nobody has hit a bar you would recognise as one. When the authors say the models could not really hit it, the tables agree.

The one number the threshold claim is about

Strip the paper to the quantity that is actually a property of the scaffolding and you get one comparison. SEIG minus VIGA (VLM-only): same backbone, same tasks, same metrics, both VLM-only, one staged and one monolithic. That difference is the staging, and it is the thing the revisit says goes to zero.

At Opus 4.7 the staging is worth +1.25 dB PSNR and +15.5% DINO on NeRF synthetic, +1.13 dB and +12.3% on Edit3D. Two things in that panel deserve more attention than the paper gives them. SSIM goes the wrong way on both sets — the unstaged baseline is the best of the three, and on Edit3D SEIG is third. And the LPIPS win on NeRF synthetic is 0.3506 against 0.3493: a 0.4% improvement, which is one of the "five out of six metrics" the paper claims. The staging is a real effect with a clear signature: the semantic scores move most (DINO, CLIP, DreamSim), PSNR moves with them, LPIPS barely registers on one set, and SSIM — the pixel-structure metric — goes backwards on both. Four convincing columns on NeRF synthetic, three on Edit3D. Not six.

There is also a row the paper does not have, and its absence matters. There is no no-harness baseline. VIGA VLM-only is itself an agentic write–render–compare–revise loop. Nobody measured "prompt the VLM once for a Blender program." So the claim "below the threshold the harness helps a lot" is, in this paper, a claim about one harness beating another harness — not about scaffolding beating none.

What the revisit does and does not establish

The revisit is a social-media post. That is not a knock; practitioners' reports about their own systems are useful and this one is being made against interest. But it should be read as a different kind of object from the paper, and four things follow.

1. No number. "Helps a lot" and "almost not at all" describe a shape. The paper has six metrics and two datasets; the revisit reports none of them. The right-hand panel above is the point: a step, a knee and a straight line all fit those words.
2. No bar. The threshold is stated as a property of models ("below it", "above it") but the quantity being thresholded was never defined.
3. The original was not a model sweep. "None of the models we tested" refers to work that is not in the paper. The published record contains exactly one backbone. So the before-state of the comparison is unpublished, and so is the after-state.
4. "Mostly the same across all model providers" rests on no provider list and no per-provider figure. The paper's cross-provider evidence is zero rows.

Now the part that cuts the other way, and it is stronger than any of the four above. The paper already located the bottleneck in the model. Discussing its own result, it says the pattern is "consistent with the finding from BlenderGym and IR3D-Bench that visual precision, not tool orchestration, is the dominant bottleneck in current agentic 3D pipelines."

If the binding constraint is the model's visual precision, then better visual precision dissolves the gap. That is not a reversal — it is the paper's own diagnosis running forward a few months. The authors wrote down the mechanism that would make their harness redundant, in the paper that introduced it, and then reported that it happened. Whatever else is missing, they are not guessing.

Scaffolding that was already worth less than nothing

Before any threshold argument, this paper contains a cleaner result about scaffolding that almost nobody quotes. VIGA full is VIGA plus SAM and SAM-3D — specialist segmentation and single-image 3D reconstruction models, bolted on to pre-segment and pre-reconstruct objects before the agent loop runs. VIGA VLM-only is the same system with those models switched off. Same backbone in both.

Adding the specialist models makes it worse on five of six metrics on NeRF synthetic and five of six on Edit3D. The paper gives the mechanism: the VLM agent "may overwrite the texture of the 3D objects generated by SAM-3D," and single-view lifting drags its own failure modes in. The paper points at the humanoid in Figure 3, where VIGA full exhibits the classic Janus failure — frontal facial features duplicated onto the back of the head — which neither VLM-only configuration produces, because both build the figure compositionally out of primitives instead of lifting a single-view mesh.

expand

Hold that next to the warning. This is scaffolding that did not need to be eaten by a stronger model, because it was already worth less than nothing at the model they had. "All of today's scaffolding will eventually get eaten" is the optimistic version.

Substitute, supply, constrain

Here is the distinction I want to test. Some scaffolding exists because the model cannot do something. Retry loops, output parsers, chain-of-thought prompting, JSON-mode wrappers, ReAct templates — every one of those was an answer to a deficiency, and every one of them has been absorbed. That kind gets eaten, and the warning is simply right about it.

Other scaffolding exists because I need a guarantee the model cannot give me. A finite action space, a typed schema, a validator that runs before anything ships. Its value was never "the model cannot do this." A better base model does not make an unbounded action space safe.

Taking SEIG apart, the binary turns out to be missing a bucket. A render is neither. When the verifier executes the Blender code and looks at the output, it is not compensating for a weakness and it is not forbidding anything — it is supplying a fact the model could not have had from the inside. Test results, compiler errors and tool responses are all this third thing, and they are most of what people point at when they say "the harness."

The honest count: three of eight are pure substitutes, and they include the mechanism the paper credits for its entire result. The staged decomposition exists because "pretrained VLMs struggle to reconstruct all scene factors simultaneously" — that is the paper's own sentence, and it is a capability claim about a checkpoint. The four rollouts are best-of-n bought against variance. The round budget is a cap on context degradation. Every one of those premises is a thing that scaling fixes.

So on this harness, the warning is right about the part that matters most. I would have preferred a different answer.

The part that is a permission

Two of the eight rows are pure constraints, and one of them is worth drawing, because it is the clearest example I have found in a paper of a rule enforced on the model rather than read by it.

Read the material row. Material refinement does not merely tend to leave the geometry alone; the agent is handed a tool that can only express material edits. The guarantee is not in a prompt the model reads, it is in the surface the model can reach — which is the same placement agent-harness argued for from the other direction: a permission check the agent can edit is not a permission check.

What this buys is visible in the pipeline's intermediate output. Because no stage may undo an earlier one, every stage boundary is a committed, coherent, editable Blender program.

expand

That property is not a capability substitute. Relighting works because lights are a separate committed stage output; per-object editing works because objects were built independently and only later assembled; physics works because the result is "a structured collection of separately addressable meshes" rather than a fused implicit surface, needing no remeshing or watertighting. None of that is waiting on a better model. A stronger VLM emitting one entangled blob is still an entangled blob.

Constraints do not get eaten. They get repriced.

This is where the two-bucket defence of the corpus stops working, and the paper hands me the evidence.

A constraint's value is the consequence of the event it forbids, multiplied by how often that event would otherwise happen. A stronger base model moves exactly one of those terms. It overwrites an accepted stage less often; it does nothing at all to what happens when it does. Frequency is a model property. Consequence is a deployment property. So the expected value of the guarantee falls, slowly, and never reaches zero.

But the constraint has a second term, and that one moves the wrong way. What you gave up to get the guarantee is worth more when the model is better. SEIG's limitations section is the receipt, and it describes its own permission matrix:

That failure exists because the composition stage is not allowed to edit geometry. The ten frozen cells in the matrix above are the guarantee and the failure mode, and they are the same ten cells. The paper even names the fix — "additional global refinement passes that revisit and jointly optimize earlier scene factors" — and its cost, "substantially increased computational cost and inference time."

Now run the base model forward. A model good enough to hold all four factors at once is precisely the model that would use a global pass well. The ban costs nothing when the model cannot do the thing anyway; it costs a great deal when the model could. So the constraint tier is not safe. It does not get absorbed — it gets outvoted, when the opportunity cost of the ban rises past the expected cost of the event it prevents.

Which is a real concession to the warning, arrived at from the other side. Substitutes get eaten. Constraints get audited. Both roads end with less scaffolding than you have now; only the reasoning differs, and the reasoning is what tells you which ones to keep.

The ones that survive an audit are the ones where the forbidden event is expensive in a way capability does not touch: money moved, data deleted, a claim published. There, frequency falling from 1% to 0.001% changes the arithmetic and not the decision.

Running the test on this site

Three of this site's own arguments, cut in half. The pattern in that figure is the finding: each piece welded a capability argument to a guarantee and shipped them as one claim. The left column is depreciating in every row. The right column is not, and in every row this site stated it less clearly than the left.

The webmcp-windtunnel line I keep coming back to is the cleanest case. A tool refused a bad argument and answered with its four legal values; the next step used one. The article's own gloss: that recovery is "a property of the code on the website, not of either model." That is exactly right and it is exactly the kind of thing a better base model does not supply. But the same article's headline was a cost multiplier between two checkpoints, which is the most perishable number in it.

And there is a measured result in this corpus that contradicts the warning's universal quantifier outright. The harness effect held the model fixed and swapped only the orchestration layer across six models: cost per task fell for every single one, by roughly a third to 61%, and the paper's mechanism list is cache-shape discipline, incremental compaction, context offload, durable execution and failure-spend governance. Not one of those is a capability the base model can absorb, because none of them is about the model. Prompt caching pays out when your prefix is byte-stable across turns; that is a fact about the bytes you send and the provider's billing, and a smarter model does not make your prefix stable.

Set that beside the other edge. Agent harnesses records STOP improving results on GPT-4 and degrading them on GPT-3.5 and Mixtral — Weng's reading being that "the base model must be capable enough to improve the mechanism." So substitute scaffolding helps inside a window: too weak and the model cannot use it, too strong and it does not need it. Both edges move right over time. A harness built on a substitute has a shelf life equal to the width of that window, and this paper's revisit is a report of the upper edge arriving.

The take

The warning is right, and it is right about more of this paper than I expected before I sorted the parts. SEIG's headline mechanism is a capability substitute in its authors' own words, its bottleneck is a model property by its own diagnosis, and the gains it measures are concentrated in the semantic metrics that a better visual model would be expected to move. Reporting that against your own contribution is good practice and I would rather have this finding than not.

What the revisit does not have is a measurement. The staging gain is a concrete, published, two-dataset number at one backbone; the claim that it goes to zero at a stronger one is two adjectives. The experiment that settles it is the same evaluation set, the same six metrics, a different backbone — a weekend's work for anyone with the code, which is not released.

The distinction survives contact, with one correction and one caveat. The correction: it is three buckets, not two, and the middle one — execution results, tool responses, the render — is most of what people actually mean by a harness and is the one nobody argues about. The caveat is the one that costs this site something. Constraints are not a safe place to have put your engineering. They do not get eaten, but the thing you traded away to get them appreciates with every checkpoint, and a constraint whose price you never wrote down will get removed by someone who has only ever seen its cost.

So the rule I would actually apply, going forward: for every bounded action space, typed schema and validation gate, write the sentence json-render wrote in its README — what this makes impossible. If that sentence names something a stronger model would have done well, the guarantee is on a clock. If it names something you never wanted to happen at any capability level, it is not scaffolding. It is the specification.

Source: Thinking in Blender: Staged Executable Inverse Graphics with Vision-Language Models (Guangzhao He, Rundong Luo, Wei-Chiu Ma, Hadar Averbuch-Elor — Cornell University), arXiv:2606.02580, 1 June 2026. Figures 2, 3 and 7 are reproduced from the paper for commentary; all other diagrams are mine, computed from the paper's published tables or drawn from its method text. There is no project page, no code release and no video; the quoted revisit is a social-media post by the authors, not a paper, and is treated as such throughout. Table values are quoted as published; the relative gains are my arithmetic on them.