2026-08-23 · 8 min · multimodal · image-generation · architecture · open-weights · explainer
For years the standard unified multimodal stack has had two pre-trained bottlenecks bolted to either end of a language model: a vision encoder that turns pixels into semantic tokens so the model can see, and a VAE that turns latents back into pixels so it can draw. Both are trained separately, both impose a representation the language model had no say in, and most of the field's argument about unified models is really an argument about those two choices.
NEO-unify, from SenseTime with NTU, removes both. Its slogan is exactly that blunt: No VE! No VAE! Images go in as patch embeddings and come out as patch embeddings, through one backbone that also handles words — autoregressive cross-entropy for text, pixel flow matching for vision, one representation space shaped by the model itself.
SenseNova-U1.5-8B-MoT is the latest checkpoint built on it: 18B parameters in bf16, Apache 2.0, doing text-to-image and native image editing, with a demo Space you can drive.
| Model | sensenova/SenseNova-U1.5-8B-MoT · Apache 2.0 · 18B params, bf16 |
| Architecture | NEO-unify — encoder-free, native Mixture-of-Transformer |
| Objectives | AR cross-entropy for text · pixel flow matching for vision, in one backbone |
| Reconstruction | 31.56 dB / 0.85 SSIM on MS COCO 2017 against Flux VAE's 32.65 / 0.91 |
| Editing | 3.32 on ImgEdit at 2B with the understanding branch frozen |
| Reference config | cfg_scale=4.0, timestep_shift=3.0, num_steps=50 |
| Code | OpenSenseNova/SenseNova-U1 |
- parameters
- 17.53B
- repo size
- 85.29 GB
- architecture
- NEOChatModel
- license
- apache-2.0
- downloads
- 8.1K
- likes
- 216
- files
- 18
What gets removed
The reconstruction row is the one that decides whether any of this is possible. A VAE exists because getting pixels back out is hard and a decoder trained for nothing else does it well — so a model that never had one should be badly worse. It is 1.09 dB worse, after ninety thousand pretraining steps at 2B parameters, with the reconstruction coming out of a generative pathway attached to a frozen understanding branch.
What that buys is not fidelity, it is freedom. When the representation space is shaped by the model rather than inherited from two separately-trained components, the argument about which representation is right for understanding versus generation stops being an architecture decision and becomes something the training run settles. The team’s own framing — stepping beyond representation arguments — is the accurate description of what removing both bottlenecks is for.
The objection to this design writes itself: a VAE exists because getting pixels back out is hard, and a decoder trained for nothing else does it well. A model that never had one should be visibly worse.
It is 1.09 dB worse. NEO-unify at 2B reaches 31.56 dB PSNR and 0.85 SSIM on MS COCO 2017 after ninety thousand pretraining steps, against Flux VAE's 32.65 and 0.91. And the reconstruction comes out of a generative pathway attached to a frozen understanding branch — the model is recovering fine-grained visual detail from a representation that was not being updated to help it.
That is the load-bearing result. Once near-lossless inputs demonstrably support both semantic understanding and pixel-level fidelity in the same space, the pre-trained encoder and the pre-trained decoder stop being necessary and start being constraints.
The editing result follows from it and is stranger. With the understanding branch still frozen, a 2B NEO-unify reaches 3.32 on ImgEdit after 60k mixed training steps on public T2I and editing data — throwing all condition context through the frozen understanding pathway while the generative pathway produces the new image. Editing is the task where a unified model should most need both halves to co-adapt, and half of it is nailed down.
The MoT part
The other half of the design is the backbone: a native Mixture-of-Transformer, with understanding and generation as branches that share attention rather than as separate models.
The finding reported there is negative in a useful way. Jointly training both branches on the same mid-training and SFT data sources, even at low data ratios and loss weights, leaves understanding stable while generation converges faster. The team's phrase is "minimal intrinsic conflict".
That is worth flagging because the opposite is the field's working assumption. Generation and understanding are usually treated as objectives that fight — one wants to compress toward semantics, the other wants to preserve detail — and a great deal of architecture exists to keep them apart. If the conflict is small once you remove the two pre-trained bottlenecks, then some of that architecture was managing a problem the bottlenecks created.
They also report better data-scaling efficiency than Bagel, reaching higher performance with fewer training tokens. That is the claim I would most want to see plotted with axes rather than asserted, and it is asserted.
What 1.5 adds
The release notes six improvements, and the two I would look at first are text rendering — legible Chinese and English in posters and infographics, which is the capability most models are still bad at — and reliable native editing, meaning preservation of subject identity and unedited content across local, text, multi-reference, insertion and replacement edits.
Both of those are the same underlying property from different sides: how much of an existing image survives a pass through the model. An architecture with no VAE round-trip is structurally better placed there, because there is no lossy latent to squeeze the unedited parts through.
A finding from actually running it
This is a small finding and I like it disproportionately, because it is the kind that only appears when someone runs the model instead of reading the card. The reference configuration says fifty steps. A fixed-seed A/B says twenty-eight keeps composition, prompt adherence and text rendering — the capability most likely to fall apart first, and the one this release specifically improved — while losing only micro-texture in landscape and skin, and nothing measurable when editing.
Which means the correct default depends entirely on what you are doing. Editing, where the model is preserving most of an existing image, has no use for the extra twenty-two steps at all. Detail-critical generation does. That distinction is not in the model card, and there is no reason it would be — the card documents what was validated, not what someone later found by looking.
The model card's reference configuration is 50 denoising steps. The demo Space ships 28, and documents why: a fixed-seed A/B found that 28 keeps composition, prompt adherence and text rendering intact — losing only micro-texture in landscape and skin, and nothing measurable when editing — while running about 1.8× faster.
Since 50 ÷ 28 = 1.79, the speedup is exactly the step ratio: sampling cost is linear in steps with no fixed overhead worth modelling, which makes the step count a clean quality-versus-latency dial.
The part that generalizes is the asymmetry. Editing, where the model preserves most of an existing image, has no use for the extra twenty-two steps at all; detail-critical generation does. That distinction is not in the model card and there is no reason it would be — a card documents what was validated, not what someone later found by looking.
What I would want next
"Better data-scaling efficiency than Bagel" needs a curve. It is the strongest strategic claim in the architecture post — that removing pre-trained priors helps scaling rather than costing it — and it is one sentence next to a figure. Encoder-free designs have historically lost on sample efficiency precisely because they cannot inherit a frozen encoder's pretraining; if that has reversed, the plot is the paper.
The reconstruction comparison is at 2B and 90k steps. Both numbers are early-checkpoint numbers, offered as an existence proof rather than a converged result, and the post is clear about that. But the interesting question is what the gap does with scale: does an 18B model close the 1.09 dB, or is there a floor that a purpose-built decoder will always sit below?
The MoT split is not published. SenseNova-U1.5-8B-MoT is 18B parameters, which strongly suggests the 8B names one branch. How the parameters divide between understanding and generation, and how much attention they actually share, is the architecture question a reader most wants answered — and the model card points at a blog post that describes the paradigm rather than this checkpoint.
Benchmarks are images of charts. The model card's evaluation section is a radial plot and a combined figure, both as pictures. That is normal and it is still a shame: nobody can put these numbers in a table next to anything else without transcribing them by eye.
Why the framing matters more than the checkpoint
The sentence I keep coming back to is not about performance:
We return to the first principles: building a model that directly engages with native inputs — pixels and words.
Multimodal AI got built as a translation problem — encode this modality into that one's space, decode it back — and almost every design decision inherited that shape. NEO-unify's bet is that the translation layers were load-bearing only because nobody had removed them, and that the representation argument dissolves once one model owns both ends.
A 1.09 dB reconstruction gap and a frozen-branch editing score are not proof of that. They are the minimum evidence required to take it seriously, which is a different and more useful thing at this stage. The checkpoint is Apache 2.0 and the inference code is public, so it is checkable — and the step-count finding above is what happens when somebody checks.