Intern-S2: a 397B model that reads the raw page

2026-07-17 · 8 min · multimodal · scientific-ai · mixture-of-experts

Why read this

A summary of Intern-S2's raw-page pretraining, science tokenizer and model-card benchmarks; the wins are vendor and partly internal benchmarks.

Open code or weights

Vision & multimodalNeeds datacenter GPUsApache-2.0Practitioner model

How this was scored

Is it new?: 1 of 3: An incremental tweak
Can I trust it?: 0 of 3: Restates claims
Can I run it?: 2 of 3: Open code or weights with real limits
Will I understand it?: 1 of 3: Partial mechanism
Can I act on it?: 1 of 3: General advice
Will it last?: 1 of 3: Relevant for months
Does it affect many?: 1 of 3: A specialist community
Only here?: 0 of 3: Same as the press release

Score 29 of 100, ranked 566 of 577 rated articles. Each question is answered 0–3 by hand, and a 3 is rare. How articles are scored

Intern-S2-Preview-397B, from InternLM (Shanghai AI Lab), is a 397-billion-parameter multimodal foundation model built for one thing: science. Not "science" as a benchmark category bolted onto a general chatbot — science as the training objective, down to how the model tokenizes a molecule and how it reads a figure off a paper.

The headline is a shape, not a single number. On general knowledge, math, and agentic coding, Intern-S2 sits at frontier parity — a 397B model trading blows with much larger closed systems, usually a hair behind. On specialized science — multi-omics, molecular reasoning, material generation, protein-binder design — it leads every frontier model, often by 4× or more. That gap is the whole story, and it's the payoff of three specific design choices.

The lineage

Intern-S2 is the third step in a line, and each step is worth naming because Intern-S2 inherits all of it:

Intern-S1 — a 235B mixture-of-experts on a Qwen3 backbone plus a 6B InternViT vision encoder, continuously pretrained on 5T tokens, more than half of it scientific.
Intern-S1-Pro — scaled to a trillion-parameter MoE with 512 experts and 8 active per token, added Fourier Position Encoding (FoPE) and explicit time-series modelling.
Intern-S2-Preview-397B — the most capable of the family. (There is also a lightweight Intern-S2-Preview-35B, continued-pretrained from Qwen3.5 with a shared-weight multi-token-prediction head, a KL loss, and chain-of-thought compression.)

The MoE math from S1-Pro is the standard sparsity trade. With $k$ of $N$ experts firing per token,

\theta_{\text{active}} \;=\; \frac{k}{N}\,\theta_{\text{expert}} \;+\; \theta_{\text{shared}}, \qquad k = 8,\; N = 512,

so a trillion-parameter model pays for only ~22B activated parameters per token. FoPE and time-series modelling let it ingest sequences from $10^0$ to $10^6$ points — the kind of range a seismograph or a mass spectrometer actually produces.

Three ideas that matter

Strip away the scale and Intern-S2 is three ideas working together:

A vision-language pretraining paradigm that learns directly from raw pages of scientific literature — no OCR-and-parse preprocessing step in front of the model.
A dynamic tokenizer that natively represents molecular formulas, protein sequences, and seismic signals as meaningful units rather than subword debris.
Large-scale multi-task reinforcement learning across more than 20 scientific domains, trained jointly, which also happens to sharpen general reasoning.

Take them in order.

Reading the raw page

A conventional document pipeline flattens a page to a string before the model ever sees it: OCR recovers the words, a layout parser guesses the reading order, and the figures and equations are dropped on the floor. The text model then learns from a transcript that has already thrown away the thing you care about — how this curve relates to that caption and that variable.

Intern-S2 skips the transcript. It "learns directly from raw pages of scientific literature, jointly modelling symbolic semantics and visual relationships in a shared representation space without intermediate parsing." The vision encoder maps text, figures, and equations into one representation, and both a symbolic-semantics head and a visual-relations head read off that same space.

learning from the raw page

Intern-S2 reads the page the way you do: text, a figure, and an equation are encoded together into one representation, so the model can learn that this curve belongs to that caption and that variable. Symbolic semantics and visual relationships are modelled jointly, in the same space.

The consequence is that a plot and the sentence that references it are learnable as a single object, not two disconnected streams. For scientific literature — where the argument often lives in the figure — that is the difference between a model that reads the paper and one that reads a description of the paper.

The dynamic tokenizer

A tokenizer is a vocabulary learned on a corpus, and a standard BPE vocabulary is learned on natural-language text. Hand it a SMILES string or a protein sequence and it splits where its merge statistics say to split — which has nothing to do with where the meaning is. The aromatic ring in aspirin gets smeared across three tokens; a run of amino acids gets merged into a chunk that no longer corresponds to any residue.

dynamic tokenizer · one input, two vocabularies

Intern-S2 dynamic tokenizer

8 meaningful tokens

atoms, carbonyls and the benzene ring stay whole

standard BPE tokenizer

8 subword fragments

the aromatic ring c1ccccc1 is split across 3 tokens

A BPE vocabulary learned on natural-language text has no reason to respect a chemical bond, a residue, or a sample's magnitude. It splits where its merge statistics say to split, so the units that carry scientific meaning end up smeared across token boundaries. Intern-S2's tokenizer emits those units directly — the difference between a model that sees a molecule and one that sees a string that happens to look like one.

Intern-S2's dynamic tokenizer emits scientifically-meaningful units directly: an atom, a bond, a residue, a waveform sample each become a token the model can address. This is not cosmetic. If a residue's identity is spread across a token boundary, the model can't attend to that residue cleanly — the representation is fighting the tokenizer. Native tokenization is what lets Intern-S2 treat a molecular formula, a protein, or a time series as a first-class input instead of a string that happens to look like one.

Multi-task RL across 20+ domains

The last piece is post-training. Intern-S2 runs large-scale reinforcement learning across more than 20 scientific domains jointly, rather than fine-tuning a separate model per task. Training the domains together is what gives the model its leading general-reasoning scores as a side effect: the same optimization that teaches it multi-omics and material chemistry also rewards careful, multi-step reasoning that transfers.

It deploys on the usual high-throughput stacks — LMDeploy, vLLM, and SGLang — with a 256K-token context for text reasoning and 64K tokens for multimodal input. It's genuinely strong at generative science: biomolecular interaction design and material-structure generation, not just question answering.

The benchmarks

Here is the shape, in one chart. Flip between the two task families and watch Intern-S2's dot move from just behind the best competitor to far ahead of it.

Intern-S2 vs. best competing model

Intern-S2best competitorleads 3/4 · median gap +27.5

On specialized science the picture inverts: Intern-S2 opens gaps of 30–50 points on multi-omics, molecular reasoning and material generation — routinely 4× the nearest frontier model. MolecularIQ is the honest exception, where GPT-5.5 still leads.

General tasks: frontier parity

On general benchmarks Intern-S2 rarely wins outright, but it rarely loses by much — which is the remarkable part for a 397B model standing next to the largest closed systems. It posts MMLU-Pro 89.75 (Gemini-3.1-Pro leads at 91.00), HMMT-2026 91.57 (GPT-5.5 at 97.06), MMMU-Pro 80.46, and SWE-Bench-Multilingual 81.67 — effectively tied with GLM-5.2's 82.00.

Benchmark table of general tasks comparing Intern-S2 against Qwen3.5-397B-A17B, DeepSeek-V4-pro, Kimi-K2.7-Code, GLM-5.2, GPT-5.5, Gemini-3.1-Pro, and Claude-Opus-4.8 across MMLU-Pro, SimpleQA-Verified, AdvancedIF, HMMT-2026, MMMU-Pro, ChartQAPro, SkillsBench, TerminalBench, SWE-Bench-Pro, and SWE-Bench-Multilingual. — General-task benchmarks: Intern-S2 at frontier parity with the largest closed and open models (Intern-S2-Preview-397B model card, 2026).

On factual recall it clearly clears the open field even where it trails the closed leader — SimpleQA-Verified is a good example:

SimpleQA-Verified (%)

Gemini-3.1-Pro

75.6

Intern-S2

69.9

GPT-5.5

64.3

Qwen3.5-397B

54.8

DeepSeek-V4-pro

46.6

020406080

The honest read: it trails the very top closed models on the hardest coding and knowledge benches — TerminalBench 2.1 67.42 vs Claude-Opus-4.8's 84.60, SWE-Bench-Pro 61.56 vs 69.20. Parity, not conquest.

Scientific tasks: dominance

Now the inversion. On specialized science the gaps stop being fractions of a point and start being multiples. Biology-Instructions (multi-omics) is the clearest case: Intern-S2 scores 56.92 where the next-best frontier model manages 13.87, and most models land between 4 and 10.

Biology-Instructions · multi-omics (%)

Intern-S2

56.92

Gemini-3.1-Pro

13.87

0204060

Material-structure generation tells the same story: MP20 67.88 against a next-best of 16.75, with most models between 1.5 and 16. Molecular reasoning (Mol-Instructions 52.37 vs GPT-5.5's 40.49), remote sensing (XLRS-Bench 51.97), microscopy VQA (MicroVQA 68.81), and biomolecular interaction design (ProteinBinder-9 4.36 vs a best competitor near 2.4) all land the same way. Roughly:

\frac{56.92}{13.87} \approx 4.1\times, \qquad \frac{67.88}{16.75} \approx 4.05\times.

It is not a clean sweep, and that's worth saying: on MolecularIQ, GPT-5.5 still leads (76.41 vs Intern-S2's 61.49). But across the science suite as a whole, a 397B model beats GPT-5.5, Gemini-3.1-Pro, and Claude-Opus-4.8 — the payoff of the raw-page pretraining, the dynamic tokenizer, and multi-domain RL compounding.

What I make of it

The specialization is the product. Most "science" models are general chatbots with a domain fine-tune. Intern-S2 pushes science into the tokenizer and the pretraining objective, and the benchmark gaps show the difference that makes — 4× is not a prompt-engineering delta.
Parity at 397B is the quiet achievement. Matching Gemini-3.1-Pro and Opus-4.8 on general tasks with a fraction of the (public) scale, while dominating science, is a stronger statement than any single scientific score.
Trust the shape, verify the numbers. The parity-vs-dominance pattern is convincing and mechanistically motivated. The internal-benchmark wins want independent replication before I'd quote the exact multiples as settled — but even halved, the lead holds.

Sources: the Intern-S2-Preview-397B model card and the Intern-S1 project (InternLM / Shanghai AI Lab). Benchmark numbers are quoted as reported on the model card; several scientific benchmarks are internal.