Intern-S2: a 397B model that reads the raw page

Intern-S2-Preview-397B, from InternLM (Shanghai AI Lab), is a 397-billion-parameter multimodal foundation model built for one thing: science. Not "science" as a benchmark category bolted onto a general chatbot — science as the training objective, down to how the model tokenizes a molecule and how it reads a figure off a paper.

The headline is a shape, not a single number. On general knowledge, math, and agentic coding, Intern-S2 sits at frontier parity — a 397B model trading blows with much larger closed systems, usually a hair behind. On specialized science — multi-omics, molecular reasoning, material generation, protein-binder design — it leads every frontier model, often by 4× or more. That gap is the whole story, and it's the payoff of three specific design choices.

The lineage

Intern-S2 is the third step in a line, and each step is worth naming because Intern-S2 inherits all of it:

• Intern-S1 — a 235B mixture-of-experts on a Qwen3 backbone plus a 6B InternViT vision encoder, continuously pretrained on 5T tokens, more than half of it scientific.
• Intern-S1-Pro — scaled to a trillion-parameter MoE with 512 experts and 8 active per token, added Fourier Position Encoding (FoPE) and explicit time-series modelling.
• Intern-S2-Preview-397B — the most capable of the family. (There is also a lightweight Intern-S2-Preview-35B, continued-pretrained from Qwen3.5 with a shared-weight multi-token-prediction head, a KL loss, and chain-of-thought compression.)

The MoE math from S1-Pro is the standard sparsity trade. With $k$ of $N$ experts firing per token,

so a trillion-parameter model pays for only ~22B activated parameters per token. FoPE and time-series modelling let it ingest sequences from $10^0$ to $10^6$ points — the kind of range a seismograph or a mass spectrometer actually produces.

Three ideas that matter

Strip away the scale and Intern-S2 is three ideas working together:

1. A vision-language pretraining paradigm that learns directly from raw pages of scientific literature — no OCR-and-parse preprocessing step in front of the model.
2. A dynamic tokenizer that natively represents molecular formulas, protein sequences, and seismic signals as meaningful units rather than subword debris.
3. Large-scale multi-task reinforcement learning across more than 20 scientific domains, trained jointly, which also happens to sharpen general reasoning.

Take them in order.

Reading the raw page

A conventional document pipeline flattens a page to a string before the model ever sees it: OCR recovers the words, a layout parser guesses the reading order, and the figures and equations are dropped on the floor. The text model then learns from a transcript that has already thrown away the thing you care about — how this curve relates to that caption and that variable.

Intern-S2 skips the transcript. It "learns directly from raw pages of scientific literature, jointly modelling symbolic semantics and visual relationships in a shared representation space without intermediate parsing." The vision encoder maps text, figures, and equations into one representation, and both a symbolic-semantics head and a visual-relations head read off that same space.

The consequence is that a plot and the sentence that references it are learnable as a single object, not two disconnected streams. For scientific literature — where the argument often lives in the figure — that is the difference between a model that reads the paper and one that reads a description of the paper.

The dynamic tokenizer

A tokenizer is a vocabulary learned on a corpus, and a standard BPE vocabulary is learned on natural-language text. Hand it a SMILES string or a protein sequence and it splits where its merge statistics say to split — which has nothing to do with where the meaning is. The aromatic ring in aspirin gets smeared across three tokens; a run of amino acids gets merged into a chunk that no longer corresponds to any residue.

Intern-S2's dynamic tokenizer emits scientifically-meaningful units directly: an atom, a bond, a residue, a waveform sample each become a token the model can address. This is not cosmetic. If a residue's identity is spread across a token boundary, the model can't attend to that residue cleanly — the representation is fighting the tokenizer. Native tokenization is what lets Intern-S2 treat a molecular formula, a protein, or a time series as a first-class input instead of a string that happens to look like one.

Multi-task RL across 20+ domains

The last piece is post-training. Intern-S2 runs large-scale reinforcement learning across more than 20 scientific domains jointly, rather than fine-tuning a separate model per task. Training the domains together is what gives the model its leading general-reasoning scores as a side effect: the same optimization that teaches it multi-omics and material chemistry also rewards careful, multi-step reasoning that transfers.

It deploys on the usual high-throughput stacks — LMDeploy, vLLM, and SGLang — with a 256K-token context for text reasoning and 64K tokens for multimodal input. It's genuinely strong at generative science: biomolecular interaction design and material-structure generation, not just question answering.

The benchmarks

Here is the shape, in one chart. Flip between the two task families and watch Intern-S2's dot move from just behind the best competitor to far ahead of it.

General tasks: frontier parity

On general benchmarks Intern-S2 rarely wins outright, but it rarely loses by much — which is the remarkable part for a 397B model standing next to the largest closed systems. It posts MMLU-Pro 89.75 (Gemini-3.1-Pro leads at 91.00), HMMT-2026 91.57 (GPT-5.5 at 97.06), MMMU-Pro 80.46, and SWE-Bench-Multilingual 81.67 — effectively tied with GLM-5.2's 82.00.

expand

On factual recall it clearly clears the open field even where it trails the closed leader — SimpleQA-Verified is a good example:

The honest read: it trails the very top closed models on the hardest coding and knowledge benches — TerminalBench 2.1 67.42 vs Claude-Opus-4.8's 84.60, SWE-Bench-Pro 61.56 vs 69.20. Parity, not conquest.

Scientific tasks: dominance

Now the inversion. On specialized science the gaps stop being fractions of a point and start being multiples. Biology-Instructions (multi-omics) is the clearest case: Intern-S2 scores 56.92 where the next-best frontier model manages 13.87, and most models land between 4 and 10.

expand

Material-structure generation tells the same story: MP20 67.88 against a next-best of 16.75, with most models between 1.5 and 16. Molecular reasoning (Mol-Instructions 52.37 vs GPT-5.5's 40.49), remote sensing (XLRS-Bench 51.97), microscopy VQA (MicroVQA 68.81), and biomolecular interaction design (ProteinBinder-9 4.36 vs a best competitor near 2.4) all land the same way. Roughly:

It is not a clean sweep, and that's worth saying: on MolecularIQ, GPT-5.5 still leads (76.41 vs Intern-S2's 61.49). But across the science suite as a whole, a 397B model beats GPT-5.5, Gemini-3.1-Pro, and Claude-Opus-4.8 — the payoff of the raw-page pretraining, the dynamic tokenizer, and multi-domain RL compounding.

What I make of it

• The specialization is the product. Most "science" models are general chatbots with a domain fine-tune. Intern-S2 pushes science into the tokenizer and the pretraining objective, and the benchmark gaps show the difference that makes — 4× is not a prompt-engineering delta.
• Parity at 397B is the quiet achievement. Matching Gemini-3.1-Pro and Opus-4.8 on general tasks with a fraction of the (public) scale, while dominating science, is a stronger statement than any single scientific score.
• Trust the shape, verify the numbers. The parity-vs-dominance pattern is convincing and mechanistically motivated. The internal-benchmark wins want independent replication before I'd quote the exact multiples as settled — but even halved, the lead holds.

Sources: the Intern-S2-Preview-397B model card and the Intern-S1 project (InternLM / Shanghai AI Lab). Benchmark numbers are quoted as reported on the model card; several scientific benchmarks are internal.