# Intern-S2: a 397B model that reads the raw page

> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/intern-s2
> date: 2026-07-17
> tags: llm, multimodal, scientific-ai, mixture-of-experts, explainer
Intern-S2-Preview-397B, from InternLM (Shanghai AI Lab), is a 397-billion-parameter
multimodal foundation model built for one thing: science. Not "science" as a benchmark
category bolted onto a general chatbot — science as the training objective, down to how
the model tokenizes a molecule and how it reads a figure off a paper.

The headline is a shape, not a single number. On general knowledge, math, and agentic
coding, Intern-S2 sits at **frontier parity** — a 397B model trading blows with much
larger closed systems, usually a hair behind. On specialized science — multi-omics,
molecular reasoning, material generation, protein-binder design — it **leads every
frontier model**, often by 4× or more. That gap is the whole story, and it's the payoff
of three specific design choices.

## The lineage

Intern-S2 is the third step in a line, and each step is worth naming because Intern-S2
inherits all of it:

- **Intern-S1** — a 235B mixture-of-experts on a Qwen3 backbone plus a 6B InternViT
  vision encoder, continuously pretrained on 5T tokens, more than half of it scientific.
- **Intern-S1-Pro** — scaled to a trillion-parameter MoE with 512 experts and 8 active
  per token, added Fourier Position Encoding (FoPE) and explicit time-series modelling.
- **Intern-S2-Preview-397B** — the most capable of the family. (There is also a
  lightweight **Intern-S2-Preview-35B**, continued-pretrained from Qwen3.5 with a
  shared-weight multi-token-prediction head, a KL loss, and chain-of-thought compression.)

The MoE math from S1-Pro is the standard sparsity trade. With $k$ of $N$ experts firing
per token,

$$
\theta_{\text{active}} \;=\; \frac{k}{N}\,\theta_{\text{expert}} \;+\; \theta_{\text{shared}},
\qquad k = 8,\; N = 512,
$$

so a trillion-parameter model pays for only ~22B activated parameters per token. FoPE and
time-series modelling let it ingest sequences from $10^0$ to $10^6$ points — the kind of
range a seismograph or a mass spectrometer actually produces.

## Three ideas that matter

Strip away the scale and Intern-S2 is three ideas working together:

1. A **vision-language pretraining paradigm** that learns directly from raw pages of
   scientific literature — no OCR-and-parse preprocessing step in front of the model.
2. A **dynamic tokenizer** that natively represents molecular formulas, protein
   sequences, and seismic signals as meaningful units rather than subword debris.
3. Large-scale **multi-task reinforcement learning** across more than 20 scientific
   domains, trained jointly, which also happens to sharpen general reasoning.

Take them in order.

## Reading the raw page

A conventional document pipeline flattens a page to a string before the model ever sees
it: OCR recovers the words, a layout parser guesses the reading order, and the figures
and equations are dropped on the floor. The text model then learns from a transcript that
has already thrown away the thing you care about — how *this* curve relates to *that*
caption and *that* variable.

Intern-S2 skips the transcript. It "learns directly from raw pages of scientific
literature, jointly modelling symbolic semantics and visual relationships in a shared
representation space without intermediate parsing." The vision encoder maps text, figures,
and equations into one representation, and both a symbolic-semantics head and a
visual-relations head read off that same space.

<VisionPretrain />

The consequence is that a plot and the sentence that references it are learnable as a
single object, not two disconnected streams. For scientific literature — where the
argument often *lives* in the figure — that is the difference between a model that reads
the paper and one that reads a description of the paper.

## The dynamic tokenizer

A tokenizer is a vocabulary learned on a corpus, and a standard BPE vocabulary is learned
on natural-language text. Hand it a SMILES string or a protein sequence and it splits
where its merge statistics say to split — which has nothing to do with where the *meaning*
is. The aromatic ring in aspirin gets smeared across three tokens; a run of amino acids
gets merged into a chunk that no longer corresponds to any residue.

<DynamicTokenizer />

Intern-S2's dynamic tokenizer emits scientifically-meaningful units directly: an atom, a
bond, a residue, a waveform sample each become a token the model can address. This is not
cosmetic. If a residue's identity is spread across a token boundary, the model can't attend
to that residue cleanly — the representation is fighting the tokenizer. Native tokenization
is what lets Intern-S2 treat a molecular formula, a protein, or a time series as a
first-class input instead of a string that happens to look like one.

## Multi-task RL across 20+ domains

The last piece is post-training. Intern-S2 runs large-scale reinforcement learning across
more than 20 scientific domains **jointly**, rather than fine-tuning a separate model per
task. Training the domains together is what gives the model its leading general-reasoning
scores as a side effect: the same optimization that teaches it multi-omics and material
chemistry also rewards careful, multi-step reasoning that transfers.

It deploys on the usual high-throughput stacks — **LMDeploy, vLLM, and SGLang** — with a
256K-token context for text reasoning and 64K tokens for multimodal input. It's genuinely
strong at generative science: biomolecular interaction design and material-structure
generation, not just question answering.

## The benchmarks

Here is the shape, in one chart. Flip between the two task families and watch Intern-S2's
dot move from *just behind* the best competitor to *far ahead* of it.

<ScienceGap />

### General tasks: frontier parity

On general benchmarks Intern-S2 rarely wins outright, but it rarely loses by much — which
is the remarkable part for a 397B model standing next to the largest closed systems. It
posts MMLU-Pro 89.75 (Gemini-3.1-Pro leads at 91.00), HMMT-2026 91.57 (GPT-5.5 at 97.06),
MMMU-Pro 80.46, and SWE-Bench-Multilingual 81.67 — effectively tied with GLM-5.2's 82.00.

<Figure
  src="/articles/intern-s2/fig1.png"
  alt="Benchmark table of general tasks comparing Intern-S2 against Qwen3.5-397B-A17B, DeepSeek-V4-pro, Kimi-K2.7-Code, GLM-5.2, GPT-5.5, Gemini-3.1-Pro, and Claude-Opus-4.8 across MMLU-Pro, SimpleQA-Verified, AdvancedIF, HMMT-2026, MMMU-Pro, ChartQAPro, SkillsBench, TerminalBench, SWE-Bench-Pro, and SWE-Bench-Multilingual."
  caption="General-task benchmarks: Intern-S2 at frontier parity with the largest closed and open models (Intern-S2-Preview-397B model card, 2026)."
/>

On factual recall it clearly clears the open field even where it trails the closed leader —
SimpleQA-Verified is a good example:

<BenchBars
  title="SimpleQA-Verified (%)"
  unit=""
  bars={[
    { label: "Gemini-3.1-Pro", value: 75.6 },
    { label: "Intern-S2", value: 69.9, highlight: true },
    { label: "GPT-5.5", value: 64.3 },
    { label: "Qwen3.5-397B", value: 54.8 },
    { label: "DeepSeek-V4-pro", value: 46.6 },
  ]}
/>

The honest read: it trails the very top closed models on the hardest coding and knowledge
benches — TerminalBench 2.1 67.42 vs Claude-Opus-4.8's 84.60, SWE-Bench-Pro 61.56 vs
69.20. Parity, not conquest.

### Scientific tasks: dominance

Now the inversion. On specialized science the gaps stop being fractions of a point and
start being multiples. Biology-Instructions (multi-omics) is the clearest case: Intern-S2
scores 56.92 where the next-best frontier model manages 13.87, and most models land between
4 and 10.

<Figure
  src="/articles/intern-s2/fig2.png"
  alt="Benchmark table of scientific tasks comparing Intern-S2 against Qwen3.5-397B-A17B, DeepSeek-V4-pro, Kimi-K2.7-Code, GLM-5.2, GPT-5.5, Gemini-3.1-Pro, and Claude-Opus-4.8 across Biology-Instructions, Mol-Instructions, MolecularIQ, SciReasoner, TOMG-Bench, MP20, ProteinBinder-9, XLRS-Bench, MicroVQA, SFE, ObsCrisis-Bench, SciCode, and SGI-Bench, with Intern-S2 far ahead on most rows."
  caption="Scientific-task benchmarks: Intern-S2 leads every frontier model on most rows, frequently by 4× or more (Intern-S2-Preview-397B model card, 2026)."
/>

<BenchBars
  title="Biology-Instructions · multi-omics (%)"
  unit=""
  bars={[
    { label: "Intern-S2", value: 56.92, highlight: true },
    { label: "Gemini-3.1-Pro", value: 13.87 },
  ]}
/>

Material-structure generation tells the same story: MP20 67.88 against a next-best of
16.75, with most models between 1.5 and 16. Molecular reasoning (Mol-Instructions 52.37 vs
GPT-5.5's 40.49), remote sensing (XLRS-Bench 51.97), microscopy VQA (MicroVQA 68.81), and
biomolecular interaction design (ProteinBinder-9 4.36 vs a best competitor near 2.4) all
land the same way. Roughly:

$$
\frac{56.92}{13.87} \approx 4.1\times, \qquad \frac{67.88}{16.75} \approx 4.05\times.
$$

It is not a clean sweep, and that's worth saying: on MolecularIQ, GPT-5.5 still leads
(76.41 vs Intern-S2's 61.49). But across the science suite as a whole, a 397B model beats
GPT-5.5, Gemini-3.1-Pro, and Claude-Opus-4.8 — the payoff of the raw-page pretraining, the
dynamic tokenizer, and multi-domain RL compounding.

<Callout type="warn">
The caveats are real. This is a **Preview**, not a final release. On the hardest general
coding and knowledge benchmarks it still trails the top closed models. Several of the most
lopsided scientific wins — MP20, ProteinBinder-9 — are **internal benchmarks**, so treat
the exact multiples as InternLM's own measurement until third parties reproduce them. And
at 397B it is heavy to self-host: frontier-scale hardware, not a workstation.
</Callout>

## What I make of it

- **The specialization is the product.** Most "science" models are general chatbots with
  a domain fine-tune. Intern-S2 pushes science into the tokenizer and the pretraining
  objective, and the benchmark gaps show the difference that makes — 4× is not a
  prompt-engineering delta.
- **Parity at 397B is the quiet achievement.** Matching Gemini-3.1-Pro and Opus-4.8 on
  general tasks with a fraction of the (public) scale, while dominating science, is a
  stronger statement than any single scientific score.
- **Trust the shape, verify the numbers.** The parity-vs-dominance pattern is convincing
  and mechanistically motivated. The internal-benchmark wins want independent replication
  before I'd quote the exact multiples as settled — but even halved, the lead holds.

---

*Sources: the [Intern-S2-Preview-397B model card](https://huggingface.co/internlm/Intern-S2-Preview-397B)
and the [Intern-S1 project](https://github.com/InternLM/Intern-S1) (InternLM / Shanghai AI
Lab). Benchmark numbers are quoted as reported on the model card; several scientific
benchmarks are internal.*