# The Skaling law: Chinchilla assumes model size and data don't interact, and they do

> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/skaling-law
> date: 2026-08-10
> tags: scaling-laws, llm, math, training, explainer
The Chinchilla scaling law is one of the most quoted equations in the field. It says the loss of a
language model decomposes into an irreducible floor plus two independent power-law terms, one for
model size and one for training data:

$$
L(N, D) = E + \frac{A}{N^{\alpha}} + \frac{B}{D^{\beta}}
$$

[**Skaling: Chinchilla's Exponents Meet Kaplan's Coupling**](https://arxiv.org/abs/2608.07222)
(arXiv 2608.07222, Mathurin Videau, Badr Youbi-Idrissi, David Lopez-Paz and Kartik Ahuja, FAIR at
Meta, 7 August 2026) points at the plus sign in the middle and observes that it is a very strong
claim nobody ever tested.

A sum of a function of $N$ and a function of $D$ has a cross-derivative of exactly zero. Not
approximately zero, not small — zero, as an algebraic identity. The additive form *asserts* that how
much a training token is worth does not depend on how big your model is. That assertion was never a
finding. It was a modelling convenience that came along for the ride.

<Callout type="note">
Personal disclosure up front, because it changes how I read this paper: four days ago I published a
[back-of-envelope estimate](/articles/ltc-gated-delta) that leaned on exactly this additive form to
argue that a small model's reported gain might vanish under a compute-matched comparison. The last
section of this piece redoes that calculation with the paper's tools. It moves.
</Callout>

## The saddle

Fit the additive law to a dense grid of trained models and the residuals are not noise. They have
structure.

<Figure
  src="/articles/skaling-law/fig1.png"
  alt="Three panels of hexagonally-packed markers over a grid of model size against training tokens. The Chinchilla panel shows a saddle pattern: strongly positive error in the bottom-left and top-right corners, negative in the top-left and bottom-right. The Skaling panel is much flatter and paler. The third panel shows Skaling's error advantage reaching sixteen-fold at the corners."
  caption="Signed percentage error of each fitted law across the (N, D) grid. Chinchilla is accurate in the interior and develops large, oppositely-signed errors toward the corners — the saddle shape you get when an N–D interaction is omitted (Videau et al., arXiv 2608.07222, Figure 1)."
/>

The paper's description of that first panel is precise: Chinchilla "is accurate in the interior of
the grid but develops large, oppositely-signed errors toward the corners, reaching several percent
where $N$ and $D$ are most imbalanced. This is the saddle-shaped residual expected when the $N$–$D$
interaction is omitted."

A saddle is the signature of a missing product term. If your model of a surface has no $xy$ term and
the true surface has one, the errors you get are positive on one diagonal and negative on the other
— which is exactly what the left panel shows. The paper backs this up with a direct measurement of
the cross-derivative $\partial^2 L/\partial N \partial D$ from local quadratic fits (their Figure 3),
finding it non-zero and structured.

## One exponent

The fix is small enough to state in a line. Kaplan's original 2020 form did couple $N$ and $D$ —
$L(N,D) = [(N_c/N)^{\alpha_N/\alpha_D} + D_c/D]^{\alpha_D}$ — but it tied the inner exponents
together through the ratio $\alpha_N/\alpha_D$, so the per-axis decay rates were no longer
independent. Chinchilla threw out the coupling to get the independence back. Skaling keeps both:

$$
L(N, D) = \left(\frac{A}{N^{\alpha}} + \frac{B}{D^{\beta}}\right)^{k} + E
$$

Chinchilla's interpretable base terms and independent inner exponents, raised to a single free outer
exponent $k$. At $k = 1$ it *is* the additive law — Chinchilla is a special case, not a rival. For
any $k \neq 1$ the cross-derivative is non-zero. And because $k > 0$, the loss is still strictly
decreasing in both arguments, so adding capacity or data can never be predicted to hurt.

<CouplingKnob />

The fitted value is what makes this more than a formality. On the Farseer grid, $k = 0.41 \pm 0.01$
— nowhere near 1, and tightly determined. Differentiating,

$$
\frac{\partial^2 L}{\partial \ln N\, \partial \ln D} = k(k-1)\,\alpha\beta\left(\frac{A}{N^{\alpha}}\right)\left(\frac{B}{D^{\beta}}\right)R^{\,k-2}, \qquad R = \frac{A}{N^{\alpha}} + \frac{B}{D^{\beta}}
$$

With $k < 1$ the factor $k(k-1)$ is negative, so the cross-derivative is negative: since
$\partial L/\partial \ln D$ is already negative, making it *more* negative means **bigger models
extract more from the same token**. That is not a surprising claim — it is roughly what everyone
believes — but the additive law is structurally incapable of expressing it.

<Callout type="note">
One honest wrinkle the paper raises itself. The coupled reducible term decays more slowly at large
scale, so it absorbs curvature the additive law can only represent through a larger $E$. Skaling
therefore pushes $E$ down — on Farseer almost to zero (0.03 ± 0.02, against Chinchilla's 0.45 ±
0.01). Since none of the runs reach the scale where loss saturates, the data fix the total loss but
not the split between a decaying term and a constant floor. So $E$ should not be read as a measured
irreducible entropy in either fit. Every other parameter is determined to within a few percent.
</Callout>

## Interpolation quality is not evidence

This is the methodological point I most want people to take away, and it is stated bluntly in the
paper: "High interpolation fit quality is not enough to validate a scaling law."

<GridStrategy />

Chinchilla achieves $R^2 = 0.995$ on the full Farseer grid. By the standard people usually apply,
that is a solved problem. Its extrapolation error is three to four times Skaling's. The failure mode
"is therefore not a poor fit to the interior, but a systematic misprediction of how the loss surface
bends away from the observed region" — which is the only thing anyone ever uses a scaling law for.
Nobody fits a scaling law to predict a run they already did.

The compute argument in the second half of that figure is the one with budget consequences. The
authors pair the coupled form with an **L-shaped sparse grid**: instead of spreading held-out points
across the whole $(N, D)$ plane, restrict training runs to the low-compute edges — a row of small
models across many data budgets, and a column of small data budgets across many model sizes. Skaling
fitted on that L-shape, at roughly a tenth of the FLOPs, extrapolates **better than Chinchilla fitted
on the entire grid** in every held-out regime. On Farseer: 0.89% vs 1.48% on larger models, 1.35% vs
1.98% on more data, 1.51% vs 2.46% far outside both.

It is also worth reading the row that is not Skaling. The nine-parameter Farseer law is *worse* than
three-parameter Chinchilla on several columns and carries huge fold-to-fold variance (±1.93 on far
extrapolation). More parameters bought instability, not accuracy. The Skaling result is a
one-parameter change that improves things, which is a different and much stronger kind of claim.

## The part that changes decisions

Everything above is about fit quality. This is about where the money goes.

The compute-optimal token-to-parameter ratio $D^\star/N^\star$ is the quantity that answers "should
the next dollar buy a bigger model or more data?" The paper recovers it two ways without assuming
any parametric law — a global Gaussian-process surrogate and a local moving-least-squares surrogate,
finding the point where the log-slopes balance — and then compares against what each fitted law
predicts.

<Figure
  src="/articles/skaling-law/fig2.png"
  alt="Two log-log panels of optimal tokens per parameter against training compute. Left: empirical optima from GP and MLS surrogates track the Skaling prediction downward while the Chinchilla prediction stays nearly flat. Right: power-law fits extrapolated to two times ten to the twenty-five FLOPs, where the empirical fits at slope minus 0.14 and minus 0.15 and Skaling at minus 0.11 fall steeply while Chinchilla at plus 0.03 rises."
  caption="Compute-optimal tokens per parameter, recovered without a parametric fit and compared against the analytic laws. The empirical exponents (−0.14, −0.15) are close to Skaling's (−0.11) and have the opposite sign from Chinchilla's (+0.03) (Videau et al., arXiv 2608.07222, Figure 6)."
/>

<AllocationDivergence />

The two model-free estimates of the optimum give exponents of **−0.14 and −0.15**. Skaling recovers
**−0.11**. The refitted additive law gives **+0.03** — the opposite sign. Inside the observed data
range all four agree, which is exactly why a good interpolation $R^2$ told you nothing. Outside it
they diverge, and the paper reports that one order of magnitude beyond the data the allocations
differ by more than 10×, with the additive law heading toward hundreds of tokens per parameter while
the empirical fits and Skaling fall to the tens.

Two caveats before anyone reallocates a training budget on this. The empirical exponent is itself
an extrapolation of a fit to a surrogate of a finite grid, and the two surrogates agreeing with each
other is weaker evidence than two independent measurements. And "the additive law has the wrong
sign" is a claim about *these* datasets, at *these* scales, with these architectures. But the sign
disagreement is not subtle, it reproduces across two independently constructed grids, and it lands
on the one number the whole scaling-law enterprise exists to produce.

## What was actually measured

Worth being concrete about the evidence base, because scaling-law papers vary enormously here.

**Farseer** is an existing public grid; the fitting set is 302 configurations totalling
~5.0×10²² FLOPs, with held-out sets for larger models (36 points, 1.5B–6.4B), more data (66 points),
and far extrapolation (7 runs at 2.3B–25B parameters on 126B–453B tokens, beyond both axes).
**SK-Grid** is the authors' own: 134 configurations, 15 model sizes from 134M to 4.9B, 16 data
budgets from 316M to 316B tokens, with far-extrapolation runs at ~10²² FLOPs on 5.8B–10.8B
parameter models. Two more datasets appear in the appendix with the same protocol and the same
ranking.

All laws are fitted identically — Huber loss in log space, L-BFGS-B with 2000 basin-hopping restarts,
analytic gradients — so the comparison is not confounded by one law getting a better optimizer. The
paper also notes that the improvement survives holding the protocol fixed, "confirming the gain
comes from the functional form rather than the protocol."

What is *not* here: no runs at frontier scale, so the far-extrapolation column tops out around 25B
parameters; no test of whether $k$ is stable across architecture families, tokenizers or data
mixtures, which is the obvious next question given that $k$ is now carrying the entire interaction;
and no mixture-of-experts models, where "model size" is ambiguous enough that it is unclear which $N$
even belongs in the formula.

## Redoing my own arithmetic

Four days ago, writing about [liquid time constants and gated delta
rules](/articles/ltc-gated-delta), I used the additive Chinchilla form to estimate a confound. The
setup: a 29M-parameter model, LTCAttention, beat its baseline by 0.062 nats on a token-matched
comparison while running 12.4% slower. I asked what the baseline would have gained if it had spent
that 12.4% on extra tokens instead, got roughly 0.061 nats, and concluded that a compute-matched
comparison might erase the entire result.

I used Hoffmann et al.'s 2022 coefficients. This paper supplies two better options: the same additive
form refitted on Farseer, and the coupled form on the same runs.

<EstimateSpread />

The three answers span 2.4×. The estimate I published was the largest of them.

Most of the movement is not the coupling — it is that Hoffmann's coefficients were fitted on a
different corpus and tokenizer, and refitting the *same* functional form on Farseer roughly halves
the answer, from 0.061 to 0.031 nats. The coupled form then trims it further, to 0.026. There is
also a pointed detail: LTCAttention's configuration sits at $D/N = 10.0$, which falls in the band
the paper reports as Chinchilla's **worst** regime — 3.47% MAPE in the optimal-ratio third, where
its pooled number hides the failure.

So the honest revision: a compute-matched baseline would probably have recovered somewhere around
**half** of LTCAttention's measured gain, not all of it. The conclusion I actually drew still stands
— the missing experiment is a wall-clock-matched run, and until someone does it the result is better
per token and undetermined per second — but I stated the confound about twice as strongly as the
evidence supports. That correction is now in the record here rather than only in my own notes.

The broader lesson is the one I would take from this paper even if I had no stake in it. Scaling-law
arithmetic is routinely used the way I used it: pull the canonical coefficients, differentiate, get a
number, cite it as though it were a measurement. It is not. It is a prediction from a functional form
fitted to somebody else's grid, and both the form and the grid are doing real work. When the answer
matters, quote the range.

## The take

The contribution is one exponent, and the reason it is a good paper rather than a small one is that
the exponent is load-bearing. It removes a structural bias that was invisible in interpolation error
and severe at the boundaries, it makes accurate extrapolation possible from a tenth of the compute,
and it flips the sign of the trend in the single number the field uses to allocate training budgets.

What it does not do is settle anything at frontier scale, where nobody has published the grid that
would test it. And there is a mild irony worth naming: the paper's own argument implies that its
$k = 0.41$ is a property of these datasets, and the honest way to use the Skaling law is to refit it
on your own runs rather than to quote 0.41 the way people have been quoting 20 tokens per parameter
for four years.

---

*Sources: [Skaling: Chinchilla's Exponents Meet Kaplan's Coupling](https://arxiv.org/abs/2608.07222)
(arXiv 2608.07222v1, Videau, Youbi-Idrissi, Lopez-Paz, Ahuja, FAIR at Meta, 7 August 2026, CC BY
4.0), read in full via the arXiv HTML rendering. Equation 3, the fitted coefficients in Table 2, the
MAPE figures in Tables 1 and 3, and the compute-optimal exponents in Figure 6 are quoted as
published. Both figures are the paper's own, flattened onto white. The cross-derivative expression,
the recomputation of my earlier LTCAttention estimate, and the sensitivity arithmetic behind the last
interactive are mine, computed from the paper's published coefficients at LTCAttention's reported
N and D. All four interactives are mine.*