WEBVTT

00:00:01.290 --> 00:00:02.378
Hi, I'm Brownie!

00:00:02.378 --> 00:00:06.732
This is Kimi K3, two point seven eight trillion parameters, running on

00:00:06.732 --> 00:00:09.271
one CPU in eight gigabytes of RAM.

00:00:10.767 --> 00:00:12.854
The trick: keep almost nothing resident.

00:00:12.854 --> 00:00:16.680
Stream the trunk, read four-bit experts straight off the drive, and

00:00:16.680 --> 00:00:18.071
memory becomes a dial.

00:00:20.483 --> 00:00:24.447
At bfloat16, it's five thousand five hundred and sixty gigabytes.

00:00:24.447 --> 00:00:26.429
Four decisions bring it down.

00:00:26.849 --> 00:00:29.628
The experts already ship at half a byte, and stay packed.

00:00:30.048 --> 00:00:33.397
Delta attention keeps a state that never grows with context.

00:00:33.817 --> 00:00:38.018
Full attention caches one small latent per position, not ninety-six heads.

00:00:38.438 --> 00:00:39.561
And the trunk streams.

00:00:39.561 --> 00:00:41.244
That one belongs to this engine.

00:00:42.592 --> 00:00:46.188
Every trunk layer runs on every token, so it streams off the

00:00:46.188 --> 00:00:49.185
drive one layer at a time, bypassing the page cache.

00:00:49.605 --> 00:00:53.679
Only sixteen of eight hundred and ninety-six experts fire per layer, cached

00:00:53.679 --> 00:00:55.715
in the same packed four-bit bytes.

00:00:56.135 --> 00:00:58.520
The matmul multiplies those nibbles directly.

00:00:58.520 --> 00:01:04.086
Decode waits on bytes, not arithmetic, so reading fewer bytes is faster, not slower.

00:01:05.483 --> 00:01:10.045
Floating-point addition isn't associative, so this engine fixes the order by hand.

00:01:10.465 --> 00:01:14.416
Four accumulators, in double precision, each take every fourth product.

00:01:14.836 --> 00:01:18.909
They combine in one fixed order that the vector path reproduces exactly.

00:01:18.909 --> 00:01:20.945
Every path agrees to the bit.

00:01:22.508 --> 00:01:26.497
So the dial works: from eight gigabytes to two hundred and twenty-four,

00:01:26.497 --> 00:01:29.489
it runs one point seven times faster, same tokens.

00:01:31.142 --> 00:01:33.679
Decode is bound by bytes moved, not arithmetic.

00:01:33.679 --> 00:01:36.216
Move fewer bytes, and memory is a dial.

00:01:39.733 --> 00:01:43.827
To recap: packed experts, a streaming trunk, and a fixed order of addition.

00:01:44.333 --> 00:01:46.434
Every source is in the full article.

00:01:46.434 --> 00:01:47.034
I'm Brownie.

00:01:47.034 --> 00:01:47.334
Bye!
