WEBVTT

00:00:01.290 --> 00:00:02.272
Hi, I'm Moss!

00:00:02.272 --> 00:00:04.562
Every part of a transformer loses information.

00:00:04.562 --> 00:00:08.488
So the hidden state must be a lossy summary of your prompt.

00:00:08.488 --> 00:00:09.143
It isn't.

00:00:10.892 --> 00:00:14.206
Two different prompts almost surely never reach the same hidden state.

00:00:14.206 --> 00:00:17.520
So a hidden state is the prompt in another basis, and

00:00:17.520 --> 00:00:18.725
can be read back.

00:00:21.400 --> 00:00:25.844
Softmax, LayerNorm and smooth activations are real-analytic, and so is the whole

00:00:25.844 --> 00:00:28.437
network, as a function of its weights.

00:00:28.857 --> 00:00:32.145
So for two prompts, the weights where they collide form a thin

00:00:32.145 --> 00:00:33.242
slice with no volume.

00:00:33.662 --> 00:00:36.986
A random initializer lands there with probability zero.

00:00:37.406 --> 00:00:39.739
And gradient steps can't squash volume onto it.

00:00:41.550 --> 00:00:45.388
The authors went hunting for collisions: over three hundred forty-three billion

00:00:45.388 --> 00:00:46.784
prompt pairs per model.

00:00:46.784 --> 00:00:47.831
They found none.

00:00:50.175 --> 00:00:54.316
SipIt starts from the hidden states at every position, say a leaked KV-cache.

00:00:54.736 --> 00:00:58.085
The model is causal, so each position depends on the prefix

00:00:58.085 --> 00:00:59.303
and one new token.

00:00:59.303 --> 00:01:00.217
Try a candidate.

00:01:00.637 --> 00:01:04.953
If its state matches the observed one, within a tolerance, that's the token.

00:01:04.953 --> 00:01:06.613
Almost surely, only one does.

00:01:07.033 --> 00:01:08.271
Append it and move on.

00:01:08.271 --> 00:01:10.251
The work is linear in the prompt's length.

00:01:10.671 --> 00:01:14.329
Gradients pick the order, so it explores under a quarter of one

00:01:14.329 --> 00:01:15.548
percent of the vocabulary.

00:01:17.133 --> 00:01:19.749
Brute force recovers every prompt too, slowly.

00:01:19.749 --> 00:01:23.486
SipIt takes about twenty-eight seconds, and is just as exact.

00:01:24.850 --> 00:01:27.495
A hidden state is not a summary of the prompt.

00:01:27.495 --> 00:01:29.346
It is the prompt, in another basis.

00:01:32.567 --> 00:01:36.668
To recap: an injective whole, collisions with probability zero, and SipIt

00:01:36.668 --> 00:01:38.159
reading the prompt back.

00:01:38.579 --> 00:01:40.557
Every source is in the full article.

00:01:40.557 --> 00:01:41.122
I'm Moss.

00:01:41.122 --> 00:01:41.404
Bye!
