WEBVTT

00:00:01.710 --> 00:00:02.737
Hi, I'm Oboe!

00:00:02.737 --> 00:00:05.475
Every FLUX before this was an image model.

00:00:05.475 --> 00:00:09.925
FLUX 3 makes image, video with sound, and robot action from one backbone.

00:00:11.558 --> 00:00:15.782
The idea: learn every modality in one model, so each constrains the others.

00:00:15.782 --> 00:00:20.332
The sound has to match the impact; the motion has to obey the mass.

00:00:22.317 --> 00:00:25.867
In training, the path from noise to data is a straight line.

00:00:25.867 --> 00:00:27.938
The network learns the velocity along it.

00:00:28.358 --> 00:00:31.492
To generate, start from noise and step along the velocity.

00:00:31.492 --> 00:00:34.626
Each step is a full pass of a huge transformer.

00:00:35.046 --> 00:00:38.746
If the field is straight, a few steps still land on the data.

00:00:38.746 --> 00:00:41.024
That's what makes twenty seconds of video tractable.

00:00:42.417 --> 00:00:46.268
Per-modality attention lets each token see only its own kind: four

00:00:46.268 --> 00:00:47.668
models in a trenchcoat.

00:00:48.088 --> 00:00:50.445
Joint attention lets any token see any other.

00:00:50.445 --> 00:00:54.275
An audio token can attend to the video frame that made the sound.

00:00:56.050 --> 00:00:59.745
Each modality goes through its own encoder, and all of them become

00:00:59.745 --> 00:01:01.284
tokens in a single sequence.

00:01:01.704 --> 00:01:06.160
One multimodal transformer runs over that whole sequence, with a text encoder

00:01:06.160 --> 00:01:08.016
conditioning it from the side.

00:01:08.436 --> 00:01:11.132
Shared decoders read each modality back out.

00:01:11.132 --> 00:01:14.212
Action is just one more lane, marked extensible.

00:01:15.800 --> 00:01:20.115
In its maker's preliminary tests, raters picked it ninety-three percent of the

00:01:20.115 --> 00:01:24.071
time over one rival, and fifty-two over another: nearly a tie.

00:01:25.850 --> 00:01:30.215
Joint attention lets modalities constrain each other; a straight flow makes twenty

00:01:30.215 --> 00:01:31.670
seconds of it tractable.

00:01:33.733 --> 00:01:38.658
To recap: a straight flow, one joint sequence, and one backbone for every modality.

00:01:39.078 --> 00:01:41.021
Every source is in the full article.

00:01:41.021 --> 00:01:41.576
I'm Oboe.

00:01:41.576 --> 00:01:41.854
Bye!
