Qwen-Image-2.1 Outpaint LoRA: 1.12% of the weights, trained to leave the picture where it was

mdjsonmcpcopy↗ai

2026-09-27 · 21 min · image-generation · diffusion · qwen · lora · fine-tuning · open-weights · explainer

Qwen-Image-2.1 has no outpainting mode. It is an instruction editor: you give it a picture and a sentence, and it draws a new picture. On 26 September ausboss published a LoRA that turns it into an outpainter, and a repost summed it up as "extends images any direction - keeps 45-93% canvas - targets 1-2 MP".

The recipe on the card is short. Pad the picture with flat gray #808080 to a canvas whose sides are multiples of 32, give the canvas to Qwen-Image-2.1 as the reference image, and start the prompt with the trigger:

Outpaint the image: replace the solid gray areas with a seamless continuation of the scene, keeping the existing picture unchanged.

The card's claim is narrower than "outpainting works". Without the LoRA, it says, Qwen 2.1 "often reframes or rescales the picture … or returns a gray frame untouched. With it, the known picture stays pixel-registered, which is what makes a clean stitch possible." This piece is about that word, registered: why an editor moves the picture at all, what 1.12% of its weights change so that it stops, and what the repost's numbers mean. I read the card, all four checkpoints' headers over HTTP range requests, and the diffusers and stable-diffusion.cpp code, then ran the LoRA on the four-core box that builds this site.

How Qwen-Image-2.1 edits

The release piece covers the architecture. Four facts from it decide how outpainting behaves.

The reference is a prefix, not an input channel. The denoiser is one stream of 32 blocks. Qwen3-VL encodes the prompt, sees the reference and reserves slots for it in the text; the reference's VAE latents are substituted into those slots, and the target's noisy latents are appended at the end. patch_size is 1 on a 16x VAE, so an H x W image is exactly (H/16) x (W/16) tokens, and each vision slot stands for 2 x 2 of them: diffusers names the constant _IMG_TOKENS_PER_SLOT = 4 (measured from the configs and code).

The mask is block-causal. Text is causal, each image block is bidirectional inside itself, and a later block sees every earlier one. The target attends to the reference; the reference never attends to the target.

expand

The reference is clean. With causal_condition: true, text and condition-image tokens are modulated from t = 0, not from the sampled noise level. The padded canvas is a finished picture at every step.

The grids line up only if the sizes match. Rotary positions have three axes, and in diffusers' words every image block "freezes the frame axis at the position reached by the preceding text and lays its tokens out on a height/width grid centred on zero". So if the reference and the target are the same size, reference token (h, w) and target token (h, w) carry the same height and width rotations and differ only on the frame axis, 16 of each head's 128 rotary dimensions. stable-diffusion.cpp builds the same centred grid (measured from both sources).

So the editor has a positional path to "the same pixel in the reference" in every layer (reasoned). What it does not have is anything that forces it to use that path. There is no mask channel saying which pixels are known, and no sampler step that overwrites them. The target starts as pure noise over the whole canvas (randn_tensor in prepare_latents), and the model paints all of it. Keeping the picture in place is a learned behaviour, learned from edit data Qwen has not published, and an edit may change the framing. A picture on a flat gray field is ambiguous as well: it could be a print on a mat, and recomposing it is a defensible reading (reasoned). The card's failures follow: a fisheye kitchen "shrank inside its frame", a street scene "was recomposed", and some runs returned the gray frame untouched.

#808080 is not an arbitrary gray. Pixels are normalised to -1 to 1, and 128 maps to 128/127.5 − 1 ≈ 0.004, as close to "no signal" as an opaque 8-bit pixel gets (reasoned). It must be opaque: stable-diffusion.cpp composites any alpha onto white before the vision encoder sees the reference (measured from conditioner.hpp), so a transparent pad arrives as white.

What "pixel-registered" buys

Registered means a pixel of the kept picture comes out at the coordinates it went in at: not shifted, not rescaled, not redrawn nearby.

It matters because nobody keeps the model's version of the known region. A VAE round trip is lossy, so every workflow on the card ends by pasting the exact original back over the kept rectangle (AusBoss's Stitch Inpaint node, over a 32 px feather). The paste is invisible only if the model's frame agrees with the original along the border. If the model moved the scene by d and scaled it by s about the centre c, a kept point p comes back at c + s(p − c) + d, and the disagreement at the border is

A shift costs the same everywhere. A rescale costs nothing at the centre and most at the corners, which is where the seam is. A feather blends intensities; it cannot move content, so a misaligned line becomes two ghosted copies across the ramp.

The card's comparison figure shows each failure in one row; two of its three rows are below.

expand

expand

The tilt test is the cleanest evidence, because a rotated picture leaves diagonal gray wedges that neither adapter trained on. In 12 renders per column, the base moved the picture by more than 2 px near the tilted edges in 12, and each adapter in 0; the largest shift was 95 px for the base and 0 px with either LoRA (reported).

expand

In PSNR on the kept area against the original, the card reports 16.4 dB without the LoRA and 34.7 dB with v2 on ten "subtle crops", and 25.5 dB against 34.0 dB on eight "big crops". In 8-bit terms 34.7 dB is an RMS error of about 4.7 levels out of 255, and 16.4 dB about 38.6 (reasoned): a small error spread over the picture, against a different picture. Why the base does worse on the subtle crops, the card does not say, and I cannot either.

What "45-93% canvas" measures

The repost's middle number is v2's training distribution: "the kept picture covers 45-93 % of the canvas (median ~74 %)", by area. It is what v2 saw, not a guarantee about outputs.

Area share converts to padding two ways. Framed evenly on every side, the canvas is z times the picture in each direction and the share is 1/z². Extended on one side only, by a fraction e of the width, the share is 1/(1 + e).

Reasoned from the card's shares. v2 is a modest-extension model whose median case is a 1.16x zoom-out; v1 "saw more extreme zoom-outs". Under ~15% kept, the card says, fills "invent a lot".

The multiple of 32 is architecture, not taste. The VAE compresses 16x and a vision slot is 2 x 2 latent tokens, so 32 px is the smallest step that keeps both grids whole. diffusers rounds every side to the nearest 32 (multiple_of = vae_scale_factor * 2), and stable-diffusion.cpp refuses a reference that is not a multiple of 32 (measured from both). Padding to the grid yourself means nothing resamples your picture on the way in. For the same reason the reference must go in at the size the output comes out; the card's ComfyUI recipe sets resolution 0, "reference and output share the canvas size". diffusers resizes each condition image to output_resolution² of area, so an explicit output size that differs puts the two grids on different scales, and the centred positions stop lining up pixel for pixel (reasoned from the pipeline code).

"Targets 1-2 MP" is the cost. The canvas is both the condition and the target, so the denoiser carries twice the image tokens of text-to-image at the same size: 4,096 each at 1024 x 1024, and 7,744 each at ai-toolkit's resolution: 1408, the 2 MP square on the 32 grid (reasoned). diffusers computes the condition's K and V once and reuses them, which the release piece measured at 2.55x on an A100. stable-diffusion.cpp's Qwen-Image-2.1 runner keeps no state between steps (measured from the source), so on a CPU every step pays for both.

Classical outpainting, for contrast

Outpainting has been done two ways before, and the LoRA is neither.

• A mask channel. The Stable Diffusion inpainting UNets take 9 input channels instead of 4: the noisy latent, the masked image's latent and the mask (measured from the 1.5 and XL inpainting configs). A public diffusers conversion of FLUX.1 Fill lists 384 input channels against 64 output channels for the same reason. The model is told which pixels are known.
• Latent blending. Keep an ordinary model and, at every step, overwrite the known region with the original noised to the current level: RePaint in pixel space, Blended Latent Diffusion in latent space. The known region is pinned by construction, and the seam is where it fails, because the model never chose that content.

Here the mask is implicit in the pixel values, a region of exactly one colour, and nothing is pinned. The card warns off the blending route: "Do not pin the known area with Set Latent Noise Mask: on Qwen 2.1 that draws a visible rectangle at the seam." I did not test why. One latent token covers 16 x 16 px, so a hard latent mask is blocky, but that is a guess (reasoned, not measured).

The training teaches two rules at once: copy where the reference is not gray, invent where it is. For v1, 924 pairs from 231 pictures, four layouts each; the target is the picture, and the source is "same canvas with everything outside the kept rectangle painted #808080" (reported). The card mentions no loss mask, and with a plain loss over the whole target a moved kept pixel costs as much as a bad fill (reasoned).

What is in the file

I read all four checkpoints' headers without downloading the weights.

All measured. Every file is BF16, an A and a B for each of 192 layers: to_q, to_k, to_v, to_out.0, img_mlp.gate_up and img_mlp.out in each of 32 blocks. Those are exactly the 192 tensors leejet's Q4_K GGUF quantises, and nothing else: no timestep embedder, no shared modulation, no txt_in. A block costs 2,490,368 parameters, and the file is 1.12% of the 7,115,124,736-parameter denoiser (reasoned).

The metadata carries the trigger verbatim, ai-toolkit 0.13.23, base qwen_image_2, and a step and epoch that check the card's dataset sizes: v1's steps 500, 1,250 and 1,500 read epochs 0, 1 and 1, which fits 924 pairs at batch 1, and v2's step 2,000 reads epoch 29, which fits 68 pairs (2,000 / 68 = 29.4; measured, and reasoned).

It is also portable, the opposite of the few-step adapter's trouble. The MLP target is already the fused img_mlp.gate_up that leejet's GGUF and Comfy-Org's weights use, with a 24,576-row B covering both halves. There is no alpha tensor, but the card says "rank 32 / alpha 32", so the correct scale is 1, which is what stable-diffusion.cpp's lora.hpp and ComfyUI's weight_adapter/lora.py apply when a file has no alpha (measured from both sources).

Where does the update live? Here is each layer type's share of the summed squared Frobenius norm of BA (measured, from the full v2 file and a streamed read of v1):

Queries and keys, the part of attention that decides where to look, get the least. That fits a model whose path to the matching reference token already exists, but it is a reading of weights, not an ablation (reasoned). A median of 10 of each layer's 32 singular values carries 90% of its update. And v1 and v2 get the same behaviour from nearly unrelated weights: the median cosine between their per-layer updates is 0.015, and their A subspaces overlap at 0.009, against 0.0078 for two random 32-dimensional subspaces of 4,096 dimensions (measured).

Running it on four CPU cores

The card's numbers all come from ComfyUI. The site has run Qwen-Image-2.1 on four CPU cores with stable-diffusion.cpp at commit 2bb7294, so I read that build's source first.

• Editing is supported. -r passes a reference, and Qwen3-VL's vision tower, which the text-encoder GGUF lacks, comes through --llm_vision. I used Qwen's mmproj-Qwen3VL-8B-Instruct-Q8_0.gguf, 752,289,728 bytes, SHA-256 matching the Hub (measured).
• A same-size reference is not resampled. The log says "resize vae ref image 0 from 416x512 to 416x512" (measured).
• The LoRA maps as shipped. diffusion_model. becomes model.diffusion_model., and img_mlp.gate_up matches the fused weight by name. The log confirms 384 of 384 tensors applied.

The picture is a public-domain NASA photograph, "Artemis I Prelaunch" by Bill Ingalls (NHQ202208310005): a rocket and two towers in silhouette against the rising sun. I scaled it to 512 px wide, cut a 512 x 416 canvas, kept the middle 384 x 320 and painted the rest #808080. That keeps 57.7%, inside v2's range, and cuts through the sun, both tower tops and the treeline, and the real photograph remains to score the fill against.

Settings follow the few-step piece, since the card's 25 steps would take three times as long: Pruna's 8-step adapter stacked with outpaint v2, both at strength 1; Pruna's 8 sigmas; Euler; CFG 1; seed 42; leejet's Q4_K denoiser and Qwen's Q4_K_M encoder; -t 4 at nice -n 19. The prompt is the trigger plus a one-line Scene:. The canvas is 0.21 MP, about a fifth of v2's smallest training size, and the stack is mine, not the card's: this tests whether the mechanism survives, not the quality. A control run dropped the outpaint LoRA and changed nothing else.

expand

expand

expand

expand

expand

expand

Scored against the photograph, before any paste-back (all measured):

With the LoRA the kept picture is where it went in. Searching shifts up to 48 px and scales from 0.84 to 1.16, then refining to 1 px and 0.01, the best fit is no shift and no scale, and one pixel off in any direction doubles the mean error, from 1.75 to at least 3.71. Its 35.64 dB sits beside the card's 34.0 to 34.7 dB for v2. The control did what the card says the base does: it recomposed, and even its best shift and scale leave 8.10 of error, because the towers are redrawn, not only moved. Its 15.53 dB sits beside the card's 16.4 dB without the LoRA. The fill-error row is my metric, not the card's, and the control's is inflated by its shift. One picture and one seed confirm the mechanism on a new runtime and a new stack, and no more.

Measured. Other authors' jobs pushed the load average as high as 10 during the first run; its last two steps, at a load near 4, are the fair ones, and its conditioning also paid for the first read of the weights into the page cache. A step carries 1,752 tokens, 88 of text, 832 of condition and 832 of target, against about 1,061 for the 512 x 512 text-to-image run in the few-step piece, whose steps took about 90 s: 1.65 times the tokens for 1.6 times the time, because nothing caches the condition here (reasoned from both logs). With both adapters the quiet steps took 172 to 174 s against the control's 144 to 151 s. I did not isolate why a rank-32 side branch costs that much, and one pair of runs cannot settle it. Peak RSS counts the memory-mapped weights, so it moves with the page cache.

What the card's numbers do and do not show

The card shows failures beside fixes on pictures neither adapter trained on, and flags its own crude metric: the gray count "counts any flat mid-gray". Its limits are a one-person release's: ten and eight pictures for v2, twelve for v1, all rendered by the author, no spread across seeds, and a "fill error" (10.2 for v2 against 16.3 without the LoRA on subtle crops) with no stated formula or unit (reported). The registration claim survives that, and is easy to check yourself. The quality numbers are an author's measurement, not a benchmark.

The take

Outpainting on an editor fails in a specific way: the model is free to recompose, and sometimes does. ausboss's fix is not a new input or a sampler trick. It is 79,691,776 parameters that teach one convention: flat #808080 means unknown, and everything else stays exactly where it is. The architecture makes that learnable, with a clean reference on the same centred grid as the target. The practical rules fall out of the same arithmetic. Pad to a multiple of 32, give the reference and the output the same size, stay near 45-93% kept if you use v2, and paste the original back at the end, because registration is what makes that paste invisible.

For the model underneath, see the release piece; for running it without a GPU, the stable-diffusion.cpp piece and the few-step piece; for how prompts reach it, the pocket rewriter; and for the block itself, the diffusion transformer explainer.