stable-diffusion.cpp: 52 model versions in one binary, and a quantiser that never touches a convolution

mdjsonmcpcopy↗ai

2026-09-26 · 29 min · image-generation · diffusion · quantization · gguf · llama-cpp · inference-optimization · open-source · on-device · systems · explainer

A repo spotlight by @Ryrenz went round on 25 September. Its pitch for leejet/stable-diffusion.cpp: image generation squeezed into one executable, no Python, the same approach as llama.cpp, running on CPU (AVX), CUDA, Vulkan, Metal, OpenCL and SYCL. About 7,161 stars, eight-hundred-odd forks, no company behind it, "basically leejet carrying it alone". And the pace: five builds, master-904 to master-908, on 23 September alone.

I cloned it at 2f88688 (26 September) with its ggml submodule and read the code instead of the README. I also read the tensor tables of the checkpoints it loads, over HTTP range requests, because that is where the memory numbers actually come from. Then I ran a CPU build and checked those numbers against it; Running it has the results, predictions first. The Qwen-Image-2.1 piece ran it too, on four CPU cores at an earlier commit, and I use its measurements where they apply.

The post mostly holds. Measured, from the GitHub API on 26 September: 7,403 stars and 827 forks, MIT, created August 2023. "One executable, no Python" holds more literally than I expected: the tokenizers are compiled in as 145.8 MB of C arrays under src/tokenizers/vocab/ (CLIP, T5, umT5, Qwen, Mistral and Gemma merges and vocabularies), and .ckpt pickles are parsed by its own src/model_io/pickle_io.cpp. The repository's eight Python files are offline conversion helpers under scripts/. Two models are exceptions: Lens and PiD refuse to start without an external tokenizer.json.

"One person" does not hold. Over the six months to 26 September the repository has 383 commits from 54 authors, and leejet wrote 197 of them, 51%. The five builds are real CI tags, one per merged commit, all committed between 16:03 and 17:45 UTC on 23 September. Two of the five are leejet's; the other three are by two other contributors. It is a lead maintainer who writes about half the commits, with 53 other people writing the rest.

expand

How a pipeline becomes ggml graphs

The code has three layers.

Blocks. GGMLBlock in src/model/common/ggml_block.hpp is a PyTorch module in C++: a map of named child blocks and a map of named parameter tensors. init() walks the tree and creates a ggml tensor for every parameter, named exactly like the checkpoint key, in a no_alloc context, so nothing is allocated until the loader knows where each weight will live. Linear, Conv2d, LayerNorm, GroupNorm, RMSNorm and Embedding are the leaves. A block's forward() computes nothing. It appends ggml ops to a graph.

Runners. GGMLRunner in src/core/ggml_runner.h owns the parameters and a workspace. compute() builds the graph, allocates, runs it on a backend and reads the result back as an sd::Tensor<float>. The conditioner, the diffusion model and the VAE are one runner each.

The pipeline. src/pipeline/diffusion_engine.cpp is ordinary host C++ that calls the runners in order: encode the prompt, loop the denoiser, decode the latent.

Which runners get built is decided without a config file. ModelLoader::get_sd_version() in src/model_loader.cpp fingerprints the checkpoint by its tensor names and a few shapes. A model.diffusion_model.joint_blocks. prefix means SD3. double_blocks. means the FLUX family, and the width of img_in then separates the variants: 384 input channels is FLUX Fill, 128 is FLUX Controls, 196 is Flex.2. A txt_in 3,584 wide is LongCat. A single_blocks.47 means FLUX.2-dev rather than klein. Wan 2.2's image-to-video and TI2V models are told apart by the product of input and output channels in patch_embedding (184,320 against 147,456). The SDVersion enum in src/model.h has 52 entries, one of them the ESRGAN upscaler.

The architecture is read the same way. FluxConfig::detect_from_weights in src/model/diffusion/flux.hpp takes the depth from the highest double_blocks.N and single_blocks.N index it sees, the width from txt_in.weight, the head count from the length of a key-norm scale, and switches Chroma on when distilled_guidance_layer exists. That is why a fine-tune with a different depth needs no new flag.

build_core_runners in src/pipeline/model_builders.cpp then pairs a conditioner with a denoiser. The conditioners say a lot about where the field went in three years:

Anima, HiDream-O1, MiniT2I and SenseNova U1.5 bring conditioners of their own. Ten LLM architectures live in src/model/te/llm.hpp, 2,975 lines. Most text encoders added since 2025 are full language models, run once per prompt with the lm_head never declared, so never loaded.

The denoiser is a UNet for SD 1.x, 2.x and XL and a DiT for everything since SD3. The VAEs are fewer than the models: sd_version_uses_flux_vae covers FLUX.1, Z-Image, Boogu and LongCat, sd_version_uses_wan_vae covers Wan, LingBot-Video, Qwen-Image, Krea2 and Anima, and the video models with sound add an audio VAE. SD 1.x and SDXL load from one checkpoint with -m; most newer models take separate --diffusion-model, --llm or --t5xxl, and --vae files.

The sampling loop runs on the host

This is the part that surprised me. The samplers in src/runtime/denoiser.hpp are not ggml graphs. They take a denoise callback and a vector of noise levels and do their arithmetic on sd::Tensor<float>, a std::vector-backed host tensor from src/core/tensor.hpp. Euler shows the whole idea:

// src/runtime/denoiser.hpp, sample_euler (error check dropped)
for (int i = 0; i < steps; i++) {
    float sigma       = sigmas[i];
    auto denoised_opt = model(x, sigma, i + 1);     // the only ggml graph run
    sd::Tensor<float> denoised = std::move(denoised_opt.pred);
    sd::Tensor<float> d        = (x - denoised) / sigma;
    x += d * (sigmas[i + 1] - sigma);
}

The graph lives inside model, which is the lambda at diffusion_engine.cpp:2375. It runs the denoiser once for the prompt and once for the negative prompt, then mixes the two with the guidance scale. One line decides whether the second run happens at all: when the guidance scale is within 1e-5 of 1.0, skip_uncond is set and the unconditional pass is skipped. That is why every FLUX example in the docs passes --cfg-scale 1.0. FLUX.1-dev takes its guidance strength as an input, through the guidance_in embedder, so at 1.0 each step costs one denoiser call instead of two.

The README lists nine sampling methods. The sample_method_t enum in include/stable-diffusion.h has 21, including DPM++ 2M SDE, TCD, LMS, iPNDM and two CFG++ variants, and scheduler_t has 17 schedules, from Karras and AYS to per-model ones for FLUX, FLUX.2 and LTX-2. The README is the conservative document here, not the inflated one.

Because every step pushes the whole latent through the whole network, a diffusion step behaves like the prefill half of LLM inference, not the decode half: the weights are read once per graph run and there is no cache that grows. One recent exception is Qwen-Image-2.1. A prefix KV cache for it landed on 23 September in #2035, switched on by default, a day after the 2bb7294 commit that the pocket-rewriter piece read. That piece's statement that sd.cpp recomputes the prompt prefix on every step was true of that commit and is no longer true of master.

Quantisation: the matrices convert, the convolutions do not

--type q4_K quantises at load time, and -M convert does the same once and writes a GGUF. Both go through one filter, ModelLoader::tensor_should_be_converted in src/model_loader.cpp. A tensor is converted only if its row length is a multiple of the target type's block size and its name is not a bias, a .scale, an embedding, a scale_shift_table, or one of FLUX's img_in, txt_in, time_in, vector_in, guidance_in and final_layer, or the UNet's time_embed. Then the blocks apply their own rules. Linear takes whatever type the loader settled on and always keeps its bias in f32. Norms are f32 in memory, though not always in a converted file (see Running it). And Conv2d::init_params hard-codes its weight to GGML_TYPE_F16, whatever the file or the flag says. A 3 x 3 kernel has a row length of 3 anyway, which no block size divides.

Block sizes come from ggml/src/ggml-common.h:

q4_0 and q4_K cost exactly the same bytes. They differ in how the 4.5 bits are spent: q4_K spends them on a scale and a minimum per 32-weight sub-block, q4_0 on one scale per block. Those rules have three consequences worth the arithmetic.

FLUX: the docs' memory table is pure weight bytes. FLUX.1-dev is 11,901,408,320 parameters (Measured, from the tensor table in the header of leejet/FLUX.1-dev-gguf's q4_0 file). 11,834,228,736 of them sit in the 304 tensors that get quantised; the other 476 tensors, 67,179,584 parameters of biases, norms and the embedder and output matrices, are stored in f32. docs/flux.md prints a "Memory" row for five types, and I can reproduce every entry to the hundredth of a MiB (Reasoned) by assuming every 2-D matrix is quantised, every bias and norm is f32, and img_in stays f32 under the K-quants because its 64-long rows do not divide 256: 12,068.09, 6,394.53, 6,395.17, 4,888.16 and 3,735.73 MiB. The table is the diffusion model's weights and nothing else. No text encoder, no VAE, no workspace.

Today's rules, and leejet's files, keep the ten embedder and output matrices in f32 rather than quantising them, which adds 180 to 224 MiB per rung. From the q4_0 file's tensor table, sd.cpp's rules predict the tensor bytes of leejet's four other FLUX files exactly; each file is that plus the same 54,176-byte header.

expand

SD 1.5: most of the UNet never quantises. From the header of the v1-5-pruned-emaonly.safetensors file the README tells you to download, the UNet is 859,520,964 parameters, and 589,027,840 of them (68.5%) are convolution weights (Measured). They stay f16 at every --type. The VAE is nothing but convolutions and norms. So the whole pipeline goes from 2.06 GiB at f16 to 1.59 GiB at q4_0 (Reasoned), 23% off, and the docs' own SD 1.x table shows the same flatness: about 2.3 G at f16, about 2.0 G at every 4- and 5-bit type (Reported).

The K-quants make it worse. SD 1.5's first two UNet levels are 320 and 640 channels wide, so 35,840,000 of the linear-layer parameters sit in rows that divide by 32 but not by 256. Under a K-quant they are left in the file's own type, which for this file is f32. The UNet comes out at 1,461,973,776 bytes at q4_K against 1,338,773,776 at q4_0 and 1,472,774,416 at q8_0 (Reasoned). On this model, choosing the "better" 4-bit format buys you an 8-bit footprint. The same effect makes the q3_K and q2_K pipelines larger than the q4_0 one. Converting an fp16 copy measured it: the q4_K file came out 51,568,992 bytes larger than the q4_0 one, exactly as predicted.

Mixed recipes are allowed. --tensor-type-rules takes regex-to-type pairs; the Ideogram4 docs use it to quantise only the attention, feed-forward and modulation weights. leejet's own Z-Image-Turbo GGUFs do the same quietly: the Q4_K file keeps 22 tensors of the context and noise refiners at q8_0 and its embedders in f32, which makes it 5.02 bits per parameter on disk rather than 4.5 (Measured, from the file's header). The patched ggml adds FP8 weights and ComfyUI's INT8 "convrot" format, and --imat-out trains an importance matrix while you generate, the llama.cpp trick for spending the bits where the activations say they matter.

expand

On a CPU the quantised weights never get expanded back to floats. The CPU backend quantises the activations on the fly, to q8_0 for the legacy types and q8_K for the K-quants, and runs integer dot products. The AVX2 path of ggml_vec_dot_q4_0_q8_0 in ggml/src/ggml-cpu/arch/x86/quants.c unpacks 32 nibbles to bytes, subtracts 8, multiplies them against 32 int8 activations, and scales the sum by the product of the two blocks' f16 scales. That is the same kernel llama.cpp uses, and the reason reading bits out of a GGUF header tells you most of what a quant costs.

Backends

The CMake options are SD_CUDA, SD_HIPBLAS (ROCm), SD_METAL, SD_VULKAN, SD_OPENCL, SD_SYCL and SD_MUSA, plus an RPC backend for offloading to another machine. The CPU backend is built with GGML_NATIVE for the build machine by default, with explicit switches for AVX, AVX2, AVX-512 and its VNNI and BF16 extensions, and AMX. ggml/src/ggml-cpu/arch/ carries kernels for x86, ARM, LoongArch, PowerPC, RISC-V, s390 and WebAssembly. That is the list the post gave, plus ROCm and Moore Threads.

Two things go beyond "it builds on everything". Placement is per module: --backend te=cpu,vae=cuda0,diffusion=vulkan0 puts each runner on its own device, and diffusion=cuda0&cuda1 splits one model's blocks across two GPUs. And the default build uses leejet's patched fork, leejet/ggml at 4bf5f60, which adds CUDA SageAttention kernels, FP8 matrix multiplies and the INT8 convrot path; SD_USE_UPSTREAM_GGML=ON trades those for stock ggml. Where a backend lacks an operator, the code asks ggml_backend_supports_op and falls back, flash attention to the explicit softmax path, INT8 convrot on Vulkan to the CPU.

Three memory tricks, each one a piece of arithmetic

Flash attention removes one matrix. Without it, ggml_ext_attention_ext in src/core/ggml_extend.cpp materialises the score matrix kq in f32: heads x L x L x 4 bytes, where L is the number of tokens. FLUX at 768 x 768 has 2,304 image tokens plus the 256 T5 tokens FluxCLIPEmbedder pads the prompt to, so 24 x 2,560 x 2,560 x 4 = 629,145,600 bytes, exactly 600 MiB (Reasoned). docs/performance.md says flash attention saves "~600mb" on FLUX at 768 x 768 (Reported). For SD2 at 768 x 768 the same formula gives 5 heads over 9,216 tokens, 1,620 MiB, against the reported "~1400mb". At 1024 x 1024, FLUX's matrix is 1,734 MiB. The docs add that flash attention slows most backends down and speeds CUDA up, and is only wired up for some backends, the CPU, CUDA and ROCm and Metal among them.

VAE tiling bounds the biggest buffer in the pipeline. The VAE decoder is the one component that works at full pixel resolution, and ggml runs a 3 x 3 convolution as an im2col followed by a matrix multiply. The SD and FLUX decoders enter full resolution with 256 channels. At the first residual block there, the block's f32 input (kept for the skip connection), its normalised f32 copy and the 256 x 9 f16 im2col of that copy are alive at once: 1,024 + 1,024 + 4,608 = 6,656 bytes per output pixel, or 1,664 MiB at 512 x 512 and 6.5 GiB at 1024 x 1024 (Reasoned, from the graph shapes). For SD 1.5, sd.cpp's own log later reported 1,664.06 MB at 512 x 512 and 3,744.14 MB at 768 x 768 (Measured). The CPU run in the Qwen-Image-2.1 piece measured that lopsidedness directly, on a different VAE: at 512 x 512, sd.cpp's compute buffers were 7.18 MB for the text encoder, 234.03 MB for the denoiser and 2,450.50 MB for the VAE (Measured, on this site).

--vae-tiling decodes 256 x 256-pixel tiles with 50% overlap and blends them. At 512 x 512 that is 3 x 3 tiles, as the docs say; at 1024 x 1024 the tiling arithmetic in src/runtime/tiling.cpp gives 7 x 7 = 49 tiles, each around 416 MiB by the same estimate, and about 3.06 times the decode work because of the overlap (Reasoned). The main decode path also retries an allocation failure with smaller tiles even when you did not ask for tiling, so running out of memory at the decode that ends a long job costs time rather than the job.

Offloading keeps the weights in RAM and stages them in. --offload-to-cpu is shorthand for --params-backend *=cpu: parameters live in system RAM, and each runner gets GPU copies on demand, which stay resident while there is room. When a graph's weights plus workspace do not fit, the runner cuts the graph into segments and runs them in order, prefetching the next segment's weights, while keeping a 512 MiB scratch reserve free on the device. --max-vram sets the budget; --params-backend diffusion=disk rereads weights from the file instead of holding them in RAM at all. The docs say offloading saves VRAM "without reducing generation speed" (Reported). That can only hold while the copies stay resident. Once every step has to re-upload the model, it costs time, and the project's own LLaDA-Image notes say so: at 512 x 512, --max-vram 3 produces byte-identical output "at roughly 2.5x the time" (Reported).

What fits in 8 GB and 16 GB

Here are the weight bytes sd.cpp would allocate for six pipelines at one --type for every part, from each checkpoint's tensor table priced with the rules above (Reasoned, GiB). The last column is the largest single part at q4_K, which is the floor for anything that has to sit on the GPU at once.

Parameter counts behind those rows, all from the files' own headers: SD 1.5's UNet 859,520,964; SDXL's 2,567,463,684; Z-Image Turbo's DiT 6,154,908,736 and its Qwen3-4B encoder 4,022,468,096; FLUX.1-dev's T5-XXL encoder 4,762,310,656 once its duplicate embedding is dropped; Qwen-Image's DiT 20,430,401,088, the same count the Qwen-Image-2.1 piece read, and its Qwen2.5-VL encoder 7,747,169,280 without the lm_head. Wan 2.2's 5B row has the oddity: its umT5-XXL text encoder, 5,680,910,336 parameters, is bigger than the 4,999,787,712-parameter video model, and its 704,688,668-parameter VAE is almost entirely convolution, so it is 1.31 GiB at every --type.

Read against an 8 GiB card: SD 1.5, SDXL and Z-Image Turbo at 4 bits fit whole, with room for their workspace. FLUX.1-dev at q4_K is 9.22 GiB, so it fits only with the text encoders on the CPU, which is what --clip-on-cpu in every FLUX example does: DiT plus VAE is 6.61 GiB. Qwen-Image at q2_K is 9.11 GiB; with its encoder on the CPU, DiT and VAE come to 6.52 GiB and leave under a gigabyte for the reserve and the workspace. At 4 bits its DiT alone is 10.74 GiB, so on 8 GiB it is an offloading job. On 16 GiB, FLUX fits whole at q6_K, 13.22 GiB. Qwen-Image's 15.22 GiB at 4 bits leaves 0.78 GiB for the reserve and the workspaces, so it wants the encoder on the CPU or offloading.

That settles one claim in the docs. docs/flux.md opens with "You can run Flux using stable-diffusion.cpp with a GPU that has 6GB or even 4GB of VRAM, without needing to offload to RAM." Against today's files, the q2_K DiT is 3,959.5 MiB and the VAE 160.0 MiB: 4,119.5 MiB of weights, over a 4 GiB card's 4,096 before a byte of workspace. Against the table the sentence was written with, it was 3,895.7 MiB, which leaves 200 MiB for everything else. Six is tight too: at q3_K the weights are 5,265.7 MiB, which leaves 878 MiB for the workspace and the 512 MiB reserve together. Read both numbers as offloading jobs now; with segmented execution that is what they are.

Pick a model, a type for each part and the flags, and see where the bytes land:

Its weight rows are exact for the files named in its caption. The two workspace rows are the largest single tensors I can derive, so treat them as floors, and segmented execution can go below its offloading estimate at the cost of a weight upload per step. For SD 1.5 the VAE row has since matched sd.cpp's log to the tenth of a MiB, and the attention row overstated what flash attention saves by 15% at 512² and 7% at 768².

Running it

Everything above was read, not run. So I ran it: a CPU build of sd-cli at 2bb7294 (22 September), four days older than the commit read above, on this site's build box. That box is four cores with AVX-512 and AMX and 15 GB of RAM, shared during these runs with a raised-priority Qwen-Image job on four threads of its own. The load average sat between 10 and 14 for the whole session, so every timing below is measured under load. Memory is not affected by the load, and memory is what this section tests.

First I diffed 2bb7294..2f88688 for every file behind a claim tested here. src/convert.cpp, src/model/common/ggml_block.hpp (Linear, Conv2d, the norms), src/model/diffusion/unet.hpp and src/model/te/clip.hpp are unchanged. tensor_should_be_converted gained one exclusion, scale_shift_table, a PixArt and LTX name no SD 1.5 tensor carries. The attention path in src/core/ggml_extend.cpp gained only an out-parameter reporting whether flash attention was used. The VAE tiling code now counts tile sizes in image pixels rather than latent pixels, but the default decode tile for an 8x VAE is 256 pixels (32 latent) in both, and the out-of-memory retry exists in both. None of it can move the numbers below.

The model is SD 1.5, because it is small: the fp16 single-file checkpoint from Comfy-Org/stable-diffusion-v1-5-archive, 2,132,696,762 bytes, SHA-256 matching the Hub's record. The f32 file the README names would not fit the disk, and an fp16 source tests the same mechanism.

The converter, predicted then measured

Before converting anything I ran my per-tensor model over the fp16 file's header and wrote the prediction down. sd-cli -M convert --type q4_0 and --type q4_K then wrote the two GGUFs, and I read each file's tensor table with my own parser.

The files are 1,625,067,152 and 1,676,636,144 bytes; the rest is a 114,976-byte header and alignment. Every per-component byte count matched to the byte, and so did the verdict: on SD 1.5 the K-quant file is 3.2% larger than the legacy one, because 320- and 640-wide rows cannot take 256-weight blocks and stay in f16.

The prediction and the census also agree on something the sections above did not say. The filter has no check on a tensor's number of dimensions, so every norm weight whose length divides the block gets quantised too: 186 one-dimensional tensors in the q4_0 file and 113 in the q4_K file, the CLIP encoder's layer norms among them (Measured). At load, LayerNorm and GroupNorm declare f32 parameters and the loader converts from the file's type to the declared one, so those gammas come back as dequantised 4-bit values (Reasoned, from load_tensors). A --type q4_0 applied at load time converts the same norms from f16 straight to f32. So a converted GGUF and the load-time --type are not quite the same model. I did not measure how much that moves an image.

In memory, the runtime buffers matched the prediction as well. sd.cpp's log reports each runner's parameter buffer:

The log's "MB" are MiB. The text encoder came to 235.06 MB from fp16 and 118.63 MB from either GGUF, and the VAE to 94.47 MB in every run, because text-to-image loads only its decoder.

One prompt, one seed, three weight files

expand

Euler A injects fresh noise at every step, so a small change in the weights changes the path and the cat, while the framing survives. Each image is a plausible sample; three images are not a quality comparison. What the runs do measure is memory:

Each peak is the parameters plus the VAE decoder's 1,664.06 MB compute buffer plus 50 to 61 MiB of process overhead. 4-bit weights save 440 to 495 MiB of a 3.7 GiB peak, 12 to 13%, because the one buffer bigger than the whole q4_0 UNet is the VAE's, and weight types do not touch it. Within a single run, step times ranged over two to fifteen times as the neighbour's load moved. The fastest single steps, 25.56 s at q4_K, 77.95 s at q4_0 and 110.36 s at fp16, were set by what the neighbour was doing at the time, so I draw nothing from them.

Flash attention and VAE tiling, measured

These runs use the fp16 file at 2 steps, since every compute buffer is sized per graph, not per step. The buffer sizes are sd.cpp's own log lines.

The VAE predictions hold to the tenth of a MiB. I had put the decoder's peak at 6,656 bytes per output pixel: 1,664 MiB at 512², 416 MiB for a 256-pixel tile, 3,744 MiB at 768². The log says 1,664.06, 416.02 and 3,744.14, with 3 x 3 = 9 tiles at 512², as the docs say, and 5 x 5 = 25 at 768², as the tiling arithmetic says. Tiling took the 512² peak down by 1,195 MiB and the 768² one by 3,326 MiB, and in both the UNet's buffer became the largest thing in the process. The price is overlap: 25 tiles of 32 x 32 latent pixels are 2.78 times the decode work of one 96 x 96 latent (Reasoned), and the tiled 768² decode took 232.20 s against 123.94 s untiled, under load. The out-of-memory retry never fired, because nothing failed to allocate.

Flash attention came in under the prediction. I expected it to remove SD 1.5's 512 MiB score matrix, 8 heads x 4,096² tokens x 4 bytes. It removed 436.83 MiB of UNet workspace, from 559.90 to 123.07. The matrix does go, but the flash path makes f16 copies of K and V, and the allocator's peak moves to another op (Reasoned, from build_kqv). That is 85% of the arithmetic, and it lines up with the docs' own SD 1.x table, where flash attention saves between 0.4 and 0.5 G at every weight type (Reported).

At 768² the prediction was 8 x 9,216² x 4 bytes = 2,592 MiB, 5.06 times the 512² matrix because the tokens grow with the pixels and the matrix with the square of the tokens. The UNet buffer without flash attention was 2,690.33 MB, and with it 276.62 MB: a saving of 2,413.71 MiB, 93% of the prediction. The bigger the image, the more of the workspace is that one matrix.

A LoRA that loaded 320 of its 448 tensors

The site's lead measured one more thing on this same binary, for the few-step Qwen-Image-2.1 piece. Pruna's few-step LoRA, PrunaAI/Pruna-Qwen-Image-2.1, is a PEFT file at rank 64 with lora_alpha 128. Against leejet's Qwen-Image-2.1 GGUF it matched 320 of its 448 tensors (Measured). There were two separate faults:

• A fused target. leejet's GGUF fuses img_mlp.gate_layer and img_mlp.proj into one img_mlp.gate_up, gate rows first, and qwen_image_2_1.hpp builds the block that way. The 64 PEFT pairs for those two projections had nothing to attach to.
• A lost alpha. lora.hpp takes its scale from a .alpha or .scale tensor in the file and otherwise uses 1.0. PEFT keeps lora_alpha only in the adapter's metadata, so the 160 pairs that did load ran at half strength: PEFT's scale is 128 / 64 = 2.

The lead rewrote the file: every lora_B doubled, and each gate_layer and proj pair fused into one rank-128 block-diagonal gate_up pair. The rewritten deltas matched PEFT's exactly, a largest absolute difference of 0.0, and 384 of 384 tensors loaded. Over quantised weights sd.cpp cannot merge the update, so it applies it at runtime as an unmerged side branch, which adds 2.4% to the multiply-adds of the projections it wraps. On this CPU that did not show: the adapter's steps took 89.89 to 91.64 s, against 90.39 and 90.50 s for the base's first two steps on the same sigmas. On a quiet machine, Q4_K DiT and Q4_K_M text encoder at 512², 8 steps with the LoRA took 771.7 s: 725.6 s of sampling, 5.6 s of text encoding and 39.6 s of VAE decode, at a peak RSS of 11.59 GiB (Measured, by the lead). The 40-step run without it, in the Qwen-Image-2.1 piece, took 3,863 s.

Half of that fix is already upstream. At 2f88688, #2057 (25 September) maps Qwen-Image-2.1's gate_layer and proj LoRA names onto the fused weight. The alpha still comes only from tensors, so a PEFT file still needs its lora_B scaled by hand.

The take

stable-diffusion.cpp is llama.cpp's method applied to a harder shape of problem. A language model is one network; a 2026 diffusion pipeline is three or four, and the text encoder can be the biggest of them. What makes 52 versions tractable is that the model is identified from its tensor names and sized from its tensor shapes, and that everything a user can tune, from placement and offload to quantisation and tiling, is applied to runners rather than to models.

The quantisation rules are the part to internalise before you pick a file. Weight types apply to matrices, not to models. A UNet that is mostly convolutions barely shrinks, a K-quant on misaligned rows can grow, and a VAE is the same size at q2_K as at f16. For a transformer denoiser the arithmetic is simple: parameters times bits over eight, plus a few hundred MiB of f32 around the edges. Run against a real build, that arithmetic predicted two converted files to the byte and the VAE's buffer to the tenth of a MiB, and it overstated only the flash-attention saving. That, a GGUF header and the planner above are enough to know what a card will hold before you download 12 GB to find out.