{
  "claim": "The thing ORPO deletes is one frozen bf16 copy of the weights. For Mistral-7B — 7,241,732,096 parameters, the model the paper actually trained — that copy is 13.49 GiB against 107.91 GiB of trainable state, so removing it saves 11.1% of model-state memory, not 50%. My arithmetic, standard mixed-precision AdamW accounting.",
  "method": "Parameter count computed from mistralai/Mistral-7B-v0.1 config.json (32 layers, hidden 4096, intermediate 14336, 8 KV heads, vocab 32000, untied lm_head) and checked against the safetensors index metadata total_size of 14,483,464,192 bytes, which is exactly 2 bytes per parameter. Per-parameter byte costs are the standard bf16-weights / bf16-grads / fp32-master / fp32-Adam-m / fp32-Adam-v breakdown. Activation memory is excluded: it is unchanged, because the reference model's forward runs under no_grad and stores nothing.",
  "source": "https://huggingface.co/mistralai/Mistral-7B-v0.1/blob/main/config.json",
  "captured": "2026-09-19",
  "note": "This is model state only, and it is the optimistic case for ORPO. Under LoRA the picture inverts — the trainable state is ~0.6 GiB, so a second frozen copy would nearly double memory — except that TRL's DPOTrainer keeps no second copy for a PEFT model: it disables the adapter and reuses the base weights as the reference. The saving is largest exactly where DPO does not pay it.",
  "columns": [
    { "key": "item", "label": "resident tensor", "align": "left" },
    { "key": "bpp", "label": "B / param", "align": "right" },
    { "key": "gib", "label": "GiB @ 7.24B", "align": "right" },
    { "key": "who", "label": "who pays", "align": "left" }
  ],
  "rows": [
    { "item": "policy weights (bf16)", "bpp": "2", "gib": "13.49", "who": "both" },
    { "item": "policy gradients (bf16)", "bpp": "2", "gib": "13.49", "who": "both" },
    { "item": "fp32 master weights", "bpp": "4", "gib": "26.98", "who": "both" },
    { "item": "AdamW first moment", "bpp": "4", "gib": "26.98", "who": "both" },
    { "item": "AdamW second moment", "bpp": "4", "gib": "26.98", "who": "both" },
    { "item": "— trainable subtotal", "bpp": "16", "gib": "107.91", "who": "both" },
    { "item": "frozen reference π_ref (bf16)", "bpp": "2", "gib": "13.49", "who": "DPO only" },
    { "item": "— DPO total", "bpp": "18", "gib": "121.40", "who": "DPO" },
    { "item": "— ORPO total", "bpp": "16", "gib": "107.91", "who": "ORPO" }
  ]
}