{
  "claim": "Across all four of SimPO's settings, ORPO has the best mean GSM8K change of any preference objective: +2.50 points against its own SFT starting checkpoint, where DPO loses 8.27 and SimPO loses 10.98. Every objective with a positive mean — ORPO, SLiC-HF, CPO — carries a supervised fine-tuning term, and every objective without one is negative. The converse does not hold: RRHF has an SFT term too and sits fifth, sunk by a single catastrophic −23.35 on Mistral-Base.",
  "method": "My arithmetic: each objective's GSM8K score in Table 9 of SimPO (arXiv 2405.14734v3) minus the SFT row of the same setting, which is the checkpoint every method in that column was trained from. SFT baselines are 28.13 (Mistral-Base), 40.49 (Mistral-Instruct), 46.32 (Llama-3-Base), 68.69 (Llama-3-Instruct). 5-shot, exact match, lm-evaluation-harness.",
  "source": "https://arxiv.org/abs/2405.14734",
  "captured": "2026-09-19",
  "note": "This corrects SimPO's own summary sentence, which reads \"except for ORPO, almost all approaches lead to consistent drops in one or more settings.\" ORPO does drop in both Instruct settings, by 3.64 and 4.10. What is true is that its drops are small and its gains are the largest — and that the objectives clustered at the top are the ones with an SFT term, which is ORPO's design argument rather than ORPO specifically.",
  "columns": [
    { "key": "method", "label": "objective", "align": "left" },
    { "key": "sft", "label": "SFT term", "align": "left" },
    { "key": "mb", "label": "Mistral-Base", "align": "right" },
    { "key": "mi", "label": "Mistral-Inst", "align": "right" },
    { "key": "lb", "label": "Llama-3-Base", "align": "right" },
    { "key": "li", "label": "Llama-3-Inst", "align": "right" },
    { "key": "mean", "label": "mean Δ", "align": "right" }
  ],
  "rows": [
    { "method": "ORPO", "sft": "yes", "mb": "+14.02", "mi": "−3.64", "lb": "+3.72", "li": "−4.10", "mean": "+2.50" },
    { "method": "SLiC-HF", "sft": "yes", "mb": "+5.61", "mi": "−0.84", "lb": "+2.50", "li": "−2.12", "mean": "+1.29" },
    { "method": "CPO", "sft": "yes", "mb": "+4.93", "mi": "−1.75", "lb": "+0.61", "li": "−1.29", "mean": "+0.62" },
    { "method": "KTO", "sft": "no", "mb": "+10.38", "mi": "−1.67", "lb": "−7.35", "li": "−11.68", "mean": "−2.58" },
    { "method": "RRHF", "sft": "yes", "mb": "−23.35", "mi": "−2.73", "lb": "−2.04", "li": "−2.58", "mean": "−7.68" },
    { "method": "DPO", "sft": "no", "mb": "−6.37", "mi": "−0.16", "lb": "−7.65", "li": "−18.88", "mean": "−8.27" },
    { "method": "IPO", "sft": "no", "mb": "−0.99", "mi": "−1.07", "lb": "−23.65", "li": "−10.46", "mean": "−9.04" },
    { "method": "SimPO", "sft": "no", "mb": "−5.92", "mi": "−5.24", "lb": "−14.78", "li": "−17.97", "mean": "−10.98" },
    { "method": "R-DPO", "sft": "no", "mb": "−10.01", "mi": "−3.49", "lb": "−7.05", "li": "−24.79", "mean": "−11.34" }
  ]
}