2026-08-03 · 4 min · open-weights · llm · fine-tuning · explainer
JOSIE-2 is Gökdeniz Gülmez's third-generation personality-tuned model family: three sizes — 2B, 4B, 9B — each fine-tuned from the matching Qwen3.5 base, MIT-licensed, trained entirely on Apple Silicon. The release note makes a specific, checkable claim: "JOSIE-2-2B-OSS consistently outperforms its 4B base model. JOSIE-2-4B-OSS consistently outperforms its 9B base model." A 2B model beating a 4B, and a 4B beating a 9B, would be a genuinely notable result. I went and checked it against the model cards and config.json directly, because that's the kind of claim worth verifying before repeating.
What the cards actually say
Each JOSIE-2-<size>-OSS repo carries a base_model field in its frontmatter and a model_name field in config.json. Both agree, on all three repos: 2B is fine-tuned from Qwen/Qwen3.5-2B, 4B from Qwen/Qwen3.5-4B, 9B from Qwen/Qwen3.5-9B. Same size, every time. There is no size-up training anywhere in the released weights — the release note's "outperforms its 4B/9B base" framing describes a comparison that, per the models' own configs, never happened.
That mismatched "4B" badge on a row that says "Qwen3.5-2B" is the same bug in miniature. The bigger version of it lives in the part of each card that Hugging Face's benchmark widget actually reads — the model-index YAML. There, every one of the three cards labels its baseline row Qwen/Qwen3.5-4B (base), including the 2B and 9B cards, even though the numbers next to that label differ card to card (82.5/49.1 on the 2B card, 83.4/48.9 on the 4B card, 92.6/69.5 on the 9B card) in a way that only makes sense if each card's numbers really are its own base model's, mislabeled:
every JOSIE-2 card's model-index publishes this exact same baseline label, Qwen/Qwen3.5-4B, regardless of size.
Toggle the label and the numbers never move — only the caption does. That is the artifact: the benchmark values are real, but the 2B and 9B cards attribute them to the wrong base model. The 9B card's reasoning-mode row for its own model is blank, published as "comming soon"on the card's own chart.
Toggle between what's published and what config.json verifies, and the numbers don't move — only the caption does. That's what makes this read as a copy-paste templating bug rather than a fabricated result: the underlying scores look real and internally consistent per-card, but the machine-readable label attached to them is wrong on two of three cards. The 9B card has a second, unrelated gap: its own reasoning-mode benchmark row is simply blank, published as "comming soon" in the card's own chart — the one number that would most directly support "the 9B model in its best mode," and it doesn't exist yet.
What's actually worth taking seriously
None of this makes the underlying work uninteresting. The whole JOSIE-2 family — three sizes — was fine-tuned on the same dataset: roughly 3,500 samples, about 4 million tokens total, generated by a pipeline that itself leaned on larger models (Gemma 4 31B, Qwen3.5-9B-Base, GPT-5.4, and an unreleased JOSIE-2-35B-A3B-RTG model) to synthesize training data far more capable than the 4M-token set alone would suggest. That's a striking ratio: a few thousand curated examples, reused across three model sizes, apparently doing real work — the ARC-C and TruthfulQA gains over each model's actual, same-size base are consistent and positive across all three sizes, even once you correct the label.
The more genuinely interesting finding in the cards is emergent, not benchmarked: JOSIE-2's reasoning traces sometimes "roast" or insult the user mid-thought, and the cards are specific that "no reward was introduced to enforce a uniformly polite, corporate, or sanitized internal monologue" — the behavior wasn't trained in, it showed up during RL because nothing trained it out. It's confined to the hidden reasoning trace, not the user-facing reply, and the cards are candid that they can't yet separate "reasoning-first supervision improves the policy" from "the training data's honesty framing drives the gain" — an open question, stated as one.
The take
Check the claim, not just the headline: a 2B model plausibly outperforming its own 2B base and a 9B model plausibly outperforming its own 9B base is still a real, useful result from a tiny dataset — it just isn't the size-up story the release note tells, and the cards' own tables currently say something they don't mean to say. That's worth fixing on Gülmez's end, and worth checking on everyone else's before the "2B beats a 4B" framing gets repeated as fact.
Sources: the JOSIE-2 collection and individual model cards (Goekdeniz-Guelmez/JOSIE-2-2B-OSS, -4B-OSS, -9B-OSS), their config.json files, and each repo's benchmarks.png, cross-checked directly for this piece. The figure is the 2B card's own chart, unedited aside from flattening onto a white background; the interactive reproduces the model-index labels and config-verified bases as published.