2026-07-24 · 6 min · agents · tool-use · code-generation · fine-tuning · open-weights · explainer
Most "new models" are not new weights. BTL-3, from Bad Theory Labs, is a clean example: it is not a from-scratch 27B model but a frozen rank-32 PEFT LoRA adapter — about 934 MB of weights — post-trained on top of a pinned revision of Qwen3.6-27B. Load the base, apply the adapter, and you get an agent tuned for coding, repository work, and structured tool use. The raw capability is Qwen's; what BTL-3 contributes is behaviour — how the model runs an agent loop and, notably, when it decides not to act.
That framing matters for reading the numbers honestly, so keep it in mind: this is a post-training result on a strong open base, released under Apache-2.0, with the base checkpoint pinned to an exact revision for reproducible loading. The card labels this frozen release "RL-0013," and the maximum RL sequence length (65,536 tokens) tells you the adapter was shaped by reinforcement learning over long, multi-step trajectories — not just supervised fine-tuning on completions.
The loop it was tuned to run
An agent model earns its keep inside a loop, not on a single completion. BTL-3's stated job is to "reason, act, inspect tool results, recover from failures, and stop when no action is required." That last clause is the interesting one. A tool-happy model that always reaches for a function call is easy to train and annoying to deploy; the harder behaviour is the route decision — recognising when a question needs a tool and when it just needs an answer. Pick a scenario and watch which path the model takes:
Every turn runs the same loop, but the route step decides what happens next. When a task needs tools, BTL-3 emits one or many calls, observes the results, and recovers if one fails. When none is needed it answers directly and stops — the behaviour BFCL scores as irrelevance. All scores are Bad Theory Labs’ own.
The four scenarios line up with the four things BFCL (the Berkeley Function-Calling Leaderboard) v4 actually measures: a single call, parallel calls fired at once, recovery when a call fails, and irrelevance — correctly declining to call anything. BTL-3's headline highlight is that last one: a self-reported 91.2% on knowing when to stay its hand. The loop is the product here; the adapter's whole point is to make Qwen3.6-27B move through it reliably.
The tool-use profile
Break BFCL v4 down by category and the shape of the model shows. It is strongest on the straightforward cases and gives ground exactly where you'd expect — when it has to compose several tools that each take several arguments:
The aggregate is 88.5% BFCL v4 AST (1097/1240 on the full official set). The 70.0% on parallel-multiple is the honest soft spot: issuing several correct calls at once, each with the right arguments, is where structured tool use is genuinely hard, and a fifth of those cases still slip. The 91.2% irrelevance number is the one worth internalising — it is the difference between an agent you can leave in a loop and one that invents work.
Coding, and where it falls off
On standard code-generation benchmarks in thinking mode, BTL-3 posts strong pass-rates — and then drops sharply on the hardest composite tasks. That gap is the useful part of the picture, not a number to bury:
HumanEval at 95.12% (156/164) and LiveCodeBench v6 at 88.1% (170/193) are the flattering figures — well-scoped "write this function" problems. BigCodeBench-Hard Instruct at 26.35% (39/148) is the sobering one: strict pass@1 on tasks that chain many library calls into one correct program is a different sport, and here the model solves roughly one in four. (BTL reports a softer 59.25% at the individual test level on the same suite — useful context, but a test-level score is not a solved-task score, so read the strict 26.35% as the real one.) These are different benchmarks at different difficulties, not a like-for-like ladder — the labels carry that.
The Compact edition
Alongside the adapter, Bad Theory Labs ships BTL-3 Compact: the complete text model packed into a single 8.39 GB native file — smaller than an 8B model stored in FP16, which works out to an effective under 2.5 bits per parameter. The claimed cost of that compression is measured on a "fresh private 100-turn tool-contract gate": Compact retained 83 of the 90 behaviours the full model completed correctly, which BTL reports as 92.2% conditional tool-behaviour retention.
Read that metric for exactly what it is. It is a private gate that BTL defined and ran, conditioned on cases the full model already passed — so it says "Compact reproduces most of what the full model got right," not "Compact loses only 8% overall." It's a reasonable internal check and a genuinely useful artifact (a 27B-class agent in 8.39 GB is easy to self-host), but it is not an independent quality measurement.
Running it
Because BTL-3 is a LoRA adapter, deployment is "load Qwen3.6-27B at the pinned revision, then apply the
adapter" — a few lines with PEFT and Transformers, or vllm serve with --enable-lora and
--max-lora-rank 32 and the Qwen XML tool parser for structured calls. The architectural context
window is 262,144 tokens (inherited from Qwen3.6's hybrid attention), though the published
benchmarks were run at a 32,768-token launch context. BTL recommends thinking mode for coding
and reasoning, which is also the mode every headline score was measured in.
The honest read
The take
BTL-3 is a modest, honest kind of release: take a strong open base, spend an RL budget teaching it to behave inside an agent loop, and ship the ~934 MB of difference under Apache-2.0. The most interesting claim isn't a coding score — it's the 91.2% irrelevance, the tuned instinct to not call a tool. For anyone assembling a private, self-hosted coding agent, that plus the 8.39 GB Compact build is a concrete, deployable proposition. Just hold the framing straight: the intelligence is Qwen's, the discipline is BTL's, and until someone outside Bad Theory Labs runs the suite, every figure is a vendor's own.
Source: the BTL-3 model card and BTL-3 Compact on Hugging Face, plus the runtime source. The card ships no figures, so the diagram here is mine; all benchmark numbers are Bad Theory Labs' self-reported values.