BTL-3: a rank-32 LoRA that turns Qwen3.6-27B into a tool-use agent

2026-07-24 · 8 min · agents · tool-calling · code-generation · fine-tuning

Why read this

A read of BTL-3's thin card: a rank-32 LoRA on Qwen3.6-27B for tool use, strong on irrelevance, weak on parallel-multiple, all numbers self-reported.

Runs on a consumer GPU

Agents & harnessesApache-2.0Practitioner model

How this was scored

Is it new?: 1 of 3: An incremental tweak
Can I trust it?: 1 of 3: Spot-checks a few numbers
Can I run it?: 2 of 3: Open code or weights with real limits
Will I understand it?: 1 of 3: Partial mechanism
Can I act on it?: 1 of 3: General advice
Will it last?: 0 of 3: News that expires in weeks
Does it affect many?: 0 of 3: A niche paper or one-off repo
Only here?: 1 of 3: Some original analysis

Score 32 of 100, ranked 560 of 577 rated articles. Each question is answered 0–3 by hand, and a 3 is rare. How articles are scored

Most "new models" are not new weights. BTL-3, from Bad Theory Labs, is a clean example: it is not a from-scratch 27B model but a frozen rank-32 PEFT LoRA adapter — about 934 MB of weights — post-trained on top of a pinned revision of Qwen3.6-27B. Load the base, apply the adapter, and you get an agent tuned for coding, repository work, and structured tool use. The raw capability is Qwen's; what BTL-3 contributes is behaviour — how the model runs an agent loop and, notably, when it decides not to act.

That framing matters for reading the numbers honestly, so keep it in mind: this is a post-training result on a strong open base, released under Apache-2.0, with the base checkpoint pinned to an exact revision for reproducible loading. The card labels this frozen release "RL-0013," and the maximum RL sequence length (65,536 tokens) tells you the adapter was shaped by reinforcement learning over long, multi-step trajectories — not just supervised fine-tuning on completions.

The loop it was tuned to run

An agent model earns its keep inside a loop, not on a single completion. BTL-3's stated job is to "reason, act, inspect tool results, recover from failures, and stop when no action is required." That last clause is the interesting one. A tool-happy model that always reaches for a function call is easy to train and annoying to deploy; the harder behaviour is the route decision — recognising when a question needs a tool and when it just needs an answer. Pick a scenario and watch which path the model takes:

agent turn · reason → act → observe → stopillustrative

scenario

BFCL Simple 93.2%

Every turn runs the same loop, but the route step decides what happens next. When a task needs tools, BTL-3 emits one or many calls, observes the results, and recovers if one fails. When none is needed it answers directly and stops — the behaviour BFCL scores as irrelevance. All scores are Bad Theory Labs’ own.

The four scenarios line up with the four things BFCL (the Berkeley Function-Calling Leaderboard) v4 actually measures: a single call, parallel calls fired at once, recovery when a call fails, and irrelevance — correctly declining to call anything. BTL-3's headline highlight is that last one: a self-reported 91.2% on knowing when to stay its hand. The loop is the product here; the adapter's whole point is to make Qwen3.6-27B move through it reliably.

The tool-use profile

Break BFCL v4 down by category and the shape of the model shows. It is strongest on the straightforward cases and gives ground exactly where you'd expect — when it has to compose several tools that each take several arguments:

BFCL v4 · category breakdown (self-reported %)

Multiple

95.5%

Simple

93.2%

Irrelevance

91.2%

Parallel

87%

Parallel-multiple

70%

050100

The aggregate is 88.5% BFCL v4 AST (1097/1240 on the full official set). The 70.0% on parallel-multiple is the honest soft spot: issuing several correct calls at once, each with the right arguments, is where structured tool use is genuinely hard, and 30% of those cases still slip. The 91.2% irrelevance number is the one worth internalising — it is the difference between an agent you can leave in a loop and one that invents work.

Coding, and where it falls off

On standard code-generation benchmarks in thinking mode, BTL-3 posts strong pass-rates — and then drops sharply on the hardest composite tasks. That gap is the useful part of the picture, not a number to bury:

Coding pass-rate · self-reported, thinking mode (%)

HumanEval

95.12%

LiveCodeBench v6

88.1%

BigCodeBench-Hard

26.35%

050100

HumanEval at 95.12% (156/164) and LiveCodeBench v6 at 88.1% (170/193) are the flattering figures — well-scoped "write this function" problems. BigCodeBench-Hard Instruct at 26.35% (39/148) is the sobering one: strict pass@1 on tasks that chain many library calls into one correct program is a different sport, and here the model solves roughly one in four. (BTL reports a softer 59.25% at the individual test level on the same suite — useful context, but a test-level score is not a solved-task score, so read the strict 26.35% as the real one.) These are different benchmarks at different difficulties, not a like-for-like ladder — the labels carry that.

The Compact edition

Alongside the adapter, Bad Theory Labs ships BTL-3 Compact: the complete text model packed into a single 8.39 GB native file — smaller than an 8B model stored in FP16, which works out to an effective under 2.5 bits per parameter. BTL publishes the byte ledger behind that figure:

A single stacked horizontal bar titled 'Where the 8.38B GGUF tensor bytes went', split into AVQ2 5.498 GB (65.60%), Affine INT4 0.919 GB (10.97%), BF16 islands 1.206 GB (14.39%), Vocabulary 0.668 GB (7.97%), Small state 0.058 GB (0.69%) and Behavior adapter 0.032 GB (0.39%). Footnote: tensor payloads 8,381,262,032 bytes, final GGUF 8,392,369,600 bytes. — How the 8.39 GB is spent: two thirds in AVQ2 vector codes, a seventh in the BF16 islands held back at full precision, and 0.39% in the rank-8 behaviour adapter that repairs what the rest broke (BTL's compression paper, Badtheorylabs/BTL-3).

The claimed cost of that compression is measured on a "fresh private 100-turn tool-contract gate": Compact retained 83 of the 90 behaviours the full model completed correctly, which BTL reports as 92.2% conditional tool-behaviour retention.

Read that metric for exactly what it is. It is a private gate that BTL defined and ran, conditioned on cases the full model already passed — so it says "Compact reproduces most of what the full model got right," not "Compact loses only 8% overall." It's a reasonable internal check and a genuinely useful artifact (a 27B-class agent in 8.39 GB is easy to self-host), but it is not an independent quality measurement.

It is also an aggregate, and BTL's own chart of it says so in the subtitle:

A horizontal bar chart titled 'Fresh sealed tool-behavior retention', subtitled 'Conditional on teacher-correct turns; the aggregate hides one weak category'. Single, Parallel, Sequential and Abstention each show 100% on n=20 teacher-correct turns; Parallel-multiple shows 30% in red on n=10; the Overall bar is 92.2% on n=90. Footnote: 83 of 90 teacher-correct turns retained, absolute score 83 out of 100. — The 92.2% broken out by category. Four categories retain everything the full model got right; parallel-multiple retains 30% of ten turns, and that single row is the whole gap (BTL's compression paper, Badtheorylabs/BTL-3).

That is the same weak spot the full model shows on BFCL — parallel-multiple, at 70.0% there — only sharper, and on ten turns rather than a benchmark suite. Read the two together and the picture is consistent: composing several calls with several arguments each is where this model is thinnest, and compression takes what little slack was left.

Running it

Because BTL-3 is a LoRA adapter, deployment is "load Qwen3.6-27B at the pinned revision, then apply the adapter" — a few lines with PEFT and Transformers, or vllm serve with --enable-lora and --max-lora-rank 32 and the Qwen XML tool parser for structured calls. The architectural context window is 262,144 tokens (inherited from Qwen3.6's hybrid attention), though the published benchmarks were run at a 32,768-token launch context. BTL recommends thinking mode for coding and reasoning, which is also the mode every headline score was measured in.

The honest read

note

Every number here is self-reported by Bad Theory Labs — there is no independent evaluation yet. The underlying capability is Qwen3.6-27B's; BTL-3 is a post-training / RL result, so credit the adapter for the loop behaviour (tool routing, irrelevance, recovery), not for raw reasoning power. Scores were measured in thinking mode at a 32K context on protocols BTL chose, the coding wins sit next to a 26.35% BigCodeBench-Hard floor, and the Compact edition's 92.2% retention is a private, conditional gate, not a public benchmark — one whose aggregate hides a 30% row. The model card ships no figures or diagrams, so the loop diagram above is my own reconstruction of the described behaviour; the two Compact charts are BTL's, from the compression paper in its runtime repo. No training-data disclosure is provided.

The take

BTL-3 is a modest, honest kind of release: take a strong open base, spend an RL budget teaching it to behave inside an agent loop, and ship the ~934 MB of difference under Apache-2.0. The most interesting claim isn't a coding score — it's the 91.2% irrelevance, the tuned instinct to not call a tool. For anyone assembling a private, self-hosted coding agent, that plus the 8.39 GB Compact build is a concrete, deployable proposition. Just hold the framing straight: the intelligence is Qwen's, the discipline is BTL's, and until someone outside Bad Theory Labs runs the suite, every figure is a vendor's own.

Source: the BTL-3 model card and BTL-3 Compact on Hugging Face, plus the runtime source, whose papers/behavior-before-perplexity/ directory carries BTL's compression paper and the two Compact charts reproduced above. The model card itself ships no figures, so the agent-loop diagram is mine; all benchmark numbers are Bad Theory Labs' self-reported values.