# Program-as-Weights: compiling a natural-language spec into a tiny local model

> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/program-as-weights
> date: 2026-07-04
> tags: llm, inference-optimization, fine-tuning, systems, explainer
Some everyday programming tasks refuse to be written as rules. Alerting a human on only the log lines
that matter, repairing malformed JSON, ranking snippets by intent, deciding whether a message is
urgent — even a regex for "parse this messy text" shatters on edge cases. The modern move is to punt:
call a large language model API and let it decide, per input. That works, but you pay for it on
**every call**, and you give up locality, reproducibility, and price.

**Program-as-Weights (PAW)** — from Waterloo, Cornell, and Harvard — proposes a different deal. Treat
that fuzzy function like source code: **compile it once** from a natural-language specification into a
compact, locally-executable neural artifact, and then *run* that artifact cheaply for every subsequent
input. The compiler is a 4B model; the artifact is a small **LoRA adapter**; the thing that executes it
is a **frozen 0.6B interpreter**. Play with the loop first — pick a spec, compile it, then feed inputs
through the interpreter:

<PawCompiler />

The headline result is the payoff of that split: a **0.6B Qwen3 interpreter** running PAW programs
**matches direct prompting of Qwen3-32B**, while using **roughly one-fiftieth of the inference memory**
and running at **30 tokens/second on a MacBook M3**.

## The compiler–interpreter system

PAW borrows the oldest idea in programming — separate the compiler from the runtime — and instantiates
it with neural parts. The **compiler** reads a function specification (plus an optional pseudo-program
sketch) and emits a parameter-efficient adapter. The **interpreter** is a small, *frozen* model that,
loaded with that adapter, executes the function on real inputs.

<Figure
  src="/articles/program-as-weights/fig1.png"
  alt="Three-panel PAW architecture. Left, the LoRA Compiler: a trainable Compiler model reads a function spec, a pseudo-program, and prefix tokens. Middle, the LoRA Mapper: mean-pooled compiler features go through an MLP that mixes a set of learned LoRA A and B basis matrices into the adapter's low-rank A and B factors. Right, the Interpreter: a frozen model runs the pseudo-program plus input through the injected LoRA to produce output."
  caption="PAW's best instantiation, Text-to-LoRA: the compiler's representation of the spec is mapped by a small MLP into a low-rank LoRA adapter (a mixture of learned basis matrices), which is injected into the frozen interpreter (paper, Figure 3)."
/>

The load-bearing piece is the **LoRA mapper**. Rather than have the compiler regress raw adapter
weights (a huge, unstructured output), it mean-pools the compiler's features and passes them through an
MLP that produces mixing coefficients over a **set of learned LoRA basis matrices** — the emitted
adapter is a combination of reusable low-rank factors. That keeps the compiler's output small and
well-conditioned, and it's what makes "spec → weights" learnable at all. (An earlier prefix-tuning
variant works too; Text-to-LoRA is just the current best.)

To train the compiler, the authors built **FuzzyBench**, a **10-million-example** dataset of fuzzy
functions spanning text processing, search and matching, custom classification, code and NL commands,
safety and verification, agentic tool use, and format repair — with clean and noisy specification
variants so the compiler learns to be robust to sloppy prompts:

<Figure
  src="/articles/program-as-weights/fig3.png"
  alt="Donut chart of FuzzyBench's 10 million examples by theme: core text processing and NLP 30%, search/matching/web intelligence 18%, custom classification and filtering 15%, code and natural-language commands 12%, safety/verification/domain knowledge 12%, agentic and tool use 8%, format repair and validation 5%."
  caption="FuzzyBench: 10M fuzzy-function examples across seven themes, released with the paper, used to train the 4B compiler (paper, Figure 2)."
/>

## Why compile at all? The economics

The reason this framing matters is cost structure. Calling a big model's API is a cost you pay on
**every input**. PAW pays a **one-time compile** — a single pass of the 4B compiler — and then each
application runs locally for almost nothing. So cumulative cost crosses over fast, and the gap only
widens. Drag the number of calls:

<CostCrossover />

That's the thesis in one picture: PAW **reframes the foundation model from a per-input problem solver
into a tool builder**. You invoke the big model once per *function definition*, get back a small
reusable artifact you own, and every *function application* after that is cheap, offline, and
reproducible. Compile a library of them and each becomes a tiny local endpoint:

<Figure
  src="/articles/program-as-weights/fig2.png"
  alt="Three-stage flow. Function Specification: three plain-language specs (classify message urgency, fix malformed JSON, remove personal information). Neural Programs: the compiler turns each into a pseudo-program plus a colored LoRA adapter (LoRA 1, 2, 3). Local Deployment: each adapter runs on a small local LM as PAW Email Triage, PAW Json Fixer, and PAW PII Redactor, processing inputs into outputs on-device."
  caption="Compile a library: each spec becomes a pseudo-program plus a LoRA adapter, deployed as a small, local, single-purpose model (paper, Figure 15)."
/>

Two practical wins reinforce it: the paper reports the adapters **quantize with no measurable accuracy
loss**, and the whole thing runs at interactive speed (30 tok/s) on a laptop — so the compiled function
is genuinely local, not a cloud dependency in disguise.

## The honest caveats

PAW is a genuinely new framing, but it buys its efficiency with real constraints, and the paper is
candid about them:

- **The compiler and interpreter are a coupled pair.** An adapter compiled for one frozen interpreter
  isn't portable to a different one — you commit to an interpreter.
- **The compiled program isn't interpretable.** Unlike source code, you can't read a LoRA adapter to
  see what the function *does*; you can only run it. "Program" is an analogy for the compile-once
  workflow, not a claim of legibility.
- **Single-step fuzzy functions.** PAW targets one-shot transformations (classify, repair, rank), not
  long multi-step agentic control flow.
- **Trained on synthetic data.** FuzzyBench is model-generated; the compiler's competence is bounded by
  the distribution of tasks it was synthesized from, and real specs can fall outside it.
- **The best adapter type is task-dependent.** Text-to-LoRA wins overall, but the paper finds no single
  parameter-efficient method dominates every task — there's still a choice to make.
- **"Matches 32B" is on FuzzyBench-style tasks.** The parity is measured on the fuzzy-function
  distribution PAW is built for, not a claim that a 0.6B model equals a 32B model in general.

## The take

What I like about PAW is that it's a *systems* idea wearing an ML paper's clothes. The interesting move
isn't a new architecture — it's noticing that we've quietly turned every fuzzy function into a recurring
API bill, and that the compile/runtime split we use for ordinary code applies here too: pay the big
model once to *build the tool*, then run the tool locally forever. The Text-to-LoRA mapper is the clever
engineering that makes "spec → weights" tractable, but the reframing is the contribution — a foundation
model as a **compiler for behaviors** rather than an always-on oracle. Whether it generalizes past
single-step functions is the open question; as a way to make a hundred small fuzzy tasks local,
private, and nearly free, it's one of the freshest ideas of the season. Code, the 4B compiler, and the
10M-example FuzzyBench are released (CC BY 4.0).

---

*Built on [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://arxiv.org/abs/2607.02512)
(Zhang, Hotsko, Kim, Nie, Shieber, Deng; University of Waterloo, Cornell, Harvard; 2026, CC BY 4.0).
Figures are reproduced from the paper; benchmark and efficiency figures (0.6B ≈ 32B, ~1/50 memory, 30
tok/s on an M3) are quoted from it. The interactive playground and cost model are illustrations of the
mechanism, not runs of the released model.*