# GEPA: optimize anything you can score and describe
> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/gepa-optimize-anything
> date: 2026-07-24
> tags: optimization, prompt-optimization, evolutionary, agents, explainer
Most ways to make an LLM system better are expensive. Reinforcement learning like GRPO needs
thousands of rollouts to squeeze a scalar reward into the weights; hand-tuning a prompt needs a human in
the loop. **GEPA** (Genetic-Pareto) makes a different bet: language is a *richer* learning signal than a
number. Instead of a policy gradient from a sparse reward, GEPA reads the system's own execution traces,
**reflects on them in natural language** to work out what went wrong, and writes a better prompt — keeping
a **Pareto frontier** of candidates so specialist wins are never averaged away. The result, per its paper,
is a prompt optimizer that turns *a few* rollouts into large gains.
That core has now been generalized twice. `optimize_anything` drops the "prompt" assumption entirely — it
optimizes any text artifact you can **score and describe** — and the new **omni** release composes GEPA
with agent-based optimizers into a single meta-optimizer. This is a walk through the mechanism, then what
each layer adds, honestly labelled.
Two sources, two sets of numbers. The GEPA mechanism and the RL comparison are from the paper *GEPA:
Reflective Prompt Evolution Can Outperform Reinforcement Learning* (Agrawal et al., arXiv 2507.19457).
The `optimize_anything` / omni interface and the Frontier-CS results are from the GEPA team's July 2026
blog post. Every number below is **author-reported** on the authors' own tasks and harness; I have not
re-run them. The interactive diagrams are illustrations of the mechanism, not measured traces.
## The core: reflect, don't reward
GEPA treats a prompt (or a whole multi-prompt system) as the thing to evolve. One turn of its loop is
small and legible — sample a candidate, run it, read what happened, and let an LLM rewrite it:
The move that makes this work is stage 4. A GRPO update collapses an entire trajectory into one scalar
advantage and nudges billions of weights; almost all of the information in *why* the run failed is thrown
away. GEPA does the opposite: it hands a **reflective LLM call** the full trace — the reasoning, the tool
calls, the tool outputs — together with the evaluator's **textual feedback**, and asks it to diagnose the
failure and propose a fix in words. "You mis-parsed the date format on the third call" is a far denser
lesson than "reward = 0.2", and it applies after a single rollout instead of thousands.
Because the feedback is language, the same loop works on any system with one or more LLM prompts, and the
gains show up fast. Across six tasks the paper reports GEPA **outperforming GRPO by 6% on average and by up
to 20%, while using up to 35x fewer rollouts** — and beating **MIPROv2**, the previous leading prompt
optimizer, **by over 10%** (for example, +12% accuracy on AIME-2025). The claim isn't that reflection is
magic; it's that natural language is a higher-bandwidth channel than a reward scalar when your optimizer is
itself a language model.
GEPA comes out of the DSPy lineage — it ships as a DSPy optimizer, and the baseline it beats, MIPROv2, is
DSPy's earlier prompt optimizer. If you already express your system as a DSPy program, GEPA is a
drop-in optimizer over its prompts; the omni post also references a "DSPy full program evolution" tutorial
that evolves the program itself, not just its instructions.
## Why a Pareto frontier, not the best-so-far
The "Genetic" half of the name is evolution — mutate, evaluate, keep the good ones. The subtle part is
*which* ones you keep. A greedy search keeps the single best-scoring candidate and mutates that. GEPA keeps
the whole **Pareto frontier**: every candidate that is the best found so far on **at least one task
instance**. Toggle between the two policies and watch what greedy throws away:
Averaging is lossy. A prompt that nails a hard sub-case but is middling overall looks worthless to a
greedy optimizer and gets discarded — along with the one trick it had figured out. By keeping the frontier
and **sampling parents from it**, GEPA preserves those specialists so their lessons can be recombined later,
and it keeps the search from collapsing into a single lineage that stalls in a local optimum. It's the same
instinct as maintaining a diverse population in a genetic algorithm, made concrete on per-instance scores.
## Optimize anything: score + describe
Once you notice that nothing in the loop actually requires the artifact to be a *prompt*, the interface
generalizes. `optimize_anything` asks for exactly two things: a **seed** text artifact, and an
**evaluator** that returns a score and some feedback. Everything else — objective, background context — is
plain English the reflective LLM reads.
```python
from gepa.optimize_anything import optimize_anything, OptimizeAnythingConfig
def evaluate(candidate: str) -> tuple[float, dict]:
score, feedback = run_judge(candidate) # your metric + any text you want the LLM to see
return score, {"Feedback": feedback}
seed = open("seed_solution.py").read()
task = dict(
evaluator=evaluate,
objective="Maximize the score for this competitive-programming problem.",
background="A sandboxed judge runs hidden tests and returns a 0-100 score.",
)
result = optimize_anything(seed, **task, config=OptimizeAnythingConfig(engine="gepa"))
```
The artifact can be a prompt, a code file, a system message, a config, a plan — anything expressible as
text and gradeable by a function. The `feedback_dict` is the whole trick: whatever diagnostic text you put
in it (a failing test's stderr, a judge's rationale, a lint report) becomes the material the reflective LLM
reasons over. Score tells the search *whether* a candidate is better; feedback tells it *why*, which is what
makes the next mutation informed instead of random.
## Omni: no single optimizer wins, so compose them
The July 2026 post starts from an inconvenient observation: on hard, open-ended problems, **no single
optimizer dominates**. GEPA's reflective mutation, an autonomous coding agent, and an agent-based proposer
each win on different problems — and each one eventually **plateaus**. The `engine=` argument turns this
into a lever: the same `optimize_anything` task can be dispatched to any of three families.
| engine | family | how it proposes |
|---|---|---|
| `gepa` | LLM-based optimizer | one reflective LLM call mutates a parent drawn from the Pareto frontier; the framework owns the loop |
| `meta_harness` | agent-based | a coding-agent proposer mutates the candidate; the framework still owns the loop |
| `autoresearch` | autonomous agent | a long-horizon agent session owns the *entire* loop — selection, proposal, and orchestration |
Across ten problems the winner is unpredictable — the paper's per-problem tally is GEPA 3, AutoResearch 3,
Meta-Harness 4 — so betting on one engine is betting wrong 60–70% of the time. Worse, when an engine
plateaus, *seeding a different engine from the stuck candidate usually breaks through*: on one problem GEPA
stalled at 54.4 after about \$1.3 of budget and switching to AutoResearch lifted it to 62.7; on another,
AutoResearch stalled at 50.0 and both other engines climbed from there to a perfect 100.
**omni** turns that into a strategy. It splits a fixed budget in two phases: **explore**, running all three
engines in parallel on a small slice (about \$5 each) and keeping the best candidate; then **continue**,
seeding a *fresh* optimizer instance with that winner and spending the rest (about \$5) to push past the
plateau — all capped at \$20 total per problem.
The composition itself is exposed as a small kit of primitives — `optimize_best_of` (parallel, keep the
top), `optimize_sequential` (chain engines), `optimize_vote` (fair cross-engine comparison), and
`optimize_adaptive_sequential` (auto-switch on plateau detection) — so omni is one policy you can write, not
a hardcoded pipeline.
## Results on Frontier-CS
The benchmark is **Frontier-CS**: ten open-ended competitive-programming problems, a \$20 budget each, using
Claude Sonnet 4.6 with medium thinking. A single zero-shot LLM call averages **7.72**, so there is real
headroom. Under a matched \$20 budget, omni tops every standalone optimizer:
The per-engine story is that omni lifts *every* base optimizer, not just the strongest — the biggest jump is
GEPA's, which nearly doubles once it stops having to break through on its own:
| optimizer | standalone | as omni | lift |
|---|---|---|---|
| GEPA | 43.8 | 61.8 | +18.0 (+41%) |
| AutoResearch | 55.4 | 63.2 | +7.8 (+14%) |
| Meta-Harness | 50.9 | 59.3 | +8.4 (+16%) |
The mechanism behind those lifts is the plateau-break — a stuck candidate handed to a different optimizer
keeps climbing:
Read these as a promising engineering result, not a settled benchmark. Frontier-CS is ten problems on the
authors' own harness with an LLM judge, and the scores are averages over a small set with real variance —
the per-problem winner already swings widely. omni also spends its budget on *three* engines plus a
continue phase, so the fair comparison is the matched \$20 cap, which the post does hold. The headline is
narrow and honest: under that budget, composing beat every single optimizer they tried — not that omni is
optimal.
## The take
GEPA is a clean idea executed in layers. The core is that **language is a denser training signal than a
reward** when the optimizer is an LLM: reflect on the trace, keep a Pareto frontier so specialists survive,
and a few rollouts go a long way — up to 20% over GRPO at up to 35x fewer rollouts, on the paper's tasks.
`optimize_anything` strips away the "prompt" assumption and leaves a genuinely general interface: *any text
artifact you can score and describe* is now optimizable, with the feedback string doing the heavy lifting.
And omni's contribution is an honest one — since no single optimizer wins and each one plateaus, **explore
across engines, then continue from the best**, which on Frontier-CS beat every standalone optimizer at a
matched budget. The caveats are the usual ones for a fresh result: small benchmark, LLM judge, provider
numbers. But the shape is compelling, and because the whole thing is `pip install` and a scoring function,
it's unusually easy for others to check on their own artifacts.
---
*Sources: [GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning](https://arxiv.org/abs/2507.19457)
(Agrawal et al., 2025) for the mechanism and the RL/MIPROv2 comparisons, and the GEPA team's
[optimize_anything goes omni](https://gepa-ai.github.io/gepa/blog/2026/07/22/optimize-anything-omni/) post
(Tan, Agrawal, Lee, Zhang, Klein, Sen, Dimakis, Zaharia, 2026) for the interface and Frontier-CS results.
GEPA is developed in the [open-source repo](https://github.com/gepa-ai/gepa) and integrates with
[DSPy](https://dspy.ai). Figures are reproduced from the post for commentary; the interactive diagrams are
mine. All benchmark numbers are author-reported.*