# MemHarness: agent memory should be reconstructed, not replayed

> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/memharness
> date: 2026-08-03
> tags: agents, memory, reinforcement-learning, llm, explainer
Most memory-augmented agents treat a retrieved experience the way a tape recorder treats a
cassette: press play, get back exactly what was stored. **MemHarness**, from a
Zhejiang University / Shanghai AI Lab team, argues that's the wrong model of memory
entirely — and points at cognitive science to say so. Human recall isn't playback; it's
reconstruction, rebuilt each time from fragments and reshaped to fit the moment. Their
agent does the same: before acting on a retrieved memory, it first critiques that memory
against what it's looking at *right now*, and rewrites it if the two don't match.

<Callout type="note">
Single paper, one lab, no third-party replication found. Everything below — every
percentage, every table — is MemHarness's own reported numbers on two benchmarks
(ALFWorld, WebShop), one 7B backbone (Qwen2.5-7B-Instruct). The paper reports no
hardware, no wall-clock latency, and no variance across seeds for any of its evaluation
numbers — see **Honesty check** near the end before you take any number as a settled
fact. The interactive diagrams below are clearly labeled: two use the paper's own
measured table values, one is an illustrative toy walkthrough of the mechanism.
</Callout>

## The failure mode: negative transfer from a memory that no longer fits

Retrieval-augmented agents work like this: finish a task, distill what happened into a
short natural-language "experience," store it, and next time a similar task comes up,
pull the closest matches back into context. The problem is *closest* is doing a lot of
work. An experience learned from one kitchen layout, one inventory state, one shopping
page, gets pasted into a context where the fridge is already full or the shelf has
something else on it — and the agent, having no reason to doubt its own memory, follows
the stale instruction anyway. MemHarness calls this the "replay" paradigm, and its
central claim is that replay's failures are systematic, not occasional: the retrieved
experience is abstract and general by construction, while the state at decision time is
concrete and constantly changing, and nothing in a replay pipeline reconciles the two.

<Figure
  src="/articles/memharness/fig1.png"
  alt="Three stacked panels. Top, 'Previous: Replay Retrieved Memory' — a memory store retrieves a memory and a history state, both injected unchanged into an input context alongside the task, with a warning that the memory may not align with the current state. Middle, 'Human Memory: Not just Replay, But Reconstruction' — a brain retrieves fragmented past experience as puzzle pieces and reconstructs them, informed by current state and knowledge, into an informed action. Bottom, 'MemHarness: Reconstruct Memory for the Current State' — the same retrieval happens, but the retrieved memory and current state pass through the agent, which critiques, aligns, and adapts the memory before it enters the final input context as reconstructed, aligned memory."
  caption="The three paradigms MemHarness is contrasting: naive verbatim replay, human reconstructive memory, and MemHarness's learned reconstruction step in between retrieval and action (Wu et al., 2026, Figure 1)."
/>

This isn't a new observation for this site — [Agent harnesses: engineering the loop
around the model](/articles/agent-harness) already named "context and memory lifecycle"
as one of the open problems in agent engineering, and pointed out that a file-backed
harness effectively turns the file system into the agent's long-term memory. MemHarness
is a concrete answer to a sharper version of that problem: it's not enough to *store*
memory durably, the harness also has to decide, at read time, whether a piece of stored
memory still applies — and if not, what to do about it. MemHarness's answer is to make
that decision itself a trained, learned skill rather than a fixed retrieval-and-paste
rule.

## Three stages: retrieve, reconstruct, act

Formally, the agent holds a memory bank $\mathcal{B} = \{m_i\}_{i=1}^N$ of entries
$m_i = (e_i, o_i^{src})$ — an abstracted experience $e_i$ paired with the **source
observation** $o_i^{src}$ it was distilled from. At each step $t$, retrieval returns the
top-$k$ closest entries, $\mathcal{E}_t = \mathcal{R}(q_t, \mathcal{B})$. A pure replay
policy conditions the next action directly on whatever comes back:

$$
a_t \sim \pi_\theta(\cdot \mid \mathcal{T}, h_t, \mathcal{E}_t)
$$

MemHarness inserts one step in between. The same policy first produces guidance $g_t$ by
critiquing the retrieved experiences against the current history $h_t$, then maps that
into final guidance $\tilde{g}_t$, and only then generates the action:

$$
g_t \sim \pi_\theta(\cdot \mid \mathcal{T}, h_t, \mathcal{E}_t), \qquad
\tilde{g}_t = f(g_t), \qquad
a_t \sim \pi_\theta(\cdot \mid \mathcal{T}, h_t, \tilde{g}_t)
$$

The reconstruction input concatenates the task, the recent history, and every retrieved
(experience, source-state) pair:

$$
x_\text{recon} = \mathcal{T} \oplus h_t \oplus \bigcup_{i=1}^{k} (e_{t,i},\, o_{t,i}^{src})
$$

and $f$ is a simple conditional: if the policy decides nothing retrieved applies, it
emits the literal token `<EMPTY>`, and $\tilde{g}_t$ falls back to a fixed self-reasoning
prompt $p_\text{self}$ instead of forcing a bad match into the action context.

<Figure
  src="/articles/memharness/fig2.png"
  alt="Three-panel pipeline diagram. Stage 1, Memory Retrieval: task, history, and current state feed a policy model that issues a query, which retrieves the top-k memories (each a strategy paired with its source observation) from the memory store. Stage 2, Contextual Memory Reconstruction: the policy model compares each memory's source observation against the current observation, then a conditional mapping either passes through the guidance or, if it is EMPTY, substitutes a self-reasoning prompt, producing final guidance. Stage 3, Action Generation: the policy model conditions on task, history, and the final guidance to produce an action, which is executed in the environment and scored by an outcome-plus-format reward, which updates the policy model via GRPO with a group-relative advantage."
  caption="The three-stage inference pipeline — retrieval, contextual reconstruction, action — trained end-to-end with GRPO on outcome-plus-format reward (Wu et al., 2026, Figure 2)."
/>

The mechanics of that middle stage — comparing a retrieved memory's source state against
the live one, and deciding pass-through / adapt / reject — are easiest to see with a toy
example. Toggle through the three cases below:

<ReplayVsReconstruct />

Note what each branch does differently from plain replay. When the state genuinely
**matches** the memory's source, reconstruction is a no-op and replay would have been
fine anyway — the interesting cases are the other two. When the state has **drifted**,
reconstruction rewrites the *target*, not just the wording, while replay keeps repeating
an instruction the environment has already invalidated. And when retrieval turns up
nothing usable, MemHarness can say so explicitly and fall back to the agent's own
reasoning — a replay pipeline has no equivalent move; it either injects a weak match or
injects nothing silently.

## Training: one policy, three roles, GRPO end to end

The same weights play all three parts — retriever-decider, reconstructor, actor — and
the whole thing is trained with **GRPO** on a sparse outcome reward plus a small format
bonus:

$$
R(\tau_i) = R_\text{outcome} + 0.1 \cdot R_\text{format}
$$

$R_\text{outcome}$ is 10 for a successful episode and 0 otherwise; $R_\text{format}$
checks that every step emits exactly one valid `<think>` block, one valid `<action>`
block, that memory is retrieved through valid `<retrieve_memory>` blocks (one to five
times per episode), and that everything is in English. Rewards are group-normalized —
sample $G=8$ rollouts per prompt and standardize against the group:

$$
A_i = \frac{R(\tau_i) - \text{mean}(\{R(\tau_k)\}_{k=1}^{G})}{\text{std}(\{R(\tau_k)\}_{k=1}^{G})}
$$

and the policy update is the standard clipped-surrogate-plus-KL objective, with clip
range $\varepsilon = 0.2$ and KL coefficient $\beta = 0.01$:

$$
\mathcal{J}(\theta) = \mathbb{E}\!\left[\frac{1}{\sum_i |\tau_i|}\sum_{i=1}^{G}\sum_{j=1}^{|\tau_i|}
\Big(\mathcal{L}^{\text{CLIP}}_{i,j}(\theta) - \beta\, \mathbb{D}_{\text{KL}}[\pi_\theta \| \pi_\text{ref}]\Big)\right]
$$

None of this is a new RL recipe — it's the same token-level, group-relative machinery
covered in [Token-level RL is a first-order approximation to the reward you actually
want](/articles/first-order-rl). What's specific to MemHarness is that the *reconstruction
step itself* is inside the RL loop and gets credit for the same sparse outcome reward as
the final action, rather than being a fixed prompt template bolted on the side. That's
also why an ablation later in this piece — replacing the trained reconstruction with a
generic, untrained LLM doing the same rewriting job — measurably underperforms: rewriting
text is not the same skill as rewriting text so that it wins the episode.

Before RL, there's a short cold-start SFT stage — 200 trajectories with GPT-5.1-generated
retrieval and reconstruction turns, plus 200 trajectory-to-memory summarization examples
per benchmark — whose only job is to teach the interaction protocol (when to emit
`<retrieve_memory>`, how to format guidance). The paper is explicit that this stage is
about "protocol and format alignment rather than task-skill acquisition," and the numbers
back that up: the cold-start model alone scores a *worse* 7.6% on ALFWorld than the
untrained base model's 14.5%, because it has learned to follow a longer protocol without
yet having learned to solve the task.

The memory bank itself lives in **Milvus**, embedded with **BGE-M3**, retrieved by
cosine similarity at $k=3$. It isn't hand-curated — during training, the policy
distills roughly half of its own generated trajectories (balanced between successes and
failures where possible) into new memory entries, so the bank grows out of the same
policy that reads from it. Write-time deduplication skips a new entry if it's
too similar (cosine $> 0.85$) to something already stored — enabled for WebShop,
disabled for ALFWorld — and retrieval-time deduplication thins a larger candidate pool
before truncating to the top-$k$.

## Does it beat the baselines

On the headline numbers: MemHarness reaches **85.2%** average success on ALFWorld's six
task categories and **75.6%** on WebShop, ahead of every baseline the paper reports —
including foundation models an order of magnitude larger:

<BenchBars
  title="ALFWorld · Avg. success rate across 6 task categories (Table 1)"
  unit="%"
  bars={[
    { label: "Qwen2.5-7B (base)", value: 14.5 },
    { label: "ReAct", value: 27.9 },
    { label: "Mem0", value: 33.5 },
    { label: "GPT-4o", value: 49.2 },
    { label: "Gemini-2.5-Pro", value: 62.1 },
    { label: "EvolveR (reproduced)", value: 70.1 },
    { label: "GRPO (no memory)", value: 76.4 },
    { label: "MemHarness", value: 85.2, highlight: true },
  ]}
/>

<BenchBars
  title="WebShop · success rate (Table 1)"
  unit="%"
  bars={[
    { label: "Mem0", value: 2.0 },
    { label: "Qwen2.5-7B (base)", value: 7.8 },
    { label: "ReAct", value: 19.5 },
    { label: "GPT-4o", value: 23.7 },
    { label: "Gemini-2.5-Pro", value: 35.9 },
    { label: "GRPO (no memory)", value: 66.1 },
    { label: "EvolveR (reproduced)", value: 72.6 },
    { label: "MemHarness", value: 75.6, highlight: true },
  ]}
/>

A few things worth being precise about here. The 16-row full table (not all shown
above) mixes closed-source frontier models (GPT-4o, Gemini-2.5-Pro), prompt-only
memory agents (ReAct, Reflexion, Mem0, ExpeL, MemP, SimpleMem), and RL-trained agents
(RLOO, GRPO, MemRL, EvolveR, and two "+GRPO" memory hybrids) — and with one exception
(**EvolveR**, explicitly marked "reproduced"), the paper doesn't say whether the other
baseline numbers are copied from those methods' original papers or re-run by the authors
under this setup. Given every RL-based and prompt-based baseline shares the same
Qwen2.5-7B-Instruct backbone as MemHarness, it reads as an in-house re-implementation for
a controlled, like-for-like comparison — which is the right thing to do for fairness, but
it also means there's no independent number to check any of them against. **Mem0** in
particular scores worse than the untrained base model on WebShop (2.0% vs. 7.8%) — plausible
for a general-purpose memory library not tuned to this task, but a reminder that "memory
system" is not automatically an improvement.

## Why raw memory can hurt: the ablation

The paper's most useful table isn't the leaderboard, it's the ablation, because it
isolates *why* MemHarness wins rather than just *that* it wins. Same policy, same GRPO
recipe throughout — only the memory wiring changes:

<AblationExplorer />

Two results here are worth sitting with. First, **RL + Raw Memory** — verbatim replay,
grafted onto the same trained policy — actually *loses* to having no memory at all on
ALFWorld (70.1% vs. 76.4%), which is the paper's sharpest evidence that unreconstructed
memory is not a free win; it can be actively confusing. Second, **w/o memory** — the
fully-trained MemHarness policy with retrieval switched off at test time — still beats
the no-memory-ever RL baseline on both benchmarks (83.0% vs. 76.4% on ALFWorld, 73.6% vs.
66.1% on WebShop). The paper reads this as evidence that training the policy to
reconstruct memories also sharpens its general reasoning, independent of whether memory
is available at inference — the reconstruction objective works partly as a training-time
signal, not only a run-time lookup.

The training curves back this up with a second, independent kind of evidence — not an
end-of-training snapshot, but what happens over the run:

<Figure
  src="/articles/memharness/fig3.png"
  alt="Line chart titled ALFWorld, x-axis Training Steps from 0 to about 150, y-axis Success Rate percent from 0 to 100. Three lines: overall Success Rate, SR when Accepted (a reconstruction was kept), and SR when Rejected (a reconstruction was rejected), all rising together from about 8% to about 85% over training, with SR when Accepted tracking closely just under overall SR and SR when Rejected running lower and noisier throughout."
  caption="Success rate over GRPO training, split by whether the episode contained at least one accepted or rejected memory reconstruction — accepted reconstructions track the rising success rate; rejected ones lag behind it (Wu et al., 2026, Figure 4a)."
/>

Trajectories where the policy accepted a reconstructed memory track the overall
success-rate curve closely; trajectories where it rejected one lag behind and stay
noisier throughout training. That's a consistency check on the whole framework: if
"accept vs. reject" were an arbitrary or miscalibrated signal, there'd be no reason for
it to correlate with which trajectories actually succeed.

MemHarness also holds up when the environment itself is unfamiliar. On ALFWorld's
out-of-distribution split — unseen room layouts and object placements — it scores
**85.9%**, while stripping reconstruction back out (raw memory injected, same OOD
environments) drops to **76.3%**, and disabling reconstruction only at test time (same
trained policy) lands at **82.4%**. The direction of every result here matches the
in-distribution ablation: verbatim replay is the worst way to use memory precisely when
the environment has changed most.

## The mechanism, under a microscope

Everything so far shows *that* reconstruction helps. The paper also runs two controlled
probes asking a narrower question: does the policy's reconstruction step actually compare
the current state against the memory's recorded source state, or is it just producing
plausible-sounding rewrites without really checking anything?

<MechanismProbe />

The **source-state ablation** answers this directly: strip $o_i^{src}$ out of the
reconstruction prompt entirely, and rejection rate barely moves — but success rate drops,
because the policy now accepts guidance it has no way to judge as stale. Swap in a
*random* memory's source state instead — a state that's guaranteed not to match — and
rejection rate jumps sharply (8.7%→13.3% on ALFWorld, 56.0%→63.3% on WebShop). That
asymmetry is the tell: removing the comparison signal doesn't change behavior much
because the policy simply can't tell anymore, while corrupting it with a wrong-but-present
signal actively triggers more rejections. The **counterfactual probe** — asking a strong
LLM to make a minimal edit to 1,000 real states so a previously-applicable memory should
no longer apply, then scoring only the reconstruction output — shows the same pattern
from the other direction: minimal edits shift outputs measurably away from "unchanged"
and toward "adapted" or "rejected" on both benchmarks, with WebShop rejecting far more
often than ALFWorld in both the matched and edited conditions (72–79% vs. 0–6%), which
the paper attributes to WebShop's longer, more heterogeneous page observations making a
fuzzy accept riskier than a clean reject.

## Honesty check

- **Self-reported, single lab, no replication.** Every number above is from this one
  paper. I found no independent reproduction, and the community-discussion page on
  alphaXiv had nothing beyond the paper's own abstract and tables at the time of writing.
- **Baselines are the paper's own reruns, not cited published numbers**, as far as the
  text discloses — with the single exception of EvolveR, marked "(reproduced)" in the
  table. That's a reasonable design for a fair, same-backbone comparison, but it also
  means none of the sixteen rows in Table 1 have an outside number to be checked against.
- **No hardware, no latency, no cost.** The paper never states what GPUs it trained or
  evaluated on, never reports wall-clock time, tokens/sec, or dollars, and never measures
  the added inference cost of the reconstruction step itself. Reconstruction is a second
  full decode pass through the same 7B policy on every step where memory is retrieved
  (retrieval decision, then reconstruction, then action) — at minimum one extra
  generation versus a direct-replay or no-memory baseline — and that overhead is not
  quantified anywhere in the paper.
- **No variance, no seeds.** Every success-rate and rejection-rate number is reported as
  a single figure with no standard deviation, confidence interval, or multi-seed spread
  disclosed for the evaluation runs.
- **Two benchmarks, one model scale.** ALFWorld and WebShop are both well-worn,
  relatively short-horizon (15–50 step) simulated environments; the paper does not test
  a larger backbone, a real-world tool-using agent, or a benchmark with a genuinely
  different observation modality. The conclusion names this directly: "future work will
  explore scaling to larger models and open-ended environments" — which is the authors'
  own way of saying this hasn't been tried yet.
- **No explicit Limitations section.** The paper has no dedicated limitations
  discussion; what's above is reconstructed from the ablations, the conclusion, and what
  the method section does and doesn't measure.
- **What is solid:** the mechanism probes (source-state ablation, counterfactual editing)
  are a genuine attempt to falsify the "it's just fluent rewriting" explanation, and they
  point the same direction from two independent angles. That's better methodological care
  than a bare leaderboard table, even without outside replication.

## The take

The idea underneath MemHarness is simple enough to state in a sentence — compare the
memory's source state to the current one before you trust it — and the paper's real
contribution is making that comparison a *trained* skill inside the same policy, credited
by the same sparse outcome reward as the action itself, rather than a hand-written
heuristic bolted onto retrieval. The ablations back the framing better than the
leaderboard does: raw memory replay measurably *loses* to no memory at all on one
benchmark, and the reconstruction-trained policy keeps a chunk of its advantage even with
memory switched off entirely, which says the training signal is doing more than teaching
better lookups.

What it hasn't shown yet is whether any of this survives outside two small, well-studied
simulators at 7B scale, and whether the extra reconstruction pass is worth its
unmeasured latency cost in a setting where that matters. "Reconstruct, don't replay" is a
good design principle for any agent harness that reads back its own memory. Whether this
specific recipe for teaching it — GRPO, a `<EMPTY>` escape hatch, a Milvus bank refreshed
by the policy's own trajectories — is the way to get there past ALFWorld and WebShop is
still an open question the paper itself doesn't claim to answer.

---

*Built on [MemHarness: Memory Is Reconstructed, Not Replayed](https://arxiv.org/abs/2607.28272)
(Wu et al., 2026; arXiv:2607.28272). Figures are reproduced from the paper for commentary.
The interactive diagrams use the paper's measured table values except where marked
illustrative; see the Honesty check above for what is and isn't independently verified.*