# Contextual Retrieval, with a runnable repro and a browser playground
> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/blog/contextual-retrieval
> date: 2026-07-17
> tags: rag, retrieval, claude, embeddings, playground
[Anthropic's Contextual Retrieval post](https://www.anthropic.com/engineering/contextual-retrieval)
has been on my to-read list for a while. It targets the oldest bug in RAG, so I finally sat down,
built a dependency-free reproduction, and wired up a browser playground. This is the write-up.
## The problem: chunks forget where they came from
Standard RAG splits documents into chunks and indexes each chunk on its own. That destroys context.
Anthropic's example is perfect:
> `The company's revenue grew by 3% over the previous quarter.`
Which company? Which quarter? The chunk can't say. A query like *"how did ACME's Q2 revenue change?"*
has **no terms to match** against that chunk — the words "ACME" and "Q2 2023" live in the *document*,
not the *chunk*. Both lexical (BM25) and semantic (embedding) retrieval miss it.
## The fix: let Claude situate each chunk
Contextual Retrieval prepends a short, chunk-specific context to each chunk **before** you embed it and
**before** you build the BM25 index — Anthropic calls these **Contextual Embeddings** and **Contextual
BM25**. The context is generated by Claude, given the whole document. The same chunk becomes:
> `This chunk is from an SEC filing on ACME corp's performance in Q2 2023; the previous quarter's revenue was $314 million. The company's revenue grew by 3% over the previous quarter.`
Now the owner and the period are *in the chunk*, so retrieval can find it.
## Watch it happen
Here is the whole pipeline in your browser — the document, its chunks, the chunks with context prepended,
then retrieval. The scoring is a real BM25 and a TF-IDF cosine (a stand-in for an embedding model); the
contextual prefixes stand in for Claude's output. Step to **4 · retrieve**, pick a query, and watch the
right chunk climb from the standard column to the contextual one:
Toggle between BM25, embeddings, and their fusion. The pattern holds across all three: the answer chunk
is buried on standard chunks and near the top once each chunk carries its context.
## The one prompt that does the work
This is the prompt from the post, verbatim. It runs once per chunk, with the full document supplied as
context:
```text
{{WHOLE_DOCUMENT}}
Here is the chunk we want to situate within the whole document
{{CHUNK_CONTENT}}
Please give a short succinct context to situate this chunk within the overall document for the purposes of improving search retrieval of the chunk. Answer only with the succinct context and nothing else.
```
The obvious worry is cost: you re-send the whole document once per chunk. **Prompt caching** kills that.
Cache the document once and every chunk in it reads from the cache, which is why Anthropic quotes a
one-time **$1.02 per million document tokens**. The document is the stable prefix, so it takes the
`cache_control` breakpoint; the chunk and instruction vary and come after it:
```python
from anthropic import Anthropic
client = Anthropic()
CONTEXT_PROMPT = """Here is the chunk we want to situate within the whole document
{chunk}
Please give a short succinct context to situate this chunk within the overall document \
for the purposes of improving search retrieval of the chunk. Answer only with the \
succinct context and nothing else."""
def situate(doc: str, chunk: str) -> str:
resp = client.messages.create(
# One Claude call per chunk. For a large index you'd typically drop to a
# cheaper model like claude-haiku-4-5 — which is what Anthropic's ~$1.02 /
# 1M-doc-token estimate assumes — trading a little context quality for cost.
model="claude-opus-4-8",
max_tokens=200,
messages=[{
"role": "user",
"content": [
# The whole document is the stable prefix: cache it ONCE, then every
# chunk of this document reads from the cache instead of re-paying for it.
{"type": "text",
"text": f"\n{doc}\n",
"cache_control": {"type": "ephemeral"}},
{"type": "text", "text": CONTEXT_PROMPT.format(chunk=chunk)},
],
}],
)
return "".join(b.text for b in resp.content if b.type == "text").strip()
```
Verify the cache is actually working: `resp.usage.cache_read_input_tokens` should be non-zero on every
chunk after the first for a given document. If it's zero, something upstream is mutating the document
bytes (a timestamp, non-deterministic JSON) and invalidating the prefix.
## A dependency-free repro
To convince myself the lift was real and not marketing, I wrote a ~180-line pure-stdlib script: naive
chunking, BM25 from scratch, a TF-IDF cosine as an embedding stand-in, and reciprocal rank fusion. The
retrieval core is small — here is BM25 and the fusion step:
```python
class BM25:
def score(self, query, i):
dl, tf, s = len(self.docs[i]), self.tf[i], 0.0
for t in query:
if t not in tf:
continue
num = tf[t] * (self.k1 + 1)
den = tf[t] + self.k1 * (1 - self.b + self.b * dl / self.avgdl)
s += self.idf(t) * num / den
return s
def rrf(rankings, k=60): # reciprocal rank fusion of BM25 + embedding rankings
scores = Counter()
for ranking in rankings:
for rank, item in enumerate(ranking):
scores[item] += 1 / (k + rank + 1)
return [i for i, _ in scores.most_common()]
```
Then I index the chunks two ways — plain, and with a one-line context prepended — and measure where the
correct chunk lands for a couple of "which entity, which period" queries. The actual output:
```text
query method plain rank ctx rank
------------------------------------------------------------------------------------
How did ACME Corp revenue change in Q2 2023? bm25 4 2
How did ACME Corp revenue change in Q2 2023? emb 4 1
How did ACME Corp revenue change in Q2 2023? hybrid 4 2
What happened to Beta Industries revenue in Q3 2023? bm25 5 2
What happened to Beta Industries revenue in Q3 2023? emb 5 1
What happened to Beta Industries revenue in Q3 2023? hybrid 5 2
recall@1 (fraction of queries where the right chunk ranks #1):
bm25 plain 0/2 contextual 0/2
emb plain 0/2 contextual 2/2
```
Contextualizing the chunks moves the answer from rank **4–5** to rank **1–2** across BM25, embeddings, and
their fusion — and embeddings recall@1 goes from **0/2 to 2/2**. On this toy corpus fusion lands the answer
at #2 rather than #1 (a small-N artifact of RRF), which is a good reminder that the fusion win is an
*aggregate* effect — which is exactly what Anthropic's real evaluation measures.
## What the real evaluation found
On Anthropic's benchmark (top-20 retrieval failure rate, i.e. `1 - recall@20`), stacking the techniques
compounds:
- **Contextual Embeddings** alone: `5.7% → 3.7%` — a **35%** cut in the failure rate.
- **+ Contextual BM25**: `5.7% → 2.9%` — **49%**.
- **+ reranking**: `5.7% → 1.9%` — **67%**.
The lexical half matters more than you'd guess — BM25 nails exact identifiers (error codes, ticker symbols,
function names) that embeddings smear together, so contextualizing *both* indexes and fusing them beats
either alone.
## Things worth copying from the post
- **Retrieve top-20, not top-5/10.** Anthropic found 20 the most performant cut for the final context.
- **Rerank the shortlist.** Retrieve ~150 candidates, then rerank down to 20 for the answer prompt — that's
the step that takes the failure rate from 2.9% to 1.9%.
- **Embedding model matters.** Gemini and Voyage embeddings were the standouts in their tests.
- **Chunking still matters.** Size, boundary, and overlap all move the numbers — Contextual Retrieval sits
on top of good chunking, it doesn't replace it.
- **A domain-tuned context prompt beats the generic one.** The template above is a floor, not a ceiling.
Be honest about what this costs. Contextual Retrieval adds a Claude call **per chunk** at index time —
cheap per token with caching, but real latency and spend when you're indexing millions of chunks, and it
has to re-run when documents change. My repro is a minimal reproduction of the *core mechanism* (context
lifts rank); the `35 / 49 / 67%` figures are Anthropic's, on their corpus. And my playground scores with a
TF-IDF cosine, not a real embedding model — it shows the *shape* of the effect, not production numbers.
## The takeaway
The move is almost embarrassingly simple: spend a cheap, cached Claude call per chunk to write down the
context a human would need to make sense of it, then index that. It attacks the failure at its source
instead of papering over it downstream, it helps lexical and semantic retrieval at the same time, and — as
the little repro above shows — you can watch the right chunk climb the rankings the moment the context goes
in.
---
*Source: [Introducing Contextual Retrieval](https://www.anthropic.com/engineering/contextual-retrieval)
(Anthropic). The prompt and the `35 / 49 / 67%` numbers are theirs; the repro and the browser playground are
mine, and the playground's scoring runs entirely client-side.*