{"title":"Contextual Retrieval, with a runnable repro and a browser playground","description":"Anthropic's Contextual Retrieval fixes the oldest RAG bug — chunks that lose their context — by having Claude write a one-line context for each chunk before you index it. I built a dependency-free repro that reproduces the ranking lift, plus an in-browser playground so you can watch the right chunk climb.","date":"2026-07-17","tags":["rag","retrieval","claude","embeddings","playground"],"draft":false,"kind":"blog","slug":"contextual-retrieval","body":"[Anthropic's Contextual Retrieval post](https://www.anthropic.com/engineering/contextual-retrieval)\nhas been on my to-read list for a while. It targets the oldest bug in RAG, so I finally sat down,\nbuilt a dependency-free reproduction, and wired up a browser playground. This is the write-up.\n\n## The problem: chunks forget where they came from\n\nStandard RAG splits documents into chunks and indexes each chunk on its own. That destroys context.\nAnthropic's example is perfect:\n\n> `The company's revenue grew by 3% over the previous quarter.`\n\nWhich company? Which quarter? The chunk can't say. A query like *\"how did ACME's Q2 revenue change?\"*\nhas **no terms to match** against that chunk — the words \"ACME\" and \"Q2 2023\" live in the *document*,\nnot the *chunk*. Both lexical (BM25) and semantic (embedding) retrieval miss it.\n\n## The fix: let Claude situate each chunk\n\nContextual Retrieval prepends a short, chunk-specific context to each chunk **before** you embed it and\n**before** you build the BM25 index — Anthropic calls these **Contextual Embeddings** and **Contextual\nBM25**. The context is generated by Claude, given the whole document. The same chunk becomes:\n\n> `This chunk is from an SEC filing on ACME corp's performance in Q2 2023; the previous quarter's revenue was $314 million. The company's revenue grew by 3% over the previous quarter.`\n\nNow the owner and the period are *in the chunk*, so retrieval can find it.\n\n## Watch it happen\n\nHere is the whole pipeline in your browser — the document, its chunks, the chunks with context prepended,\nthen retrieval. The scoring is a real BM25 and a TF-IDF cosine (a stand-in for an embedding model); the\ncontextual prefixes stand in for Claude's output. Step to **4 · retrieve**, pick a query, and watch the\nright chunk climb from the standard column to the contextual one:\n\n<ContextualRetrievalPlayground />\n\nToggle between BM25, embeddings, and their fusion. The pattern holds across all three: the answer chunk\nis buried on standard chunks and near the top once each chunk carries its context.\n\n## The one prompt that does the work\n\nThis is the prompt from the post, verbatim. It runs once per chunk, with the full document supplied as\ncontext:\n\n```text\n<document>\n{{WHOLE_DOCUMENT}}\n</document>\nHere is the chunk we want to situate within the whole document\n<chunk>\n{{CHUNK_CONTENT}}\n</chunk>\nPlease give a short succinct context to situate this chunk within the overall document for the purposes of improving search retrieval of the chunk. Answer only with the succinct context and nothing else.\n```\n\nThe obvious worry is cost: you re-send the whole document once per chunk. **Prompt caching** kills that.\nCache the document once and every chunk in it reads from the cache, which is why Anthropic quotes a\none-time **$1.02 per million document tokens**. The document is the stable prefix, so it takes the\n`cache_control` breakpoint; the chunk and instruction vary and come after it:\n\n```python\nfrom anthropic import Anthropic\n\nclient = Anthropic()\n\nCONTEXT_PROMPT = \"\"\"Here is the chunk we want to situate within the whole document\n<chunk>\n{chunk}\n</chunk>\nPlease give a short succinct context to situate this chunk within the overall document \\\nfor the purposes of improving search retrieval of the chunk. Answer only with the \\\nsuccinct context and nothing else.\"\"\"\n\ndef situate(doc: str, chunk: str) -> str:\n    resp = client.messages.create(\n        # One Claude call per chunk. For a large index you'd typically drop to a\n        # cheaper model like claude-haiku-4-5 — which is what Anthropic's ~$1.02 /\n        # 1M-doc-token estimate assumes — trading a little context quality for cost.\n        model=\"claude-opus-4-8\",\n        max_tokens=200,\n        messages=[{\n            \"role\": \"user\",\n            \"content\": [\n                # The whole document is the stable prefix: cache it ONCE, then every\n                # chunk of this document reads from the cache instead of re-paying for it.\n                {\"type\": \"text\",\n                 \"text\": f\"<document>\\n{doc}\\n</document>\",\n                 \"cache_control\": {\"type\": \"ephemeral\"}},\n                {\"type\": \"text\", \"text\": CONTEXT_PROMPT.format(chunk=chunk)},\n            ],\n        }],\n    )\n    return \"\".join(b.text for b in resp.content if b.type == \"text\").strip()\n```\n\n<Callout type=\"tip\">\nVerify the cache is actually working: `resp.usage.cache_read_input_tokens` should be non-zero on every\nchunk after the first for a given document. If it's zero, something upstream is mutating the document\nbytes (a timestamp, non-deterministic JSON) and invalidating the prefix.\n</Callout>\n\n## A dependency-free repro\n\nTo convince myself the lift was real and not marketing, I wrote a ~180-line pure-stdlib script: naive\nchunking, BM25 from scratch, a TF-IDF cosine as an embedding stand-in, and reciprocal rank fusion. The\nretrieval core is small — here is BM25 and the fusion step:\n\n```python\nclass BM25:\n    def score(self, query, i):\n        dl, tf, s = len(self.docs[i]), self.tf[i], 0.0\n        for t in query:\n            if t not in tf:\n                continue\n            num = tf[t] * (self.k1 + 1)\n            den = tf[t] + self.k1 * (1 - self.b + self.b * dl / self.avgdl)\n            s += self.idf(t) * num / den\n        return s\n\ndef rrf(rankings, k=60):  # reciprocal rank fusion of BM25 + embedding rankings\n    scores = Counter()\n    for ranking in rankings:\n        for rank, item in enumerate(ranking):\n            scores[item] += 1 / (k + rank + 1)\n    return [i for i, _ in scores.most_common()]\n```\n\nThen I index the chunks two ways — plain, and with a one-line context prepended — and measure where the\ncorrect chunk lands for a couple of \"which entity, which period\" queries. The actual output:\n\n```text\nquery                                                method   plain rank  ctx rank\n------------------------------------------------------------------------------------\nHow did ACME Corp revenue change in Q2 2023?         bm25              4         2\nHow did ACME Corp revenue change in Q2 2023?         emb               4         1\nHow did ACME Corp revenue change in Q2 2023?         hybrid            4         2\n\nWhat happened to Beta Industries revenue in Q3 2023? bm25              5         2\nWhat happened to Beta Industries revenue in Q3 2023? emb               5         1\nWhat happened to Beta Industries revenue in Q3 2023? hybrid            5         2\n\nrecall@1 (fraction of queries where the right chunk ranks #1):\n  bm25    plain 0/2   contextual 0/2\n  emb     plain 0/2   contextual 2/2\n```\n\nContextualizing the chunks moves the answer from rank **4–5** to rank **1–2** across BM25, embeddings, and\ntheir fusion — and embeddings recall@1 goes from **0/2 to 2/2**. On this toy corpus fusion lands the answer\nat #2 rather than #1 (a small-N artifact of RRF), which is a good reminder that the fusion win is an\n*aggregate* effect — which is exactly what Anthropic's real evaluation measures.\n\n## What the real evaluation found\n\nOn Anthropic's benchmark (top-20 retrieval failure rate, i.e. `1 - recall@20`), stacking the techniques\ncompounds:\n\n- **Contextual Embeddings** alone: `5.7% → 3.7%` — a **35%** cut in the failure rate.\n- **+ Contextual BM25**: `5.7% → 2.9%` — **49%**.\n- **+ reranking**: `5.7% → 1.9%` — **67%**.\n\nThe lexical half matters more than you'd guess — BM25 nails exact identifiers (error codes, ticker symbols,\nfunction names) that embeddings smear together, so contextualizing *both* indexes and fusing them beats\neither alone.\n\n## Things worth copying from the post\n\n- **Retrieve top-20, not top-5/10.** Anthropic found 20 the most performant cut for the final context.\n- **Rerank the shortlist.** Retrieve ~150 candidates, then rerank down to 20 for the answer prompt — that's\n  the step that takes the failure rate from 2.9% to 1.9%.\n- **Embedding model matters.** Gemini and Voyage embeddings were the standouts in their tests.\n- **Chunking still matters.** Size, boundary, and overlap all move the numbers — Contextual Retrieval sits\n  on top of good chunking, it doesn't replace it.\n- **A domain-tuned context prompt beats the generic one.** The template above is a floor, not a ceiling.\n\n<Callout type=\"warn\">\nBe honest about what this costs. Contextual Retrieval adds a Claude call **per chunk** at index time —\ncheap per token with caching, but real latency and spend when you're indexing millions of chunks, and it\nhas to re-run when documents change. My repro is a minimal reproduction of the *core mechanism* (context\nlifts rank); the `35 / 49 / 67%` figures are Anthropic's, on their corpus. And my playground scores with a\nTF-IDF cosine, not a real embedding model — it shows the *shape* of the effect, not production numbers.\n</Callout>\n\n## The takeaway\n\nThe move is almost embarrassingly simple: spend a cheap, cached Claude call per chunk to write down the\ncontext a human would need to make sense of it, then index that. It attacks the failure at its source\ninstead of papering over it downstream, it helps lexical and semantic retrieval at the same time, and — as\nthe little repro above shows — you can watch the right chunk climb the rankings the moment the context goes\nin.\n\n---\n\n*Source: [Introducing Contextual Retrieval](https://www.anthropic.com/engineering/contextual-retrieval)\n(Anthropic). The prompt and the `35 / 49 / 67%` numbers are theirs; the repro and the browser playground are\nmine, and the playground's scoring runs entirely client-side.*\n","readingTimeMins":7,"url":"https://ai.thesatyajit.com/blog/contextual-retrieval"}