back to the blog

blog/retrieval-throwing-away-its-own-gains.md
measurementretrievalmethod

Our retrieval was throwing away a third of its own gains

A better embedder measured 3.1 points better alone and produced nothing end to end. We found the stage eating it by counting, not testing. The proof was the number that did not move.

We swapped in a code-specialised embedder. Alone it measured 3.1 points better at finding the right file, p=0.004, surviving correction across eight comparisons. Run end to end, it produced nothing.

We counted instead of testing

Take every query the better embedder fixed. Sort those queries by what the pipeline did with the gain. Three buckets fall out: redundant, propagated, lost.

A third were lost. 33.3 percent against 2.2 percent, Fisher exact p=0.00041, odds ratio 22.

The fusion was eating it

We fused keyword search and meaning search by rank agreement. Both arms rank the candidates. A result scores well when both put it near the top.

That has a failure mode we never measured. A file only the meaning arm finds cannot reach the top 4, however confidently that arm ranks it. The keyword arm has to agree first. Our better embedder was better at exactly the files the keyword arm would never surface.

The variable that did not move

Across the three buckets the keyword rank of the correct file swings: median 0.5, then 8.5, then 27.0. Kruskal-Wallis H=16.53, p=0.00026.

The meaning rank stays flat. 2.0, then 2.5, then 2.0. H=0.12, p=0.943.

That flat row is a negative control. It is the variable that must not differ if the explanation holds.

The reranker never fired

We built a reranker for this case. It fired on 0 of the 8 lost queries.

It triggers when the meaning arm is unsure. The lost bucket is where the meaning arm is confident and the keyword arm disagrees. The rescue was gated by the inverse of the failure.

The fix

Retrieve a pool from both arms. Rerank the whole pool. Take the top 4.

Code retrieval goes 89.2 to 93.1 percent, p=0.007.

Then we ran it on LoCoMo, which shares no vocabulary with code. This is evidence recall, not QA accuracy. 0.675 evidence recall@10 on LoCoMo, on Estelle's own harness, where plain BM25 scores 0.496.

It generalised harder off-domain than on.

The subtraction

The control moved too. A random decoy rose 2.0 to 2.6 percent over the same change.

Netted against that drift the gain is +3.3 points, not +3.9. We quote the smaller one.

The conversation number needs no subtraction. +10.1 points is ours against ours, one harness, one set of questions, before and after. The 9x-more-expensive embedder was not adopted.

Why we trust the flat row

Because we have watched our own instruments lie.

In one night we found four inert. A contract test pinned to the response half, while the drift lived in the request. A release check that asserted a tag existed, never that it was current. A function named ground() returning 0 on every branch. A ceiling with 36 points of slack that nothing could reach.

All four were green. Each covered one half of a contract and went silent about the other. A broken test goes red and someone investigates. A test green about the wrong half looks exactly like safety.

So we prove an instrument can fail before we trust it. Break the thing it watches for, watch it go red, record that.

Our reviewer finds about a third of the defects we inject into real third-party commits, against a false-alarm floor of 2 to 5 percent replicated on three samples. That is not a good number. We publish it because we know which half it covers.

Why this exists

A graph tells you where the code is. It cannot refuse a claim that is not true.

Estelle runs a deterministic gate, zero model calls, that checks a change against the repository's real symbol graph and blocks an invented package or API before it merges. Retrieval finds the code. The gate decides whether what you were told about it is true.

methodscripts/diagnose_fusion_loss.py and scripts/compare_embedder_arms.py
measured
Whether the right file reaches the top 4 for a natural-language query, across four retrieval configurations, and where a measured single-stage gain is lost.
corpus
awesome-llm-apps (583 queries) for code, and LoCoMo, a public conversational dataset sharing no vocabulary with code. Neither is ours.
n
583 code queries. The fusion-loss partition is 27 queries, 10 redundant, 8 propagated, 9 lost.
control
The dense rank of the gold file across the three buckets. It had to stay flat if the story was true. It did (median 2.0 / 2.5 / 2.0, p=0.943). For pool-rerank, a random decoy that rose 2.0 to 2.6 percent alongside the intervention.
not shown
UPDATED 13 August 2026: pool-rerank was measured on eval arms and NOT in production when this was published on 12 August. Pool-20 and the payload cap shipped and were verified on prod on 13 August, so the paid tiers now run it. Standing limits: lost against propagated alone is underpowered (p=0.081, n=9 against 8). Two corpora, one retrieval task. Says nothing about generation quality.