We swapped in a code-specialised embedder. Alone it measured 3.1 points better at finding the right file, p=0.004, surviving correction across eight comparisons. Run end to end, it produced nothing.
We counted instead of testing
Take every query the better embedder fixed. Sort those queries by what the pipeline did with the gain. Three buckets fall out: redundant, propagated, lost.
A third were lost. 33.3 percent against 2.2 percent, Fisher exact p=0.00041, odds ratio 22.
The fusion was eating it
We fused keyword search and meaning search by rank agreement. Both arms rank the candidates. A result scores well when both put it near the top.
That has a failure mode we never measured. A file only the meaning arm finds cannot reach the top 4, however confidently that arm ranks it. The keyword arm has to agree first. Our better embedder was better at exactly the files the keyword arm would never surface.
The variable that did not move
Across the three buckets the keyword rank of the correct file swings: median 0.5, then 8.5, then 27.0. Kruskal-Wallis H=16.53, p=0.00026.
The meaning rank stays flat. 2.0, then 2.5, then 2.0. H=0.12, p=0.943.
That flat row is a negative control. It is the variable that must not differ if the explanation holds.
The reranker never fired
We built a reranker for this case. It fired on 0 of the 8 lost queries.
It triggers when the meaning arm is unsure. The lost bucket is where the meaning arm is confident and the keyword arm disagrees. The rescue was gated by the inverse of the failure.
The fix
Retrieve a pool from both arms. Rerank the whole pool. Take the top 4.
Code retrieval goes 89.2 to 93.1 percent, p=0.007.
Then we ran it on LoCoMo, which shares no vocabulary with code. This is evidence recall, not QA accuracy. 0.675 evidence recall@10 on LoCoMo, on Estelle's own harness, where plain BM25 scores 0.496.
It generalised harder off-domain than on.
The subtraction
The control moved too. A random decoy rose 2.0 to 2.6 percent over the same change.
Netted against that drift the gain is +3.3 points, not +3.9. We quote the smaller one.
The conversation number needs no subtraction. +10.1 points is ours against ours, one harness, one set of questions, before and after. The 9x-more-expensive embedder was not adopted.
Why we trust the flat row
Because we have watched our own instruments lie.
In one night we found four inert. A contract test pinned to the response half, while the drift lived in
the request. A release check that asserted a tag existed, never that it was current. A function named
ground() returning 0 on every branch. A ceiling with 36 points of slack that nothing could reach.
All four were green. Each covered one half of a contract and went silent about the other. A broken test goes red and someone investigates. A test green about the wrong half looks exactly like safety.
So we prove an instrument can fail before we trust it. Break the thing it watches for, watch it go red, record that.
Our reviewer finds about a third of the defects we inject into real third-party commits, against a false-alarm floor of 2 to 5 percent replicated on three samples. That is not a good number. We publish it because we know which half it covers.
Why this exists
A graph tells you where the code is. It cannot refuse a claim that is not true.
Estelle runs a deterministic gate, zero model calls, that checks a change against the repository's real symbol graph and blocks an invented package or API before it merges. Retrieval finds the code. The gate decides whether what you were told about it is true.