When the Correct Answer Is Nothing, What Does Your Pipeline Return?

Benjamin Nweke described a failure that took him three weeks to notice. He had turned on Structured Outputs for a pipeline that parsed payment confirmation messages into transaction records, and his reconciliation job started flagging a small, steady stream of mismatches — a couple of percent of a week’s volume. Not crashes. Not malformed rows. The amount and the sender matched; the date was off.

When he put the raw messages next to the extracted records, the pattern was clean: every mismatched transaction came from a message that never mentioned a date at all. The schema declared transaction_date as required. The model could not return nothing, so it returned something — usually the date the extraction job happened to run.

Nothing in that pipeline was broken. The JSON was valid. The types checked. The only thing wrong was that a value existed where no value should have.

I recognised it immediately, because I had just published the same bug wearing different clothes.

The same failure, a different component

I maintain a small on-premise retrieval system as an open-source reference implementation: documents that can’t leave the perimeter, no GPU, an answer costing tens of seconds on CPU. Repeated questions about the same regulations are the obvious case for a cache, so I added a semantic one. If two questions mean the same thing, serve the stored answer and skip generation entirely. Cosine similarity between question embeddings, threshold at 0.92.

A reader asked me to prove the threshold was safe. It wasn’t, and I wrote up why — including the three errors I found in my own method along the way.

But the finding that matters here isn’t about embeddings. It’s about what the cache does when the correct answer is none of the above. A stored question and an incoming question can be near-identical strings with opposite correct answers — can staff with exclusive dedication teach at universities against without — and the cache has no way to return “this is a different question.” Its only outputs are a hit and a miss. There is no third option, and a hit skips both retrieval and the model.

So: a required schema field cannot say this isn’t in the source. A similarity cache cannot say this isn’t the same question. Two different components, two different teams, the same missing return value.

Reliability mechanisms work by removing degrees of freedom

Both of these components were added to make a system more reliable, and both succeeded at what they were added for. Structured Outputs eliminated an entire class of parsing failure. The cache eliminated a real latency cost.

They work the same way: by narrowing what the system is allowed to emit. A schema narrows the output to a fixed shape. A threshold narrows a continuous score to a binary decision. That narrowing is the point — it’s what makes the downstream code simple.

But one of the degrees of freedom that gets removed is abstention.

This is what makes the failure mode hard to catch. Constraint violations are loud by construction: a malformed response fails a parser, a type error fails a check, a missing key throws. A forced value violates nothing. It passes every gate the pipeline has, because every gate was built to check shape and the problem is provenance.

And the two mechanisms differ in a way that matters for triage. A required field that gets filled leaves a record you can inspect afterwards — Nweke found his by putting source messages next to extracted rows. A cache hit leaves a log line and nothing else. The model was never called, so there is no generation to compare against anything, and nothing in the trace distinguishes a correct hit from an incorrect one. The user sees a fluent, sourced, confident answer to a question they didn’t ask.

What the cache measurement actually showed

That embedding models handle negation badly is documented. NevIR finds most retrieval models, including state-of-the-art ones, rank at or below chance on document pairs that differ only by a negation. A 2019 biomedical study found the inversion directly, across sentence embedding models that predate the current transformer generation: mean cosine similarity ran higher for negation and antonym pairs than for sentences human experts rated as highly similar.

What I needed to know was narrower and operational: does a threshold exist that rejects the near-identical-but-opposite pairs while still accepting genuine rephrasings? To test it I built pairs of Spanish administrative questions by hand — not drawn from any query log — in categories: negations that must be rejected, and paraphrases that must be accepted.

The first version of that experiment was wrong, and the way it was wrong is the useful part.

My negation pairs differed by a single token: con against sin, incluye against excluye. My paraphrase controls differed by most of their tokens. So when the negations scored high and the paraphrases scored low, I had no way to distinguish an embedder failing at semantics from an embedder succeeding at surface form. A function that counted shared words would have produced the same table.

The fix was a second control population: paraphrases that differ by one token, the same surface distance as the negations, with the opposite required behaviour. Running the same nine negation pairs against both controls with bge-m3:

Two things in that figure are worth separating.

The first is that the uncontrolled comparison was confounded by lexical distance. Same embedder, same adversarial pairs, and the verdict moves from uninformative to nearly separable depending on the one variable the second control was built to change. The 0.3556 doesn’t reach significance on an exact permutation test (p=0.4376), so the honest reading of the top panel is that the score carries no usable information — not that it’s reliably inverted. The 0.9333 does reach it (p=0.0070, nine negations against five paraphrases per control).

The second is that “nearly separable” is not “separable.” The ordering is right — an AUC of 0.9333 says the model usually ranks a genuine paraphrase above a negation. But the lowest accepted pair still sits below the highest rejected one, so the margin is negative. The best available threshold costs one error out of fourteen, and it sits a billionth above the highest negation in the set, 0.96247: any lower at all and it admits a pair with the opposite correct answer. Drop the lowest paraphrase from the sample and the margin changes sign.

A ranking and a cut point are different things, and a cache needs the cut point. For the other embedder I tested, nomic-embed-text on the same Spanish pairs, the error-minimising configuration was the degenerate one that accepts nothing — no cache at all.

That is a statement about one hand-built corpus and two embedders, not about embedding models in general. It was enough to decide what ships.

Both fixes have holes

Nweke’s proposed fixes are the right shape. Making fields nullable removes the pressure to invent: if the date isn’t in the message, the model can now say so, and the decision about whether to infer it moves back into your code where it belongs. And asking the model for the exact source text backing each value gives a reviewer something concrete to check — if a value is filled but its evidence field is empty, the hallucination is visible in the data itself.

I want to note two things about that second pattern, not to score points but because they’re the kind of gap that survives review — including mine.

The ordering is load-bearing, and the published code doesn’t have it. The class declares value first and evidence second; the sentence directly beneath it explains that placing evidence first matters because keys generate in sequence — the model has to write down what it’s looking at before it commits to an answer. That’s the mechanism that makes the pattern more than decoration. Copy the class as shown and you get the audit trail without the forcing function: the model commits to a value, then writes the justification.

The retry loop is bypassed by its own trigger. The retry function calls the SDK’s parsing helper with response_format=Transaction, on a line that sits outside the try block. The except ValidationError wraps a second, separate validation of the raw response string. But the helper runs that same Pydantic validation internally, so a response that fails validation raises before control ever reaches the handler — and the second validation cannot fail if the first one passed. The loop that exists to recover from validation failures never runs on them.

Neither of these is an argument against the pattern. They’re an argument for the next section.

Why this is hard to see even when you’re looking for it

The retry loop is the easy case. One mocked invalid response and a test catches it. Bugs are tractable; that’s what makes them bugs.

My control and Nweke’s field ordering aren’t like that.

Mine was wrong for a day, and I would have defended it if someone had raised it — right up until I sat down and wrote out the two populations I was comparing. His ordering principle is stated correctly one sentence after code that doesn’t follow it.

Neither of us was careless. We were both actively working on exactly this problem, and in both cases the gap is between a stated principle and an implementation that doesn’t carry it — which no test you’d think to write reaches, because the code does exactly what it says it does. It just doesn’t do what you said it did.

That’s the reason this failure mode deserves a name rather than a fix. The fix is easy once you see the instance. Seeing the instance is the work.

Three questions per component

What I do now, for each component in a retrieval or extraction pipeline:

What does this return when the correct answer is nothing? Not what does it return on bad input — what does it return when the input is fine and the answer genuinely doesn’t exist. A required field returns a fabrication. A binary threshold returns the closer of two wrong options.

Can it emit absence at all? If the return type has no representation for “not found,” the component will fabricate under pressure, and no prompt engineering changes that. Nullable types, an explicit not_found sentinel, a third state alongside hit and miss.

Is there a signal when it happens? This is where the two cases diverge sharply. Nweke could reconstruct his by comparing sources to rows. A cache hit logs its similarity score and the original question, but nothing in that record separates a correct hit from an incorrect one. Mine ships disabled by default because I found no safe threshold; the absence of any signal that separates a bad hit from a good one is why I haven’t tried to tune my way out of that.

None of this is specific to LLMs, incidentally. It’s the same reasoning behind nullable columns and explicit sentinels in ordinary data modelling. What changed is that a component which fabricates under pressure is now fluent enough that the fabrication reads as an answer.

The part I haven’t solved

A cache that fails loudly instead of silently would solve most of this, and I don’t have a clean design for one. The obvious move — a confirmation step before serving a hit — is what I tested two cross-encoders for, and neither produced a usable cut point; one of them was significantly inverted on the English pairs.

The less obvious move is to change what gets cached. Caching retrieval results rather than generated answers keeps the model in the path: a near-miss degrades which documents get read rather than fabricating a response. That isn’t immunity — wrong documents are still wrong documents — but it moves the failure from invisible to visible, and visible failures are the ones you can build on.

That trade seems right to me and I haven’t measured it. Which is the honest place to stop.

···

References

Nweke, B. (2026). Your LLM Can Return Perfect JSON and Still Be Wrong. Towards Data Science. https://towardsdatascience.com/your-llm-can-return-perfect-json-and-still-be-wrong/

Weller, O., Lawrie, D., & Van Durme, B. (2024). NevIR: Negation in Neural Information Retrieval. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics, 2274–2287. https://aclanthology.org/2024.eacl-long.139/

Blagec, K., Xu, H., Agibetov, A., & Samwald, M. (2019). Neural sentence embedding models for semantic similarity estimation in the biomedical domain. BMC Bioinformatics, 20, 178. https://doi.org/10.1186/s12859-019-2789-2

The pairs, scripts, raw scores and the plotting code for the figure are in the experiment directory of the project repository: https://github.com/psychohub/rag-onpremise/tree/main/docs/experiments

···

This is a personal open-source project. It describes no institutional deployment and uses no data from any production system; the question pairs were written by hand for the experiment. Written in Spanish, self-translated, with Claude as a language editor; the experiments, the measurements and the mistakes are mine.