of this series (Designing a Persistent Knowledge Layer That Refuses to Guess), I built a persistent knowledge layer next to a RAG pipeline: an evidence layer for grounding, a structured knowledge layer for accumulated understanding, and an orchestrator that decided which of the two a question needed. I deployed it on Azure, seeded it with a synthetic insurance corpus engineered to break retrieval systems in six specific ways, and measured what happened.
The architecture held up. The contradiction register refused to answer a question the firm itself had not settled. The effective-date scoping kept a rule that took effect on 1 March from being applied to a loss dated 20 February. The live model walked a four-hop reasoning chain with citations at every step.
But I ended that article with a list of things I would build next, and two of the entries on that list have not left me alone since.
The first was a number. When I ran all twenty-one documents through live extraction on the deployed stack, the model proposed concepts that my alias-based entity resolution could only partially merge. The result was 149 concept objects where the curated graph has 19 – a roughly 7x fragmentation factor. I wrote at the time that this was “the concrete argument for embedding similarity and LLM adjudication in the resolution chain.” It is. But sitting with the number longer, I realized it argues for something bigger. The knowledge layer I built stores relationships, but nothing in the system actually traverses them at retrieval time. The graph is consulted like a filing cabinet, not walked like a graph.
The second was an incident, not a number. During live validation I asked the deployed system an unscripted question — why a particular claim had been triaged the way it was, and my first phrasing got intercepted by the contradiction gate, because the claim’s evidence brushed against a contested concept. The gate did its job. But the routing in front of the gate had made a judgment call about how to retrieve, based on keyword markers in my question, and it occurred to me that I had built a system whose retrieval quality depended on how a question happened to be worded.
This article is about fixing both. It is the result of the design work I did after previous work was shipped, and it changes my own architecture in a way I want to be upfront about: the query router from Part 1, the wiki-first, evidence-first, hybrid decision I drew as a flowchart and defended in prose is actually the component I am retiring. Not because it was wrong for what Part 1 was doing, but because I now think there is a better answer to the question it was trying to answer, and the better answer also unlocks three capabilities the router never could: real multi-hop traversal, time-aware relationships, and contradictions that are discovered instead of curated.
I will keep the previous approach, and present it in three parts, the same way as before.
Part I is the design revision. Vendor-neutral, like last time. It covers the principle that replaces the router (retriever versus filter), the always-fused retrieval pipeline, bitemporal edges, ingest-time contradiction detection, and the entity-resolution subsystem that everything else stands on.
Part II grows the Azure stack. The resource group from Part 1 is still running, and it turns out to contain most of what this design needs — including a change feed I provisioned and never used. One new engine joins it.
Part III walks the insurance corpus through the new design — live. The same Ostermere Mutual documents, the same contradiction, the same claims — and the questions the new architecture can answer that the old one could not. The implementation is deployed on a fresh Azure resource group, and every request and response quoted in Part III was actually sent to and returned by that system, the same standard Part 1 held itself to.
What this article deliberately defers: the reflection loop that derives higher-level insights, the full write-path architecture in production depth, the agent-tool layer, and the governance backlog. Each of those deserves its own treatment, and stacking them here would produce a survey instead of an argument.
Everything in the dataset is synthetic. Ostermere Mutual does not exist. Neither does the regulator, the policy, the claims, the people, the wind zones or the figures. Nothing here is insurance, legal, underwriting or claims advice.
Naming note: current Microsoft documentation uses Microsoft Foundry for the platform previously called Azure AI Foundry. As in Part 1, I use Foundry throughout.
The complete Part 1 project (application, infrastructure, dataset and the generated Obsidian vault) is available at github.com/mcekikj/persistent-knowledge-layer, under the MIT license — that repository stays frozen as the artifact Part 1 describes. The Part 2 implementation lives in its own repository at github.com/mcekikj/knowledge-graph-fusion.
Contents
Part I – The design revision
- Where Part 1 Ended, Honestly
- The Router Was the Weak Link
- Retriever versus Filter
- Always Fused, Then Grounded
- Time Moves Inside the Edges
- Contradictions: From Curated to Discovered
- Entity Resolution: The Subsystem the Graph Stands On
- One Truth, Projected
- The Ablation Is the Acceptance Test
- When Routing Is Still the Right Answer
Part II – Growing the Azure Stack
- What Is Already Running
- The Second Engine: Cosmos DB for Apache Gremlin
- Teaching the Index About the Graph
- The Projection Worker, and the Ontology Ostermere Already Has
- Reranking the Union
- What the Additions Cost
Part III – The Corpus, Traversed
- The Multi-Hop Question, Answered Properly
- The Question Part 1 Could Not Ask
- Would the System Have Found con-001 on Its Own?
- To Sum It All Up
References
Part I – The design revision
1. Where Part 1 ended, honestly
A quick recap for readers joining here, and an honest accounting for readers returning.
Part 1’s architecture had three layers. An evidence layer (chunks, vectors, full text in a search index) answered what source material is relevant right now. A knowledge layer (concepts, typed relationships, decisions, comparisons, contradictions in a document store) answered what has the system already worked out. An orchestrator sat in front of both and decided, per question, which layer to consult: wiki-first for definitions and prior decisions, evidence-first for exact wording, hybrid for cross-source synthesis.
Three things about that design were validated live and survive unchanged into this article:
- The contradiction gate. When retrieval touches a concept that two current documents genuinely disagree about, the system returns both positions with their sources and effective dates, names the accountable owner, and does not answer. This is still, in my view, the single most valuable behavior in the whole design, and nothing in this article weakens it.
- Effective-date scoping. A question about a claim is answered against the documentation in force on the date of loss, not the documentation in force today.
- Provenance. A knowledge page is never a source; it is always traceable to one.
And three things were weaker than the article’s prose, which I said at the time and can now do something about:
- Entity resolution stopped at alias matching, and the live deployment measured the consequence: 149 machine-extracted concepts against 19 curated ones.
- Relationships were stored but not traversed. The wiki search ranked objects by term overlap, the 28 typed relationships in the corpus were rendered on pages and walked by humans in Obsidian, but no query ever followed an edge.
- The router decided how to retrieve based on keyword markers in the question — and a system whose retrieval strategy depends on question wording has a soft spot exactly where it should be most dependable.
2. The router was the weak link
The router made sense when I built it. Not every question needs both layers, and sending everything everywhere looked wasteful. So the orchestrator inspected the question for markers — “compare” and “why” suggested synthesis, “exact” and “quote” suggested evidence, and picked a mode.
The problem is not that the heuristics were crude, although they were, and I said so at the time. The problem is structural, and it took the live deployment to make me see it: the router makes retrieval quality a property of the question’s wording instead of a property of the system.
Two users asking for the same fact with different phrasings could take different paths through the system and get differently-grounded answers. A question that deserved relational context would not get it unless it happened to contain a synthesis marker. And when I imagined replacing my keyword heuristics with a classifier, or with the model itself choosing a retrieval tool, the structural problem stayed exactly where it was — I would have been delegating the same judgment call to a smarter judge, when the design question was whether the call should exist at all.
There is also a quieter cost that I only appreciated when I started designing evaluation for this system. A router multiplies the space you have to test. Every retrieval quality question becomes three questions. Did it work in wiki mode, in evidence mode, in hybrid mode, and every failure investigation starts with “which path did this take?” A fixed pipeline is deterministic, and deterministic things are testable, abatable, and cheap to reason about.
So the revision starts from a different premise: run everything, always, and let the results compete on evidence. If a question has no useful relational context, the traversal stage contributes little and costs little. If it has no strong semantic matches, the graph anchors carry it. The system does not guess which case it is in, instead, it finds out.
That premise immediately raises a fair objection regarding does “run everything” mean every storage technology earns a place in the pipeline? That is the next section, and the answer is no.
3. Retriever versus filter
Any system that remembers things eventually faces three shapes of question.
What is similar to X – a similarity question.
What is connected to X – a relationship question.
What was true about X in March, and what changed – a time question.
Each shape maps naturally onto a different storage technology — vector indexes, graph databases, time-series stores — and the tempting conclusion is that a serious knowledge system needs all three, wired together behind an orchestrator that fires parallel sub-queries and intersects the results. Architectures like that exist. They work. They also front-load enormous cost. Three engines to operate and secure, cross-store consistency machinery, and a merge problem, fusing rankings from engines whose scores are not comparable. That is genuinely hard to get right. And the parallel-query-and-intersect pattern quietly drops good answers: a document that is a perfect semantic match but falls just outside a time window vanishes from the intersection entirely.
The test that cuts through this is one question per modality: does it produce ranked evidence that must be fused, or does it narrow and reweight someone else’s ranking?
- Semantic and keyword search produce ranked evidence. Retriever.
- Time, in this system, never ranks anything on its own. “In force on the date of loss” and “prefer current over superseded” are filters and boosts applied to a relevance ranking. Part 1 already treated time this way — the as_of parameter constrained candidates, it never ordered them. Time is a filter.
- The graph sits in between. It does not rank by itself, but it does something no filter does. It expands. It turns one relevant hit into a connected neighborhood, and the paths it returns carry a relational signal that should influence the final ranking.
A modality that filters does not earn an engine, it earns a field and a scoring function. A modality that expands and contributes evidence earns an engine only because the primary engine cannot do the job — and search services genuinely cannot traverse.
Applied to this system, keyword, semantic, and time-as-recency all fit inside one search service. The graph earns a second engine. A time-series database earns nothing, unless the day comes when time produces ranked evidence of its own, trend and anomaly signals, which for a knowledge layer over policy documents may be never.
Two engines, not three. And one commitment that outlives every engine choice: a unified interface. One query object in, one result contract out, and the caller never learns which store answered. That is what lets backends be added, swapped, or removed as query patterns prove them, without anything above the interface changing.
4. Always fused, then grounded
With the router gone and the engines chosen, retrieval becomes one fixed pipeline that every query runs, top to bottom, no modes:
A few of these steps deserve a closer look, because each one fixes a specific weakness from Part 1.
Step 1 carries anchors, not just text. Each chunk in the search index stores the IDs of the entities it mentions. A hybrid search result therefore already knows where it enters the graph, no bridging lookup, no name matching at query time. This one schema decision is what makes two engines feel like one system.
Step 2 is mandatory, and that is the point. In most graph-augmented designs, traversal is an option, but a mode a router selects or a tool an agent may call. Here it is a fixed stage. When the graph has nothing useful near a query, it contributes little and the search results carry the answer. In this case, the pipeline degrades gracefully instead of branching. And because traversal always runs, its contribution is measurable — you can ablate it, which becomes important in section 9.
Step 4 is one reranker over the whole union. This matters because search scores and graph proximity are not comparable numbers, and gluing two ranked lists together by interleaving is a coin flip dressed as engineering. A single reranking stage that sees both the semantic evidence and the relational paths can do what neither ranking could alone, letting a connected-and-relevant object outrank a slightly-more-similar-but-isolated one.
Step 7 is where Part 1’s gate lives on. Grounding assembles the final bundle: the ranked objects, the relationship paths that connect them, the citations, and, unchanged from Part 1, the contradiction warnings. Fusion replaces the router. It does not replace the gate. If the bundle touches a contested concept, the system still presents both positions, names the owner, and declines to answer as though the matter were settled.
The model, in other words, never receives a pile of disconnected snippets. It receives a small, ranked, connected, attributed subgraph rendered as context. Its job shifts from inventing connections to interpreting documented ones, which is precisely the job a language model is good at, and precisely the shift that keeps grounded answers grounded.
One implementation note for the skeptical reader: when a query clearly names a concrete anchor (“everything connected to claim CLM-1042”), the orchestrator may invert the order internally — traverse first, then push the connected IDs down as a filter into the vector query. That is a performance optimization, invisible at the interface. There is deliberately no search-only or graph-only mode for a caller to select, because the moment such a mode exists, something upstream has to decide when to use it, and that something is a router wearing a new name.
5. Time moves inside the edges
Part 1 handled time with a single mechanism: sources carried effective dates and supersession links, and the as_of parameter filtered retrieval to the documentation in force on a given date. It worked, for example it kept a March rule from being applied to a February loss, but it worked at the granularity of documents. The knowledge layer’s relationships had no time on them at all. An edge either existed or it did not.
The revision moves time to where the knowledge actually lives: every relationship edge carries a validity window, valid_from, valid_to (open-ended if current), alongside the timestamp when the system learned the fact. Event time and ingest time, both recorded that the model bitemporal databases have used for decades, applied to a knowledge graph’s edges.
When a fact changes, the old edge is not rewritten. It is expired — its valid_to is closed, and a new edge opens. History is preserved by construction, which is the same non-destructive stance Part 1 took toward superseded documents, now applied to individual relationships.
Three question shapes fall out of this model almost for free, and none of them was answerable in Part 1:
- As-of: which inspection rule applied on 20 February 2026? Traverse only edges valid at that timestamp. Part 1 could filter documents this way, now the relationships themselves answer.
- Timeline: how did the roof inspection requirement evolve? The ordered sequence of an entity’s edges by validity window, from the 2025 general rule, through the January analysis email, to the March H3 threshold.
- Diff: what changed in underwriting guidance this quarter? Edges whose validity opened or closed inside a window as added, superseded, expired.
For an insurance knowledge base, these are not exotic queries. They are the questions a claims professional, an auditor, and a compliance officer respectively ask every week. Part 1 could tell you what was in force on a date. This design can tell you what changed, when, and show its work.
6. Contradictions: from curated to discovered
I have to be honest about how con-001 — the trace-and-access contradiction that anchors Part 1’s strongest demo — got into the system. I put it there. The corpus was engineered so the conflict existed, and the contradiction object was curated into the knowledge store with its statements, its owner, and its status. The gate was real and automatic, the detection was me.
Bitemporal edges make detection automatic, and the mechanism is almost embarrassingly simple once the edges carry time. At ingest, after a new fact is extracted and its entities are resolved, check it against the currently-valid edges on the same subject:
The policy in the middle is where the judgment lives, and it inherits Part 1’s governance stance wholesale. A clear supersession, a newer version of the same rule from the same authority expires the old edge automatically, because that is versioning, not disagreement. A genuine conflict where two current sources, different authorities, incompatible statements is surfaced, recorded as an unresolved contradiction object with an accountable owner, and never resolved by the machine. Recency is not applicability; Part 1’s rule, now enforced at write time instead of curation time.
What I find most satisfying about this design is that the detection costs no new infrastructure and no new concepts. It is a policy applied to machinery the system already has: typed edges, validity windows, and the contradiction object Part 1 already defined. The gate that made Part 1’s best demo work does not change at all, it just stops depending on me to load it.
7. Entity resolution: the subsystem the graph stands on
Now the unglamorous part, and the reason this article exists at all: 149 concepts where the curated graph has 19.
Every promise made so far — traversal that expands a hit into a neighborhood, edges that carry time, contradictions detected on the same subject, rests on one assumption that “the same subject” is one node. If “Actual Cash Value,” “ACV,” “cash settlement basis,” and “depreciated value” become four nodes, traversal fragments, timelines split across duplicates, and contradiction detection never fires because the conflicting facts sit on different vertices. Over-merging is worse. Collapse two genuinely different concepts and every path through the merged node is corrupted.
Part 1 resolved entities by alias matching — exact ID, then title, then a declared alias list. It handled the four ACV synonyms because I had declared them. The live deployment then demonstrated, at a 7x rate, what happens with synonyms nobody declared.
The revision treats resolution as a first-class pipeline with a shape I want to describe precisely, because the shape is what controls the cost:
The two-threshold decision is the heart of it. High-confidence matches link automatically. Clear non-matches create a new entity. Only the gray zone in between, and the width of that zone is a tunable cost lever, goes to the model for adjudication, with the mention, its local context, and the candidate’s profile in the prompt. In Part 1’s live run, alias matching alone would have sent essentially every undeclared synonym to “create new,” which is exactly the 149 we measured. Embedding similarity in the blocking and scoring stages is what catches “cash settlement basis” without anyone having declared it.
And because resolution will sometimes be wrong, merges are modeled as reversible same_as redirect edges rather than destructive rewrites — the same supersede-don’t-overwrite stance as everywhere else in the design. A wrong merge is an edge you can expire, not a corruption you have to excavate.
The live deployment put a first number on this pipeline. The identical corpus that fragmented into 149 concepts under Part 1’s alias, only resolution produced 120 under the new resolver, a twenty percent reduction, measured like-for-like on the same documents. Not a solved problem since 120 is still six times the canonical 19, and the residue is extraction phrasing that no similarity threshold can bridge (section 19 shows a concrete case). But the mechanism moves the number in the right direction, and now there is a number to move.
One operational lesson from the same run, filed under things a diagram never shows: the blocking stage’s candidate embeddings must be cached. My first live seed re-embedded every canonical name for every extracted mention, where mentions times entities instead of entities, and turned a four-minute seed into a stall I had to kill. One embedding per entity, cached forever, restored the arithmetic. The kind of bug that is obvious the moment you say it out loud, and invisible until you deploy.
One recommendation I will repeat in the implementation notes: prototype this subsystem early and measure it (precision and recall against a labeled set) before committing to the extraction pipeline at scale. Part 1 accidentally produced the perfect labeled starting set: 149 machine-extracted concepts whose correct disposition against the 19 canonical ones is known. Resolution quality is the leading indicator of graph quality, and therefore of everything the graph promises. If this number is bad, nothing downstream can be good.
8. One truth, projected
Retrieval is the read side. A short section on the write side, because the graph has to come from somewhere, and because Part 1’s stack contains a mechanism I provisioned and never used.
The design is a read/write split. One document store is the durable system of record — every source, every wiki object, verbatim, with its dates. The two retrieval stores, which are the search index and the graph, are derived projections. A change feed on the system of record drives a projection worker that updates both. One authoritative write, projections follow, accordingly.
This one move resolves the problem that sinks most multi-store designs (keeping the stores consistent) by never asking them to be independently written. There is no dual-write to get wrong and no reconciliation job to babysit. The cost is eventual consistency measured in seconds, which for a knowledge layer over policy documents is an easy trade.
I am keeping this section deliberately short. The production depth of the write path, where we have the idempotent projections, replay, poison handling, rebuild procedures, deserves its own article, and this one has a different spine. What matters here is the shape: verbatim record first, projections derived, nothing dual-written.
9. The ablation is the acceptance test
Part 1 earned whatever credibility it has by deploying and measuring instead of asserting. This design commits to the same standard, and it names the experiment that could kill its centerpiece.
The always-fused pipeline makes one strong claim that graph traversal improves retrieval, where the bundle with relational expansion beats the bundle without it. Because traversal is a fixed stage rather than a routed option, that claim is directly testable: run the same golden query set through the full pipeline and through a search-only variant with the traversal stage disabled, and compare.
- A golden query set over the corpus, semantic questions, entity-anchored questions, relational multi-hop questions, temporal questions, and out-of-scope questions that should return nothing, each labeled with the objects the bundle must contain, and for relational queries, the paths it must surface.
- Recall and ranking metrics on the grounded bundle, before the model ever sees it — did the needed evidence make it in, and near the top?
- The fused-versus-search-only ablation as the acceptance test for the graph itself. If fusion does not measurably improve recall or relational-path correctness, the graph is not earning its cost, and the fix goes into extraction and resolution before anything expands.
- Resolution precision and recall tracked as a first-class metric, because section 7 is the leading indicator for everything else.
A design that names the experiment that could falsify it is a design you can check. So here is the first run of that experiment, on the live deployment, with a small golden set of eight questions and sixteen expected evidence items:
| Metric | Fused | Search-only |
| Recall of expected items | 0.75 | 0.75 |
| Relationship paths per bundle (avg) | 10.0 | 0 |
The honest reading: on recall, at this corpus scale, the ablation is inconclusive, where twenty-one documents are small enough that hybrid search alone already reaches what it is going to reach, and the graph’s recall contribution cannot show up. What the graph measurably adds at this scale is the second row, where an average of ten documented, time-valid relationship paths per bundle that search-only retrieval cannot produce at all. The recall half of the acceptance test needs a corpus large enough for entry-point misses to occur, and the path-correctness half needs the labeled golden set grown well beyond eight questions. Both are queued. The design said the ablation decides whether the graph earns its cost; the first measurement says the cost is currently earned by grounding quality, not recall, and says it with numbers rather than enthusiasm.
10. When routing is still the right answer
Part 1 had a section titled “when you should not build this,” and I would rather keep that habit than acquire a religion.
Keep a router, or plain single-mode retrieval, when the corpus has no meaningful relational structure to traverse; when questions arrive in one dominant shape (“find me the paragraph”) and synthesis is rare; when latency budgets are so tight that a traversal stage plus a union rerank cannot be afforded even at one to two hops; or when there is no capacity to build and measure entity resolution, because an always-fused pipeline over a fragmented graph amplifies noise on every single query instead of only on routed ones. Fusion makes the graph’s quality everyone’s problem. That is a feature exactly when you are prepared to make the graph good.
This design targets the corpus scale of this series with tens of documents today, plausibly tens of thousands of objects for an early production system. The always-fused commitment and the two-engine topology deserve re-examination at a scale where traversal fan-out and rerank token costs dominate. At that point selective traversal stops being a router smell and becomes an engineering necessity. The interface is designed so that change would be invisible to callers which is the whole argument for the interface.
Part II – Growing the Azure stack
11. What is already running
Everything above is deliberately portable. Now the part where continuity pays off, because Part 1 did not end as a diagram, but as a running resource group, and this design was shaped partly by looking at what that resource group already contains.
| Already deployed (Part 1) | Role in this design |
| Azure AI Search (free tier) | The retriever – hybrid BM25 + vector + RRF; gains an entity-anchors field and an entities index |
| Cosmos DB for NoSQL | Already the knowledge store – becomes formally the system of record, and its change feed (provisioned all along, never consumed) becomes the projection backbone |
| Microsoft Foundry (gpt-5-mini + embeddings) | Extraction with structured outputs, resolution adjudication, and the new union reranker |
| Container Apps + FastAPI | The unified interface and orchestrator – the pipeline of Figure 3 replaces the router inside KnowledgePipeline |
| Blob Storage, Key Vault, managed identity, App Insights | Unchanged – the no-keys identity model extends to the new pieces |
One new engine joins, one worker process, and two index changes. That is the entire delta, and it is worth pausing on why it is so small: the retriever-versus-filter test kept time out of a third engine, and the read/write split reuses a change feed the stack already had. Architecture that grows by one engine at a time is architecture you can keep operating.
12. The second engine: Cosmos DB for Apache Gremlin
The graph needs an engine because the one thing a search service cannot do is traverse. Within the Azure boundary, staying inside the identity model and the operational habits Part 1 established, the natural fit is Cosmos DB for Apache Gremlin — the graph API of the same database family already running as the system of record.
What the graph stores, in this design:
- Entity vertices – the canonical concepts of the knowledge layer: roof-age-risk, trace-and-access, high-wind-zone-h3. One vertex per resolved entity, which is why section 7 gates everything.
- Object vertices – lightweight references to sources, decisions, and contradictions, keyed to the same IDs the search index and the record use.
- Typed edges with validity windows – narrows_rule_for, supersedes, contradicts, applies_to, each carrying valid_from, valid_to, ingested_at, and the source IDs that evidence it.
Traversal at query time is bounded and typed from the entry-point anchors, one to two hops, following only edge types relevant to the query intent, collecting both the connected objects and the paths themselves, because the paths are part of the evidence the reranker and the model will see.
Two operational notes from designing against Cosmos Gremlin specifically. First, traversal consumes request units in proportion to fan-out, which is the practical reason hops stay bounded at one to two and edge types are filtered, therefore the cost model punishes undisciplined traversal quickly. Second, the serverless capacity mode fits this workload the same way it fit Part 1’s store, where traversals are bursts around queries, and idle costs nothing.
13. Teaching the index about the graph
The search index changes in two small ways that carry most of the fusion design.
Every chunk gains an entities field — the collection of canonical entity IDs mentioned in that chunk, stamped at projection time after resolution. This is what makes step 1 of the pipeline produce anchors instead of just text: a hybrid search hit arrives already knowing where it enters the graph. Without this field, fusion would need a name-matching lookup between the stores on every query — the seam would show.
A second, small index appears, the entities index. One document per canonical entity (name, type, aliases, and a name embedding) serving exactly one purpose, the blocking stage of entity resolution. When extraction produces a mention, the resolver runs a filtered hybrid lookup against this index to get its candidate set. It is a resolution tool, not a retrieval surface, because queries from users never touch it.
Time, meanwhile, stays where the retriever-versus-filter test put it, and that is inside the main index as filterable fields (effective_date, superseded_on) and scoring adjustments — the Azure AI Search machinery Part 1 already used for as_of, now shared with edge-level validity in the graph.
14. The projection worker, and the ontology Ostermere already has
The write path lands on Azure with one addition that an Azure Functions worker on the Cosmos change feed. Every write to the system of record, either it is a new source, an extracted concept, an edge, flows through the feed to the worker, which projects it into the search index (chunk, vector, entity anchors) and the graph (vertices, typed time-valid edges). Part 1’s demo seeded synchronously because it was easy to run locally; the change-feed worker is that same projection logic given a production shape, and the demo mode keeps a synchronous fallback so the repository still runs with no Azure credentials at all.
The part of this section I enjoy most is the ontology, because it turns out Part 1 already built it without calling it that. Typed extraction needs a target vocabulary, the set of relationship types the model is allowed to emit, and generic related_to edges are precisely the thing that makes knowledge graphs disappointing. Part 1’s curated corpus contains twenty-eight hand-written relationships using types like narrows_rule_for, scoped_exception_to, requires_validation_by, contradicts, supersedes, commonly_confused_with. That hand-curated vocabulary is the ontology: the extraction prompt targets it with structured outputs, a validation pass rejects out-of-schema edges, and the curated relationships become both the few-shot examples and the first regression set for extraction quality.
That is the quiet advantage of having built the knowledge layer by hand first. You do not have to invent an ontology in a workshop, you promote the one your own curation already proved useful.
15. Reranking the union
Step 4 of the pipeline needs one honest paragraph of Azure specifics, because there is a trap here I want to flag before someone falls into it.
Azure AI Search ships a built-in semantic ranker, and the natural assumption is that it handles the reranking stage. It cannot! Not because it is weak, but because it only reranks its own query results. The union in this pipeline is hand-assembled: search hits plus graph-pulled objects that never went through the search query at all. No search service reranks documents it did not retrieve.
So the union rerank is external. A scoring call to a Foundry-deployed model that sees each candidate with its relational path rendered alongside its text, and orders the union by combined semantic and relational evidence. A reranker that can read “connected to the query’s anchor through narrows_rule_for, two hops” as evidence will make different, and for relational questions, better decisions than any similarity score.
There is a cost silver lining that carries over from Part 1: because the built-in semantic ranker goes unused, the search service’s free tier remains viable for the proof stage, exactly as it was in Part 1’s deployment. The rerank cost moves to model tokens, where it is proportional to use instead of provisioned per month.
16. What the additions cost
Part 1 measured its running costs live ($0.16 for the whole validation period, roughly $0.05 a day idle), and the additions keep the same discipline: serverless where possible, provisioned nowhere.
| Addition | Mode | Expected cost behavior |
| Cosmos DB for Gremlin | Serverless | RUs per traversal burst; idle near zero; bounded hops keep per-query RUs small |
| Azure Functions (projection) | Consumption | Free grant covers a demo corpus many times over |
| Entities index | Inside existing search service | No new service; counts against the free tier’s document quota |
| Union rerank | Foundry tokens | The one genuinely new variable cost – proportional to query volume and bundle size |
| Extraction + gray-zone adjudication | Foundry tokens | Ingest-time, one-off per document; the two-threshold design exists to keep the gray zone narrow |
The honest summary: the fixed-cost delta is close to zero, and the two things that can actually cost money (rerank tokens and gray-zone adjudication) are both governed by knobs the design exposes on purpose (bundle size, hop bounds, threshold width). The implementation article will put measured numbers against each, the way Part 1 did.
Part III – The corpus, traversed
The corpus does not change. The twenty-one Ostermere Mutual documents, the contradiction, the claims, the wind zones — all of it carries over from Part 1, which is precisely what makes it useful here. The questions that exposed Part 1’s limits can now be replayed against the new design. And they were literally. The design above is deployed on a fresh resource group next to Part 1’s, the corpus was reseeded through the fused ingestion path (live extraction, live edge extraction against the ontology, live resolution, projection into a real Gremlin graph — 75 vertices and 165 edges from 21 documents), and every output quoted below came back from that deployment.
One number before the walkthroughs, because it validates the section this article leans on hardest. Under Part 1’s alias-only resolution, live extraction produced 149 concepts against 19 canonical ones. Under the new resolution pipeline — embedding similarity in blocking and scoring, two thresholds, gray-zone adjudication, the same corpus produced 120. A twenty percent reduction in fragmentation from the same documents, measured like-for-like. Not yet 19, since the direction and the mechanism are confirmed, and the remaining gap is extraction phrasing, which the next walkthrough makes concrete.
17. The multi-hop question, answered properly
Part 1, section 5.6, described multi-hop as a failure mode: “Why was this claim triaged Level 1?” needs the claim notes, then the triage guideline, then the water-damage concept, then the policy clause and top-k similarity ranks, it does not traverse. Part 1’s live deployment answered the question anyway, but it did so on the strength of the model reasoning over whatever chunks happened to rank, the hops lived in the prompt, not in retrieval.
Under fusion, the same question retrieves differently. Hybrid search surfaces the claim notes as entry points, carrying anchors for CLM-1042, claim-triage-level, and sudden-accidental-water-damage. Traversal walks one hop: the triage level’s edge to the guideline that defines it, the water-damage concept’s scoped_exception_to relationship with the wear-and-tear exclusion, the claim’s contrasts_with link to CLM-1155. The bundle the model receives is the chain itself. Four objects and the typed paths connecting them rather than four chunks that happen to co-rank.
The difference shows up in what can go wrong. In Part 1, if one chunk of the chain missed the top-k, the model had a gap to bridge by inference. Under fusion, the chain is pulled in because it is connected, and the reranker sees the connection as evidence. Retrieval does the walking; the model does the explaining.
Here is the live run. The request:
{ "question": "Explain why the CLM-1042 water loss was placed at triage Level 1
under the property claim triage process.", "top_k": 6 }The grounded bundle came back with fourteen typed paths alongside the ranked objects. A sample of the paths array, verbatim:
provisional-operational-view-level-1-triage -explained_by-> initial-repair-estimate
(valid 2026-04-18..open)
required-initial-information-intake -applies_to-> first-notice-of-loss-fnol
(valid 2026-02-15..open)
localized-corrosion -applies_to-> supply-pipe-connector-failure
(valid 2026-04-18..open)And the model’s answer walked the chain with citations, exactly as designed:
Summary why CLM-1042 was triaged Level 1, Triage criterion: Level 1 applies where damage is minor, there is no safety concern or continuing water entry, and estimated exposure is below EUR 5,000 [INS-SYN-002]. Facts from the file that meet those criteria: damage was limited (wet kitchen ceiling board; moisture in a limited section of the upstairs bathroom floor) …
Notice the vertex names in those paths: provisional-operational-view-level-1-triage, first-notice-of-loss-fnol. These are live-extracted provisional entities, named by the model’s own phrasing — the graph a real extraction pipeline builds is messier than the one you curate, and the paths carry that texture honestly.
18. The question Part 1 could not ask
Here is a question no version of Part 1 could answer, and every insurance auditor asks: “How did the roof inspection requirement evolve, and what exactly changed on 1 March 2026?”
Part 1 could answer as-of questions — which rule applied on date X — by filtering documents. But evolution is a question about relationships over time, and Part 1’s edges had no time on them. Under the bitemporal model, the corpus’s own data answers it as a timeline traversal on the roof-age-risk vertex:
- 2025-06-01 – edge opens: general inspection threshold above 20 years (INS-SYN-003).
- 2026-01-22 – edge opens: the rationale relationship — the analysis email explaining why a 15-year threshold was coming, with its 3.4x displacement finding (INS-SYN-018).
- 2026-03-01 – edge opens: threshold above 15 years, scoped to H3 new business (INS-SYN-004). The general edge stays open — it still holds outside H3. Scoped supersession, Part 1’s first failure mode, expressed as two concurrently-valid edges with different scopes.
And the diff form — what changed in Q1 2026, is the same traversal filtered to edges whose validity opened in the window: the H3 threshold, the zone-classification ownership, the recoverable-depreciation change from policy v1.1 to v1.2. Each with its source attached, each still carrying Part 1’s discipline that the answer cites documents, not itself.
The live run returned twenty-four time-valid edges for the roof-age-risk neighborhood, ordered by validity, and the model produced exactly the chronological narrative the design promised:
Summary timeline (chronological by effective date). I cite the governing source IDs in brackets and preserve each rule’s scope. 2025-06-01 – Baseline rule: a roof inspection is normally requested when the primary roof covering is more than 20 years old (roof-age-inspection-threshold) …
And the diff form (temporal_mode: “diff”, as_of: “2026-01-01”) returned only the edges whose validity opened in the window, each tagged opened or superseded.
One honest artifact from the live data, worth recording because it is a real bitemporal modeling lesson: the curated edge high-wind-zone-h3 narrows_rule_for roof-age-risk shows valid_from: 2025-06-01 in the live graph — the date of its earliest evidencing source, not the date the narrowing took effect (2026-03-01). My reconstruction heuristic for pre-bitemporal curated edges uses the earliest source date, and for supersession-flavored relations that is semantically wrong; the later date is the truth. Reconstruction of validity windows for edges that predate bitemporal capture needs per-relation-type rules, not one heuristic. Noted, and queued for the fix.
I want to be precise about what is new here. The facts were always in the corpus. Part 1 retrieved them individually just fine. What was missing was the shape of the question — evolution and change are traversals over time-valid edges, and no amount of better ranking produces them from a system whose relationships do not know when they were true.
19. Would the system have found con-001 on its own?
The question I most wanted to answer with this design: if the contradiction detection of section 6 had existed when the corpus was first ingested, would con-001 have been discovered instead of curated?
Walk the ingest. The Claims Handling Manual (INS-SYN-012, effective 2025-11-01) arrives, and extraction produces an edge: trace-and-access covered_as_standard, limit EUR 5,000, valid from November. Later the Endorsement Catalogue (INS-SYN-008, effective 2026-01-01) arrives, and extraction produces: trace-and-access optional_paid_endorsement, HS-TA-01, EUR 24 premium.
The detection check asks: does the new fact conflict with a currently-valid edge on the same subject? Both edges attach to the same vertex — but only if resolution correctly lands “trace-and-access costs” from a claims manual and “Trace and Access (HS-TA-01)” from a product catalogue on one entity. Assume it does (and note, again, everything routes through section 7). The two edges are incompatible, standard cover and paid endorsement cannot both describe the same policy, and the supersession test fails on both criteria that would allow an automatic expiry: different authorities wrote them, and the newer document does not claim to replace the older one. Genuine conflict. The system opens a contradiction object, attaches both statements with their sources and dates, assigns the owner, and the gate arms itself.
That was the design-time prediction. What actually happened on the live deployment was better in one way and worse in another, and both are worth reporting.
Better: I never had to run the replay. The detector fired during the corpus seed itself, unprompted, and this object appeared in the register next to the curated con-001:
{
"id": "con-auto-trace-and-access-cost-recovery-coverage_basis",
"status": "unresolved",
"statements": [
{ "source_id": "INS-SYN-008", "effective_date": "2026-01-01",
"statement": "trace-and-access-cost-recovery coverage_basis
endorsement-hearthmere-catalogue" },
{ "source_id": "INS-SYN-012", "effective_date": "2025-11-01",
"statement": "trace-and-access-cost-recovery coverage_basis
trace-and-access-costs-are-covered-as-standard-under-hearthmere-
up-to-eur-5-000-..." }
],
"why_not_resolved": "Both sources are current and come from different authorities;
the newer document does not supersede the older one. Detected at ingest - the
system records the conflict and does not pick a side."
}The live model extracted incompatible coverage_basis claims from the two documents, resolution landed them on one vertex, the supersession test failed on the different-authority criterion, and the machine opened an unresolved contradiction with both statements and their dates. Nobody curated it. The policy found it.
Worse — and this is the caveat the design-time version of this section predicted, observed in the wild: look at the subject vertex. It is trace-and-access-cost-recovery, a provisional entity named by the model’s own phrasing, not the canonical trace-and-access the curated graph uses. Resolution correctly put both conflicting mentions on one vertex, which is why detection fired at all, but not on the canonical one. The auto-detected contradiction and the curated one describe the same real conflict from two vertices that resolution should eventually merge. Detection is a policy on top of extraction and resolution quality; sections 6 and 7 are one investment, not two features. The live run proved both halves of that sentence in a single object.
The auto-detected statements are also visibly rawer than the curated ones, rendered triples rather than quoted prose. That is what ingest-time detection looks like at this maturity, and I would rather show it than polish it: the mechanism works, and the finishing quality is extraction work, not architecture work.
20. To sum it all up
Part 1 argued that a RAG system needs somewhere for understanding to accumulate, and built the place. This article is about what that place must become once you take its own contents seriously: the relationships deserve traversal, the traversal deserves to run on every query, the edges deserve to know when they were true, and all of it stands on entity resolution done properly.
In practical terms, the revision is four commitments:
The router asks: which store should answer this question?
Fusion answers: both, always — and the results compete on evidence.
Part 1’s edges said: these two things are related.
Bitemporal edges say: they were related from March, until the update — and here is what changed.
Part 1’s contradiction was curated. The detector discovers it at ingest.
Part 1’s resolution matched aliases. The pipeline earns the graph its connections.
And one thing deliberately does not change: the gate. When the evidence touches a conflict the organization has not resolved, the system still returns both positions, names the accountable owner, and declines to answer as though the matter were settled. Fusion changes how evidence is gathered. It does not change the honesty of what happens when the evidence disagrees with itself.
The Azure delta is one engine, one worker, and two index changes on the stack that has been running since Part 1, the implementation lands in the same repository, and the next article will validate these designs the way Part 1’s were validated: deployed, queried live, measured, and reported with the failures left in.
Thank you for continuing this exploration with me. Part 1 ended by saying the application was no longer searching a pile of documents but slowly building a reviewable model of a domain. This part is about letting the system walk that model, along its relationships, backward through its history, and straight into its own contradictions, and I strongly believe that is the difference between a knowledge layer that stores understanding and one that uses it.
Disclosure: I am a Microsoft MVP. This article reflects my own independent work and opinions; Microsoft had no involvement in or review of its content. All Azure usage described is based on public documentation and my own deployment.
References
[1] (Part 1 link) – the first article in this series, with the accompanying repository at github.com/mcekikj/persistent-knowledge-layer
[2] P. Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020), NeurIPS 2020
[3] D. Edge et al., From Local to Global: A Graph RAG Approach to Query-Focused Summarization (2024), arXiv
[4] Microsoft, Hybrid Search Overview – Azure AI Search (2026), Microsoft Learn
[5] Microsoft, Relevance Scoring in Hybrid Search Using Reciprocal Rank Fusion (2026), Microsoft Learn
[6] Microsoft, Introduction to Azure Cosmos DB for Apache Gremlin (2026), Microsoft Learn
[7] Microsoft, Change Feed in Azure Cosmos DB (2026), Microsoft Learn
[8] Microsoft, Azure Cosmos DB Trigger for Azure Functions (2026), Microsoft Learn
[9] Microsoft, Add Scoring Profiles to a Search Index (2026), Microsoft Learn
[10] Microsoft, Structured Outputs with Azure OpenAI (2026), Microsoft Learn