A FAQ is already the answer, pre-written and paired with its question. Ask “What is my deductible?” and the right response is a lookup away: the support team wrote it, word for word, months ago. Run the FAQ through the same embed-and-retrieve pipeline as a raw PDF and you throw that structure away, often returning a worse match than a plain lookup would. When the source is already question-and-answer, the RAG has to treat it that way.
This article is a bonus in Enterprise Document Intelligence, a series that builds an enterprise RAG system from four bricks. FAQ as RAG is the case where you get to design the corpus: every brick inverts, parsing is trivial, retrieval doubles as a cache, and few-shot prompting becomes a retrieval problem too.
🧭 New to the series? Every article in this series sits on our two Towards Data Science author pages, Angela Shi and Kezhan Shi. That is the shortest way to see what is covered and where this one sits.
📓 Runnable companion notebooks are on GitHub: doc-intel/notebooks-vol1.
Pull the logs of a customer-support chatbot a few weeks after launch and a pattern shows up: most user queries are variations of the same fifteen questions. “How do I cancel?”, “Can I end my policy early?”, “How do I stop coverage?” are three phrasings of one underlying question, with one answer the support team already wrote two years ago. The system is paying generation cost on every query, when the answer was already on disk.
This is the FAQ problem, and it is not what most RAG tutorials prepare you for. The standard framing assumes a chaotic corpus you inherit (PDFs, scans, contracts) and parsing is half the battle. The FAQ inverts that. You write the corpus. The structure is whatever you decide it should be. The four bricks of the pipeline reshape themselves around that fact, and one of them (generation) gets cheaper than the literature suggests.
This bonus article walks the four bricks one more time, on a fifteen-entry synthetic FAQ for a fictional home-insurance product. The point is not to build an FAQ chatbot. The point is to show how much the architecture changes when the corpus is yours.
1. Why the FAQ is a different problem
In the rest of the series the corpus is the constraint. An expert wrote the contract a decade ago, the PDF was scanned at 200 dpi, the page numbers do not line up with the printed ones, and the system has to recover meaning from all of that. Most of the engineering goes into recovering structure that someone else lost.
In the FAQ case, structure is upstream. The team curating the FAQ chooses the schema, the granularity (one Q-A per concept), the canonical phrasing of each question, the wording of each answer, the tags. Nothing has to be recovered because nothing was lost. The implication for each brick is direct.
The rest of the article walks the four bricks in order.
2. Parsing is trivial when you author the schema
The “parsing” step on an FAQ is loading a structured file. There is no PDF, no layout reconstruction, no OCR. The team that owns the FAQ defines a schema once and lives with it.
What the table looks like in practice, on the fifteen-entry example used throughout this article:
The work spent on parsing in Articles 5 (document parsing) and 10 (adaptive parsing) of the main series does not apply here. What does apply is something the main series spends less time on: versioning the corpus. An FAQ entry changes when the product changes. The team needs to know which version of an answer was returned to a user on a given date. That is corpus-management work, not parsing work, and the series covers it in Article 19 (storage). The FAQ is a fast-moving case of the same problem.
3. Question parsing as cache lookup
The job of question parsing on a generic document is to map a user’s phrasing to the document’s vocabulary (Article 6, question parsing). On an FAQ it shifts: the question is whether the user query corresponds to any of the canonical questions we have already curated. Three outcomes are possible, and the system should know which one it is in before doing anything else.
-
Direct match: The user query and a canonical question mean the same thing. Return the canonical answer verbatim. No generation needed.
-
Adjacent match: A canonical question is closely related but not identical. The canonical answer is a starting point, possibly with a thin LLM rewrite.
-
Miss: No canonical question is close enough. The query is outside the FAQ, or it is a new question the team should add.
The same retrieval primitive answers all three. The differences are in the similarity threshold and what happens next.
Classification is half the work. The other half is what the system does once it knows which outcome it is in. Three outcomes deserve three different actions, and the router is the single function that owns that dispatch.
The three branches carry three very different cost profiles. A direct hit is single-digit milliseconds and zero LLM tokens. An adjacent hit costs one embedding call plus one LLM completion, and the prompt is bounded (system + three Q-A pairs + user query, typically under 1000 tokens). A miss is the cheapest of the three at runtime but the most expensive over the life of the product: each logged miss represents a small piece of editorial work the FAQ team should do.
Three useful observations from a real run on this example.
Direct matches are conservative: The threshold for “direct” sits high (0.92 in this example) so the system only short-circuits to the canonical answer when the user really did ask the same question. False direct matches break user trust quickly (“the bot answered the wrong question with high confidence”).
Adjacent matches are most of the traffic. Real user queries phrase things differently, narrow the scope, or combine two FAQ topics. The canonical answer is a useful starting point but rarely the final answer. This is where the dynamic-few-shot pattern in section 5 earns its place.
Misses surface gaps in the FAQ: A query that lands in “miss” with low similarity to every canonical question is a signal: either the FAQ is incomplete, or the user is asking about something off-product. Both deserve to be logged and reviewed by the team that owns the corpus.
4. Retrieval as the cache
Once the cache outcome is decided, retrieval is mostly done. The top match is either the answer (direct), or an answer plus its few neighbours (adjacent), or it is set aside (miss). The interesting design choice is what to return alongside the top match.
A generic RAG system retrieves passages. An FAQ system retrieves complete Q-A pairs: the canonical question, its answer, and the tag. This matters because the Q-A pair is the unit of meaning in this corpus, and it is also the unit the generation step needs in the adjacent case (both the question and the answer land in the prompt).
Run on a user query close to an existing canonical question (“Does my policy cover fire damage?”), the top-5 comes back with the adjacent canonical hit on rank 1 and four neighbours that become the few-shot context in section 5:
A few engineering points worth being explicit about.
The corpus is static at query time: Embeddings on the canonical questions are computed once at FAQ-publish time and cached. A user query needs exactly one embedding call and one matrix multiply against the cached corpus. Latency budget is single-digit milliseconds for retrieval, regardless of FAQ size up to several thousand entries.
Versioning the embedding cache: When an FAQ entry’s wording changes, its embedding changes too. The cache key has to include the canonical question text (or a hash of it) so that stale embeddings cannot survive an edit. The same logic applies to the embedding model itself: changing models invalidates the entire cache.
Hybrid scoring matters more on small corpora. Fifteen entries leave plenty of room for cosine to be ambiguous. Adding a BM25 score and combining the two (Article 9, hybrid scoring) on the canonical question text catches direct lexical hits that the embedding alone misses. The combined score is the one used to decide direct / adjacent / miss.
5. Generation, and the case for dynamic few-shot
Few-shot prompting (giving the LLM a handful of worked examples of question + answer before the live query so it can follow the pattern) is usually a static engineering artifact: a senior engineer writes three example Q-A pairs into the system prompt, the prompt ships with the build. It works, and it ages badly: as the FAQ evolves, the static examples drift, and the prompt becomes a hidden source of stale instructions.
The FAQ-as-RAG setup makes a different option natural. The retrieval step already produced the top-k canonical Q-A pairs for the current user query. Instead of static engineered examples in the system prompt, the user prompt is built at query time with those retrieved pairs as in-context examples. The few-shot examples are dynamic, retrieved per query, drawn from the current FAQ. When the FAQ is updated, the examples update for free.
To see what static few-shot looks like next to it, the two patterns live side by side below. The contrast is the entire argument for the dynamic version.
A side-by-side of the three regimes on the same query makes the difference concrete.
What this buys, beyond the obvious “answers stay in sync with the FAQ”:
Scope discipline: A generic LLM with no examples drifts into general internet-grade answers (“typical home insurance covers…”). Examples drawn from the specific FAQ keep the tone, the numbers, and the brand voice consistent with the team’s curated answers.
Cheaper than people expect: The prompt grows by a few hundred tokens per query (k=3 short Q-A pairs). For most chat models the cost difference between zero-shot and dynamic few-shot is small relative to the quality difference.
Free contradiction detection: When the LLM’s answer disagrees with the retrieved examples, that disagreement is observable in the logs. It is a clean signal that either (a) the user query has slipped outside what the FAQ covers, or (b) the FAQ itself has internal contradictions that the team should resolve.
6. The FAQ grows from the question stream
Everything so far has assumed the FAQ corpus is ready on day one. That assumption is wrong. Writing an exhaustive FAQ in advance is real work, and doing it well means anticipating questions that have not been asked yet, in vocabulary that has not been used yet. Few teams manage that and stay current. The honest design starts from the opposite premise: the FAQ is incomplete by construction, and the system is built to close the gap as the gap is observed.
6.1 Miss routes to a person, not to generic RAG
The instinct from the rest of the series would be: when the FAQ misses, fall back to RAG over the underlying product manuals or CGV. That works mechanically. It also bypasses the actual problem. Someone has to decide what the canonical answer is for a question the FAQ does not cover, and that someone is a domain expert, not an LLM reading a manual.
The architecture: the miss outcome from the classifier routes the query into an expert queue. A support specialist (the same person who wrote the existing entries) reviews the question, writes the canonical answer, and the new Q-A pair lands in the FAQ corpus. Next time that question (or one close enough) comes in, it lands in direct or adjacent. The system never invents an answer it does not have; it shows the gap.
6.2 What “frequently asked” finally means
Most FAQ projects guess at which questions will be frequent and curate around those guesses. After three months of production logs, the guesses are usually wrong: half the curated entries get one or two hits, and the top-five questions the team is receiving never made it onto the list.
A query-stream-driven FAQ inverts the order. The team starts with whatever it has, observes which miss patterns recur, ranks them by frequency, and promotes the high-frequency ones into canonical entries. Stale entries that never get hit are retired. The list of canonical questions ends up reflecting what users ask, not what the team predicted they would ask. “Frequently asked” stops being a guess and becomes a measurement.
The signal needed is cheap: each route_query call writes a row to a query log with the user query, the classifier outcome, the matched qid (or none), and the similarity. A weekly job clusters miss queries by embedding proximity, ranks the clusters by size, and returns the top-N to the expert queue. The team writes one canonical answer that covers the cluster, and N queries that were missing tomorrow are direct or adjacent matches.
6.3 The expert in the loop, not replaced
Three places where a person is doing work the system cannot do:
-
Writing a canonical answer for a new question. The expert decides what the company’s position is, the wording, the numbers, the exceptions. The system has no way to invent that.
-
Approving borderline adjacent matches. The classifier hands an LLM-adapted answer back to the user, but the expert queue gets a sample of those for review. If the adapted answer drifts from the canonical one in ways that matter, the expert tightens the canonical Q-A or the threshold.
-
Retiring entries that have gone stale. The product changed, the policy was updated, the regulation moved. Someone has to find that out and pull the entry, or rewrite it.
This is the series’s central position applied to the FAQ case. The system exists to amplify the expert’s work, by reusing every curated answer thousands of times, by surfacing the questions that need expert input, by keeping the answers consistent across users. It does not exist to replace the expert with a model that hallucinates plausible-sounding answers for queries the team has never discussed.
7. Where this stops and the main series picks up
The FAQ case looks simple because of the inversion. The standard problems are still there, just pushed into a different layer.
Corpus governance is now the hard problem. The structure work that Article 5 (parsing) and Article 10 (adaptive parsing) do on parsing, Article 17 (classification) and Article 19 (versioning) do on those, all happens upstream at FAQ-edit time. Who can edit an entry, how versions are tracked, how stale answers are retired: all of it is real work. The FAQ does not eliminate the cost; it relocates it.
Listing and synthesis questions still apply. “What are all the exclusions?” needs every matching Q-A pair: a sweep over the corpus, not a top-k (the N best-scoring ones). Top-k is structurally wrong for listing because it stops as soon as it has enough candidates, not when it has found everything. Article 12 (listing) develops this pattern in detail.
Evaluation is still per-failure-mode. The framing of Article 20 (evaluation), that aggregate metrics lie and per-question-type metrics tell the truth, matters more here than in generic RAG because the failure modes are different. False direct matches are the canonical failure for an FAQ system and are invisible to an aggregate recall metric.
8. Conclusion
The FAQ case is what every brick of the pipeline looks like when you get to design the corpus on purpose. Parsing is a Pydantic load, question parsing is a similarity threshold, retrieval is a precomputed matrix-vector product, generation is a format() call. The work does not disappear; it moves up, into the FAQ schema, the editorial discipline, the versioning of curated answers, the threshold tuning between direct and adjacent hits.
Two patterns generalise back to the main series: caching what the corpus answers (any system serving the same questions repeatedly), and dynamic few-shot (retrieval applied to the prompt). When someone describes their use case as “we have a list of questions our users keep asking”, that is an FAQ, and the structural advantage should not be thrown away by feeding the questions through generic RAG.
9. Sources and further reading
The FAQ-style sentence-pair similarity the cosine threshold uses is Reimers and Gurevych (Sentence-BERT, EMNLP 2019). The retrieval-based few-shot selection behind the dynamic few-shot pattern is Liu et al. (What Makes Good In-Context Examples for GPT-3?, ACL 2022). The broader landscape (retrieval-augmented and tool-augmented LMs) is in Mialon et al. (Augmented Language Models, TMLR 2023). The article’s pattern: FAQ-as-cache + dynamic few-shot, exact-match short-circuit before the four bricks ever run, and the same FAQ rows reused as an in-context example bank for the residue.
Earlier in the series:
-
Document Intelligence: series intro. What the series builds, brick by brick, and in what order.
What works, what breaks
-
Baseline Enterprise RAG, from PDF to highlighted answer. The four-brick pipeline end to end: PDF in, highlighted answer out.
-
Embeddings Aren’t Magic: The Predictable Failure Modes of RAG Retrieval. Where embedding similarity wins (synonyms, typos, paraphrase), where it predictably breaks (unknown terms, negation, term-vs-answer relevance), and how to use it anyway.
-
RAG is not machine learning, and the ML toolkit solves the wrong problem. Why chunk-size sweeps and finetuning optimize the wrong thing; route by question type instead.
-
From regex to vision models: which RAG technique fits which problem. Two axes, document complexity and question control, that pick the technique for each case.
-
10 common RAG mistakes we keep seeing in production. Ten production mistakes, organized brick by brick, with the fix for each.
-
Document parsing
-
Building Document Structure with Loop Engineering: Recovering a PDF’s Outline from Body Typography for RAG. Rebuilding the outline from body typography when the PDF ships no contents page at all: six signals, one bounded loop.
-
Before Full Agentic RAG: Know How You Decide, and the Parsing Methods You Pick From. The parsing methods as a catalogue, and the decision of which to run, before handing the loop to an agent.
Generation
-
Loop Engineering for RAG Generation: Iterate top-k One at a Time. Reading the retrieved pages one at a time instead of all at once, and what that buys when the answer sits in only one of them.
-
Most RAG Hallucinations Are Extraction Errors: Seven Patterns for a Typed Generation Contract. Seven recurring ways a model gets the extraction wrong, and the typed contract that catches each one.
-
Loop engineering for RAG generation: an LLM cascade from a cheap local model up to a hosted flagship. Starting on a cheap local model and escalating only when the answer does not hold up, measured.
One-document pipelines
-
Prompt Engineering Isn’t Enough: How Four Bricks of Context Engineering Stop RAG Hallucinations. Why a better prompt does not fix a wrong page, and what each of the four bricks contributes to the context instead.
-
Cut an Enterprise RAG Pipeline’s Latency and Cost by Calling the LLM Less, Not by Buying a Faster Model. Cutting a pipeline’s latency and cost by calling the model less often and cheaper, not by buying a faster one.
-
RAG workflow and loop engineering: the dispatcher that decides when to loop and when to stop. Feedback loops, bounded iteration, and the dispatcher, composed into one workflow.
-
Loop engineering for RAG: the small loops inside each step, the big loops across the pipeline. The two scales of loop: small bounded loops inside each brick, big generation-triggered loops across them.
-