Retrieve One Row from a Table, Not the Whole Table: Row-Level Chunks for RAG

A table on paper is a bounded region: a rectangle of cells with a header row on top and body rows below. A retrieval system that treats the whole rectangle as one chunk collapses that structure. The chunk either matches (and the generation model receives every row, most of them irrelevant to the question) or it does not (and the generation model receives nothing, because the one relevant row is buried under all the other rows the scorer had to average over).

The mismatch is between the document’s unit of information (one rectangle) and the reader’s unit of question (one row). A question like “cap for vehicle theft?” asks about one row. A question like “which events are covered?” asks about the whole rectangle. Both are legitimate. A retrieval brick that only offers the rectangle answers the second question and mishandles the first.

The fix is not to pick one scale over the other, but to make both scales available and let the dispatcher pick. That is what this article builds, on top of the parsing brick’s existing output.

Two steps turn a parsed table into a retrievable index: read the shape the parser emits, then serialize each body row into its own chunk.

Docling and Azure Document Intelligence both emit tables as markdown-pipe lines inside line_df. A table with 4 body rows becomes 6 lines: a header row (| Col1 | Col2 | ... |), a separator row (| --- | --- | ... |), and one body row per data row, in reading order. No extra _type column survives the public line_df contract. The pipe pattern in text is the signal.

The rules of the pipe format are the same across docling and Azure Layout, and light enough to detect without a per-parser branch:

• A pipe row starts and ends with |, with at least one interior pipe.
• The separator row (interior of dashes, colons and pipes) marks the split between the header and the body.
• Consecutive pipe rows on the same page, or across an immediate page break, belong to the same table.
• Anything else, including a non-pipe line in between, resets the group.

The four rules above are the whole contract. The row-level primitive below reads only line_df and needs nothing else from the parser.

The serialize_table_rows function turns each body row of every table into a retrievable chunk. A pandas.DataFrame in, a pandas.DataFrame out, the same shape Article 7B’s retriever uses, so the two bricks compose over the one line_df. The whole function is a small scan: group the pipe lines into tables, read the header off the row above the separator, and emit one output row per body line.

def serialize_table_rows(line_df: pd.DataFrame) -> pd.DataFrame:
    """One retrievable chunk per body row of every table."""
    lines = line_df.sort_values(["page_num", "line_num"])
    tables = group_contiguous_pipe_rows(lines)   # runs of | ... | lines

    out = []
    for tid, table in enumerate(tables, start=1):
        sep = first_separator_row(table)         # the | --- | --- | line
        headers = split_cells(table[sep - 1])    # the row above the separator
        headers, body = fold_multirow_header(headers, table[sep + 1:])
        for row_idx, line in enumerate(body):
            cells = split_cells(line.text)
            out.append({
                "table_id": f"t{tid}",
                "page_num": line.page_num, "line_num": line.line_num,
                "column_headers": headers, "row_cells": cells,
                # headers travel with the values, so a match reads like a sentence
                "row_serialized":
                    " | ".join(f"{h}: {c}" for h, c in zip(headers, cells)),
            })
    return pd.DataFrame(out)

The function returns a new frame, not a mutation of line_df. The parsing brick’s frame stays the point of truth for the document’s geometry (bbox, position, page); the row-level frame is a second index on top of it. Downstream retrievers that never enable row-level indexing keep working unchanged. The ones that do read the second frame explicitly, so nobody has to check a chunk_type tag.

The serialization format. Each body row becomes col: val | col: val | .... The choice is deliberate: an LLM reads that shape as naturally as a paragraph, and the retriever that keyword-matches or embeds paragraphs today can index row_serialized values with no change. The reader who receives a match sees a self-explaining line (the column names travel with the values) instead of a naked cell.

Column-value pairs also compose with the standard citation contract from Article 8. When the generation model quotes a specific value, the quote lands on the body row’s (page_num, line_num), exactly like any other span.

The row-level index is a second scale, not a replacement. The dispatcher picks the scale from the question, and two worked examples (the guarantees table from the intro, then a real published paper) show the payoff.

Three question shapes cover most of what users ask a table, and each maps to a scale.

• Targeted lookup (“cap for vehicle theft?”, “BLEU for the Transformer big model?”, “salary of the deputy CFO?”). Route to the row-level index. Return one row plus its column headers. Generation receives exactly what it needs.
• Synthesis (“which events are covered?”, “what layer types does the paper compare?”, “list every executive on the compensation grid?”). Row-level still fires, but every row of the same table matches. The dispatcher notices the shared table_id and widens back to the whole table before generation.
• Mixed (“summarise the vehicle-theft cap and compare it with the fire cap”). Two rows on the same table, generation stitches the answer from both. The row-level index makes this cheap; a whole-table dump would have worked too, but at a higher token cost.

The widening rule from row-level to whole-table is worth naming: when k body rows of the same table_id all match the question, and k covers most of the table, treat the match as table-level. In practice, a threshold like k / n_body_rows ≥ 0.6 catches synthesis queries. Below that threshold, the row-level answer is the honest one. A synthesis pretending to summarise on the strength of 2 out of 40 rows would be a hallucination the arbiter of Article 7C should already refuse.

Back to the car-insurance contract that opened this article. Its guarantees table has one row per covered event and four columns: the event, its cap, its deductible, and the eligibility condition. Here is a slice of it (the full contract runs to about forty rows).

The question from the intro, “what is the cap for vehicle theft?”, is a targeted lookup. Serialize the table, then keyword-match on the serialized rows.

row_df = serialize_table_rows(line_df)          # one chunk per body row
hit = row_df[row_df["row_serialized"].str.contains("Vehicle theft")]
# 1 row, 122 chars:
#   "Covered event: Vehicle theft | Cap: 25,000 EUR |
#    Deductible: 500 EUR | Eligibility condition: Tracker installed and active"

The match returns one row of 122 characters: the Vehicle theft event with its cap, deductible and condition, nothing else. The whole eight-row slice is 943 characters, a 7.7× ratio already. On the full forty-row contract the same single-row answer is roughly a 40× saving, since the ratio is just the row count. The generation model reads the one line it needs, so it cannot answer with the fire cap or the vandalism deductible by mistake.

The guarantees table is hand-built to match the intro; here is the same mechanism on a real published PDF. The Attention paper has four tables. Table 1 on page 6 is the compact case that shows the mechanism end to end. It has four body rows, one per layer type: Self-Attention, Recurrent, Convolutional, Self-Attention (restricted). The columns are Layer Type, Complexity per Layer, Sequential Operations, Maximum Path Length. Here it is as printed in the paper:

Two questions test the two scales.

Targeted query. “What is the complexity of self-attention per layer?”. Keyword-match on row_serialized returns the Self-Attention row (page 6, line 4), 124 characters of context. A looser match also surfaces the Self-Attention (restricted) variant on line 7; either way the retriever hands back the one or two rows that match, not the whole rectangle. The whole Table 1 is four rows totalling 528 characters, a 4.3× ratio for the single-row hit on this compact four-row table. The gap tracks the row count, the same way it did on the guarantees table in the previous section.

Synthesis query. “What layer types are compared?”. Keyword-match on Layer Type: returns four hits, all on table_id = t1. Every row of the table matches, so the dispatcher widens back to the whole table. Same outcome as a table-level retriever, no regression.

Two practical concerns remain: the one table shape the serializer has to handle itself, and how the row-level index plugs into the existing retriever.

Real parsers do not always produce a single header row. On the Attention paper Table 2 (page 8, BLEU and Training Cost), the printed header spans two lines: Model / BLEU / Training Cost on top, then EN-DE / EN-FR under both BLEU and Training Cost. Docling flattens that into a first pipe row with blank cells (Model, BLEU, , `Training Cost (FLOPs)`,), a separator, then a body row that is really the second header line, before the numbers begin.

A single-line serializer reads the flattened first row as the header and the second header line as the first body row. A keyword search for EN-DE then hits that label-only row before the numeric rows below it. This is the one table shape the primitive has to handle itself, because it is common and the failure is silent.

The fix stays inside the serializer, in the fold_multirow_header step the core loop already calls. The signal is the blank cell: a header cell is empty only when the parser flattened a cell that spanned several columns. When that happens, forward-fill the spanned label rightward, then check whether the first body row is really a sub-header (all labels, no digits, with real numbers on the row below). If it is, concatenate the two header levels column by column and drop the sub-header line from the body.

def fold_multirow_header(headers, body):
    """Fold a spanned two-line header into one labelled header row."""
    if "" not in headers or len(body) < 2:   # nothing to fold
        return headers, body

    headers = forward_fill(headers)      # a merged cell blanks the columns it spans
    subline = split_cells(body[0].text)
    is_subheader = (
        len(subline) == len(headers)
        and not any(has_digit(c) for c in subline)               # all labels
        and any(has_digit(c) for c in split_cells(body[1].text)) # numbers below
    )
    if is_subheader:
        headers = [f"{t} {s}".strip() for t, s in zip(headers, subline)]
        body = body[1:]                  # the sub-header line is not a data row
    return headers, body

On the real docling parse of Table 2, this turns the header into Model, BLEU EN-DE, BLEU EN-FR, Training Cost (FLOPs) EN-DE, Training Cost (FLOPs) EN-FR, drops the label-only line, and serializes the numeric rows against the merged labels. A search for EN-DE now lands on the data rows, where it belongs.

The guard is what makes this safe to run on every table. It fires only when the header carries a blank cell and the suspected sub-header is all text and the row beneath it carries digits. An ordinary single-line-header table, or an all-text glossary table, keeps every body row untouched. Upstream is still the better place to normalise multi-row headers, since a parser that emits one logical header row fixes it for every consumer, not just retrieval. Until that lands, the serializer no longer breaks silently on the shape.

The row-level frame is the parallel index from section 2.2, wired into the retriever by a single activation.

• row_df = serialize_table_rows(line_df) is computed once and cached next to line_df (output//parsing/row_df.parquet).
• The retriever gains a use_row_level: bool flag alongside use_toc, use_keywords, use_dense. On, the row-level frame is indexed with the same keyword and embedding primitives that already run on paragraphs.
• The question parser (Article 6) sets the flag by question shape: targeted_lookup on a document with tables turns it on, synthesis leaves it off, mixed turns it on and lets the widening rule from section 3.1 do the fusion.

Nothing about the paragraph-level retriever changes.

Tables in a document carry structured information that a paragraph-level retriever averages away. This article added a second retrieval scale on top of the existing line_df contract: serialize_table_rows(line_df) → row_df turns each body row of every detected table into a chunk paired with its column headers, keyed on (page_num, line_num) for citation. The dispatcher picks the scale by question shape. Targeted queries route to row-level, synthesis queries widen back to the whole table, mixed queries stitch two or three rows. On a slice of the insurance guarantees table a targeted lookup returns one 122-character row where the whole slice is 943, and on the Attention paper Table 1 the same move sends 124 characters against 528; the ratio is about the row count, so a full contract saves far more. Multi-row headers, the one shape a single-line serializer gets wrong, are folded inside the serializer: a spanned header is forward-filled and its sub-header line concatenated into one labelled row, guarded so an ordinary table is never touched. The row index sits alongside line_df, not inside it. The parsing brick’s contract is untouched, and every existing consumer of line_df keeps working unchanged.