Traceability in document extraction: every data point should point to the page and line it came from

Extracting data from a document with a language model is now relatively simple. The hard part is being able to answer, months later and in front of an auditor, where each of those data points came from. In regulated sectors such as anti-money-laundering compliance or identity verification, a value without a source is a value that cannot be defended. That is why our work starts from a simple rule: no data point leaves the system without indicating the document, page and line it was taken from.

Why "the AI said so" is not acceptable

A language model can read a tax status certificate or articles of incorporation well, but it can also be wrong with apparent confidence: confusing the tax address with a branch address, picking up the date of an earlier deed or filling in a value that does not appear in the document. The problem is not only the error, but that without a source nobody can tell a correct value from an invented one without rereading the whole file.

In a regulated review, the person who approves is accountable for the decision. If the system hands them an RFC, an incorporation date or the name of an attorney-in-fact, they need to be able to verify it in seconds. A value with a source is checked with one click; a value without a source forces them to repeat the work that automation was supposed to save.

That is why we separate two functions: language models extract and draft, but they do not decide. Decisions are made with explicit rules, configurable thresholds and, in the end, the approval of a person.

What it means for a data point to have a source

Traceability is not attaching the full PDF next to the result. It means every extracted value travels with its precise origin:

• Document: which of the files in the case file contains it.
• Page: the page where it appears.
• Line and region: the exact line of text and its location on the page, so the reviewer sees it highlighted.
• Source type: not every value comes from the text. A value read from a QR code indicates the page and the area of the code; a value obtained from an external lookup records when and which service was queried.

With this, the review screen stops being a list of fields and becomes a list of verifiable statements. The reviewer does not have to trust the system: they can check it.

Audit: reconstructing how a decision was reached

A case file reviewed today may be questioned much later, whether by an internal audit, by the authority or by a client disputing a decision. To respond, keeping the final result is not enough; you need to be able to reconstruct the process. In practice, that requires a few design conditions:

1. Results that are never overwritten. If a document is processed again, the new result is added next to the previous one with its own date. Nothing is deleted or rewritten.
2. Pinned versions. Each run records which version of the configuration, the extraction instructions and the rules it used. That makes it possible to explain why a value was extracted a certain way at a certain time.
3. Logged access. Viewing the source document behind a value is also logged, because case files contain sensitive personal and tax information.

An extracted value is only as reliable as the ability to verify it. Without a source, automation shifts the risk; with a source, it reduces it.

Human review and discrepancies between documents

Traceability becomes more valuable when the case file contains several documents that should agree. An ID, a tax status certificate, a proof of address and articles of incorporation rarely say exactly the same thing: abbreviations, names with or without a second surname, addresses written differently or, sometimes, data that genuinely does not match.

When the system cross-checks names, CURP and RFC across documents and finds a difference, the reviewer's question is immediate: which one is right? If each value points to its page and line, the reviewer can see both fragments side by side and decide with evidence. Without a source, the discrepancy can only be flagged as "review", and someone has to search by hand for where each version appears.

This approach also helps calibrate trust in automation. When values can be compared against their source, it is possible to measure how often the system is right for each document type and each critical field, and to decide with data what can be approved with less intervention and what always requires careful review.

How we apply it

In Loony, our document intelligence platform, traceability is a design principle, not an add-on. Every extracted value, every QR code reading and every external lookup is stored with its source, and the reviews of the complete case file (coherence, completeness and cross-checking data between documents) produce results in the same shape. The operator reviews, checks the source when needed and approves; the system keeps the history of how each result was reached.

If your team reviews case files and needs every data point to be defensible in an audit, Write to us.