Every fact cites its page, or it is dropped.

The problem is not extraction. It is trust.

A personal-injury case arrives as a file: emergency room notes, orthopedic visits, physical therapy logs, imaging reports, pharmacy records and bills, often from six or more providers, often thousands of pages. Someone has to read all of it and write a chronology, the dated list of what happened to the patient. It takes days, and the same visit turns up three times because three providers each recorded it.

A language model can read those pages in minutes and produce a plausible chronology. Plausible is the danger. An attorney who puts a fact in a demand letter has to be able to point at the page it came from, and a fact that cannot be found on any page is worse than no fact at all. So the design rule for MedChron is simple to state: every extracted fact carries a citation to the page it came from, and a fact whose citation cannot be verified is dropped, not shown.

The pipeline

Files are deduplicated first, before any model sees them: the same scanned document arrives twice more often than you would think. Pages are then run through OCR and handed to extraction prompts on Gemini, in stages, on BullMQ workers so a two-thousand-page file does not block a two-hundred-page one. Each stage writes its output with the page range it read.

Deduplication of facts is the hard part. I rebuilt this stage twice. The first version asked the model to merge duplicates. It merged too eagerly and occasionally invented a date to reconcile two entries. The version that shipped puts deterministic merge guards ahead of the model: entries can only be merged when provider, date window and category agree, and the model is asked to choose between candidates, never to write a new one.

Citations that check themselves

A citation is a page number plus the span of text the fact was drawn from. A resolver checks each one against the record: the page must exist, the span must be found on it within a tolerance for OCR noise, and the date on the fact must agree with the page header where one exists. Anything that fails is dropped, and the drop is logged so we can see which prompts are producing unverifiable output.

This costs recall. It is the right trade. The chronology is a working document for a lawyer, and a shorter list of facts that all survive scrutiny is worth more than a longer one with a few that do not.

How we know it works: replay evaluation

Prompts change. Models change. Without a fixed measure, every change is a guess. We keep a set of source records with known-good extractions and replay each prompt version against them, comparing the output field by field.

The clearest example is medications. Half of the extracted medication rows had no start date, which broke treatment timelines downstream. I rewrote the medications prompt, added post-extraction validators, and replayed. Rows missing a start date fell from 53% to 26%, with no fabricated drug names and no fabricated citations in the output. The remaining 26% are mostly records that genuinely do not state a start date, which is a fact about the records, not the prompt.

• Every prompt lives in an append-only registry with a version, so an evaluation result always names the prompt it measured.
• Patient identifiers are scrubbed before the chronology and bills prompts run.
• Chi, the in-product assistant, answers questions about a patient only with cited facts from that patient’s file, and went through three QA rounds and 28 reported defects before release candidate, two of them data-isolation findings that were fixed and re-tested.

What I would tell someone building this

Decide what happens to an unverifiable fact before you write the first prompt. Put deterministic guards ahead of the model wherever you can, and let the model choose rather than compose. Build the replay evaluation before the second prompt version, because you will want to compare them. And treat every number you report as something an attorney might one day ask you to defend.