Extracting data from a PDF

This is the most requested thing and the least checked: hand it a PDF — a payslip, a blood test, a delivery note, a report — and ask for the data in a table. It works, it saves a long stretch of your afternoon, and it fails in one very specific way that you need to know about before you trust it.

Two PDFs with the same name and nothing in common

The three rules

1. Name the fields, one by one

"Pull out the data" lets the model decide what counts as data. "Pull out basic pay, supplements, gross total, total deductions and net pay" leaves nothing to its judgement — and if one of them is absent, you notice, because a row is missing.

2. Require "not legible" instead of a guess

This is the "what to do when it does not know" piece from lesson 1, and here it decides everything. Without it, an unreadable field comes back filled in.

3. Make the arithmetic add up against the paper

It is the cheapest check there is and the one that catches the real error, because a misread figure almost never adds up with the rest of the document:

A prompt that works

Here is a payslip. Extract ONLY these fields, as a table:
period · grade · basic pay · supplements (one per row)
· gross total · total deductions · net pay.

If a field cannot be read with confidence, write "not legible".
Do NOT infer it from the others and do NOT work it out yourself.

At the end, tell me whether the gross total matches the sum of the
items, and whether the net matches gross minus deductions.

That last paragraph is what turns the extraction into something checkable: you ask for the arithmetic and all you have to do is see whether it says it adds up.

And the PDF belongs to somebody

For the specific case of your own payslip, this site has a tool that does exactly this and compares what it extracted against the collective agreement tables — the difference being that the document does not go into a general-purpose chat, the comparison is not improvised by a model, and a pharmacist reviews the report.

A full example, and what is wrong with it

The prompt above ends by asking it to check the sums. This example shows why that check, written the way it is, is worth nothing — and what to do instead. It is the most expensive failure in the whole lesson and you cannot see it by reading the reply.

It is given a photo of a payslip (not a text PDF: a phone photo, which is how nearly all of them arrive) together with the full prompt above.

And now, what is wrong with that reply:

1. The check is circular, which is why it always adds up. It added up what it read itself, not what is printed on the paper. The base salary on the original was €1,240.00 and it read €1,249.00: from there, all its numbers are consistent with each other and none of them is the payslip's. Asking it to check its own figures is asking it to correct itself with the very data it has just invented — "everything adds up" comes out no matter what. And that "everything adds up correctly" is precisely what stops anybody looking at the paper again.
2. A smudged 4 and a 9 look alike. Nine euros a month. In a photo with a shadow, the stroke of a closed 4 is almost a 9. The model does not "hesitate": it picks the one that fits and writes it with the same poise as every other digit. It does not come out in italics, with a warning, or in smaller type. Nine euros a month is €126 a year counting the extra payments — and in a back-pay claim covering three years, it is the whole claim. On a blood test, the same error moves a value across a reference range.
3. The supplements arrive merged into a single figure. You asked for "supplements, one per row" and it returned €170.90 in one piece. The total is the same, so the sums add up and nothing looks missing — but what has been lost is which part was the absorbable "on account of the agreement" supplement and which was seniority, which is not absorbable. Precisely the split that decides whether that payslip is right or wrong. The error is not in a number: it is in a row that does not exist.
4. And it has not written "not legible" once. You asked for it explicitly and the photo has a shadowed corner. The "what to do when it does not know" instruction is the key piece of the whole of Level 2, and here writing it is not enough: the model does not perceive a smudged digit the way you do, because it is not looking at a doubtful image — it is producing the likeliest text given what it sees. The way to test whether the instruction works is to feed it a document with a deliberately covered patch. If it says nothing, it is not applying it.

The check that does work differs by one phrase, and it changes everything: "tell me what the TOTAL GROSS box says on the document, and compare it with the sum of the items you extracted". Now there are two separate sources — what is printed in one box and the sum of others — and a discrepancy becomes possible. With the payslip above it would have said: sum 1,419.90, box 1,410.90, does not match. And now you know a digit was misread, and where.

The general rule holds for any document: a check is only worth something if it can fail. If the answer is "it adds up" no matter what, you were not checking anything.

And if your pharmacy is not like that

When it does not work first time

Before you call this learned

• Before uploading a PDF I check whether I can select text inside it.
• I name the fields I want, one by one.
• I require "not legible" and I check that it actually uses it.
• I make the totals add up before accepting any figure.
• If the document belongs to somebody else, I do not upload it.

← Previous: summarising a paper Next: patient-facing material →