How We Built AI Invoice Parsing for PDFs, Scans, and Photos

When people talk about invoice OCR, they usually focus on text recognition. In reality, that is only part of the job.

For vporiadku, our invoicing and bookkeeping product for businesses in Slovakia, the harder problem was taking messy supplier invoices — PDFs, scans, phone photos — and turning them into a draft expense that a business can actually book. Reading the text was one step. Figuring out what each piece of text meant was the real challenge.

Why Rules Alone Were Not Enough

Invoices are inconsistent by nature. The invoice number might be in the top-right corner on one file, near the footer on another, and hidden behind a local label on a third. The same applies to dates, VAT values, totals, and supplier details.

A rule-based approach — regex patterns, keyword matching, and heuristics — works on clean invoices from a handful of suppliers. The moment layouts become less predictable, accuracy drops fast. The issue is not a lack of text. The issue is mapping raw text to the correct business fields.

The Pipeline We Settled On

1. Upload a PDF, photo, or scanned invoice
2. Read the PDF's own text layer when it has one
3. Strip the parts of that text that are not invoice data
4. Send what remains — or, for a scan or photo, the image itself — to a model with a strict extraction schema
5. Check the model's output against business rules
6. Save the result as a draft expense that the user reviews before booking

Read the Text Layer First

Most supplier invoices are PDFs generated by another invoicing program, so they already contain text. Extracting it happens locally and costs nothing, which is why it always comes first. Only a document without usable text — a scan or a phone photo — goes to a multimodal model that reads the image directly. The image path costs roughly two orders of magnitude more than the text path, so the order of these steps matters.

This step also taught us something. After a library upgrade, text extraction quietly started failing on every PDF, and every invoice fell through to the expensive image path. Nothing looked broken: the results were still correct. Only the cost was wrong. Since then, every scan reports which path it took, so a silent fallback shows up in the logs instead of on the bill from the AI provider.

Send the Model Less

The text layer of a real invoice is mostly not invoice data: terms and conditions, covering notes, signature blocks. That text adds nothing to field extraction, and it is exactly where incidental personal data lives. Before anything leaves our servers, vporiadku drops those prose lines and caps the length of the text. The minimisation is a per-workspace setting that is on by default. It is a data-minimisation measure first, and it also makes each call cheaper.

A Strict Output Schema

The prompt had to be strict. The more generic the prompt, the more generic the output. The version that worked asks for a fixed JSON object with exactly the fields a Slovak business needs:

• supplier name, IČO, DIČ, and IČ DPH
• invoice number, issue date, due date, and delivery date
• variable and constant symbols for the bank transfer
• IBAN and SWIFT
• line items with quantity, unit price, total, and VAT rate
• subtotal, VAT amount, total, and currency

The model also has to list the fields it is unsure about instead of guessing, and to rate its own confidence. Both travel with the draft expense, so uncertainty stays attached to the data rather than hidden in a confident-looking number.

Multi-Language Invoices

vporiadku is built for businesses in Slovakia, but their suppliers are not all Slovak: invoices also arrive in Czech, German, Hungarian, or English.

The language itself was rarely the hardest part. Formatting differences caused more trouble than vocabulary: date formats, decimal separators, VAT labels, and currency placement all vary. So instead of treating it as a translation problem, we normalise values into one internal format and describe the expected formats in the schema.

Never Trust the Model's Arithmetic

Even when the extracted result looks right, we check it:

• the line totals are added up and compared with the stated subtotal; a difference of more than five cents flags the result and keeps our own sum next to the model's
• an IBAN that fails its checksum is marked as uncertain instead of being accepted
• amounts, quantities, and the number of line items are bounded before anything is written to the database, so a hallucinated twenty-digit total cannot break the save

Users do not just need automation. They need enough confidence to trust the output without re-checking everything from scratch.

Budget Before the Call

Every AI call in vporiadku runs against a spending ceiling in euros. The expected cost of a call is reserved before the provider is contacted, and a call the budget cannot cover does not start. Because the text path and the image path differ so much in cost, the reservation is made only after the cheap local extraction has decided which path the document takes.

Pay by Square Integration

One Slovakia-specific feature turned out to be genuinely useful: Pay by Square.

For invoice recipients, scanning a QR code in a banking app is much easier than typing payment details. vporiadku generates the real Slovak BySquare format — LZMA compression, a CRC32 checksum, and base32hex encoding — not the similar-looking Czech format. We check generated codes by decoding them back with an independent library.

It was not the most technically complex part of the system, but it delivered obvious value very quickly. Those features are often worth more than the clever ones.

What We Learned

First, extracting fewer fields well is better than extracting everything badly.
Users mostly care about the fields they actually use.

Second, OCR is only one layer of the problem.
The bigger challenge is turning unstructured documents into reliable business data.

Third, AI extraction needs guardrails.
Without checks and review, it is too easy to let bad data pass through.

Fourth, watch the cheap path.
A fallback that still produces correct results can stay invisible for a long time while it multiplies the cost.

And finally, real invoices are always messier than sample files. Designing for that early saves a lot of pain later.