A third of incoming invoices arrived unreadable — phone photos with keystone distortion, faded thermal prints, scanner shadows, out-of-order attachments. No model was going to fix that. So we built a vision agent that inspects each page, picks a bespoke enhancement recipe, and only then hands off to labelling and training. Everything downstream got faster the moment the inputs got cleaner.
Every attempt to train an in-house invoice reader hit the same wall — dirty inputs meant slow, expensive labelling; slow labelling meant slow iteration; slow iteration meant a model that never surpassed the commercial OCR it was supposed to replace.
The enhancer agent broke that loop from the top. It takes a raw page, inspects it, decides which pathology dominates, and runs a bespoke recipe — perspective correction for phone photos, CLAHE for faded thermal prints, morphological background subtraction for scanner artefacts, page-splitting and reordering for multi-page attachments.
Below a legibility threshold, it doesn't guess — it escalates back to the sender for a re-capture. That refusal is what closes the loop on inbound quality over time.
4 recipesAdaptivePer-page quality scoreSender re-capture loopCPU-only
A mediocre extractor on clean images beats a good extractor on dirty ones — every time. Invest in the inputs before you invest in the model.
The enhancer is the first agent every page meets. Everything downstream — labelling throughput, verifier accuracy, weekly fine-tune quality — inherits whatever it produces.
A month of intake was classified by the failure mode that dominated each page. Four pathologies covered 96% of the previously-unreadable pile.
Held at an angle, half-shadow, keystone distortion pulls the page into a trapezoid. Text lines curve across the frame.
The receipt has sat in a glovebox for months. The ink has evaporated into a narrow, low-contrast band that no threshold can separate.
Binding-edge shadow, dust specks, and streaks. A global threshold either loses the text near the binding or turns the whole page into noise.
Three pages of one invoice, mis-scanned upside-down, out of order, with a stray cover sheet. Line-item continuity is lost across the joins.
Shares total 96%. The remaining 4% — the genuinely destroyed — are the only pages that still route to a human on arrival.
The enhancer isn't a fixed pipeline of filters. It runs a per-page decision — read the page, classify the dominant pathology, apply that pathology's recipe, and then score the result. If the score doesn't clear the bar, it doesn't fake it.
Each recipe is a tight sequence of CV operations tuned to one specific failure. Below, the transform that each recipe performs — with the input state on the left and the recovered state on the right.
The agent finds the four page corners (contour detection, then Hough-line refinement) and derives the homography that maps the distorted quadrilateral to a rectangle. Only then does adaptive thresholding kick in — on a page that finally has straight text lines.
A global histogram stretch smears the noise as much as it lifts the ink. CLAHE (Contrast-Limited Adaptive Histogram Equalisation) equalises in tiles and caps the amplification per tile — the ink separates from the paper without blowing up the speckle.
A large morphological OPEN estimates the smooth "paper" background (which contains the shadow but not the text). Subtract that from the page and the shadow disappears — leaving a uniformly-lit surface that a global threshold handles cleanly. Speckle removal follows with a small median.
The file is segmented into individual pages; each page runs orientation-detection (Tesseract OSD) and rotates upright; page-number strings ("p. 2 of 3") are extracted and used to sort. The reader downstream receives one coherent, ordered document instead of a shuffled deck.
Every enhanced page carries a legibility score. Below the bar, the agent doesn't send half-recovered text downstream — it replies to the sender asking for a re-capture, and the corpus gets cleaner over time.
Every recovered page is now a training example — the enhancer directly grew the corpus.
We froze the extractor model and ran it against the same document mix, once with the enhancer in front and once without. Every metric that mattered moved.
On the phone-photograph subset — where perspective distortion made the extractor nearly useless — accuracy jumped 51 points. On scanner artefacts, where the extractor was already limping along, the lift was 33 points.
Across the whole messy pile the aggregate lift settled at +23 points of first-pass field accuracy, from the same extractor weights.
Clean inputs meant cheap labels. Cheap labels meant faster iteration. Faster iteration meant the in-house reader caught up to — and then surpassed — the commercial OCR inside a quarter.
| Before — Preprocessing off | After — Enhancer agent in front |
|---|---|
| ~⅓ of pages effectively bypassed the OCR — the commercial product refused them. | Enhancer recovers 88% of that previously-unreadable pile. |
| First-pass field accuracy on the messy mix: ~72%. | Same extractor, enhanced inputs: 95% (+23 pts). |
| Labelling time: ~8 minutes per invoice (humans starting from raw). | Labelling on enhanced pages: under 90 seconds per invoice. |
| Fine-tune cadence: quarterly, blocked on labelled data. | Weekly fine-tune loop, no longer starved for training examples. |
| Bad pages silently made it into the ERP — verified downstream. | Sub-threshold pages refuse to proceed — automated re-capture ask instead. |
"We spent six months trying to squeeze more accuracy out of the extractor. Then we spent four weeks fixing the inputs — and got more lift than the six months combined. The unlock was staring at us in the raw folder the whole time."
The individual CV operations are decades-old. What made the enhancer valuable was the agent that decided which operation to run — a single filter chain on everything would have blurred the wins.
The re-capture ask isn't the enhancer failing — it's the enhancer preventing bad data from poisoning downstream training. It also nudged the inbound corpus to get cleaner over time.
We shipped Recipe 1 (phone) in week 2 and measured its impact before starting Recipe 2. That kept us honest — no recipe went live until we'd proven the previous one moved the number it was supposed to.
Each recipe is a versioned artefact with its own gold-set. When a downstream regression appears, we can bisect not just the model but the enhancement pipeline that produced its training data.
Two extensions are in flight. First: a recipe for handwritten annotations (initials, stamps, marginalia) that today confuse the extractor. Second: applying the same "inspect-then-recipe" pattern to non-invoice documents where dirty inputs are the bottleneck.