Handwritten forms and regional languages: what document AI can do

What document AI does with a handwritten form

Document AI turns a scanned or photographed form into structured data. It finds the fields, reads what is written in each one, and passes the values to the system that acts on them. On printed text this is mature work. On handwriting it is harder, and on handwriting in several scripts it is harder again.

The useful question for a buyer is which fields can be read reliably enough to act on, which need a check, and which should go to a person every time. A good project answers that field by field, with evidence from your own forms.

Scan quality sets the ceiling

No model recovers information that the image does not hold. Faint ink, folds across a field, stamps over a signature box, skewed pages, shadows from phone cameras and low-resolution scans all cut accuracy before any reading starts.

Before you evaluate a vendor, look at how your forms are captured. Often the cheapest accuracy gain is at the counter or the scanning station: a consistent scanner setting, a flat page, a full-page capture. Build an image quality check into the pipeline so that a poor scan is flagged at the start, while the paper is still at hand and can be scanned again.

• Resolution and contrast good enough to read by eye
• Whole page captured, straight and flat
• Stamps, staples and folds kept clear of key fields
• Poor images rejected at capture, with a reason

Handwriting variation and mixed scripts

Handwriting varies by person, by pen and by hurry. Some writers stay inside the boxes and many do not. Digits get overwritten, corrections are struck through, and names are shortened in ways only a local clerk would recognise.

Regional languages add more. A single form can carry a printed template in one language, a name in a regional script, an amount in numerals and an account number in Latin digits. Some scripts use joined characters and vowel marks that sit above or below the line, where they collide with a box border or the line above. Some characters look alike when written fast. People also write the same name in different scripts on different forms, so transliteration matters when you match records.

Ask for results on each script and each field type separately. An average across everything hides the weak spots, and the weak spots are where your errors will come from.

• Results reported per script
• Results reported per field type: names, amounts, dates, numbers
• Test forms from many writers and locations
• A plan for names written in more than one script

Validate fields against data you already hold

Reading is half the job. The other half is checking what was read against what you know. An account number has a fixed length and often a check digit. A branch code must exist. A date must be plausible. An amount in figures should agree with the amount in words. A customer name should match the name on the account, allowing for spelling and transliteration differences.

These checks catch errors the model cannot see on its own, and they turn a doubtful reading into a confirmed one. They also catch mistakes made by the person who filled in the form, which no reading model will fix.

• Format, length and check-digit rules
• Lookups against master data
• Cross-checks between fields on the same form
• Name matching that tolerates spelling and script differences

Confidence thresholds and routing uncertain cases to people

Each reading comes with a confidence score. Treat it as a signal to calibrate on your own forms, never as a guarantee. Set thresholds per field, based on the cost of a mistake. A wrong amount or account number on a money transfer is serious. A wrong value in an optional remarks field is minor.

Use three lanes. High confidence with passed validation goes straight through. Middle confidence goes to a reviewer, who sees the image of that field next to the proposed value and confirms or corrects it. Low confidence or failed validation goes to full manual handling. Design the review screen with care, because a reviewer who can clear a field in seconds makes the whole process work. Keep every correction: it is your best training and test data.

We built automated processing of physical mandates and other money-transfer forms for an Indian bank, reading handwritten text in Indian regional languages. It has run with high accuracy and near-zero failures since launch. That is the result of one engagement, with its own forms and controls. Yours will need its own evidence.

Measure accuracy honestly

Character accuracy flatters. A form with one wrong digit in an account number is a failed form, however many characters were right. Measure at field level and at document level, on a test set the model has never seen and that matches real intake, with bad scans, rarer scripts and messy handwriting included.

Then measure the whole process, since validation and review catch some of the reading errors. Track the straight-through rate, which is the share of forms that needed no human touch. Track the error rate among those straight-through forms. Nobody checked them, so this is the number that matters most. Sample again at regular intervals after launch, because forms, pens and staff change.

• Field-level and document-level accuracy
• Results split by script, field and scan source
• Straight-through rate
• Errors found in a sample of straight-through forms
• Review time per form

What to do next

Collect a few hundred real forms, including the worst scans. List every field, the cost of a wrong value, and the data you could validate it against. Ask any vendor for field-level results on that set. If you want a wider view first, our free AI readiness report starts with six questions.