Anyone who has worked with property records knows the pile: inspection reports, appraisals, permits, tax records and certificates, some typed, some scanned from paper, all in different layouts. Founders usually ask the same question first: can software read these and turn them into data?
It can, and the first version is simpler than the vendor pitches suggest. This guide is for founders deciding what to build first when they want to OCR property documents or extract data from property documents. It draws on a property-transparency platform built from concept to production by one engineer, Sikandar: frontend, backend, document pipeline and deploy. The client is not named. The numbers below are settings that were chosen, not measured results.
What should a document pipeline do before any AI?
Four boring things, in order: take the upload, read the text, file the document, and attach it to the right property. Each step is plain code you can test, and together they cover most of what a first release needs. AI belongs after these work, because a model is only as useful as the text and structure you hand it.
Here is the whole path, as a picture:
| Step | What happens | Needs AI? |
|---|---|---|
| 1. Upload | The file goes straight to cloud storage, not through your app server | No |
| 2. Read the text layer | Pull the text the PDF already contains | No |
| 3. OCR scanned pages | Turn page images into text, only where there is no text | No |
| 4. File by rules | Keyword rules set the type and a confidence score | No |
| 5. Property timeline | The document shows on the property it was attached to | No |
Illustrative diagram.
Two details of the upload step matter. First, files go directly to cloud storage, so a large PDF or a video never ties up your server. In the platform this guide is based on, images, PDFs and video could all be uploaded, and very large files could be added as links. Only the PDFs were read for text. Second, decide early how a document finds its property. There, the user picked the address on a map and the document was attached to that property. The contents were not used for matching. Reading an address out of a messy scan and trusting it is harder than asking the person already looking at the right property.
One limit to state honestly: in that build the map loaded only the properties in view, and search was by location, not by document contents. Filing documents and showing them on a property timeline is useful on its own. Full-text search across everything is a separate feature you can add once the text is stored and trusted.
When do you need OCR, and when is the PDF's own text enough?
Try the PDF's own text first, and use OCR only where that text is missing. Many PDFs are made digitally and already contain real text. Reading it is fast, free and exact. OCR, or optical character recognition, is software that reads the shapes of letters in a picture. It is slower, uses more memory and makes mistakes, so it is the fallback, not the default.
You can often tell the difference by looking at what comes back. A scanned page is just an image, so reading its text layer returns almost nothing. The pipeline in this build worked like this:
- Read the text layer first. PyMuPDF, an open-source PDF library, pulled each page's own text.
- OCR the whole document if the text is tiny or garbled. If the total was under 100 characters, or the text looked garbled, Tesseract, an open-source OCR engine, ran on every page.
- Otherwise OCR only the thin pages. A page with under 50 characters was treated as a scan and sent to OCR on its own.
- Keep OCR only if it helps. The OCR result replaced the original only when it produced more text.
- Clean up. Unicode normalisation fixed ligatures (such as the single "fi" character) and whitespace was collapsed.
The garbled check is worth copying. Some PDFs have a text layer that looks present but is nonsense, often from a bad earlier scan. The build flagged that case with two simple signals: an unusual share of punctuation, and a lack of common English words.
Memory deserves a line of its own. OCR on a big document can use a lot of memory, and a small, cheap server will fall over. Here Tesseract rendered one page at a time at 150 DPI and freed the memory after each page, and a full OCR run was capped at 20 pages. There was also a 50 MB file cap for OCR and 50 KB of stored text per document. Your limits will differ. The habit carries over: cap every step and decide what happens when a file goes over.
How do you file documents by type without a model?
With keyword rules that score each document and a cut-off below which the answer is "Unclassified". A rule here is a list of words and phrases that strongly suggest one document type, matched on whole words so that a short term does not fire inside a longer, unrelated one. Each rule adds to a confidence score for its type, and the best score wins, as long as it clears the threshold.
The build filed each document by type or by building system, with a confidence score. As an example, one strong home-inspection trigger was enough to give a score of 0.9. Anything below the threshold was not guessed. It was filed as "Unclassified", which is a real, visible category rather than a hidden failure.
A small example makes the idea concrete. These rules and scores are invented for illustration:
| Trigger found in the text | Filed as | Score |
|---|---|---|
| "home inspection report" | Inspection | 0.9 |
| "building permit" plus "issued" | Permit | 0.8 |
| One weak word, nothing else | Unclassified | 0.3 |
Example figures for illustration. Tune the words and scores on your own documents.
Three habits kept this reliable:
- Detect on the full text. Filing ran on the complete extracted text, before it was trimmed for storage, so a keyword on page 30 still counted even though the stored copy was shortened.
- Prefer "Unclassified" to a wrong label. A wrong label can mislead a reader. An unfiled document is just waiting for a person.
- Add special handling for sensitive types. Appraisals were recognised by keywords and hidden from public view automatically, so the safe default did not depend on a person remembering.
How do you keep it running when files are bad?
Assume files will fail and design so that a failure is visible, retried and recoverable. Real uploads include encrypted PDFs, corrupted files, blank scans and storage hiccups. A pipeline that works on your ten clean test files will meet all of these in the first week of real use.
The recovery rules in this build were deliberately plain:
- A queue with retries. Work went onto a Redis queue, which a background worker processed, with 3 retries per job.
- Recovery at startup. Files left stuck in "processing", for example after a restart, were put back on the queue when the system started.
- A fallback if the queue is down. The app fell back to ordinary background tasks rather than dropping the upload.
- Retrying downloads. Fetching a file from storage was retried 3 times, with a new link each time, because links expire.
- Encrypted PDFs skipped. A locked file cannot be read, so it is passed over rather than allowed to crash the job.
- A failure never hides the file. Any failure marked the document "Unclassified", so it still appeared on the property.
That last rule is the one to insist on. If a failed document silently vanishes, you lose trust and nobody knows why. If it appears as "Unclassified", a person can open it and decide.
A public record also needs a way for readers to push back. The platform let users flag a document as spam, inappropriate, false information or a privacy problem, and an admin resolved each flag. If strangers can see the documents, this is not optional.
For documents the pipeline cannot read well, add a small review step where a person checks the text or the label. Handwriting, damaged scans and inconsistent formats from different local offices are the usual reasons. Our guide on the review queue for AI extraction covers how to set one up.
When is it time for an LLM or a vendor?
When your rules stop being enough for the question you now want answered. A pipeline like this one tells you what kind of document you have and shows it on the right property. It does not read inside the document and pull out the roof age, the permit date or the amount owed. That is where teams usually add a large language model (LLM) or a document-extraction vendor, as a later stage on top of the text you already have.
Vendors in this space pitch extraction into structured data without a template, often with a confidence score for each field. That is a real benefit when you have many layouts and no time to write rules for each. The same weak spots remain, though: scans, handwriting and inconsistent formats from different offices are where any tool, vendor or model, makes mistakes. So whichever you pick, keep a human check for the fields that matter.
| Your need | Usually enough |
|---|---|
| Know what each document is and show it on the property | Text-first pipeline and keyword rules |
| A few fixed fields from one or two known layouts | Rules or simple patterns, plus review |
| Many layouts, fields pulled out of free text | An LLM or a vendor, plus review |
| Low volume, no engineers on hand | A vendor, to avoid building anything |
| Documents too sensitive to send to a third party | Your own pipeline, or a model you host |
Send a model clean text, not raw scans, and treat its answer as a suggestion: check it against rules and show the source next to it. Our guides on what AI features cost and running an LLM on your own server help with the cost and privacy side of that choice.
The sensible order is the one in this guide: a plain pipeline first, a person for the hard cases, and an LLM or vendor once you can name the exact fields you need and have real documents to test them on. To scope the first stage, start with a Build Plan: from $500, ready in 2 to 3 days, credited in full toward the build. A first version like this is what an MVP build covers; the later stage is AI automation. See how this was done in the property-transparency platform case study.
Questions founders ask
Not for a first version. Reading the PDF text, running OCR on scanned pages and filing documents with keyword rules covers a lot. Add an LLM or a vendor later, when you can name the exact fields you need pulled out of the text.
A digital PDF already contains real text, which can be read quickly and exactly. A scanned PDF is only a picture of a page, so OCR has to read the letters from the image. OCR is slower and makes mistakes, so use it only where there is no text layer.
File it as "Unclassified" and still show it. A visible, unlabelled document can be checked by a person. A wrong label can mislead readers, and a document that vanishes after a failure destroys trust.
Not reliably. Scans, handwriting and mixed formats are where OCR tools, vendors and models all make the most mistakes. Plan a human review step for documents and fields that matter.
No. A simple, dependable option is to have the user pick the property, for example an address on a map, and attach the document to it. Reading addresses from messy scans and trusting them is a harder problem to add later.
Technical partner for real-estate and fintech founders: decides with you what to build, launches it and stays on after launch. Top Rated Plus on Upwork with 100% Job Success. Builds MVPs, rescues stalled builds and hardens AI prototypes.
See client work →