OCR Fallback for Scanned Grant Applications with Tesseract

This guide is part of the PDF Grant Application Parsing section within the broader Data Ingestion & Grant Parsing Workflows framework, and it solves one narrow problem: a grant application arrives as a scanned image with no embedded text layer, so the coordinate-based extractors have nothing to read — how do you recover the text with OCR without letting a low-confidence guess masquerade as a fact an auditor will later rely on?

The text-layer probe in the parent stage is the trigger. When it finds no embedded text, it does not throw the document away and it does not attempt to parse pixels for table coordinates. It hands the file to this fallback, which rasterizes each page, cleans it, runs Tesseract, and — critically — reads the engine’s own per-word confidence back out so that any page below a fixed floor is routed to a human instead of being trusted. The recovered text then enters the exact same schema contract the born-digital path produces, carrying the original SHA-256 source digest unchanged.

When to Use This Approach #

Reach for this fallback only when the text-layer check has already failed. Three preconditions must hold:

• The document is a scan, not born-digital. The upstream PDF Grant Application Parsing stage probes the first pages with pypdf; if extract_text() returns essentially nothing, the file is an image and belongs here. If a text layer exists, OCR is the wrong tool — parse it directly with the table extractor, which is faster and lossless.
• The output feeds a regulated artifact. OCR is probabilistic, so recovered figures eventually reconcile against 2 CFR §200.302 financial-management records. That regulation requires records to be traceable to source documents, which is why the SHA-256 digest computed on the original scan is preserved verbatim through every stage and why a low-confidence page is flagged rather than silently accepted.
• You accept a manual-review branch. No confidence floor makes OCR error-free. This design assumes some pages will fall below the floor and be queued for a person; if your workflow cannot absorb that, OCR is not a fit and the document should be re-requested as a native PDF.

Table structure recovery, currency normalization, and downstream reconciliation are explicitly out of scope here. This stage recovers text, not tables — geometric table reconstruction from OCR word boxes belongs to the parent section; canonical field translation belongs to Field Mapping & Normalization; and triage of flagged pages belongs to Error Categorization & Logging. This stage runs synchronously, one document at a time, so page ordering and the audit trail stay deterministic.

Step-by-Step Implementation #

The reference implementation targets Python 3.10+. Tesseract itself is a system binary, so install it at the OS level and pin the Python bindings alongside pdf2image, which shells out to Poppler for rasterization:

pip install "pytesseract==0.3.10" "pdf2image==1.17.0" "opencv-python-headless==4.10.0.84" "pypdf==4.2.0" "numpy==1.26.4" "Pillow==10.4.0"
# system dependencies:
#   Debian/Ubuntu:  apt-get install tesseract-ocr poppler-utils
#   macOS:          brew install tesseract poppler

One contract to internalize before writing code: confidence is an output Tesseract reports per word, not a knob you tune. image_to_data returns a parallel conf list scored 0–100; you read it back to decide whether a page is trustworthy. You never ask Tesseract to “be more confident.”

Step 1: Confirm the empty text layer and preserve the source digest #

The handoff from the parent stage is a file path and the assertion that the document is a scan. Re-confirm that here — defensively, because trusting an upstream flag without checking is how a born-digital PDF gets needlessly (and lossily) OCR’d — and pin the SHA-256 digest of the original bytes. That digest is the immutable identifier 2 CFR §200.302 traceability depends on, and it must travel unchanged all the way to the schema contract.

import logging
import hashlib
from pathlib import Path
from typing import Dict, List
from pypdf import PdfReader

OCR_LOGGER = logging.getLogger("grant_ingestion.ocr")
OCR_LOGGER.setLevel(logging.INFO)
_handler = logging.StreamHandler()
_handler.setFormatter(logging.Formatter("%(asctime)s | %(levelname)s | %(message)s"))
OCR_LOGGER.addHandler(_handler)


def confirm_scanned_source(pdf_path: Path) -> Dict[str, object]:
    """Confirm the empty text layer and pin the source digest before OCR."""
    resolved = pdf_path.resolve()
    with open(resolved, "rb") as fh:
        digest = hashlib.sha256(fh.read()).hexdigest()

    reader = PdfReader(str(resolved))
    embedded = "".join(page.extract_text() or "" for page in reader.pages[:3])
    if len(embedded.strip()) > 50:
        raise ValueError(
            f"PDF carries a {len(embedded.strip())}-char text layer; "
            "parse it directly and do not OCR."
        )

    manifest: Dict[str, object] = {
        "source_digest": digest,
        "source_path": str(resolved),
        "page_count": len(reader.pages),
        "ingestion_route": "ocr_fallback",
    }
    OCR_LOGGER.info("OCR route confirmed | digest %s… | pages %d", digest[:10], len(reader.pages))
    return manifest

The > 50 character floor mirrors the upstream probe exactly, so the two stages agree on what “no text layer” means. A ValueError here is a routing bug worth surfacing, not swallowing.

Step 2: Rasterize pages at 300 DPI with pdf2image #

Tesseract’s documented minimum for reliable character segmentation is 300 DPI; below that, small budget-line type blurs into unrecoverable strokes. Rasterize every page to a PIL image at a fixed DPI so the same scan produces the same pixels on every run.

from pdf2image import convert_from_path
from PIL import Image


def rasterize_pages(pdf_path: Path, dpi: int = 300) -> List[Image.Image]:
    """Rasterize each page to a PIL image at a fixed DPI for reproducibility."""
    if dpi < 300:
        OCR_LOGGER.warning("DPI %d is below the 300 floor; small type may not resolve", dpi)
    pages: List[Image.Image] = convert_from_path(str(pdf_path), dpi=dpi, fmt="png")
    OCR_LOGGER.info("Rasterized %d page(s) at %d DPI", len(pages), dpi)
    return pages

Keep dpi fixed across a corpus rather than tuning per document — a variable DPI makes two runs of the same file non-comparable, which breaks the reproducibility an auditor expects.

Step 3: Preprocess each page for the OCR engine #

Raw scans carry skew, uneven lighting, and JPEG speckle that all degrade recognition. Convert to grayscale, deskew by the dominant text angle, and binarize with an adaptive threshold — a global cutoff fails when one corner of the scan is darker than the other. OpenCV handles all three cheaply.

import cv2
import numpy as np


def preprocess_for_ocr(image: Image.Image) -> np.ndarray:
    """Grayscale, deskew, and adaptively threshold a page for Tesseract."""
    gray = cv2.cvtColor(np.array(image), cv2.COLOR_RGB2GRAY)

    # Deskew using the minimum-area rectangle over the dark (ink) pixels
    coords = np.column_stack(np.where(gray < 128))
    angle = cv2.minAreaRect(coords)[-1]
    angle = -(90 + angle) if angle < -45 else -angle
    if abs(angle) > 0.1:
        h, w = gray.shape
        matrix = cv2.getRotationMatrix2D((w / 2, h / 2), angle, 1.0)
        gray = cv2.warpAffine(
            gray, matrix, (w, h),
            flags=cv2.INTER_CUBIC, borderMode=cv2.BORDER_REPLICATE,
        )

    # Adaptive threshold tolerates uneven scan lighting a global cutoff would clip
    binary: np.ndarray = cv2.adaptiveThreshold(
        gray, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 31, 15
    )
    return binary

The 31 block size and 15 constant are the levers to tune when output is over- or under-thresholded: a larger block averages lighting over a wider area, and a larger constant erodes faint speckle at the cost of thin strokes.

Step 4: Run Tesseract and capture per-word confidence #

Use image_to_data, not image_to_string — the former returns the per-word conf list you need for gating, the latter throws it away. Pass an explicit config so runs are reproducible: --oem 1 selects the LSTM engine and --psm 6 tells Tesseract to assume a single uniform block of text, which fits a dense application page.

import pytesseract
from pytesseract import Output

TESS_CONFIG = "--oem 1 --psm 6 -l eng"


def ocr_page(binary: np.ndarray) -> Dict[str, object]:
    """Run Tesseract and return recovered text plus per-word confidences."""
    data = pytesseract.image_to_data(binary, config=TESS_CONFIG, output_type=Output.DICT)

    words: List[str] = []
    confidences: List[float] = []
    for token, conf in zip(data["text"], data["conf"]):
        conf_value = float(conf)
        if token.strip() and conf_value >= 0:  # Tesseract emits -1 for non-text boxes
            words.append(token)
            confidences.append(conf_value)

    mean_conf = round(sum(confidences) / len(confidences), 2) if confidences else 0.0
    return {
        "text": " ".join(words),
        "word_confidences": confidences,
        "mean_confidence": mean_conf,
    }

Filtering the -1 sentinel matters: Tesseract emits it for layout boxes that contain no text, and including those would drag the page mean toward a meaningless number. An empty page yields a 0.0 mean, which the gate below correctly treats as a failure.

Step 5: Gate on the confidence floor and route low pages to manual review #

This is the load-bearing step. A page whose mean confidence sits below the floor is not handed downstream — it is flagged and queued for a person. The floor of 80.0 is conservative for financial documents where a transposed digit is a compliance event; lower it only if your reviewers are drowning and the field values are low-stakes.

from dataclasses import dataclass, field

CONFIDENCE_FLOOR = 80.0


@dataclass
class PageOcrResult:
    page_number: int
    text: str
    mean_confidence: float
    status: str = field(default="")


def gate_on_confidence(
    page_number: int,
    ocr_result: Dict[str, object],
    floor: float = CONFIDENCE_FLOOR,
) -> PageOcrResult:
    """Route pages below the confidence floor to manual review; never trust silently."""
    mean_conf = float(ocr_result["mean_confidence"])
    text = str(ocr_result["text"])
    if mean_conf < floor:
        OCR_LOGGER.warning(
            "Page %d confidence %.2f below floor %.2f; routing to manual review",
            page_number, mean_conf, floor,
        )
        return PageOcrResult(page_number, text, mean_conf, status="MANUAL_REVIEW")

    OCR_LOGGER.info("Page %d cleared OCR at %.2f confidence", page_number, mean_conf)
    return PageOcrResult(page_number, text, mean_conf, status="ACCEPTED")

The flagged page still carries its recovered text so a reviewer sees Tesseract’s best guess — but its status prevents that guess from being treated as verified data. The WARNING line is the audit record of the routing decision.

Step 6: Feed recovered text into the schema contract #

Accepted pages fold into the same manifest shape the born-digital path emits, so downstream stages never learn or care that OCR was involved. The source_digest from Step 1 passes through untouched, binding the recovered text to exactly one input file, and any flagged page sets the manifest’s overall status to MANUAL_REVIEW.

def assemble_ocr_manifest(
    source_manifest: Dict[str, object],
    pages: List[PageOcrResult],
) -> Dict[str, object]:
    """Fold accepted OCR text into the shared schema contract, digest preserved."""
    flagged = [p.page_number for p in pages if p.status == "MANUAL_REVIEW"]
    accepted = [p.page_number for p in pages if p.status == "ACCEPTED"]
    recovered_text = "\n".join(p.text for p in pages if p.status == "ACCEPTED")

    manifest = dict(source_manifest)  # carry source_digest through unchanged
    manifest.update({
        "recovered_text": recovered_text,
        "pages_accepted": accepted,
        "pages_flagged": flagged,
        "validation_status": "MANUAL_REVIEW" if flagged else "PASSED",
    })
    OCR_LOGGER.info(
        "OCR manifest sealed | digest %s… | accepted %d | flagged %d",
        str(manifest["source_digest"])[:10], len(accepted), len(flagged),
    )
    return manifest

A PASSED manifest is the only thing Field Mapping & Normalization will consume without a human in the loop; a MANUAL_REVIEW manifest is the handoff contract into Error Categorization & Logging.

Verification #

Confirm the fallback behaves deterministically with four checks:

1. The confirm gate rejects born-digital input. Feed a PDF that does carry a text layer and assert confirm_scanned_source raises ValueError — proof that OCR never runs on a document the direct extractor should handle.
2. The confidence gate flags low pages. Pass an ocr_result with mean_confidence below the floor and assert the returned PageOcrResult.status == "MANUAL_REVIEW", and that a page at or above the floor returns "ACCEPTED".
3. A flagged page taints the whole manifest. Assemble a manifest from one accepted and one flagged page and assert validation_status == "MANUAL_REVIEW" and that the flagged page number appears in pages_flagged.
4. The digest survives end to end. Assert assemble_ocr_manifest returns the same 64-character source_digest it received, so re-hashing the original scan reproduces it byte-for-byte — the bind an auditor uses under 2 CFR §200.302.

A clean run emits one INFO line per accepted page and one WARNING per flagged page; ship those to a write-once tier so the routing trail is replayable.

manifest = confirm_scanned_source(Path("scanned_foundation_application.pdf"))
results: List[PageOcrResult] = []
for page_number, image in enumerate(rasterize_pages(Path(manifest["source_path"])), start=1):
    ocr = ocr_page(preprocess_for_ocr(image))
    results.append(gate_on_confidence(page_number, ocr))

sealed = assemble_ocr_manifest(manifest, results)
assert len(sealed["source_digest"]) == 64
assert sealed["validation_status"] in {"PASSED", "MANUAL_REVIEW"}

Common Errors & Fixes #

Related #

• Parent section: PDF Grant Application Parsing
• The born-digital counterpart that runs when a text layer exists: Extracting Tables from Grant PDFs Using pypdf and Camelot
• Choosing an engine once a text layer is present: Camelot vs pdfplumber for Grant Budget Tables
• Where flagged and failed pages are triaged: Error Categorization & Logging