Tier 2: visual-only Word forms

⛔ DEACTIVATED (2026-08-31). Tier 2 is turned OFF in production. A document that merely looks like a form (underlined headings, tabular layouts, stray rules) cannot be reliably distinguished from a real one, so field inference produced false positives. The product now converts Tier 1 only (documents with real Word form markup).

Enforced at two layers, both hard-coded — no env flag re-enables it:

• workers/batch/src/sqs-consumer.ts passes detectVisual: false, detectVision: false (the old DOCX_DETECT_VISUAL / DOCX_DETECT_VISION env vars are no longer read).
• services/docx2acroform/app.py ignores the X-Detect-* request headers.

The detector code (detect.py, detect_vision.py), CLI --detect-* flags, the eval harness, and scripts/verify-tier2-enable.py are kept dormant so the capability can be revived if the detector’s precision ever clears the bar below. The rest of this document describes that (currently inactive) design.

Problem

docx2acroform today converts Tier 1 documents — Word files that contain real form markup (sdt content controls, legacy FORMTEXT/FORMCHECKBOX/ FORMDROPDOWN fields, or Office signature lines). extract_fields() finds those, place.py drops AcroForm widgets at their positions, and tag_forms.py wires them into the structure tree.

A large share of real-world “forms” carry no form markup at all. They only look like forms:

• Name: ___________________ (runs of underscores)
• underlined blank spans (Label: followed by underlined whitespace)
• empty cells in a bordered table meant to be filled in
• drawn rectangles / lines used as write-in boxes
• checkbox glyphs (☐, ❑, [ ], ( )) or small drawn squares

For these, extract_fields() returns [] and the CLI prints “No form fields found — nothing to do” and produces no PDF. Tier 2 is the capability to detect these visual affordances and place real, fillable, tagged AcroForm fields on them.

What we already have (and what we don’t)

Grounded in a read of the two nearest systems:

• services/flyer-tagger — does NOT detect fields from appearance. It only tags existing AcroForm widgets (page.widgets()). But its Python label-association primitives are directly reusable:
  • inventory.build_inventory(page) → PageInventory with LineItems (text + bbox + direction) via PyMuPDF get_text("dict").
  • formfields.nearest_label_line(lines, rect) / nearest_label(...) — 36 pt directional label lookup (left / right / above), with alphanumeric and bare-line-number (7a, 25c) rejection already built in.
  • formfields.humanize_field_name(name) — fallback naming.
• FORM_HYBRID_VISION (workers/api, TypeScript) — does NOT detect fields from appearance either; it refines labels/layout of already-extracted fields with Claude. But it is a proven reference for the optional vision path: per-page render → screenshot + compact text-layer TSV → Claude Sonnet → structured output, with PDF↔image coordinate mapping (htmlY = pageHeight − pdfY − textHeight) and recordCost() cost tracking.
• docx2acroform itself — place.create_widgets() already places widgets from spec.page + spec.rect, and tag_forms.tag_form_fields() already tags any widget into the structure tree. Tier 2 reuses both unchanged — it only needs to produce FieldSpecs with page/rect pre-filled.

Net: the detector is new code, but placement, tagging, reporting, and label association are existing building blocks.

Architecture

Tier 1 places by locating text markers; Tier 2 places by detected rects. The two share the back half of the pipeline.

                 ┌── Tier 1: extract_fields() finds markup ──┐
 input.docx ──►  │  mark → LibreOffice → locate → redact     │──► place → tag → out.pdf
                 └───────────────────────────────────────────┘
                 ┌── Tier 2: no markup ──────────────────────┐
 input.docx ──►  │  LibreOffice render (no markers) →        │──► place → tag → out.pdf
                 │  detect affordances on rendered PDF →      │    (create_widgets +
                 │  FieldSpec{page,rect,type,label,inferred}  │     tag_form_fields,
                 └───────────────────────────────────────────┘     both unchanged)

New module: docx2acroform/detect.py — detect_visual_fields(pdf_path, use_vision=False, threshold=DEFAULT_THRESHOLD) -> DetectResult (a struct of fields / candidates / notes / cost_usd), run on the LibreOffice-rendered PDF. It sets page/rect/field_type/tooltip and a new source="inferred" plus a confidence. create_widgets() then places them (skipping locate_markers/ redact_markers, which are marker-only). Optionally redact the underscore/rule glyphs the field covers so the widget sits on a clean background.

Detection strategy

Two complementary detectors, cheapest first:

A. Geometric (primary — deterministic, free, offline)

1. Underscore runs — text spans of ≥ N consecutive _ (tune N≈4). Field rect = union of the underscore chars’ bboxes, height from the run’s font. Label: immediately left/above → tooltip via nearest_label_line.
2. Tab-leader / underlined blanks — get_text("rawdict") spans flagged underline, or trailing whitespace after a Label: with an underline rule.
3. Drawn rules & boxes — page.get_drawings() horizontal lines (write-on rules) and rectangles (write-in boxes) not part of a table border.
4. Table cells — empty cells in a bordered table; label from the header row or the adjacent label cell. (Word tables survive to the PDF as drawn lines + cell text; detect via get_drawings() grid + text occupancy.)
5. Checkboxes — glyphs ☐ ❑ [ ] ( ) in text, or small (~8–14 pt) square drawn paths → CHECKBOX; label to the right (checkbox-style).

B. Vision (optional — higher recall on messy layouts, gated + costed) Reuse the FORM_HYBRID pattern: render each page to an image, build the compact text/rect TSV, ask Claude Sonnet 4.6 for {type, bbox, label, options, required, confidence} per field, map bboxes back to PDF space, place. Gated by a flag; every call recorded to cost_ledger under a new operation type docx-form-detect. Used to fill gaps when geometric recall is low, or as the sole detector when --detect=vision.

False-positive controls (never place a phantom field)

The tool’s existing philosophy is “never silent.” Tier 2 extends it to “never confidently wrong”:

• minimum underscore-run length; ignore single _ and decorative rules in headers/footers (exclude top/bottom margin bands).
• signature-line rules (long full-width rules under “Signature”) → SIGNATURE intent, not a plain text field — or leave to Tier 1 sig detection.
• dedupe overlapping detections (a bordered cell around an underscore run = one field, not two); geometric + vision agreement raises confidence.
• per-field confidence; a threshold below which a field is reported as a candidate but NOT placed (surfaced for user confirmation) rather than silently created or silently dropped.
• hard cap per page with a logged notice if exceeded (no silent truncation).

Reporting & fidelity

Every inferred field is flagged distinctly so nothing masquerades as authored:

• FieldSpec.source = "inferred" (alongside sdt / legacy).
• detected_by: underscore | rule | table-cell | checkbox-glyph | vision.
• confidence: 0–1.
• report gains fields_inferred and a per-field inferred: true; the fidelity_summary grows an inferred bucket. The forms-app FidelityReport surfaces an “Inferred fields — please verify” section (mirrors the existing unplaced/skipped callouts).

Test corpus & acceptance

• Synthetic fixtures (extend tests/fixtures.py): underscore blanks, a bordered fill-in table, tab-leader lines, checkbox glyphs, and a decorative underscore doc (must yield zero fields — the false-positive gate).
• Real corpus: collect visual-only public forms (no fldChar/sdt); label a ground-truth set. Track precision/recall per detector; gate the Phase 3 auto-flip on ≥ 0.9 precision on the geometric pass — we would rather miss a field the user adds than invent one.

Phasing (chosen: hybrid detector, opt-in, place-and-flag)

1. Phase 1 — hybrid MVP (detect.py, behind an opt-in flag --detect-visual / DOCX_DETECT_VISUAL): a geometric detector (underscore runs, tab-leaders, checkbox glyphs) and a Claude Sonnet vision pass that fills gaps / resolves ambiguous layouts, their outputs merged and deduped (agreement raises confidence). Vision calls recorded to cost_ledger under a new docx-form-detect operation type (+ migration). Above-threshold fields are placed and flagged “inferred — verify”; below-threshold ones are reported as candidates, not placed. Ships with synthetic fixtures and a real-corpus precision gate.
2. Phase 2 — depth & tuning (shipped): drawn write-in boxes (get_drawings rectangles; small square → checkbox, wider → text field) and empty bordered table cells (PyMuPDF find_tables, labeled by column header / row label), with table-border rules/boxes attributed to the cell rather than double-counted. Remaining for a later pass: threshold tuning against the real corpus and wider checkbox/rule coverage.
3. Phase 3 — service wiring + UI surfacing (shipped) / auto-tiering + confirm-reject (gated / deferred):
   • Shipped: the batch executor (workers/batch/src/docx-form-executor.ts) now sends X-Detect-Visual/X-Detect-Vision when the deployment enables them (DOCX_DETECT_VISUAL / DOCX_DETECT_VISION env, both default off in sqs-consumer.ts), carries fields_inferred / inferred_candidates / detection_notes and per-field inferred/detected_by/confidence/ tagged through transformReport onto the job row, and books the vision spend under a dedicated docx-form-detect cost_ledger row (migration 271
     • COST_OPERATION_TYPES). The forms FidelityReport surfaces an “Inferred fields — please verify” banner, per-field Inferred badges, a “possible fields we did not add” candidate list, and any detection notes. A markup-free doc that yields only candidates now says how many were spotted instead of a flat “add content controls”.
   • Auto-tiering (implemented): the executor sends X-Detect-Auto (CLI --detect-auto) when DOCX_DETECT_VISUAL=1, so Tier 2 runs only when Tier 1 finds no markup — an already-marked-up form is never polluted with inferred guesses (verified: Laura’s 371-field form adds 0 inferred under auto-tiering, vs 33 with plain --detect-visual). Explicit --detect-visual still runs alongside for testing.
   • Enable step: set DOCX_DETECT_VISUAL=1 in .env.node-server on the .4 host and run scripts/rebuild-server.sh (rebuilds the docx2acroform sidecar with the current detector + recreates the api-node consumers). Nothing changes for existing conversions until this flip.
   • Deferred: the interactive confirm/reject round-trip (promote a candidate to a placed field and re-generate) is a follow-up; today candidates are surfaced read-only for the user to act on manually.

Rollout stays opt-in (default off) until the enable step; when on, it is auto-tiering — Tier 2 augments only markup-free documents.

Reusable-code map

Need	Reuse	Location
text-line geometry	build_inventory / LineItem	flyer_tagger/inventory.py
label association	nearest_label_line, nearest_label	flyer_tagger/formfields.py:90
fallback naming	humanize_field_name	flyer_tagger/formfields.py:70
bare-line-number / dot-leader reject	regex filters	flyer_tagger/formfields.py:49,99
widget placement	create_widgets (from spec.rect)	docx2acroform/place.py
structure tagging	tag_form_fields	docx2acroform/tag_forms.py
vision prompt + coord map + cost	reference pattern	workers/api/.../form-hybrid-converter.ts, form-text-layer.ts

The flyer-tagger primitives live in a separate Python package (services/flyer-tagger/flyer_tagger). Options: (a) copy the ~120 lines of label/inventory geometry into docx2acroform (keeps the CLI dependency-free and self-contained — preferred for a small, stable surface), or (b) factor a shared packages/pdf-a11y-py lib. Recommend (a) now, (b) if a third consumer appears.

Decisions (2026-08-30)

1. Detection approach: hybrid from the start — geometric + Claude vision in Phase 1, merged/deduped (agreement raises confidence).
2. Rollout: opt-in flag first (default off) through Phases 1–2; automatic when Tier 1 finds no markup in Phase 3, once precision is proven.
3. Inferred-field UI: place + flag “verify these” above the confidence threshold; below-threshold fields listed as candidates, not placed.
4. Precision bar for the Phase 3 auto-flip: ≥ 0.9 precision on the real corpus (decided 2026-08-30). Below that, Tier 2 stays opt-in — we would rather miss a field the user adds than invent one.

Effort (rough)

• Phase 1 hybrid MVP (geometric + vision, migration + cost op-type, opt-in flag, place-and-flag reporting, fixtures + precision harness): ~4–5 days.
• Phase 2 drawings/tables depth + tuning: ~2–3 days.
• Phase 3 auto-tiering + forms-UI confirm/reject: ~2–3 days. Independently shippable; Phases 1–2 stay behind the opt-in flag.