Bulk PDF Remediation — Build Brief

Reference document for epic #3150 and issues #3151–#3157. Every one of those issues links here by path.

Status legend used throughout: ✅ merged · 🚧 in flight · ⬜ not started.

---

1. Problem

The dashboard is built for one document at a time. A university with 40,000 published PDFs and an April 2027 deadline cannot use it. We need a bulk path: watch a folder, remediate what can be remediated without a human, and route the remainder to a review queue — with an evidence trail defensible to counsel.

Compliance deadlines (ADA Title II / DOJ rule, Federal Register 2026-07663):

Deadline	Applies to
April 26, 2027	Populations ≥ 50,000
April 26, 2028	Populations < 50,000

---

2. The product boundary: two tiers, not three

The codebase already names this split — flyer_tagger/preflight.py has Route = Literal["in-place", "reflow"]. We adopt it and rename it in customer terms:

• Polish — the pixels are the product. Output renders identically; accessibility added invisibly underneath.
• Convert — the content is the product. Re-authored as accessible HTML. Layout necessarily discarded.

The criterion is pixels, not text layer:

A document needs convert if any pixel has to change to make it accessible. Everything else is polish.

This is easy to get wrong in the obvious direction. “Image-only” does not imply convert — an image-only page can be tagged <Figure> with alt text, or handled by OCR mode writing /ActualText over the untouched raster, both pixel-identical and therefore both polish. Conversely a document with a perfect text layer and 3:1 body contrast does need convert, because the fix moves pixels. Text-layer presence selects a strategy inside polish; only the pixel question selects the tier. See §7.2 — this distinction was mis-stated in an earlier draft and is the single easiest error to make in this system.

2.1 pdfpolish and flyer-tagger are one capability

They are separated by history, not by a product boundary. Both preserve pixels. Both spend money on vision. Polish is one capability with two entry conditions:

Tag tree	Strategy	Today’s implementation
Usable tree exists	repair	pdfpolish / polishTaggedPdf()
No usable tree	build	flyer-tagger Lambda

Either way: same verify gate, same report schema. flyer-tagger is not deleted — it is demoted from a product to a strategy behind polish.

2.2 ⚠️ Never destroy an existing tag tree

The build path is unconditionally destructive today:

• inject.py:20 — pre-existing BDC/BMC/EMC/MP/DP markers are stripped
• structure.py:838 — root.StructTreeRoot = struct_root, unconditional overwrite
• preflight.py — has no notion of whether the document is already tagged

Correct for an untagged Canva flyer (nothing to lose); catastrophic in bulk. A document with a decent-but-imperfect tree routed into the build path silently discards human-authored structure — including manual remediation the customer already paid a vendor for. At 40,000 documents that is unrecoverable.

Rule: repair is the default whenever a usable tree exists. Rebuild is the exception — it requires a recorded justification, keeps the discarded tree recoverable, and is rejected outright if it scores worse than what it replaced.

Three guards, not a threshold (#3151, merged): repair-first · rebuild-must-beat- original · discarded-tree-recoverable.

2.2a What flyer-tagger actually does to an existing tree (current code)

The §2.2 warning describes the emission step and is still literally true — structure.py:1118 does root.StructTreeRoot = struct_root unconditionally, so the output’s tree is always newly built, never the source’s tree extended.

But the source tree is no longer ignored. Since #2291 the pristine /StructTreeRoot is captured once, before the overwrite (cli.py:456 — it must be held, because build_structure rebinds pdf.Root.StructTreeRoot on the first page, so re-reading it on page 2+ would walk the new partial tree and silently disable every prior). Four mechanisms then feed it back in:

Issue	Module	What it preserves
#2291	source_prior.py	Segmentation + roles. Each inventory line carries src = "B<n>:<role>", exposed to the vision model as an sr hint and enforced on headings in the retry loop. The author’s own block boundaries beat any geometry heuristic.
#2387	source_field_order.py	Author’s declared form-field order, extracted before /Tabs=/S is set. Otherwise field order is re-derived geometrically, which scrambles stacked field rows (e.g. the f1040 header).
#3208	structure.py:815	List-ness. If the source knew a run was a list, <LI> is restored when analysis had reclassified it to <P>.
#3214	structure.py:685	Table header cells. Source <TH> pins the TH/TD call and /Scope — necessary because Sonnet-4.6 is not bit-deterministic even at temperature=0, and the header call was observed to flip between runs on the same table.

So the accurate description is “rebuild informed by the source prior,” not a blind overwrite. All four are best-effort and degrade to “no hints” rather than failing a page.

In the bulk path, however, none of this assessment is actually consulted. flyer-tagger has a real three-state verdict — tree_assess.assess_tree() returns absent / present-and-usable / present-and-broken, where “broken” means any of:

• not linked (no /ParentTree, or no page/annot /StructParents), or
• element_count <= 1 (an empty /Document shell), or
• total_marks == 0 (tree carries no marked content), or
• content_coverage < COVERAGE_FLOOR (0.80, provisional)

The bulk state machine does not call it. It dispatches on audit.score.hasTagStructure (bulk-remediation-state-machine.ts:256), which the scorer computes as a raw byte-string presence test — containsPattern(pdfStr, '/MarkInfo') && containsPattern(pdfStr, '/StructTreeRoot') (pdf-accessibility-scorer.ts:665). Not a parse. Just “do both tokens appear in the file.”

preflightVerdictFromAudit then maps hasTags ? 'present-and-usable' : 'absent' — it is structurally incapable of emitting present-and-broken.

Consequence: a document with a present-but-broken tree is dispatched to repair, because the tokens are there. Repair cannot fix a tree that is not linked to content, so it fails to reach conformance and falls through to review.

This is fail-safe on the destruction axis — it can never rebuild over an existing tree, which is exactly the §2.2 hazard — and it is free in API terms. But it converts an API cost into a reviewer-hours cost, and reviewer time is the scarcer resource. See §6.4a. The code comment at bulk-remediation-state-machine.ts:124-126 already flags this: wire the real flyer_tagger.preflight verdict when #3155 lands.

One exception to “fail-safe,” worth closing with the same work. The token test is an AND. A document carrying /StructTreeRoot but lacking /MarkInfo scores hasTagStructure: false and is dispatched to build — which overwrites the tree it does have. That combination is not hypothetical: 6.2-1 (MarkInfo/Marked=true) is fix #14 on the census ladder. The corpus contains documents with a real tree and no /MarkInfo, and today every one of them takes the destructive path.

(The related compressed-object-stream concern does not apply: the scorer inflates /ObjStm payloads and appends them before matching, so a tag tree inside a compressed object stream is still seen — provided inflateSync is available, which it is on the Node batch worker. It silently no-ops on an edge runtime.)

2.2b ⚠️ Gap: authored content is NOT preserved

Everything harvested above is structural — segmentation, roles, field order, list-ness, header-ness. What is not harvested is the authored content: the text a human actually typed, which is the expensive half.

Confirmed absent from the build path (no source-/Alt read exists in structure.py or cli.py):

Attribute	What it is	Preserved?
/Alt	alt text on Figure elements	❌ no
/ActualText	text replacement for a run	❌ no
/E	expansion of an abbreviation	❌ no
/Lang	per-element language (multilingual docs)	❌ no
/T	authored title on a structure element	❌ no
Link /Contents	alternate description for a link annotation	❌ no
/Headers / /Scope	beyond the two anchored geometries (#3214)	partial

This is precisely the “manual remediation the customer already paid a vendor for” hazard in §2.2, and it is the high-value half. A document routed to build loses its human-written alt text and receives machine-written alt text in its place.

The existing guards do not catch this:

• The score will not catch it — veraPDF 7.3-1 checks that /Alt is present, never that it is good. Replacing a careful human description with a generic machine one scores identically, or better.
• The rebuild-must-beat-original guard compares scores, so it passes.
• The pixel diff is zero — nothing visual changed.

So a silent quality regression can pass every gate we have. At 40,000 documents, against a customer who has already paid for partial remediation, this is the most damaging failure mode in the system — and the least visible.

Recommendation (not yet filed): extend source_prior into a source authored-content prior that harvests the attributes above keyed by MCID → block, and re-applies them to the newly built tree, with generated alt text used only as a fallback where the source had none. Add a specific non-regression assertion to the #3151 guard set: never replace a non-empty source /Alt (or /ActualText, /E, /Lang) with a generated value. That is a content-level guard, orthogonal to the score-level and pixel-level guards that already exist.

Note the interaction with §7.2: the 60% A+B population is defined by missing authored items — overwhelmingly alt text. Any customer who has had a prior remediation pass will have documents where that alt text is present and human-written. Those are exactly the documents this gap destroys.

2.3 The polish promise, stated precisely

“Polish does not change a pixel” is verified, not asserted — flyer_tagger/verify.py renders before/after at 150 dpi and diffs. Reference fixture: 0 differing pixels across ~4,800 veraPDF checks.

Three operations would break it and are classified rather than hidden:

Operation	Effect	Default
recolor.py	rewrites fill colour of a low-contrast text run; geometry untouched	opt-in
border_recolor.py	rewrites stroke colour of a low-contrast border; geometry untouched	opt-in
flatten.py	bakes widget appearances, drops form layer — gives up interactivity	opt-in

So polish has two modes: polish:strict (default — paint ops only ever wrapped, pixel identity by construction) and polish:appearance (opt-in, permits recolors; render diff becomes bounded rather than zero). flatten is a separate consent.

2.4 Polish is not the “free tier”

Polish already runs vision for alt text, describes links, and in build strategy makes a vision call per page. Cost does not define the tier — the pixel promise defines the tier. Measured production cost confirms this: polish-build is currently the most expensive operation per document (§6).

Scope principle: polish does everything machine-checkable that can be done in place — which is very nearly everything. If a check is machine-verifiable and fixable without moving a pixel, the answer is not “route to a human” — it is extend polish.

---

3. Where documents come from

3.1 Customer bucket watch ✅ (primary)

s3_integrations (migration 111) — one row per watched folder:

• provider: aws · r2 · b2 · s3_compatible
• input_prefix / output_prefix
• credentials: AssumeRole (AWS) or KMS-sealed access keys. Postgres never sees plaintext; decryption happens only in integration-creds.ts.
• detection: event (workers/r2-event-receiver) or poll (s3-poll-worker.ts)

A schema CHECK refuses an output_prefix at or under input_prefix. This is load-bearing: it is what prevents our own outputs re-entering the pipeline as new input. Do not relax it.

3.2 SharePoint / OneDrive ✅

Graph ingest rides the same s3_ingest queue, dispatched on type: 'graph_ingest' (epic #1833). For a university this is likely higher volume than S3 — their PDFs live in SharePoint and on the website, not in a curated bucket.

3.3 URL list / Siteimprove import ⬜ (not built)

Siteimprove crawls the customer’s site and inventories every PDF it finds as a URL, not a file — with page count, discovered-on pages, and its own accessibility flags. That inventory is exportable as CSV/XLSX and available via their REST API. Their own PDF remediation is a partner referral, which is precisely the seam we sell into.

A URL list inverts the writeback model. Bucket and SharePoint ingest both write the fixed file back where the original lived, preserving folder structure (s3-integration-writeback.ts) — that is what makes them hands-off. A Siteimprove export gives us https://university.edu/docs/handbook.pdf and no write target.

This makes URL-list a different product shape, not a third adapter:

	Bucket / SharePoint	URL list (Siteimprove)
Loop	closed	open
Writeback	in place, folder structure preserved	none — download bundle
Ends at	file replaced	human re-uploads via their CMS
Deliverable	remediated files	remediated files + put-it-back manifest

Recommendation: build URL-list ingest first as a discovery and triage product — point it at a Siteimprove export and return “here are your 12,400 PDFs, X% already pass, Y% we can fix untouched, Z% need a human, here is the cost.” That report is what sells the bucket integration, and it reuses the #3154 dry-run estimator wholesale.

Caveat to state to customers: a crawler’s inventory is what the site links. It is not what is in the CMS. Expect the bucket to contain files Siteimprove never saw.

---

4. The pipeline

ingest → sha256 → dedupe? → triage (audit) → route
   passthrough → writeback original        → done
   polish      → repair | build → verify   → done | review
   convert     → (policy-gated, off today) → review
   review      → human queue
   paused      → budget stop-loss (#3154)

The concrete gap this epic closed: s3-ingest-consumer.ts fired one unconditional convert per ingested object. The work was replacing that sendMessage(convert) with a state machine.

4.1 Routing ✅ (#3152)

packages/shared/src/bulk-routing.ts — a pure, table-driven router. No I/O, no infrastructure. Four routes, seven named reason codes. It consumes the preflight verdict rather than reimplementing the classifier, so the single-document and bulk paths can never disagree.

The pass gate. A veraPDF-clean document is NOT automatically finished. Pass-through requires all of:

1. veraPDF PDF/UA-1 clean, and
2. verapdfRan — the validator actually ran, and
3. structural score ≥ per-folder threshold (default 95), and
4. zero high-severity manualRisks

Fail-closed. A validator that did not run, or a manual-risk analysis that was degraded or never ran, can never produce a pass-through — it produces manual-risks-unverified → review. manualRisks exists precisely because conformance checkers pass documents a screen-reader user cannot read.

Reason code	Route
clean-and-low-risk	passthrough
high-severity-manual-risks	review
manual-risks-unverified	review
polishable-in-place	polish
reflow-convert-permitted	convert
convert-not-permitted	review
not-remediable	review

4.2 Nothing is ever silently skipped

“Skip” is two different things and the system separates them:

• A clean document is passed through — copied to the output bucket, recorded as done, counted as compliant, given a provenance record.
• A document we cannot fix becomes a review row with a reason.

Neither is a silent drop. A batch report reading “8,000 files, 3,000 skipped” is worthless to an ADA coordinator.

4.3 Dedupe on content, not path ✅

remediation_items keys on content_sha256, not source_key. Customers re-run the same folder monthly as the deadline approaches; unchanged content must cost near zero. This also makes a budget-paused batch resumable for free (#3154) and is the single highest-leverage decision in the schema.

4.4 Keep-best guard ✅

“Best” is seeded with the original, so a polish attempt that regresses the score is never shipped. Every attempt is recorded in attempts[] — a downgrade is never silent. MAX_ATTEMPTS = 2.

---

5. The audit contract — what runs where

5.1 Two optional checks exist in the interactive audit

apps/audit/src/components/PdfAuditPanel.tsx exposes two checkboxes, wired to the deep and ai flags on the audit job message:

Checkbox	Flag	What it is	Metered?
”Also check color contrast and form-field visibility (beta — slower, renders every page)“	deep	#2968 Phase 2 raster tier: color-contrast, non-text-contrast. Needs a live headless browser to rasterize every page.	No — shipped explicitly unmetered. Cost is compute and wall-clock, not tokens.
”Also check math/equation alt text and screen-reader coherence with AI (beta — slower, uses AI)“	ai	#2968 Phase 3 + #3162: math-alt, screen-reader-coherence, alt-quality, text-quality, visual-quality, link-quality — now six checks, not two.	Yes — real model calls, booked per-check to cost_ledger.

Both default off. They are orthogonal by design: a raster check failing outright is a different failure mode than an AI call failing, and folding metered AI spend under the already-documented unmetered deep flag would silently change what that flag means.

5.2 Neither runs in the bulk path — in triage or verify

bulk-remediation-executor.ts :: auditBytes() calls exactly three things:

1. runVerapdfSummary(bytes)
2. scorePdfAccessibility(bytes, { mode: 'full', verapdf })
3. analyzeManualReviewRisks(bytes) — note the signature takes only bytes; it has no deep/ai options to pass

There is no deep branch and no ai branch. So:

The post-remediation audit is byte-for-byte the same function as the pre-remediation audit, and both are weaker than what a customer gets from the interactive audit UI.

verifyPasses() deliberately re-runs the same route() gate over the remediated bytes and demands it return passthrough — the output must clear the exact bar an untouched document would. That symmetry is correct and should be preserved: if verification were weaker than intake, we would ship remediated files that fail their own pass gate.

5.3 The defect this creates

A document can pass bulk verify, be written to the customer’s output bucket and counted as compliant — and then fail the interactive audit the customer runs on that same file, because the interactive audit can see contrast and alt-text quality failures the bulk gate never looked for.

That is a defensibility problem, not just a coverage gap. It must be closed before the batch evidence report (#3157) is sold as evidence.

5.4 Recommendation: deep at triage, ai at the pass gate

The two flags belong in different places, for different reasons.

deep must run at triage — it is the tier classifier

This is not an optional quality upgrade. Per §7.2, the polish-vs-convert decision turns on “does any pixel have to change,” and contrast / form-field visibility is the only instrument we have that measures it. veraPDF cannot see contrast at all.

Without deep at triage, the router is structurally blind to the criterion that defines the tier. Today it routes on text-layer presence (preflightVerdictFromAudit derives route: hasText ? 'in-place' : 'reflow'), which §7.2 shows is the wrong axis. A document with a clean text layer and 3:1 body text routes to polish, polishes “successfully,” passes the gate, and ships — still unreadable.

Affordable because deep is unmetered: no tokens, only browser render time. At 40,000 documents that is a throughput constraint, not a budget one — size the browser pool accordingly, and cap pages via analyzeDeepPdfRisks({ maxPages }) (default MAX_DEEP_AUDIT_PAGES, 50 since #3385) and bound wall-clock with renderDeadlineMs. A run cut short by the deadline reports fewer pagesRendered and must be routed the same way as a capped one. pagesRendered < pagesTotal is already reported honestly in PdfDeepAuditStatus and is never a clean pass for the unrendered pages; the router must treat a capped or failed deep run as fail-closed → review, on the same principle as manual-risks-unverified.

ai stays off at triage

1. Mostly wasted there. The AI checks are diagnostic — they discover alt text is missing or meaningless. In bulk, polish is about to write that alt text anyway. Paying to diagnose what the next step fixes unconditionally is pure loss at 40,000×.
2. Wrong side of the cost curve. Triage runs on 100% of documents; the pass gate runs on a fraction. ai is genuinely metered (§6).

Population	deep	ai	Why
Triage, all documents	on	off	deep is the polish/convert classifier; ai would diagnose what polish is about to fix
Passthrough candidates	on	on	exactly where a machine-conformant-but-inaccessible document escapes unexamined
Verify, post-polish, before done	on	on	detects “alt text present but meaningless” — the failure veraPDF structurally cannot see. But see §5.5: at default severities the gate will not act on it.
Review queue items	on	on	feeds the reviewer’s checklist

5.5 ⚠️ The severity ceiling — why ai at the pass gate does not yet work

The pass gate counts high-severity manualRisks only. Of the six AI checks, only four can emit high, and the conditions are narrow:

Check	Can emit high?	High-severity condition
alt-quality	✅	figure AI confirms is meaningful has no alt text at all (missing ? 'high' : 'medium'). Fires only when meaningful && !altHelpful && confidence ≥ 0.5
math-alt	✅	likely equation on a page where no figure/formula has alt text. Downgrades to medium if the page has any alt text, or an alt-texted element’s page is unresolved
screen-reader-coherence	✅	model-assigned, over orphaned-caption · heading-mismatch · alt-text-contradiction · reading-order-break · other (max 8 findings)
link-quality	✅	model-assigned — link text does not describe its destination
text-quality	❌	medium for an unmarked-language passage, low for a heading that may not describe its section. Documented as needs-review, never a hard AA fail
visual-quality	❌	use-of-color and image-of-text both hardcoded 'low'

The problem: the failure we are most exposed to is alt text that is present but wrong — because polish writes alt text on every Figure, so /Alt is always present after a build pass. That case scores medium, and medium does not block. A document polish just filled with generic or inaccurate alt text passes veraPDF 7.3-1, passes the structural score, passes deep (pixels are identical by construction in strict mode), and passes the gate — with alt-quality findings sitting in manualRisks, unread.

So enabling ai at the pass gate as §5.4 describes would detect the defect and then ship it anyway.

The severity is correct at triage and wrong at verify, because the meaning differs by context:

• At triage, “unhelpful alt text” is a pre-existing condition. medium is right — it is a finding, not a blocker.
• At verify, “unhelpful alt text” means we just wrote that. It is our own defect, on a document we are about to certify. It should block.

Required: a context-dependent threshold — the verify gate must treat alt-text-quality (and math-alt-quality) medium as blocking, while triage keeps today’s behaviour. Do not fix this by raising the severity globally; that would flood the triage review queue with pre-existing findings and invert the cheapest-first ordering (§8.1).

Note for #3156: coherence risks carry page: null by design — they are relative to the announced tag-order stream, not a page. The reviewer checklist cannot pin them to a page and must present them as document-level items.

Consequence for the router

bulk-routing.ts currently has no contrast input. Wiring deep at triage means extending PreflightVerdict (or adding a sibling signal) with the deep verdict and adding reason codes for “pixels must change → convert” and “deep audit did not run → review.” That is a change to the pure router, and it is the correct place for it — the decision must be visible and testable, not buried in the executor.

---

5.6 Per-document confidence score (spec)

The customer-facing promise is “we report confidence, not a checkmark.” This is the spec behind that claim. Nothing composes it today — every input exists; the composition does not.

The governing idea: confidence is COVERAGE, not quality

The structural score already answers “how good is this document.” Confidence answers a different question: “how much of the accessibility space did we actually examine, and how strong was the method?” A document can score 100 and warrant low confidence (only 15 of its 385 pages were looked at), or score 60 with high confidence (we examined everything and it is genuinely bad).

Keep them separate. Never blend them into one number — the whole value of the claim is that it is honest about its own limits.

1. Requirement areas

Confidence is computed per area, then composed. Areas map to what a complaint actually alleges, not to the internal check layout:

Area	Deterministic source	Judgment source
Structure (tags, headings, lists, title)	veraPDF + scorePdfAccessibility	—
Document language	veraPDF 7.2-33/34	text-quality (language of parts)
Reading order	tag order walk	screen-reader-coherence
Tables	TH/TD/scope/Headers present	complex semantics → human
Images / alt text	/Alt present (veraPDF 7.3-1)	alt-quality — is it meaningful
Links	tagged + /Contents	link-quality — does it describe the target
Forms	field labels, /TU	—
Contrast	deep raster: color-contrast, non-text-contrast	—
Math / equations	—	math-alt
Images of text · use of colour	—	visual-quality
Resize / reflow	❌ nothing measures this today	—

2. Per-area verdict — four states, and the fourth is the important one

verified   — deterministic check ran, on adequate coverage
judged     — AI check ran; carries the model's own confidence where available
             (alt-quality exposes verdict.confidence, floor 0.5)
n/a        — the document contains no such content
unexamined — check did not run, ran degraded, or was page-capped

n/a is what makes the score honest in the customer’s favour, and it is the half competitors’ “manual review required” boilerplate gets wrong. A policy memo with no figures, no tables, no forms and no math has four fewer places to be uncertain — its confidence should be higher, not penalised for checks that were never relevant.

Applicability is deterministic and free — the risk model already walks the structure tree. Decide it from content presence (any Figure → alt-text area applies; any Table → tables; any Link; any form field; math-alt candidate detection; any text → contrast). Never infer n/a from “the check found nothing” — that conflates absent content with clean content, and it is the one error that would make the score a lie.

3. Page coverage is a first-class term

Every vision and raster check caps at 15 pages (§6.4b). On UT Southwestern’s 385-page outlier that is 3.9% coverage — and a naive score would report it as confidently as a 2-page memo.

pageCoverage = pagesActuallySampled / pagesTotal
pagesActuallySampled = deepAudit.pagesRendered − deepAudit.pagesSkippedForRotation

pagesSkippedForRotation is already reported for exactly this reason: those are real pages that were rendered but never sampled, and they are not a clean pass. Per-page areas (contrast, math, visual-quality, alt-quality) have their assurance multiplied by pageCoverage. Whole-document areas (structure, language, links, forms) do not — veraPDF reads the entire file.

4. Composition: gate on the weakest link, never average

areaAssurance:  verified 1.00 · judged 0.75 · judged-low-confidence 0.50
                · unexamined 0.00 · n/a excluded from the calculation

confidence = min( areaAssurance × pageCoverageFactor ) over APPLICABLE areas

Use a minimum, not a weighted mean. A mean lets a strong structural score paper over a contrast check that never ran — precisely the hole §5.4 exists to close. The minimum makes one unexamined applicable area cap the whole document, which is the correct and conservative behaviour.

Report the band plus the binding constraint, never a bare number:

Band	Condition
Certified	every applicable area verified, page coverage 100%
High	every applicable area verified or judged; coverage ≥ 90%
Moderate	all applicable areas examined; coverage ≥ 50%
Low	coverage < 50%, or any area judged-low-confidence
Insufficient	any applicable area unexamined

Insufficient is a gate, not a grade — it pairs with the existing fail-closed rules (manual-risks-unverified, a deep run that did not complete). A document cannot ship as compliant at Insufficient.

5. The legal-risk view — Larry’s point, made reportable

Areas are not equal in exposure. Report a risk-weighted subset separately, so the deterministic strength is not diluted by soft areas:

Complaint-critical areas — contrast, forms, tables, structure, reading order — are all verified deterministically. Image-description quality is judged at 0.87 mean confidence. Math and language-of-parts: not applicable.

That sentence is defensible, specific, and better than any competitor’s “manually reviewed.” It is also the honest shape of the claim: certify what is measured, qualify what is judged, name what does not apply.

6. Known gap to disclose, not hide

Resize / reflow (WCAG 1.4.4, 1.4.10) is measured by nothing today. Under this spec it is unexamined wherever it applies, which caps affected documents at Insufficient. That is the correct answer and it will be uncomfortable — the fix is to build the check (it pairs naturally with the §7.2a convert criterion, since reflow failure is a convert trigger), not to drop the area from the model.

7. Where it surfaces

• remediation_items — a confidence JSONB: band, per-area verdicts, page coverage, and the binding constraint
• Per-file sidecar manifest (§9.2)
• Batch report (§9.3) — distribution of bands across the batch
• Review queue — Insufficient items sort toward the top; the binding constraint is the reviewer’s first checklist item

8. Acceptance

• A document with no figures/tables/forms/math reports those areas n/a and is not penalised
• A 385-page document reports its true page coverage and cannot reach High
• One unexamined applicable area forces Insufficient regardless of score
• The band is reproducible from the stored audit result — no hidden state
• Confidence is never substituted for the pass gate; both are recorded

6. Cost model, and how to measure it

6.1 We have no production data for the AI checks

Queried against prod (vuvwmfxssjosfphzpzim): there are zero cost_ledger rows for any audit-* AI operation type. The only audit-report rows carry $0.00. The AI audit has effectively never run at volume in production.

So the honest answer to “how much more expensive are those checks” is: we do not know yet, and we do not have to guess — the instrumentation to find out is already built and shipped.

6.2 The measurement is already wired

pdf-audit-executor.ts books every AI check to cost_ledger under a distinct operation type:

Check	operation_type
math-alt	audit-math-alt
screen-reader-coherence	audit-coherence
alt-quality	audit-alt-quality
text-quality	audit-text-quality
visual-quality	audit-visual-quality
link-quality	audit-link-quality

analyzeAiPdfRisks() returns costEntries per check, and PdfAiAuditStatus reports estimatedCostUsd — bucket D’s meaningful “how much ran” is dollars, not pages.

The procedure:

1. Take a stratified sample (~200 docs) from the UT corpus spanning page counts and the census classes.
2. Run the #3154 dry-run/triage pass with deep: true, ai: true.
3. Query:

   SELECT operation_type,
          COUNT(DISTINCT file_id)                       AS docs,
          ROUND(AVG(estimated_cost_usd)::numeric, 5)    AS avg_per_call,
          ROUND(SUM(estimated_cost_usd)::numeric, 4)    AS total_usd
     FROM cost_ledger
    WHERE operation_type LIKE 'audit-%'
    GROUP BY 1 ORDER BY total_usd DESC;

4. Regress per-document cost against page count to get a projection formula for the estimator.

Until that runs, no AI check goes on by default in bulk. This is the same fail-closed discipline as the pass gate.

6.3 What we do know — measured production cost per document

From cost_ledger, rolled up per document (not per call — per-call averages badly understate multi-call operations):

Operation	Docs	Avg / doc	Median	p95	Max
flyer-tagging (polish build)	114	$0.706	$0.314	$3.551	$4.091
conversion (convert)	2,740	$0.078	$0.006	$0.342	$6.343
visual-polish	472	$0.002	$0.002	$0.003	$0.007
polish repair (pdfpolish)	—	$0.000	—	—	—

Repair is deterministic (pikepdf / pdf-lib, no AI) and costs nothing — the executor now returns costCents: 0 for it explicitly.

6.4 ⚠️ This inverts an assumption in the epic

Issue #3154 states “Convert is considerably more expensive.” The measured data says the opposite: polish-build averages ~9× convert per document ($0.706 vs $0.078).

Three caveats, all pointing the same way — the gap is real but much smaller than raw figures suggest:

1. Both distributions are heavily right-skewed (median ≪ mean), so averages are long-tail-driven.
2. The populations are not matched — see §6.4c, the flyer-tagger sample is dense marketing flyers, not corpus-representative text documents.
3. The measured rate predates two optimizations: a −30% reduction already landed, and batch inference is a further −50%. Together that is $0.706 → ~$0.247/doc, which is the same order as convert rather than 9× it.

The durable conclusion is narrower than “polish is expensive”: the budget model must not assume polish is the free tier, and the stop-loss must gate polish-build. It is not a five-figure-accident tier once optimized.

Practical consequences:

• The stop-loss must gate polish-build, not just convert. (The current executor correctly gates all paid work and lets free passthrough/dedupe proceed even when the ceiling is hit — that is right: a compliant document should still clear the backlog.)
• The dry-run estimator must price by tier mix. Note the cost lives one level below the route: the router branches on content (hasText), while the repair-vs-build strategy branches on tag tree (hasTagStructure). The tag tree does not decide where a document goes — it decides what it costs once it gets there. Practically all API spend is the untagged-and-has-text population routed to build: 30.1% of the corpus — 452 of 1,500 documents, 5,817 pages (cross-tabulated from document-detail.csv, not inferred).
• Prompt caching on the flyer-tagger Haiku lane is a live, unrealized saving (see docs/admin/audits/2026-09-17-tiered-routing-ab.md) — the tiered router is safe but buys ~0 today because the Haiku lane does not emit the ephemeral cache breakpoint.

6.4a Two cost axes, not one

Bulk spend splits across two resources that are easy to conflate:

Axis	Driven by	Paid in
API cost	strategy dispatch (hasTagStructure) — build vs repair	dollars, ~$0.71/doc on build
Human cost	the crudeness of that same signal (§2.2a) — broken trees dispatched to repair, failing through to review	reviewer hours

The second axis is invisible in cost_ledger and therefore absent from the stop-loss. A batch can sit comfortably under its dollar ceiling while generating a review queue nobody has the staff to clear.

Wiring assess_tree into bulk triage moves documents from the second axis to the first — it raises API spend and lowers review volume. Whether that is a good trade is a per-customer question (a university with student workers may prefer the queue; one without may not), which argues for making it a per-folder policy knob rather than a global default.

6.4b Corpus cost estimate — derived without making a single model call

Every term that drives token count is knowable statically: vision input is a closed-form function of page geometry, every text input is hard-capped in source, and output is bounded by the response schemas. So the corpus can be priced before spending anything. Script: estimate_audit_cost.py (census document-detail.csv + constants read from source). Corpus: 1,500 documents, 27,203 pages, median 3 pages, mean 18.1.

Inputs, and where each number comes from

Term	Value	Source
Vision image tokens/page	1,454	(612 × 1.5) × (792 × 1.5) / 750, render scale 1.5 from pdf-to-png.ts
Vision page cap	15	MAX_MATH_ALT_PAGES = MAX_ALT_QUALITY_PAGES = MAX_VISUAL_QUALITY_PAGES — one MODEL CALL per page, so raising it raises the bill; each is settable per call
Deep-audit page cap	50	MAX_DEEP_AUDIT_PAGES — raised from 15 by #3385 (it bounded a browser launch per page; one lease now covers the whole call). Unmetered, settable via analyzeDeepPdfRisks({ maxPages })
Coherence stream cap	30,000 chars	MAX_STREAM_LENGTH, screen-reader-simulator.ts:71
Text-quality cap	3,000 chars	MAX_BODY_TEXT_CHARS
Link-quality cap	80 × 200 chars	MAX_LINKS_JUDGED × MAX_LINK_TEXT
flyer-tagger rate	15,882 in + 12,758 out per page	measured, cost_ledger (113 docs / 334 pages)
AI-audit model	Claude Haiku 4.5, $1/$5	GEMINI_FLASH_MODEL → shim DEFAULT_FLASH_TARGET

Results

Pass	Per page	Per doc	Corpus (1,500 docs)
deep raster audit	$0.0000	$0.0000	$0.00 — zero tokens, no model call
ai audit (all 6 checks, upper bound)	$0.0031	$0.0558	$83.64
flyer-tagger build — flyer-measured rate	$0.2390	—	$1,390.36
— after the −30% already landed	$0.1673	—	$973.25
— + batch inference −50%	$0.0837	—	$486.62

Build figures are over the 5,817-page build population only, not all 27,203.

The answer to “will two audit passes add to the cost”

Essentially no. Two full AI passes over the entire corpus cost $167, against $487 for optimized build — and the recommended posture (§5.4) does not run AI on the whole corpus. Restricted to verify on the build population it is $25.70, about 5% of build spend. deep adds nothing in dollars at all.

The structural reason: flyer-tagger’s cost is 80% output tokens, billed at 5× the input rate. The audit checks emit small bounded verdicts (≤8 coherence findings, ≤12 link verdicts), so their output is ~9% of their input. Audit is input-dominated and cheap; remediation is output-dominated and expensive. They are not comparable workloads, and audit cost was never the thing to worry about. Optimizing remediation output is the whole game.

Where flyer-tagger’s output tokens actually go — not the JSON

A natural response to “80% output” is to compact the wire format. That is the wrong lever, and the codebase already establishes why (analyze.py, #3131 / #3135):

“on the text-layer path the compact JSON envelope is tiny — a moderately dense page’s structured output is only a few hundred tokens, yet the job averages several thousand OUTPUT tokens per page. The gap is adaptive thinking: extended-thinking tokens are billed at the OUTPUT rate and, on Sonnet 4.6, adaptive thinking defaults to high effort.”

So of ~12,758 output tokens/page, the parsed JSON is a few hundred. The schema is already minimal — single-char keys (t, x), inventory entries referenced by index rather than re-transcribed, and structured-output json_schema mode. A better serialization would optimize perhaps 3% of the output bill.

The real levers, in order of size:

Lever	Status	Note
Thinking effort high→medium	✅ landed (#3135, 2026-09-16)	the dominant term; FLYER_TAGGER_EFFORT env-configurable
Batch inference −50%	⬜	applies to the whole bill
Tiered model routing (Haiku on safe pages)	✅ safe, ⬜ unrealized	buys ~0 until the Haiku lane emits a cache breakpoint
Prompt caching on the Haiku lane	⬜	unlocks the tiered-routing win
Wire-format compaction	—	already done; negligible remaining

Exception — OCR mode. The OCR wire schema requires t = the exact transcription of every line, so its output scales with page text volume rather than with thinking. There the output genuinely is content — irreducible, not format overhead. Any per-page output-token model must treat text mode and OCR mode separately.

Is the measured rate pre- or post-optimization?

Pre. 105 of the 114 measured documents (314 of 334 pages) predate #3135 (2026-09-16). So applying the −30% on top of 12,758 out/page is legitimate and not double-discounting. The 9 documents after the change show higher per-page tokens (15,521 out), but at n=24 pages that is document-mix noise — the −30% is not yet confirmed in production data. Confirming it is part of the re-measurement in §6.4c.

The 15-page cap also makes AI audit nearly flat per document: a 200-page report costs the same vision spend as a 15-page one. That is why per-page cost is so low on a corpus with an 18-page mean — and it is also a coverage limitation to state honestly, not a pure win (page 16+ is never looked at).

⚠️ cost_ledger will under-report AI audit by ~1.9×

The AI checks are billed under the alias gemini-3-flash-preview ($0.50/$3.00 in llm-cost.ts), but the shim actually serves Claude Haiku 4.5 ($1.00/$5.00). The corpus pass truly costing $83.64 would be recorded as $44.42. Since #3154’s stop-loss compares cost_ledger spend against the ceiling, the ceiling is ~1.9× looser than it appears for any AI-audit spend. Fix the alias pricing before wiring AI checks into the bulk path.

Confidence

Solid: image tokens (closed-form), all caps (read from source), flyer-tagger rate (measured). Estimated: output tokens per call, and ~1,800 chars/page of body text for the coherence stream. Both are second-order — vision input dominates the audit total. The flyer-tagger rate is the least reliable number, for the reason in §6.4c.

6.4c ⚠️ The flyer-tagger rate is measured on an unrepresentative sample

The 15,882-in / 12,758-out per-page rate comes from 113 documents averaging 3.0 pages — flyer-tagger’s design population of dense marketing flyers, which is what it has been tested on. It is not representative of the corpus, whose documents are text-heavy and average 18.1 pages.

Direction of the bias is unclear and should not be guessed:

• Output tokens scale with elements emitted per page. A dense flyer has many small positioned elements; a text page has fewer, larger blocks — arguing the measured rate over-states text documents.
• But a long text document has more pages, and cost is per-page, so totals rise even if per-page falls.

Treat $0.0837/page (optimized) as an upper bound pending re-measurement on a corpus-representative sample. The estimator re-runs in seconds once a better per-page rate exists — it is a single constant.

6.5 Budget governance ✅ (#3154, migrations 305/306)

• team_bulk_settings.monthly_budget_cents — NULL = unlimited
• stop_loss_enabled (default TRUE)
• Budget checked before each paid step, not tallied after
• Exceeding the ceiling pauses (state paused), does not fail — work is preserved and resumes next period or when the ceiling is raised
• Alert fires at most once per calendar month (budget_alerted_period)
• team_bulk_period_spend_cents(team, since) does the rollup

The recurring smoke-account depletion (hourly paid runs draining the balance to 0 → convert 402) is the small, survivable version of exactly this failure mode. A 10,000-document folder misrouting into build is the five-figure version.

---

7. Expected coverage — what fraction each step resolves

7.1 Source

UT System census, 1,500 PDFs (tools/pdf-inventory/census-results/ut-system-2026-08/). 1,386 validated by veraPDF; 114 not (83 image-only, 31 too large / parse-failed).

Class A = structural, machine-repairable with no author input. Class B = content, needs a human or AI to write something.

7.2 ⚠️ Two independent axes — do not conflate them

The census measures PDF/UA-1 machine conformance via veraPDF. That is one axis. The polish-vs-convert decision runs on a different axis, and the census is silent on it.

	Axis 1 — what structure is missing	Axis 2 — does any pixel have to change
Decides	deterministic repair vs AI authoring, within polish	polish vs convert — the tier itself
Driven by	PDF/UA clause failures	contrast, font size, images-of-text, reflow, table restructuring
Measured by	veraPDF — the census	the deep raster audit — never run at corpus scale
Known?	✅ yes, §7.4	❌ no

The census explicitly did not test colour contrast, font size, reading-order sanity, alt-text quality, table semantics or real screen-reader behaviour. Those are WCAG matters and veraPDF cannot see them.

⚠️ PDF/UA-1 and WCAG are different standards, and the law requires WCAG

This is the most consequential thing in the brief to get straight:

• 0 of the 106 PDF/UA-1 clauses in the class map mention contrast. Not one. PDF/UA-1 governs structure, tagging and semantics. Contrast is WCAG 1.4.3 / 1.4.11 — a visual criterion PDF/UA does not address at all.
• The DOJ Title II rule requires WCAG 2.1 AA, not PDF/UA-1.

Therefore: a document can be 100% PDF/UA-1 conformant and still fail the standard the law actually names. veraPDF returning zero errors does not discharge the obligation, and any vendor implying otherwise — including us — is overclaiming.

Concretely, this bounds §7.2d: “92.3% reachable by polish” is a PDF/UA-1 statement. Extending it to WCAG AA requires the contrast result we do not have.

No document in the 1,500-document census has ever been tested for contrast. That was a defensible scoping decision — the census was deliberately Tier 0 + 1 (deterministic, zero AI cost) so a hostile reader could re-run it — but it means the corpus picture is half a picture, and the missing half is plausibly the most prevalent failure class. On the web, low contrast is the single most common WCAG failure by a wide margin (WebAIM’s annual survey finds it on roughly four in five home pages). Whether PDFs behave the same way is unknown, and knowable cheaply.

Class C (“image-only”) is an axis-1 fact and does not mean convert. It describes text-layer presence, which selects a strategy inside polish, not a tier:

• An image-only page that is a photo or diagram → tag it <Figure>, write alt text, done. Pixel-identical. Polish.
• An image-only page that is an image of text → flyer-tagger’s mode="ocr" recognizes the text from the render and attaches structure via /ActualText over the untouched raster (cli.py). Still pixel-identical. Still polish — for the screen-reader-access problem. It does not resolve WCAG 1.4.5 (images of text), 1.4.4 (resize) or 1.4.10 (reflow), so such a document may separately fail axis 2.

7.2a The actual convert criterion — three tiers, not two

The criterion is not “does any pixel change” but “does any pixel have to MOVE.” Colour and geometry are different risks, and the middle tier is where most of the value is:

Tier	What changes	Layout	Method	Consent needed
polish:strict	nothing visual	identical	deterministic + AI authoring	none
polish:appearance	pixel colours only	identical — nothing moves, resizes, reflows or repaginates	fully deterministic (WCAG contrast maths is exact)	brand approval
convert	geometry	lost — re-authored	re-authoring	reviewed before ship

Contrast belongs in tier 2, not tier 3. recolor.py rewrites a text run’s fill colour and border_recolor.py a border’s stroke colour, both with geometry untouched. Every glyph stays exactly where it was, so nothing can overflow its box, collide, or repaginate — everything still fits, by construction. And the fix needs no model call: the required contrast ratio is arithmetic, and picking a compliant shade in the same hue is deterministic.

That makes contrast cheap to fix and safe to fix — the opposite of what its prevalence would suggest. The only reason it is opt-in is brand consent, not layout risk. That is a far easier customer conversation:

“We will darken this body text from #767676 to #595959. Nothing moves.”

versus re-authoring the document.

What genuinely forces convert

Only problems whose fix moves geometry:

Problem	Why colour changes cannot help
Font size below minimum	enlarging text displaces everything around it
Reflow / resize (WCAG 1.4.10, 1.4.4)	a fixed-layout PDF cannot reflow at all
Images of text (WCAG 1.4.5)	the text must become real text
Tables needing visual restructuring	the table itself must be re-laid-out — partial, see below
Contrast inside a raster image	recolor operates on vector paint ops, not baked pixels

This list is materially shorter and rarer than the previous framing implied, and it means contrast prevalence does not reduce polish reach — a contrast-failing document moves from polish:strict to polish:appearance, not to convert. §7.2d’s 92.3% likely survives the contrast measurement; what changes is the mode mix within polish, not the polish/convert boundary.

On tables specifically: polish does build table structure — _build_table emits TH/TD, apply_headers_ids handles complex tables, and #3214 anchors header cells to the source tree. So many table-header problems are fixed in place. The residue is tables whose visual design carries no header to mark, or whose semantics need the table itself rebuilt — those are convert or review, and they are also the “complex table semantics” line in the reviewer residue (§8.3).

7.2b What we know, and what we must stop claiming

Route	Prior estimate	Status
Passthrough (already conformant)	1%	0.1% — measured
Polish — deterministic only	30%	33% — measured (axis 1)
Polish — needs AI authoring	10%	60% — measured (axis 1)
Manual only (oversized, encrypted, parse-fail)	—	2% — measured
Convert (pixels must change)	50%	❌ UNMEASURED

The axis-1 numbers are real. The single largest population is “structurally fixable but for one or two authored items” — and §7.2d measures exactly what those items are: every one of them is something the pipeline already generates automatically, most often a missing link description. Combined with Class A, 92.3% of the corpus is reachable by polish on the clause axis. AI authoring quality gates only the subset needing figure alt text — 25.1% of the corpus, not 60%; the other 58% of near-miss documents are repaired by deterministic code with no model call at all. See the decomposition in §7.2d.

The convert share is not known and must not be quoted. An earlier draft of this brief claimed “convert is ~5%, only image-only documents need re-authoring.” That was wrong: it read a text-layer statistic as a tier statistic. Every axis-1 bucket above can also fail axis 2 — a document with perfect structure and 3:1 body text is a convert candidate the census scores as a near-miss.

Since contrast is plausibly the most prevalent accessibility failure of all, the convert share could be large. Treat it as unknown until measured.

7.2d What the near-miss documents are actually missing — and why it matters

The 60% “A+B near miss” bucket is the largest population in the corpus, and its value depends entirely on what the B items are. Measured, not assumed.

The clause map contains only 5 Class-B clauses out of 106. Of those five, three ever appear in the corpus at all — and all three are items our pipeline already generates automatically:

B clause	What it is	Near-miss docs failing it	We generate it?
7.18.5-2	Link alternate description (/Contents)	694 (77.5%)	✅ link_describe.py
7.3-1	Alt text on Figure tags	376 (42.0%)	✅ vision alt text
7.18.1-3	Form field /TU label	50 (5.6%)	✅ form-label jury
7.11-1	Embedded-file Desc	0	—
7.18.6.2-2	Media-clip Alt	0	—

Most near-miss documents fail exactly one B clause (76.5%); 22.1% fail two. No near-miss document fails a clause outside the class map.

100% of near-miss documents (896 of 896) fail only B clauses we already generate automatically.

Polish reach

	Documents	Share
Class A — structural only, no authored text	488	32.5%
+ near-miss whose B items are auto-generated	896	59.7%
= reachable by polish	1,384	92.3%

That 92.3% is the same ceiling the clause ladder reports (92.4% of the full 1,500 — the 114 unvalidated documents cannot be cleared by clause fixes at all). Polish reaches essentially the entire validatable corpus.

Per institution, the spread is wide and worth knowing before a pitch:

Institution	A only	+ auto-B	Polish-reachable
UT MD Anderson Cancer Center	11.1%	88.9%	100.0%
UT Rio Grande Valley	74.0%	24.4%	98.3%
UT Tyler	29.5%	68.2%	97.7%
UT El Paso	22.9%	69.2%	92.1%
UT Permian Basin	29.9%	52.1%	82.0%
UT Southwestern Medical Center	27.4%	52.5%	79.9%

⚠️ Decomposition by FIX MECHANISM — q gates far less than it appears

“Can we fix the near-miss authored items mechanically?” splits the population in a way the A/B class map does not. Two of the three live B clauses are repaired by fully deterministic code — no model call, therefore no q risk at all:

B clause	Share of near-miss	Repaired by	Deterministic?
7.18.5-2 link /Contents	77.5%	link_describe.py — humanize_url, extract_title_from_html, compose_link_description	✅ yes — no model, no prompt
7.18.1-3 form /TU	5.6%	form-label-jury.ts — “No LLM calls — deterministic label extraction + keyword matching”	✅ yes
7.3-1 figure /Alt	42.0%	vision	❌ no — this is the only one needing AI

Splitting near-miss documents by whether they need figure alt at all:

Population	Docs	Of near-miss	Of corpus
Needs only deterministic text (link and/or form)	520	58.0%	34.7%
Needs AI-authored figure alt	376	42.0%	25.1%

Consequence: q gates 25.1% of the corpus, not 60%. An earlier draft of this brief said “AI authoring quality gates ~60% of the corpus.” That was wrong — it applied q to the whole near-miss population when 58% of it is repaired by deterministic code.

The corrected model:

H1 = ( conformant + classA + nearMiss_deterministic + nearMiss_ai × q_alt ) × (1 − c)

q_alt	base	H1 at c=20%	H1 at c=30%
100%	92.3%	73.9%	64.6%
90%	89.8%	71.9%	62.9%
70%	84.8%	67.9%	59.4%
50%	79.8%	63.8%	55.9%
30%	74.8%	59.8%	52.4%

The part carrying no q risk whatsoever is 67.3% of the corpus. Even if machine-written alt text were acceptable only 30% of the time, the base only falls from 92.3% to 74.8% — a 17.5-point swing, not the catastrophic sensitivity the earlier model implied.

This substantially de-risks Measurement B. q is still needed to state a calibrated confidence, and it remains a prerequisite for a public “90% confidence” claim — but it is no longer the load-bearing unknown. c is.

The surprise: link descriptions, not alt text

7.18.5-2 at 77.5% is nearly twice as prevalent as missing alt text. The single most common authored-content gap in the corpus is links with no alternate description — and it is also the cheapest of the three to generate well, since the link target and surrounding text are both available deterministically. Any prioritisation that leads with alt text is optimising the second-biggest problem.

⚠️ Two caveats that bound this claim

1. This is PDF/UA clause space only — axis 1 (§7.2). It says nothing about contrast, type size, or reflow. “92.3% reachable by polish” means 92.3% can reach machine conformance without authored human text. It does not mean 92.3% need no pixel changes. A document in this set can still fail contrast and land in convert. The convert share (§7.2c) remains unmeasured and this result does not change that.
2. Auto-generated ≠ verified. Conformance is satisfied the moment /Alt and /Contents are present, but whether the generated text is good is a judged verdict in the §5.6 confidence model, never verified. The conformance claim is strong; the quality claim carries a confidence level. This is exactly the population §5.5 is about — and the reason the verify gate must treat “present but unhelpful” as blocking.

7.2e ✅ MEASURED — UT Southwestern, 2026-09-18

First real run of the deterministic census (deterministic-census.ts, spread sampling, concurrency 3, 15.6 min on 10.1.1.15). 213 documents, UT Southwestern Medical Center. Single institution — corpus-wide numbers are still pending.

c is far lower than the sensitivity analysis assumed

Clean raster runs (denominator)	204
Did not run — unexamined, excluded, never counted clean	9
Fails contrast	27
c	13.2%
Fully examined (not page-capped)	179 / 204 (87.7%)

§7.7 estimated a 67% H1 needed c ≈ 22-27%. Measured c is half that.

Almost all of it is text contrast — 26 documents (84 findings) on color-contrast versus 2 documents on non-text-contrast. That is recolor.py territory: geometry-safe, deterministic, no model call (§7.2a).

⚠️ Contrast is NOT independent of clause class — the (1 − c) model is void

Population	n	Fails contrast
Class A — structural only	58	10.3%
Near-miss, deterministic text only	83	2.4%
Near-miss, needs AI figure alt	32	59.4%
Image-only (C)	31	0.0%
All classified	204	13.2%

A 59.4-point spread. Documents needing figure alt text fail contrast at 25× the rate of documents needing only link/form text.

That is mechanistically sensible and should have been anticipated: a document with figures is a designed document — brochure, flyer, report — where someone chose colours. A document whose only authored gap is a link description is typically plain text on white.

Every (1 − c) table earlier in this brief is therefore an approximation. Compute H1 from the joint counts instead; the summarizer now does.

H1 from the joint data

Needs no AI and is pixel-clean	133 / 204 = 65.2%
H1 at q_alt = 100%	71.6%
H1 at q_alt = 90%	70.9%
H1 at q_alt = 70%	69.7%
H1 at q_alt = 50%	68.4%

H1 moves only 3.2 points across the entire plausible range of q_alt.

The reason is the non-independence above: the 32 documents needing AI alt text are also the ones that mostly fail contrast (59.4%), so they were largely excluded from the pixel-perfect population regardless of how good the generated alt text is. q_alt is nearly irrelevant to H1 on this account. Measurement B drops from “prerequisite” to “nice to have” for the H1 claim specifically — though it is still required to state a calibrated confidence, and it still matters for H2.

Both headline numbers, measured

Claim	Target	Measured (UTSW)
H1 — no human, pixel-perfect, ≥90% confidence	67%	70.9%
H2 — of the remainder, without pixel fidelity	90%	≤85.5%

H1 beats the target. H2 lands slightly under it, bounded by the 9 documents whose raster check could not run at all — with those resolved, H2 rises.

Caveats that travel with these numbers

1. One institution, n=204. c carries roughly ±5% at this sample against UTSW’s 1,198 discovered documents. Quote the interval.
2. Still an upper bound on H1, for three independent reasons: type size is unmeasured, raster-baked contrast is not separated from vector, and 9 documents went unexamined.
3. The pure tier covered only 65 of 213 documents (30.5%) — the other 148 have no tag structure, so their tag quality is unassessable. Not a clean result; those are exactly the build-path documents.
4. UTSW is the weakest institution on polish reach in the corpus (79.9%, §7.2d). These numbers are likely conservative relative to the system average.

Pure-tier profile (65 analyzed)

Finding	Docs
reading-order-divergence	43 (66.2%)
hidden-text	36 (55.4%)
tag-nesting	32 (49.2%)
structure-mismatch	26 (40.0%)
alt-text-quality	5 (7.7%)
ambiguous-link-text	5 (7.7%)

234 high-severity findings. Reading-order divergence dominates — the defect veraPDF passes silently and a screen-reader user hits first. Notably, existing authored text is rarely bad (7.7% each for alt and link text), which is a weak positive signal for q_alt.

7.2c How to measure the convert share

The instrument already exists and is unmetered — the deep raster audit (color-contrast, non-text-contrast). Cost is browser render time, not tokens.

1. Take the census sample (or a ~300-doc stratified subset).
2. Run the audit with deep: true (no ai needed for this question).
3. Classify each document: does it carry a contrast or font-size failure that polish:appearance recolor cannot reach?
4. That share, plus images-of-text and reflow failures, is the convert share.

Until this runs, the dry-run estimator (#3154) cannot price a batch, because tier mix is the dominant cost term (§6.3). This measurement blocks the estimator.

7.8 The remediation ladder — measured, with confidence intervals

The commercial question in its most useful form: what share is conformant after each successive capability is applied? Cumulative, each stage adding one capability. Measured on UT Southwestern (n=204 classifiable of 213), 95% Wilson intervals.

Tool: remediation-ladder-v2.py.

⚠️ This section was rebuilt 2026-09-18 and the numbers moved a long way down. The previous table (stage 1 = 65.2%) came from remediation-ladder.py, which scored documents on veraPDF clause class and contrast only. It never read the six AI checks, so it silently assumed every WCAG criterion those checks cover — 1.1.1 alt quality, 1.4.1 use of colour, 1.4.5 images of text, 2.4.4 link purpose, 1.3.2 reading order — was already satisfied on every document. It was not a sampling error; the ladder was not asking the question. Do not quote the old table.

As measured today

Stage	Capability added	n	Share	95% CI	Pixels
0	nothing — already conformant	0	0.0%	[0.0%, 1.8%]	—
1	deterministic tag-tree fixes	23	11.3%	[7.6%, 16.3%]	identical
2	+ AI tag-tree fixes (alt, math, order)	51	25.0%	[19.6%, 31.4%]	identical
3	+ surgical content fixes (contrast recolor)	55	27.0%	[21.3%, 33.4%]	colours only
4	+ reflow / convert	204	100.0%	[98.2%, 100.0%]	layout discarded

That leaves 149 documents at stage 4. The decomposition is the important part, because most of it is not a property of the documents:

Blocker	Docs	Sole blocker
math-alt check never completed	60	51
genuine image-of-text (validated)	63	11
use-of-colour (1.4.1)	48	32
image-only (class C)	31	3

Two of the four blockers are our defects, not the customer’s documents

1. The math-alt check fails on tall pages (#3241, #3242). 60 documents have a page taller than 2000pt; all 60 failed the check. Only 9 of the 153 normal-height documents did. findMathAltIssues fails closed when every examined page fails to render — correct behaviour — but the render is failing because the rasterised page exceeds Bedrock’s 8000px limit. 51 documents are sent to convert for this reason and no other, a quarter of the classifiable corpus. This is the single largest lever in the system and it is entirely ours.

2. image-of-text is ~50% false positives (#3244, #3245). Validated against each PDF’s own text layer, 46 of the 109 flagged documents have no genuine image-of-text at all. That correction is already applied to the table above; before it, stage 1 read 7.8% and stage 3 read 17.2%. The remaining 63 are an upper bound — a paraphrased claim scores as genuine by construction (#3244), and the decorative-duplicate case is not filtered yet.

Sensitivity: what the ladder becomes as those are resolved

Scenario	Stage 1	Stage 2	Stage 3
A as measured today	11.3% [7.6, 16.3]	25.0% [19.6, 31.4]	27.0% [21.3, 33.4]
B + tall-page render fixed (#3241)	36.3% [30.0, 43.1]	50.0% [43.2, 56.8]	52.0% [45.1, 58.7]
C + use-of-colour repairable by recolor	36.3% [30.0, 43.1]	50.0% [43.2, 56.8]	67.6% [61.0, 73.7]

Scenario B is a bug fix, not a new capability, and it is worth 25 points at stage 1. Scenario C is an open engineering question and must not be assumed: WCAG 1.4.1 is violated when information is carried by colour alone, and the fix is a non-colour cue. Sometimes that is an underline on a link, which draws under existing glyphs and moves nothing — stage 3. Sometimes it is inserting the word “required” or an asterisk, which reflows the line — stage 4. Both patterns are present in this corpus and the census cannot yet tell them apart, so the ladder holds all 48 at stage 4 by default. Splitting that population is the highest-value unbuilt measurement in this brief.

⚠️ Stage 4 is not really 100%

Within the 204 documents that could be classified, yes. Against all 213, counting the 9 whose raster check could not run at all (#3230 — permissions- encrypted, not corrupt), the ceiling is 95.8% [92.6%, 97.6%]. Those 9 are unexaminable, not clean.

What the ladder says now

1. Nothing passes today. Zero of 204, upper bound 1.8%.
2. The deterministic stage is no longer the headline. It is 11.3% today and 36.3% with #3241 fixed — not the 65.2% previously claimed. The old number survived only because the AI criteria were not being counted.
3. Our own reliability is the dominant term. 51 sole-blocked documents from a render bug and 46 false-positive documents from an uncalibrated detector together outweigh every genuine content blocker in the corpus.
4. Contrast recolor adds 2 points, not 13. Once the AI criteria are counted, most contrast-failing documents are blocked on something else as well, so repairing contrast alone rarely finishes a document. It remains cheap and worth having; it is not a headline capability.
5. §7.7’s commercial headline needs re-deriving. The “67% with perfect pixel fidelity” claim traced to the old stage-1/stage-3 figures. A defensible restatement is available only under scenario C, which is unproven.

Caveats that travel with this table

• One institution. UTSW is the weakest in the corpus on polish reach (79.9%, §7.2d), so these are likely conservative against the system average.
• “Conformant,” not “accessible.” These stages measure PDF/UA clauses, contrast, and the six AI-checked WCAG criteria. Type size and reflow behaviour are not verified here (§7.9).
• Image-only documents are assigned to stage 4, which is conservative: some are photographs polish could tag, and OCR mode can attach structure over an untouched raster.
• Fail-closed twice. A document whose raster check never ran is excluded from the denominator; a document with an incomplete AI check cannot be certified below stage 4. Both are deliberate and both cost us headline percentage.
• 2.4.6 heading-descriptiveness is excluded — it is WCAG AAA, out of scope for an AA ladder.

7.9 WCAG coverage differs sharply by stage — and stage 4 is strong

§7.8’s stages 0-3 are built from veraPDF clauses plus the raster contrast checks. PDF/UA conformance is a floor, not the standard the law names (§7.2). But the coverage picture is very different above and below stage 4, and an earlier draft of this section got stage 4 badly wrong.

Stage 4 (reflow / convert) runs 13 browser-based WCAG checks

convert.ts:4137-4166 runs, on every converted document’s HTML output:

checkFocusOrder · checkResizeText · checkTextSpacing · checkKeyboardAccessibility · checkFocusVisible · checkNontextContrast · checkVisualAccessibility · checkOrientation · checkInputPurpose · checkFormInteraction · checkHoverContent · checkAiQuality · checkReflow

Resize and reflow ARE verified, at three widths:

Check	Criterion	How
checkResizeText	1.4.4 Resize Text	zoom:2 at 1280px for clip detection, then a 640px viewport — the true 200% equivalent. The code is explicit about why both: under zoom:2 Chromium reports scrollWidth in zoomed units while innerWidth stays in layout units, so comparing them measures the zoom delta, not reflow. A genuine test resizes the viewport and lets the page lay out.
checkReflow	1.4.10 Reflow	re-renders at 320 CSS px (REFLOW_PROBE_WIDTH), the WCAG narrow-viewport threshold

So: 1280 → 640 → 320. checkReflow also implements the SC 1.4.10 exception properly (allowlisting pre, table, figure, svg, canvas, and descendants of overflow-x: auto|scroll, per W3C G206/C37/C38), and distinguishes status: 'error' from passed: false so a Puppeteer crash reports the criterion skipped, not violated — the same fail-closed discipline the pass gate needs.

Measured coverage of WCAG 2.1 A+AA at stage 4

From wcag-criteria-map.ts (50 criteria):

	Count
Dedicated browser check	15
Dedicated AI check	7
axe-core / quality-score signal only	7
No signal of any kind	21

All 21 with no signal are marked typicallyNA for converted documents — audio, video, captions, pointer gestures, motion actuation, timing, multi-page navigation. There is no criterion that is both applicable and unmeasured in the convert path.

Criteria with a dedicated check: 1.1.1, 1.3.1, 1.3.2, 1.3.4, 1.3.5, 1.4.1, 1.4.4, 1.4.5, 1.4.10, 1.4.11, 1.4.12, 1.4.13, 2.1.1, 2.1.2, 2.4.3, 2.4.6, 2.4.7, 3.1.2, 3.2.1, 3.2.2, 3.3.1, 3.3.2.

Note 9 criteria marked typicallyNA are checked anyway (keyboard, focus traps, form interaction) rather than assumed — conservative, and right.

Stages 0-3 are the weak ones, not stage 4

A document that stays a PDF gets veraPDF plus raster contrast. It does not get the 13 browser checks, because they need HTML to run against. For those stages:

Criterion	Stage 0-3	Note
1.4.3 / 1.4.11 contrast	✅ raster
1.3.1, 3.1.1, 2.4.2	✅ veraPDF	presence, not meaningfulness
1.1.1 alt meaningful	❌	needs the AI audit (§5.4)
2.4.4 link purpose, 1.3.2 sequence, 2.4.6 headings, 3.1.2 language of parts	❌	needs the AI audit
1.4.1 use of colour, 1.4.5 images of text	❌	needs the AI audit
1.4.4 resize, 1.4.10 reflow	❌	arguably N/A: a fixed-layout PDF cannot reflow by construction. Decide deliberately — do not silently score it either way.

So the WCAG gap is concentrated in stages 0-3, and closing it is exactly the §5.4 work of wiring the AI audit into the PDF pass gate. Stage 4 already has what those stages lack.

This also sharpens a commercial point: converting produces a document we can say far more about. Polish preserves the pixels; convert produces something whose accessibility we can actually evidence across 29 criteria.

Cost of closing the stage 0-3 gap

From the static token model (§6.4b), Haiku 4.5 rates, all six AI checks:

	UT Southwestern (219)	Full corpus (1,500)
AI audit, before only	$7.89	$83.64
before + after	$15.79	$167.28
Remediation so there is something to re-audit (optimised)	$43.18	$488.63

$0.036 per document. The audit is not the expense — remediating is, by ~6×.

A full before/after on every document is unnecessary: ladder membership follows from the before audit plus a finding→stage map (the §7.2d method). The after audit only calibrates — does the fix clear the finding, and is generated alt text accepted (q_alt). Both are sample questions. ~$26 total for a UTSW WCAG ladder.

⚠️ Sequencing: do not run this before #3227

The AI checks scrape JSON from free text, and an unparseable response degrades to an empty finding list (§5.5). A failed check is indistinguishable from a clean one, which would bias every WCAG number optimistic, in our favour, invisibly. Fix the fail-closed behaviour first. Fix #3219 too, or the run’s own cost books ~1.9× under.

Coverage against the most-litigated WCAG failures

The criteria that actually appear in ADA complaints, mapped to what we verify:

Litigated failure	Criteria	Stages 0-3 (stays a PDF)	Stage 4 (converted)
Low colour contrast	1.4.3, 1.4.11	✅ raster deep audit	✅ axe + checkNontextContrast
Missing / inadequate alt text	1.1.1	⚠️ presence only (veraPDF 7.3-1); quality needs the AI audit, not yet wired to the gate (§5.4/§5.5)	✅ checkAiQuality
Keyboard navigation failures	2.1.1, 2.1.2	❌ no HTML to drive	✅ checkKeyboardAccessibility
Missing form labels	3.3.2, 1.3.5	✅ form-label jury, /TU	✅ checkFormInteraction, checkInputPurpose
Inaccessible multimedia	1.2.1-1.2.5	❌	❌ asserted N/A, never checked — #3231
Non-descriptive link text	2.4.4	✅ deterministic findAmbiguousLinkText + AI link-quality	✅ link-name + AI
Inaccessible documents / PDFs	—	the product	the product

Six of seven are covered. Two findings from tracing it:

The typicallyNA design is sound where a rule can fire. It is a fallback, not an override — determineConformance overrides the N/A whenever any custom rule for that criterion emitted a signal. And the emitters are right: checkKeyboardAccessibility pushes keyboard-reachable / no-keyboard-trap unconditionally with pass/fail, so a passing keyboard check reports Supports, not N/A. checkFormInteraction guards on fm.hasForms, so “no forms” is a verified N/A. Keyboard — a litigated criterion — is handled correctly.

Multimedia is the exception, and it is the litigated one. The 1.2.x criteria carry customRuleIds: [], so no rule can ever fire and the override can never trigger. They are unconditionally reported Not Applicable with the asserted remark “Converted documents do not contain audio or video content” — without looking. The verification machinery exists (CONTENT_NA_MAP, detectContentSignals) and is wired only for the web-page profile. Filed as #3231.

That is a factual claim we would have to defend, on a contested criterion, in a document written for counsel. It is also a small fix.

Two honest caveats

1. typicallyNA is a profile assumption, not a verified fact. A converted document could contain content engaging 1.3.3 or 2.5.x. The map is explicit that the 'web-page' profile may assume nothing; 'converted-document' may, because the source is non-interactive. Defensible — but it is an assumption, and the confidence score (§5.6) should surface it as n/a by profile rather than as verified.
2. Some criteria need human judgment no machine makes. “WCAG compliant” is not fully machine-verifiable and we should never claim it is. What we can state per document is every criterion verified deterministically, every one machine-judged with a confidence, and every one not assessed — named. That is §5.6, and this section is the strongest argument for building it.

7.3 Important caveat on reading the axis-1 numbers

“Class A” means the document fails only Class A clauses. It does not guarantee polish actually fixes all of them — that depends on polish’s clause coverage, which the ladder in §7.4 measures. Treat the axis-1 shares as an upper bound conditioned on polish covering the relevant clauses, and remember they say nothing about axis 2.

Also: only 0.1% pass today, and machine-conformance is necessary, not sufficient. We never call a document “accessible” — we say machine-conformant. Every customer-facing surface must preserve that distinction.

7.4 The ladder — cumulative pass rate per failure mode fixed

Question: if I fix failure mode x, what percentage of documents then pass everything (zero remaining PDF/UA-1 failures)?

A document “passes” only when every clause it fails is in the fixed set — this is maximum set-coverage, not a frequency ranking. Order below is the best found by randomized greedy (400 restarts), i.e. roughly the cheapest path up the curve.

Percentages are of the 1,386 validated documents. Against the full 1,500 the ceiling is 92.4%, since the 114 unvalidated documents can never be cleared by clause fixes alone. Bars are a visual aid only — the numeric column carries the data.

Fix #	Clause	A/B	What it is	Cumulative pass
1	7.1-10	A	Set DisplayDocTitle=true	0.1%	▏
4	7.1-9	A	Add dc:title to XMP metadata	8.0%	███
5	7.3-1	B	Alt text on Figure tags	10.1%	████
8	7.18.1-2	A	Annotation Contents or Alt	18.8%	███████
10	7.5-1	A	Valid role map	27.3%	███████████
14	6.2-1	A	MarkInfo/Marked=true	33.5%	█████████████
16	7.2-33	A	Language for metadata	45.7%	██████████████████
17	7.18.3-1	A	Annotations inside structure tree	63.0%	█████████████████████████
20	7.4.2-1	A	Heading numbering order	73.0%	█████████████████████████████
24	7.18.4-1	A	Form field TU/description	79.3%	████████████████████████████████
25	7.18.1-3	B	Annotation alt in structure	80.7%	████████████████████████████████
28	7.10-1	A	Table structure (TH/TD/Headers)	86.4%	██████████████████████████████████
32	7.2-22	A	—	90.4%	████████████████████████████████████
39	7.21.3.2-1	A	—	95.3%	██████████████████████████████████████
54	7.2-4	A	—	99.1%	███████████████████████████████████████
67	7.2-9	A	—	100.0%	████████████████████████████████████████

Knee points:

Target	Clauses to fix
50% of validated docs pass	17
80%	25
90%	32
95%	39
99%	54
100%	67

The expensive stretch is 95% → 100%: 28 more clauses for 5 points of coverage, each affecting 1–6 documents in the entire corpus. Everything from fix #45 down has single-digit frequency — that is the long tail to leave to manual remediation, and the natural place to draw the automation boundary.

Read this as the polish roadmap. Fix #17 (7.18.3-1, annotations inside the structure tree) is the single biggest jump in the curve — 45.7% → 63.0%. Only three of the first 25 fixes are Class B; the rest need no author at all.

7.5 Tier-0 quick wins (near-universal, trivially fixable)

Finding	Share of corpus
No tag tree at all (untagged → build strategy)	35%
No document language set (/Lang) — a one-line fix	38%
DisplayDocTitle not set	67%
Image-only scan, no text layer (needs OCR — Class C)	5%

The 35% untagged figure is the headline cost driver, with a correction: only the untagged documents that also have a text layer reach the expensive build path (~30% — see §6.4). The remainder are image-only, route reflow, and cost nothing today because convert is disabled. Untagged is also the largest single input to review volume, via the broken-tree dispatch described in §2.2a.

7.6 Exposure framing (for sales, not a quote)

Failing documents in the sample span ~17,611 pages. At prevailing market rates of $5–8/page that is $88,055–$140,888 for the sample alone, of which $27,502 (the structural-only pages) is avoidable through automated repair. Present as prevailing-rate context only.

---

7.7 The two commercial headline numbers

These are the claims the offering is sold on, so they get defined precisely and backed by measurement rather than asserted.

1. “We remediate X% of your documents with no human interaction, at ≥90% confidence, with perfect pixel fidelity.”

2. “Of what remains, we remediate Y% at ≥90% confidence, without perfect pixel fidelity.”

What each number is made of

H1 = ( already-conformant + Class A + near-miss·q ) × ( 1 − c )
H2 = ( remainder − genuinely-blocked ) / remainder

Term	Value	Status
already-conformant (D)	0.1%	✅ measured
Class A — structural only, no authored text	32.5%	✅ measured
near-miss — needs only auto-generated text (§7.2d)	59.7%	✅ measured
genuinely blocked — oversized / encrypted / parse-fail	2.1%	✅ measured
near-miss needing only deterministic text (§7.2d)	34.7%	✅ measured
near-miss needing AI figure alt	25.1%	✅ measured
q_alt — share of AI-alt documents whose generated text clears the bar	—	❌ UNMEASURED
Everything except q_alt and c is already in hand.

c is the load-bearing unknown, not q_alt. Because q_alt applies only to the 25.1% needing figure alt (§7.2d), even a pessimistic q_alt moves the base by ~17 points, while c multiplies the whole thing. Measure c first.

H1 sensitivity — what the number would be

q ↓ · c →	10%	20%	30%	40%	50%
100%	83.1%	73.8%	64.6%	55.4%	46.2%
95%	80.4%	71.5%	62.5%	53.6%	44.7%
90%	77.7%	69.1%	60.4%	51.8%	43.2%
85%	75.0%	66.7%	58.3%	50.0%	41.7%
80%	72.3%	64.3%	56.3%	48.2%	40.2%

A 67% claim requires:

If AI acceptance q is…	…then contrast/size failure rate c must be
100%	27.4%
95%	25.0%
90%	22.4%
85%	19.6%
80%	16.6%

So 67% is a defensible claim if roughly one document in four fails contrast or type size and the generated text is accepted nine times in ten. Both are plausible. Neither is measured.

H2 — the remainder

The remainder consists of contrast/size failures (→ polish:appearance or convert), near-miss documents that missed the AI bar (→ human-assisted), and image-only documents (→ OCR mode or convert). Only the oversized / encrypted / parse-fail band (2.1% of the corpus) is genuinely unreachable.

At the target scenario (q = 90%, c = 22.4%, H1 = 67%): remainder = 33.0%, H2 = 93.6%. H2 is ≥ 87% across every cell of the sensitivity grid, and rises as c rises — a larger remainder is diluted by the fixed 2.1% blocked band.

Larry’s 90% for H2 is conservative. Combined reach at the target scenario is 97.9% of all documents, with 2.1% genuinely blocked.

The pixel-fidelity constraint is what costs us — not automation

Worth separating, because it is a pricing and positioning lever:

• “No human interaction, any fidelity” includes contrast repair, which is deterministic and needs no reviewer (§7.2a). That number is D + A + NM·q — 86.2% at q = 90%, independent of c.
• “No human interaction and pixel-perfect” is the same population times (1 − c) — 67% at c = 22.4%.

The ~19-point gap between them is bought entirely by the pixel-fidelity promise. If a customer will accept colour-only changes — nothing moves, resizes or repaginates — the automatic number jumps from 67% to 86%. That is the strongest argument for making polish:appearance default-on with a colour diff (open question 7), and it should be presented to the customer as their choice, not ours.

Measurement A — contrast census (gives c)

• Run the existing deep raster checks (color-contrast, non-text-contrast) over the census corpus
• $0 in model tokens — browser rasterization and pixel arithmetic only; ~2.7 browser-hours for 1,500 documents at the 15-page cap
• Blocker: corpus PDFs are not local (corpus/ holds only a README); they are on R2 (accessible-dev-assets), and document-detail.csv also carries every source URL and sha256
• Also must capture type size below the minimum, which currently nothing measures, and separate contrast failures in vector text (recolorable) from those baked into a raster (not recolorable → convert)

Measurement B — AI authoring quality calibration (gives q)

This is the one nobody has scoped, and it is the harder of the two. We cannot claim “90% confidence” about machine-written text without measuring how often a human accepts it.

• Sample near-miss documents, run the real generation, and have a human grade each generated item: acceptable / unacceptable
• Weight the sample by real prevalence, not intuition (§7.2d): link descriptions appear in 77.5% of near-miss documents versus alt text at 42.0%. A calibration weighted toward alt text would measure the wrong thing.
• Expect q to be high, because the dominant item is the most tractable one: a link’s target URL and surrounding text are both deterministically available, so the generator is not guessing the way it is with a photograph
• Output: an acceptance rate per B clause, which is what converts a judged verdict in the §5.6 confidence model into an actual probability

Until Measurement B exists, the §5.6 confidence bands are ordinal, not probabilistic — we can say “High” but not “90%”. The headline claim as worded requires a calibrated number, so B is a prerequisite for making it publicly.

8. Load distribution — the human review queue (#3156) ⬜

8.1 Model: lease-based pull, not assignment

“Press a button, get the next document that needs review” — a real work queue, not ORDER BY created_at LIMIT 1.

• Claim via FOR UPDATE SKIP LOCKED, setting claimed_by + lease_expires_at. Two reviewers pressing simultaneously must never get the same document.
• Auto-release on expiry. A reviewer who closes the tab must not park a document forever.
• Sort cheapest-to-clear, not FIFO. A reviewer clearing 40 one-click alt-text documents moves the batch’s completion percentage further than 3 heroic table rebuilds — and completion percentage is what the customer watches.

Why pull-with-lease rather than assignment: reviewers are interchangeable, task cost varies enormously, and people quit mid-shift. Assignment creates stranded queues behind whoever is on vacation. The exceptions are filters, not a different model — “documents from this folder” (a department reviews its own), and a skill filter once the reason-code histogram shows tables and reading-order genuinely need different people.

8.2 Four terminal outcomes

Outcome	Effect
Approve	→ output bucket, state done
Fix-and-approve	editor, then re-verify through the same pass gate
Escalate to convert	→ convert tier (requires policy allowConvert)
Reject as unfixable	state rejected + reason code

manualRisks is the reviewer’s checklist — ordered, dismissible items with page and location. The reviewer works a list instead of hunting. /tags and /inspect already exist as the guided surfaces; this feeds them a task list.

8.3 Every rejection reason is a feature request against polish

Polish’s scope principle is that it does everything machine-checkable that can be done in place. The genuine residue reaching a human is:

• contrast / font-size (needs pixel changes → polish:appearance)
• judgement calls on meaning (is the alt text true, not merely present)
• complex table semantics
• documents with no text layer
• genuinely ambiguous reading order

If a reviewer is repeatedly fixing something machine-verifiable and pixel-safe, the answer is not more reviewer throughput — it is extending polish. The reason-code histogram is the roadmap for which capability to build next.

8.4 Team scoping ✅ (migrations 305/306, landed 2026-09-18)

The shared work queue is the first genuinely multi-user feature in the bulk path. The ownership layer it needs has landed:

• remediation_items.team_id + remediation_events.team_id (nullable — personal integrations stay NULL and never surface in a team console; ON DELETE SET NULL so tearing down a team never destroys evidence)
• team_bulk_settings — budget, stop-loss, allowed_email_domains
• is_team_member() / is_team_admin() — SECURITY DEFINER, granted to authenticated for RLS evaluation
• resolve_bulk_team(user) — attributes work to the pool the user draws from; hardened in 306 against a malformed credit_source stranding every bulk file
• RLS: team members read their team’s items; all writes stay service-role
• States extended with paused and rejected
• Partial index remediation_items_team_review_idx keeps the queue scan small

Still missing — the lease layer itself: claimed_by, lease_expires_at, the claim RPC, reason codes, and the cheapest-first ordering signal. See §10 for the scoped work item.

---

9. Evidence and reporting (#3157) ⬜

Remediation without a provenance record is not defensible. For many buyers the evidence is worth more than the remediated files.

9.1 Fleet console

The dashboard stops being a conveyor belt and becomes a fleet console. Per folder and per batch:

• documents by state · tier mix · cost burn against ceiling
• review queue depth + estimated hours to clear
• compliance percentage over time
• downloadable batch report

9.2 Sidecar manifest

Alongside each output file, write <name>.gevell.json: what was found, what changed, which tier and mode, before/after scores, pixel-diff result, whether a human reviewed it and who.

9.3 Batch report

What an ADA coordinator hands to counsel ahead of the deadline. Per the established per-URL summary rule: the report opens with a summary — every folder scanned, document counts and compliance percentage per folder — above the detailed per-document findings.

The report is a customer-facing document and must itself be WCAG 2.2 AA compliant.

9.4 Append-only event log ✅

remediation_events is append-only so any document’s history is fully reconstructible. This is the evidence substrate; do not add an UPDATE path to it.

---

10. Status and open work

#	Item	Status
3151	Polish/flyer-tagger parity, non-destruction guards	✅ merged
3152	Pure routing policy + pass gate	✅ merged
3153	remediation_items + state machine (passthrough + polish:strict)	✅ merged
3154	Budget ceiling + stop-loss + dry-run estimate	🚧 migrations 305/306 landed
3155	Enable polish:appearance and convert behind per-folder policy	⬜
3156	Human review queue — lease-based claim	⬜ lease layer outstanding
3157	Fleet console, batch report, sidecar manifests	⬜
—	deep at triage as the polish/convert classifier (§5.4)	⬜ not filed — blocks correct routing
—	Measure the convert share over the corpus (§7.2c)	⬜ not filed — blocks the #3154 estimator
—	Source authored-content prior — preserve /Alt, /ActualText, /E, /Lang (§2.2b)	⬜ not filed — highest risk
—	Wire assess_tree into bulk triage — replace the hasTagStructure token test (§2.2a, §6.4a)	⬜ not filed
—	Measurement A — contrast + type-size census (§7.7) — gives c, gates both headline numbers	⬜ not filed — highest commercial value
—	Measurement B — AI authoring quality calibration (§7.7) — gives q; prerequisite for any public “90% confidence” claim	⬜ not filed — highest commercial value
—	Per-document confidence score (§5.6) — customer-facing claim already made in marketing	⬜ not filed
—	Resize/reflow check (§5.6.6) — nothing measures WCAG 1.4.4 / 1.4.10 today	⬜ not filed
—	AI checks on the pass gate (§5.4) + context-dependent verify severity (§5.5)	⬜ not filed
—	AI check cost measurement run (§6.2)	⬜ not filed
—	URL-list / Siteimprove ingest (§3.3)	⬜ not filed

10.1 Convert routing — ⚠️ this section is now STALE

An earlier draft recorded that policyFor was hardcoded to DEFAULT_ROUTING_POLICY and nothing could route to convert. #3155 landed on main while this brief was being written (“Bulk remediation: per-folder tier policy + convert/appearance routing”, commit 5b0e030f1), and bulk-remediation-executor.ts now reads per-folder policy from the database with a fail-safe fall back to the conservative default on a query error.

Re-verify before relying on anything in §§5.4, 7.2a or 7.7 that assumes convert is unreachable — and note the §10 status table below may also lag. The allowConvert / allowPolishAppearance defaults per folder are now the operative question, not whether the plumbing exists.

10.2 Decisions taken

Decision	Value	Date
Pass-gate structural score threshold	95	2026-09-17
Polish dispatch	by tag-tree state (1A) †	2026-09-17
MAX_ATTEMPTS	2	2026-09-17
Dedupe key	content_sha256, not path	2026-09-17
Multi-user unit	teams (not tenants)	2026-09-18
COVERAGE_FLOOR	0.80 — provisional	2026-09-17

† The decision record is accurate; the implementation is coarser than the decision intended. “Tag-tree state” was decided as the three-state assess_tree verdict (absent / present-and-usable / present-and-broken). What shipped dispatches on a two-state byte-string token test (hasTagStructure), which cannot express present-and-broken. Do not read this row as a description of current behaviour — see §2.2a.

10.3 Open questions

1. Does the bulk pass gate need deep + ai to be defensible? §5.3 argues yes. Needs a decision before #3157 ships the evidence report.
2. Pricing for AI checks. ai is gated auth-only today — no credit or entitlement check. Deferred deliberately; still deferred.
3. Is polish-build really 9× convert? §6.4. Needs a controlled measurement on matched documents before the estimator is trusted.
4. COVERAGE_FLOOR = 0.80 remains provisional.
5. URL-list writeback. Is a download bundle + manifest an acceptable deliverable, or does Siteimprove ingest wait for a CMS integration?
6. Should build be blocked entirely on documents carrying authored /Alt until §2.2b lands? The conservative answer is yes — route them to review rather than destroy work a customer paid for. Cheap to implement (tree_assess already walks the tree), and reversible once the prior exists.
7. Should polish:appearance be default-on? Per §7.2a the question is narrower than it first appeared: contrast repair is deterministic and geometry-preserving, so the risk is not layout breakage — it is brand consent. The real options are (a) opt-in per folder as today, (b) default-on with a per-document colour diff the customer can review, or (c) default-on only where the recolor stays inside the customer’s own supplied palette. This is a product positioning decision, not an engineering one. Given contrast is likely the most prevalent failure and among the cheapest to fix, leaving it off by default may be the single largest unforced loss of value in the system.