Automating the human-review queue
Scope: every review item currently requiring a person in the PDF → HTML → PDF pipeline. Time-based media, flashing, and moving content are excluded — they don’t survive the round-trip.
Ceiling is how far automation can go, not how far it goes today:
| Auto | Machine decides. Never enters the queue. |
| Flag | Machine decides the majority, escalates a bounded exception set. |
| Assist | Machine drafts, human always confirms. Reduces minutes, doesn’t remove the item. |
| Human | Stays. |
Confidence on prevalence: M measured (published data), E estimate with reasoning, U unknown — nobody has this data. Most rows are E or U, because the corpora everyone cites measure what static scanners detect, which is precisely the complement of this list. That gap is itself an asset: nobody holds real prevalence data on the judgment criteria, and you’d be generating it.
Run costs are marginal cost per page at August 2026 inference prices, and will fall.
The structural point, before the tables
You generate the HTML. That means a large class of criteria are properties of your template, not of any document — focus indicators, reflow, text spacing, orientation, target size, bypass blocks, consistent navigation, and most of name/role/value are determined by the generator, identically, for every document it produces.
Test those once per template release against a fixture corpus, retain the evidence, and assert them for every document downstream. That’s not a shortcut; it’s a stronger claim than per-document sampling, and it’s how a build pipeline should treat an invariant.
Roughly eight to ten criteria move from per-document review minutes to zero this way, for about two weeks of work. Nothing else in this plan has that ratio.
HTML stage
| Review item | WCAG | What automates it | Ceiling | How common | Run cost | Build |
|---|---|---|---|---|---|---|
| Reading order | 1.3.2 | A11y tree order vs. rendered geometry; then linearize and LLM-score coherence | Flag | Very high in multi-column source E | $0.005 | 3–4 wk |
| Tab order | 2.4.3 | Drive Tab via CDP, record activeElement sequence, diff against reading order | Auto | Moderate E | ~$0.001 | 1–2 wk |
| Keyboard traps | 2.1.2 | Tab 2× focusable count, detect non-escaping cycles, verify Esc and Shift+Tab | Auto | Low in generated HTML E | shares above | 3–5 d |
| Keyboard operability | 2.1.1 | Activate each control by keyboard, diff resulting DOM state against mouse activation | Flag | Low E | ~$0.002 | 2–3 wk |
| Focus visible / not obscured | 2.4.7, 2.4.11 | Pixel-diff element bbox before/after focus; occlusion test via elementFromPoint | Auto | Template-level | negligible | 1–2 wk |
| Listen (announcement quality) | 1.1.1, 1.3.1, 4.1.2 | Synthesize announcement string from a11y tree (name/role/state), LLM-evaluate adequacy | Flag | n/a — method, not criterion | $0.01 | 2–3 wk |
| Alt text adequacy | 1.1.1 | VLM on crop + page context; decorative/informative split first, then equivalence check | Flag | Near-universal in PDF-derived E | $0.01–0.04 | 3–4 wk |
| Complex images needing long desc. | 1.1.1 | Chart/diagram/map classifier gates a long-description requirement | Assist | High in gov/edu E | $0.005 | 2 wk |
| Table: data vs. layout | 1.3.1 | Content-type uniformity + header styling + VLM on rendered table | Auto | Very high E | $0.01 | part of below |
| Table: header assignment | 1.3.1 | Geometry-driven scope; headers/id proposal for spanned and nested | Flag | Very high E | $0.02 | 4–6 wk |
| Heading outline fidelity | 1.3.1, 2.4.6 | Computed font size/weight vs. tag level to find styled-not-tagged headings; LLM outline check | Auto | High E | $0.003 | 1–2 wk |
| Link purpose | 2.4.4 | Generic-text detection, then LLM purpose-from-context on the remainder | Auto | High M | $0.002 | 1 wk |
| Color as sole indicator | 1.4.1 | Desaturate and diff; plus text pass for colour-referencing prose | Flag | Moderate E | $0.002 | 2 wk |
| Sensory references | 1.3.3 | Text classifier for shape/size/position instructions | Auto | Low–moderate E | negligible | 3–5 d |
| Text contrast over images/gradients | 1.4.3 | Sample rendered pixels behind glyph bboxes, worst-case ratio | Auto | High M | negligible | 1–2 wk |
| Non-text contrast | 1.4.11 | Same engine, applied to borders and UI boundaries | Auto | High M | negligible | 3–5 d |
| Form label adequacy | 1.3.1, 2.5.3 | Visible label vs. accessible name string comparison | Auto | High where forms exist E | negligible | 1 wk |
| Form grouping | 1.3.1 | Geometric clustering + shared control names → fieldset/legend proposal | Flag | High E | $0.003 | 2 wk |
| Error handling | 3.3.1, 3.3.3 | Submit invalid data, capture response, check programmatic association and text | Flag | High where forms exist E | $0.005 | 3–4 wk |
| Name, role, value (widgets) | 4.1.2 | Trigger widget, observe ARIA state transitions against visual state change | Flag | Template-level | $0.003 | 3–4 wk |
| Status messages | 4.1.3 | MutationObserver; meaningful change outside a live region | Flag | Low for static docs E | negligible | 2 wk |
| Reflow / text spacing / orientation | 1.4.10, 1.4.12, 1.3.4 | Render at 320px and 400%; inject spacing CSS; detect scroll and clipping | Auto | Template-level | ~$0.001 | 1 wk |
| Target size, dragging alternatives | 2.5.8, 2.5.7 | Geometric measurement with exception classification | Auto | Template-level | negligible | 1 wk |
| Language of parts | 3.1.2 | Per-run language ID vs. lang attributes | Auto | Low–moderate E | negligible | 1 wk |
| Bypass blocks, multiple ways | 2.4.1, 2.4.5 | Landmark and skip-link presence; nav mechanism inventory | Auto | Template-level | negligible | 1 wk |
| Consistent nav / identification | 3.2.3, 3.2.4 | Cross-page diff of nav order and repeated-component naming | Auto | Template-level | negligible | 1–2 wk |
PDF stage
| Review item | What automates it | Ceiling | How common | Run cost | Build |
|---|---|---|---|---|---|
| Artifact vs. real content | Positional recurrence across pages + content classification | Auto | Very high E | negligible | 2 wk |
| List structure (L/LI/Lbl/LBody) | Bullet and numbering pattern detection vs. tag tree | Auto | Very high E | $0.002 | 2–3 wk |
| Tag order vs. content stream | Same engine as HTML reading order | Flag | Very high E | shared | shared |
| Field tooltips, types, tab order | Deterministic from AcroForm dictionary + field geometry | Auto | High E | negligible | 2 wk |
| Radio group association | Shared field name + geometric clustering | Auto | High E | negligible | in above |
| Link–OBJR pairing | Structural check against annotation array | Auto | Moderate E | negligible | 1 wk |
| Bookmarks match outline | Compare outline tree to heading tags | Auto | High E | negligible | 3–5 d |
| Language changes in document | Per-run language ID | Auto | Low–moderate E | negligible | shared |
| OCR quality on scanned pages | Engine confidence + VLM re-read of low-confidence regions, diff | Flag | High for scanned E | $0.01 | 2–3 wk |
| Math and formulas | Formula region detection, then MathML from source or generated alt | Assist | Low overall, high in STEM U | $0.01 | 4+ wk |
What stays human
- Authorial intent on ambiguous alt text and long descriptions. Full page context gets a model close; the residual is genuinely in someone’s head.
- Scoping. What’s in scope, sample selection, what counts as representative, whether an exception applies. Principal-level, not reviewer-level.
- Novel content patterns the models haven’t seen. Bounded and shrinking, but non-zero.
- The conformance determination and remarks. Not a detection problem. At 100% detection accuracy you still need someone to assert conformance and be accountable for the assertion.
Build order
Ranked by review minutes eliminated per engineer-week, not by criterion importance.
Phase 1 — two weeks, retires the most items per day of work
The interaction harness plus the deterministic checks. Tab order and keyboard traps share one CDP harness. Bookmarks, Link–OBJR, non-text contrast, language of parts, and sensory references are each under a week and fully deterministic. Ship the template-level battery in the same phase — focus visible, reflow, text spacing, orientation, target size, bypass blocks, consistent navigation — since those convert to per-release assertions rather than per-document findings.
Phase 2 — six to eight weeks, the volume drivers
Reading order first; one engine serves both the HTML stage and PDF tag-order verification, and it’s the largest single line item in document review. Then artifact-vs-content and list structure, which together account for most tagging-layer review time. Then the decorative/informative split on images — not full alt adequacy, just the classification, which is the cheaper half and removes most of the volume.
Phase 3 — eight to twelve weeks, expensive but high value
Table semantics is the hardest thing on this list and the most valuable; budget conservatively. Announcement synthesis, the form error-handling harness, and OCR verification follow.
Deferred
Name/role/value, status messages, and math. The first two are template-level in your pipeline and low-yield per document; math is expensive and only matters for STEM portfolios. Revisit when a customer forces it.
Cost
Marginal inference and compute for a fully-instrumented page: roughly $0.01–0.02 for text-only pages, $0.05–0.08 for pages with images and tables. Against a manual remediation market of $3–8 per standard page, that’s well under one percent.
The conclusion to draw is that compute is never your constraint, and pricing per page against a marginal cost this low is leaving money on the table. Your real costs are engineering weeks and expert minutes — which is the same answer the unit-economics model gave from the other direction.
Instrument as you go. Every item routed to a reviewer is a labeled example. Log per criterion: escalation rate, reviewer override rate, time-to-decide, inter-reviewer agreement. Promote a criterion from Flag to Auto only when confirmation rate clears a threshold across a meaningful sample, and retain the evidence. That produces the substantiation file you’d need if anyone asks you to back a claim about what your automation achieves — and after the accessiBe order, that’s a live question rather than a hypothetical.