Epic: converter-grade PDF auditing on audit.gevell.com — run every check directly on the PDF
Status (2026-09-11)
- Phase 0 — done, no code change needed. Traced
pdf-manual-risks.ts→pdf-audit-executor.ts→apps/audit’sPdfScoreReport.tsxand the printed/ exported report (audit-report-renderer.ts): both already rendermanualRisks.risksgenerically (.map()over the array, no per-type branching), so every analyzer’s findings already reached the report. Locked in with a compile-time-exhaustive regression test (PdfScoreReport.test.tsx—Record<ManualReviewRiskType, true>) so a future analyzer type can’t silently stop rendering. - Phase 1 — landed. Four new bucket-A (deterministic, zero marginal cost)
checks, pure functions over the existing
PdfRiskModel:heading-by-appearance(pdf-manual-risks-appearance.ts) — large/bold- looking text with no Hn ancestor anywhere in the tag tree; complements (never duplicates)structure-mismatch’s P-only heading-sized check.language-of-parts(pdf-manual-risks-appearance.ts) — a passage whose dominant script (Cyrillic/Greek/Hebrew/Arabic/CJK) doesn’t match its (own/inherited/document-default)/Lang. Required adding/Langcapture toStructElementNodeand a document-defaultdocLangtoPdfRiskModel.form-field-missing-label(pdf-manual-risks-forms.ts) — AcroForm fields with no/TUaccessible name, radio groups deduped to one finding. Required addingformFields(viaacroform-extractor.ts, reused as-is) toPdfRiskModel.- Screen-reader serialization (
pdf-screen-reader-serializer.ts,serializeFromStruct) — tag-order announced-text dump, exposed asManualReviewRisks.screenReaderPreviewand rendered as a collapsed “What a screen reader announces” panel in the live report. Reusable as-is for Phase 3’s screen-reader-coherence AI judgment (see that table below). - All four are wired into
collectRisks(now 16 analyzers) and covered by unit tests plus abuildPdfRiskModelintegration test for the new/Langand AcroForm plumbing.
- Phase 2 — mostly landed (color contrast + non-text contrast); images-of-text
deferred. New OPT-IN
deepflag onPOST /api/pdf-audit(query param, threaded throughPdfAuditMessage/PdfAuditResult.deepAuditRan) — off by default, so the free/fast audit is completely unchanged. When requested, the executor rasterizes pages viaAwsBrowserProvider(the SAME headless- Chromium pathaudit-executor.tsalready runs, unmetered, for every web audit — no new billing surface, matches that existing precedent rather than the epic’s original “Deep tier = metered” assumption, which turned out not to fit rasterization specifically) and runs two new checks (pdf-pixel-contrast.ts):color-contrast— reuses the web-audit pixel-contrast resolver’s proven math (contrastRatio,sampleInkAndBackground, now exported) against a crop of the rendered page, keyed off the PDF’s own fill-color operator (ground truth, more reliable than the web version’s CSS-color guess) and an estimated (not measured — glyph widths aren’t tracked) run bbox.non-text-contrast— scoped to AcroForm widget boundaries (not the general “any graphic UI element” problemwcag-nontext-contrast-checker.tssolves for DOM/CSS, which isn’t reusable for PDF at all): samples thin bands just outside/inside each widget’s exact/Rect, no estimation needed there.- Images-of-text deferred, on purpose: unlike rasterization,
image-ocr.tsspends real per-call third-party cost — Mathpix (OCR) and a classification call that, despite thegetGeminiClient()name, runs on Bedrock Claude Haiku in production viabedrock-gemini-shim.ts(#2751 ZDR cutover — Bedrock is the opt-OUT default; real Gemini only runs with an explicit non-production opt-out). No Gemini spend either way, but still real Mathpix + Bedrock cost with no existing free/unmetered path to mirror — wiring it into a currently-free audit endpoint without a rate cap or cost_ledger entry (which needs a DB migration — a bigger, more sensitive change) risks real unbounded spend. Left for a deliberate decision, not silently built. (Any Phase 3 work should call Bedrock directly — Claude Haiku/Sonnet, or Nova for a cheap first pass with a Haiku fallback perbedrock-nova.ts’s established pattern — never the Gemini SDK path.)
- Phase 3 — math-alt + screen-reader-coherence landed; alt-text-quality and
use-of-color still open. New OPT-IN
aiflag onPOST /api/pdf-audit(separate fromdeep— seePdfAuditMessage.ai’s doc comment for why), originally gated auth-only pluscost_ledgervisibility (no credit/ entitlement blocking) while the tiering/packaging question below stayed unresolved. Now also gated onaudit_runper the metering decision landed at the bottom of “Open questions” —aicharges quantity +2 on top of the base run. Two checks landed:math-alt-quality(pdf-ai-opinions.ts’sfindMathAltIssues) — first production wiring ofvision-equation-audit.ts’sauditEquations(previously 100% dead code). Vision-detects likely equations per page, checks the page for aFigure- orFormula-tagged, alt-texted element.screen-reader-coherence(findScreenReaderCoherenceIssues) — reusesscreen-reader-simulator.ts’s judgment core (now split out assimulateStream, shared with the HTML path) fedpdf-screen-reader-serializer.ts’s struct-tree stream instead of HTML.- Real spend booked to
cost_ledger(audit-math-alt/audit-coherenceoperation types, migration20260911000000_296_...) via a direct PostgREST insert inworkers/batch/src/pdf-audit-executor.ts(that context has noworkers/apiEnvfor the sharedrecordCosthelper) — awaited, attributed toresponse.modelVersion(the actually-served Bedrock model), never Gemini in production (seebedrock-gemini-shim.ts). - Still open, deliberately deferred:
- alt-text-quality (vision-judged) — needs new image-to-
Figuregeometry-matching work:ExtractedImagehas bytes but no position,EmbeddedImageRecthas position but no bytes, and neither cleanly correlates to a specific structFigureyet. - use-of-color — no identified building block (per the epic’s own check-map row: “⚙️ AI-assisted, partial”); needs scoping before any code is written.
- alt-text-quality (vision-judged) — needs new image-to-
- Reviewed via the
pr-review-toolkit3-agent parallel review (code-reviewer, silent-failure-hunter, pr-test-analyzer) against the PR diff before merge — caught and fixed a real correctness bug (math-alt-qualitychecked onlyFigure, notFormula, for alt text — flagged PDF/UA-conformant equations as having none, backwards), a real spend-loss bug (one page failing mid-batch invision-equation-audit.tsdiscarded the whole batch’s already-billed token usage — now degrades per-page instead), a CI-breaking typecheck error, and several logging/awaiting gaps in the cost-booking path. PR: https://github.com/LarryAnglin/theaccessible-platform/pull/2977 (branchclaude/pdf-audit-engine-parity-phase3→main).
- Phase 2’s images-of-text is still deferred for the same reason noted below (real Mathpix + Bedrock cost, no metering decision made yet) — unrelated to Phase 3’s AI-opinions work, just the other still-open deferred item.
- Post-Phase-2 review pass (same day): ran the
pr-review-toolkitfive- agent review against the full branch diff before it’s ever pushed/merged. Found and fixed one real correctness bug three agents converged on independently —pdf-pixel-contrast.tsfedisLargeTextraw PDF points where it expects CSS px (1pt = 4/3px), so genuinely-large 18–24pt text was held to the stricter 4.5:1 ratio instead of 3:1 — plus several more from a single pass: unknown-colorspace text silently treated as black (could false-flag light-on-dark text), a keyboard-inaccessible scrollable preview panel (WCAG 2.1.1 — this product’s own non-negotiable standard), rotated pages sampled with mismatched geometry (raster applies /Rotate, the model’s width/height didn’t),DeepPdfAuditResultrestructured to a discriminated union (free — unshipped), an unbounded screen-reader-preview text dump riding along in every conversion’s metadata, and several smaller correctness/consistency fixes (radio-group dedup in the widget-contrast check, decoded-text-length crop-width estimate, /Rect corner normalization). All fixed, tested, and verified against the real pixel math (no mocked math in the new regression tests). Left for later, with reasoning documented in code comments: Latin-script Unicode range precision, a wall-clock deadline for the per-page rasterization loop (page-count cap only, for now), and language-of-parts’ partial overlap with the existing document-language check. - Second review pass, on the fix commit itself (same day): the first
pass’s fix commit had never been reviewed — re-ran all five agents scoped
to it. Found a real regression in the crop-width fix above:
page.decodedByMcidaccumulates EVERY run sharing one MCID into one string (often a whole paragraph), so using its length for a SINGLE run’s crop estimate could overshoot far worse than the original raw.length bug it replaced. Fixed by takingmin(decoded length, raw length)— verified by temporarily reverting the fix and confirming the new regression test fails on the old code. Also found and fixed: rotated pages were rasterized and silently excluded from both raster checks with no signal anywhere — a fully-rotated document would report a clean “Includes the deep raster checks” with zero coverage, the same false-clean-bill-of-health class of bug the first pass’sfailedChecksfix addressed, just from a different cause. Now tracked aspagesSkippedForRotationand shown in the report. Also: rotation-lookup failure now fails CLOSED (null, treated as unsafe) instead of open (defaulting to0/”unrotated”);PdfAuditResult’s deep- audit fields were nested into onedeepAudit?: PdfDeepAuditStatusdiscriminated union (was 5 loosely-related flat optionals, still unshipped so still free to fix);RiskPageModel.rotationnarrowed to a0 | 90 | 180 | 270 | nullliteral union;failedChecksnarrowed toDeepAuditCheckName[]; plus doc-comment corrections and added test coverage for the previously-untested 2-checks-failed and multi-page-radio- group wording. Documented (not fixed) as a known follow-up: the same raster/model mismatch that motivated the rotation guard also applies to CropBox vs. MediaBox (see pdf-pixel-contrast.ts’s module header).
Summary
Gevell PDF (pdf.gevell.com) runs a sophisticated accessibility engine that covers almost the entire Matterhorn Protocol across both the reflow and preserve-formatting pipelines. Gevell Audit (audit.gevell.com) already shares the source-only half of that engine at full parity. This epic closes the rest of the gap without converting the PDF or building an HTML twin — every remaining check runs directly on the source PDF. Because audit is not repair, we never mutate the file; we only inspect it and report findings.
The thesis: a tagged PDF natively contains both halves of every comparison the converter used to fake with a twin —
- the structure/tag tree = the programmatic order + semantics a screen reader is handed (what AT announces), and
- the content stream = the real visual layer (glyph geometry, paint order, fonts, colors — what a sighted user sees).
Every “twin vs. PDF” check the converter did was really “proxy-for-visual vs. programmatic.” On the PDF we hold the actual visual layer, so we compare programmatic-vs-visual directly. This is the canonical way PDF/UA reading order is verified, and it is a more honest audit — it judges the PDF as delivered, not as converted.
Why this is cheap: most of the engine already exists
- The dual model already exists and already runs in audit.
buildPdfRiskModel()(workers/api/src/services/pdf-manual-risk-model.ts) produces structure elements in tag order (roles, /Alt, /Lang, MCIDs, form refs) joined to per-MCID paint order, geometry (bbox), color, render mode, and ToUnicode-decoded text. The manual-risks analyzer (analyzeManualReviewRisks, wired atworkers/batch/src/pdf-audit-executor.ts:160) runs 13 checks over it today. - The “Kendall-tau reading order” the converter runs on the twin already has a shipping
PDF-native cousin in audit:
findReadingOrderDivergencecompares tag order vs. paint/geometric order via longest-increasing-subsequence agreement (pdf-manual-risks.ts:53-54). - Rasterization is available server-side (
renderPdfPagesToBuffers/renderPdfPageToPnginworkers/api/src/utils/pdf-to-png.ts) and used across the converter. - Pixel-contrast, non-text-contrast, OCR, AcroForm extraction, embedded-image geometry, and the vision equation auditor all already exist as services.
Most of this epic is wiring existing building blocks to the PDF audit path plus a handful of new pure functions — not a ground-up build.
Architecture
One shared analyzer builds up to three models from the source PDF and runs every check as a pure function over them. No pipeline, no persisted artifacts, no mutation.
source PDF bytes │ ├─▶ StructureModel (tag tree: role, alt, lang, MCIDs, form refs — TAG order) ← buildPdfRiskModel ✅ ├─▶ GeometryModel (per-MCID glyph bbox, paint order, color, font, size) ← buildPdfRiskModel ✅ └─▶ RasterModel (page PNGs, on demand — Deep tier only) ← renderPdfPagesToBuffers ✅ │ join by MCID ──▶ each semantic node gets its physical position / color / font │ check functions → findings (never mutations)Two tiers
Per the cost-tracking standard, deterministic checks are free per-document and run always; AI opinions cost per-document model spend and are metered.
- Structural audit (always on, ~zero marginal cost). Deterministic + veraPDF +
geometry/structure checks over
StructureModel/GeometryModel. This is the source- only engine, already at parity, plus the new deterministic structure-tree checks. - Deep audit (metered / premium). Adds the raster checks (contrast, images-of-text) and the AI-assisted opinions — the same passes conversion runs (alt-text quality, use-of-color, math, screen-reader coherence). In conversion these are generate-and-apply; in audit the identical model call becomes generate-and-report (“this figure has no alt text; suggested: …”). Every AI call records cost to the usage ledger, exactly as conversion does.
Master check map
Buckets: A = source-PDF-only (deterministic) · B = needs page raster · D = needs an AI/model call (orthogonal flag).
| Check (WCAG SC) | PDF-native method | Building block | Bucket | Status |
|---|---|---|---|---|
| Reading order (1.3.2) | tag order vs paint/geometric order, LIS agreement | findReadingOrderDivergence | A | ✅ already in audit |
| Geometric order / structure mismatch / tag nesting / artifact misclassification (1.3.1/1.3.2) | tag tree vs content geometry | pdf-manual-risks-structure | A | ✅ already in audit |
| Hidden text / garbled text (1.1.1/1.3.1) | render mode + color-vs-bg + ToUnicode decode | manual-risks phase 1 | A | ✅ already in audit |
| Link integrity / ambiguous link text (2.4.4/2.4.9) | Link elems vs text vs /Rect | manual-risks | A | ✅ already in audit |
| Word fragmentation / hyphenation / no-live-text (1.3.1) | decoded MCID text patterns | pdf-manual-risks-content | A | ✅ already in audit |
| Heading hierarchy (1.3.1/2.4.6) | Hn sequence in tag tree | scorer checkHeadingHierarchy | A | ✅ already in audit |
| Table headers / scope / associations (1.3.1) | struct table walk + TH/scope/Headers | struct-table-extractor + scorer | A | ✅/⚙️ deepen |
| ”Looks like a heading, not tagged” (1.3.1) | large glyph runs (geometry) not inside an Hn element | GeometryModel (font size) | A | ✅ landed — heading-by-appearance (#2968 Phase 1; size-only signal, weight deferred) |
| Language of parts (3.1.2) | script detection vs /Lang attrs (own/inherited/doc-default) on struct spans | model text + struct /Lang | A | ✅ landed — language-of-parts (#2968 Phase 1; non-Latin-vs-presumed-Latin only, by design) |
| Form field labels + purpose (1.3.1/3.3.2/1.3.5) | widget → /TU (Lbl-struct association deferred) | acroform-extractor (reused) | A | ✅ landed — form-field-missing-label (#2968 Phase 1) |
| Screen-reader serialization (1.3.1/1.3.2/4.1.2) | dump model.elements in tag order: role+text+alt = announced stream | StructureModel | A | ✅ landed — serializeFromStruct (#2968 Phase 1), shown as a collapsed report panel |
| Alt-text presence + quality (1.1.1) | struct /Alt present? then OCR/vision judgment of the covered image | manual-risks + image-ocr + alt-text-suggester | A→D | ✅ presence / ⚙️ deepen |
| Color contrast (1.4.3/1.4.6) | render page (headless browser), estimate run bbox from GeometryModel, sample fg/bg pixels | wcag-pixel-contrast-checker (contrastRatio, sampleInkAndBackground, now exported) + pdf-to-png | B | ✅ landed — pdf-pixel-contrast.ts findColorContrastIssues (#2968 Phase 2, opt-in deep) |
| Non-text contrast (1.4.11) | AcroForm widget boundary: thin bands just outside/inside the exact /Rect, mean-color compared | new (widget /Rect is exact, unlike wcag-nontext-contrast-checker’s DOM/CSS approach — not reused as-is) | B | ✅ landed (scoped to form widgets) — findFormWidgetContrastIssues (#2968 Phase 2) |
| Images of text (1.4.5) | extract embedded images, OCR, flag text-heavy | embedded-image-geometry + image-ocr | B | ❌ deferred — image-ocr.ts spends real per-call Mathpix + Bedrock (Claude Haiku, via the Gemini-shaped shim — not actual Gemini) cost with no free/unmetered precedent (unlike browser rasterization — see Status); needs a metering/cost_ledger decision before wiring |
| Math / equation alt (1.1.1) | figure regions → vision spoken-math opinion | vision-equation-audit (auditEquations) | B+D | ✅ landed — pdf-ai-opinions.ts findMathAltIssues (#2968 Phase 3, opt-in ai) |
| Use of color (1.4.1) | semantic — colour-only cue detection | raster + model | B+D | ⚙️ AI-assisted, partial — no building block identified yet |
| Screen-reader coherence (1.3.2/2.4.6) | announced-order text → model coherence judgment | screen-reader-simulator (judgment half) | D | ✅ landed — findScreenReaderCoherenceIssues (#2968 Phase 3, opt-in ai) |
Reusable AI/opinion passes — input signatures (grounded)
Verified export signatures, and whether they drop onto PDF-derived inputs cleanly:
| Pass | Entry point | Native input | PDF reuse |
|---|---|---|---|
| Alt-text quality/suggestion | alt-text-suggester.ts — AltTextRequest (image src: data URI/URL), model claude-haiku-4-5-20251001 | image bytes | Native — feed embedded image bytes as data URI |
| Equation / math alt | vision-equation-audit.ts — auditEquations(...) (renders pages via pdf-to-png) | PDF pages | Native — already PDF-based |
| Images-of-text / OCR | image-ocr.ts — runImageOcr, classifyImageIsDocument, isMeaningfulOcr | image bytes | Native |
| Form fields | acroform-extractor.ts — extractAcroFormFields(bytes) | PDF bytes | Native |
| Pixel contrast | wcag-pixel-contrast-checker.ts — contrastRatio, parseCssRgb, PixelSamplingPage | HTML/CSS-shaped nodes | Math reusable; node-sourcing is HTML-shaped → new PDF adapter feeds glyph bboxes |
| Screen-reader coherence | screen-reader-simulator.ts — serializeForScreenReader(html) + simulateScreenReader | HTML | Judgment/prompt reusable; serializeForScreenReader is HTML-only → write serializeFromStruct(model) sibling |
| Quality score | quality-scorer.ts — scoreQuality(...) | element data | Partial — element-shaped; evaluate fit |
| Per-element confidence | confidence-scorer.ts — scoreElementConfidence(html) | HTML | Converter-internal; N/A for audit (subsumed by risk severity) |
Out of scope / not applicable to fixed-layout PDF
Report these as “not applicable to PDF,” never as pass/fail:
- Reflow (1.4.10) and text spacing (1.4.12) — reflow-context criteria for adaptable content; PDF is fixed-layout and PDF/UA does not require them. They belong to the web-audit product (already covered there via the browser checkers).
- Input purpose / autocomplete (1.3.5) — no native PDF token; infer weakly from /TU and field names. Advisory only, not conformance.
- Inherently human judgments — “is this alt text meaningful?”, “is colour the only cue?” stay manual-review flags or AI-assisted opinions (Deep tier). Auditing does not turn a manual Matterhorn checkpoint into a machine pass.
Phased plan
Phase 0 — surface what already runs (days, near-zero cost) — ✅ done
manual-risks already runs in audit; confirm all 13 analyzers reach the apps/audit
report UI rather than being computed and dropped. Likely the single biggest visible jump
for the least work.
- Trace
apps/audit/src/lib/pdf-audit-api.ts+PdfScoreReport.tsxvs. the executor’s result payload; list any findings computed but not displayed. - Result: already true, no gap found. Both the live report and the printed/exported
report render
manualRisks.risksgenerically; locked in with a regression test — see “Status” above.
Phase 1 — new deterministic structure-tree checks (bucket A, ~1–2 wks) — ✅ done
Pure functions over the existing PdfRiskModel. No raster, no AI.
- Heading-by-appearance (untagged visual headings), language-of-parts, form-label association, screen-reader serialization dump. See “Status” above for what shipped and where each check lives.
Phase 2 — raster checks (bucket B, ~2–3 wks; Deep tier) — ⚙️ mostly done
Wire existing checkers to the PDF path; add per-audit rasterization cost.
- Color contrast (glyph bboxes from GeometryModel →
contrastRatio), non-text contrast, images-of-text (embedded-image extraction + OCR). - Result: color contrast + non-text (form-widget) contrast landed, opt-in via a
new
deepflag onPOST /api/pdf-audit(unmetered — mirrors the existing, already-unmetered web-audit browser-rendering precedent, not a new billing surface). Images-of-text deferred — real per-call Mathpix + Bedrock cost (not Gemini — see “Status” above) with no such free precedent to mirror; needs a metering decision first.
Phase 3 — AI-assisted opinions (bucket D; Deep tier) — ⚙️ mostly done
Reuse conversion’s passes as generate-and-report.
- Alt-text quality + suggestion, math alt, use-of-color, screen-reader coherence (new PDF serializer + reused judgment). Record every model call to the usage ledger.
- Result: math alt + screen-reader coherence landed, opt-in via a new
aiflag (separate fromdeep), spend recorded tocost_ledger. Alt-text quality (needs image-to-Figure geometry-matching work) and use-of-color (no building block identified yet) remain open — see “Status” above.
Cost tracking
Structural audit is deterministic and free per-document. Every Deep-tier AI call and each page rasterization must log operation type, resource consumed, estimated cost, and timestamp to the usage ledger, mirroring the conversion pipeline.
Open questions
- Tiering/packaging: is Deep audit a per-document meter, a plan entitlement, or both? Researched 2026-09-11 — see “Metering/tiering research” below for grounded options.
- Should the report present AI suggestions inline (show the customer the “good” version) as a differentiator vs. competitors’ pass/fail-only reports?
- Reuse vs. fork: keep the shared services in
workers/api/src/servicesand import from the audit executor (current pattern), or extract apackages/pdf-audit-engineso Gevell CLI and other surfaces share it?
Metering/tiering research (2026-09-11)
Traced the actual entitlement system (entitlements.ts v3 — checkEntitlement/
consumeEntitlement, product → mode resolution per package) rather than the legacy
credits.ts shape it now wraps, per migration 195_entitlements_model.sql.
Grounding facts:
audit_runalready exists as a product on quota_groupaudit, explicitly scoped in the migration comment as “covers HTML + PDF audits”: Free = 10 lifetime (quota, one-time), Individual = 50¢/audit (paygo), Professional/ Team/Enterprise =unlimited.convert.ts(pdf_convert) charges a flat per-page price regardless of how much AI actually ran on that page (alt-text suggestion, quality scoring, vision equation audit, etc. all ride the same flat charge). Real spend is booked tocost_ledgerpurely for margin observability — never metered back to the customer per AI call. The CIaudit.tspath chargesaudit_runthe same way (flat, at settlement, viaconsumeEntitlement). This flat-price-absorbs-AI- variance-plus-cost_ledger-for-margin-truth pattern is the norm everywhere in this codebase, not the exception.- Gap found:
workers/api/src/routes/pdf-audit.tsnever callscheckEntitlement/consumeEntitlementat all — despite the schema’s stated intent, the interactive PDF audit (base tier and thedeep/aiflags) is currently 100% unmetered againstaudit_run, gated only byrequireAuth. The HTML/URL audit path (remediate-v2.ts,url-fetch-executor.ts) does chargeaudit_run; PDF audit was never wired to it. This is the actual near-term risk behind Phase 2/3 leaving images-of-text and the AI checks deferred — there’s no charge to extend in the first place, on any tier including Free.
Options:
- A — fold into
audit_runvia quantity (recommended). Wire the PDF audit’s base charge (consumeEntitlement('audit_run', 1, ...), quantity 1) exactly like the HTML path, then bump quantity for the opt-in tiers (e.g.deep→ 2,ai→ 3, or scaled to the actual check count) against the SAMEauditquota_group/entitlement — no new tables. Mirrors convert.ts’s own flat-price pattern: Free’s 10 lifetime audits drain faster on deep/ai, Individual’s wallet is charged proportionally more cents, paid plans keep unlimited access (same “unlimited audits” value prop they already have), andcost_ledgerkeeps margins honest exactly as it does for conversion. This unblocks images-of-text and the remaining AI checks with no migration — they just ride the same quantity bumpdeep/aialready trigger. - B — dedicated product_key(s) (e.g.
pdf_audit_deep,pdf_audit_ai) with their own quota_group(s) andpackage_entitlementsrows — a real, separately priced upsell SKU distinct from base audits, matching the epic’s original “Deep audit (metered/premium)” framing. Needs a new migration + a genuine pricing decision per package (what does Free/Individual/Professional get?) — bigger lift, and the actual reason this has stayed an open question for three phases. - C — hybrid. A package-level entitlement gates whether deep/ai can be requested at all (e.g., Individual+ only), and a per-document meter (quantity scaled to what actually ran) charges usage within that gate. This is really Option A with a package allowlist layered on top — straightforward to add to A later without a redesign.
- Not mutually exclusive: A can ship now (closes the unmetered-spend gap immediately, on every tier including Free) and B can be layered on later if Deep Audit should become an explicit, separately-marketed SKU.
Decision: Option A — landed 2026-09-11. Wired workers/api/src/routes/pdf-audit.ts
to soft-reserve audit_run (reserveUsage, quantity via the new shared
pdfAuditRunQuantity({ deep, ai }) — base 1, deep +1, ai +2) before staging;
holdKey threads through PdfAuditMessage to the executor, which settles it —
consume_entitlement + release 'captured' on a delivered complete result
(raw PostgREST RPC, mirroring recordAiCheckCost’s and routes/audit.ts’s
tryDeductCredits’s existing pattern, since workers/batch has no workers/api
Env), release 'released' (no charge) on any hard failure. A message without
holdKey (pre-existing/in-flight during deploy) is tolerated — skipped, not
thrown. This closes the gap noted above: PDF audit (base tier included) is now
metered against the SAME audit_run entitlement the HTML/URL audit already
uses, on every package tier. Unblocks images-of-text and the remaining Phase 3
AI checks — they ride the same pdfAuditRunQuantity bump, no new migration
needed when they land. Tests: pdf-audit.test.ts (gate/quantity/rollback-release,
34 tests) and pdf-audit-executor.test.ts (charge/release settlement, 27 tests),
both passing; typecheck clean on workers/api, workers/batch, packages/shared.
Reviewed 2026-09-11 via a 4-agent pr-review-toolkit pass (code-reviewer,
silent-failure-hunter, pr-test-analyzer, comment-analyzer) — found and fixed 1
critical + 4 important issues, pushed to PR #2981:
- A DB outage or missing/inactive
audit_runproduct row was falling through to “insufficient credits” (402, falsely telling the user $0 remaining) instead of a service error — fixed to mirrorconvert.ts’s gate exactly (BILLING_MISCONFIGURED/BILLING_UNAVAILABLE, 503). - Settlement now charges by what
deep/aiactually ran (deepAudit?.ran/aiAudit?.ran), not the requested flags — a tier that degraded toran:false(e.g. a fleet-wide Bedrock outage) made zero real calls and is no longer billed for it. The pre-flight reserve still uses the requested flags (must stay worst-case). consumeAuditRunChargeretries once on a transient HTTP failure and tags a definitive non-okRPC status (e.g.insufficient_fundsafter delivery — arguably the most likely real leak) with the sameREVENUE LEAKmarker as the other failure branches. Apending_credit_deductions-style reconciliation row is documented as a deliberate, not-yet-built follow-up (needs a schema change; that table is hard-coded topdf_convert).executePdfAuditnow guards its entry against an SQS redelivery of an already-terminal job — previously a crash between the charge and the SQS delete could let a redelivery overwrite a paid, deliveredcompleterecord withfailed, no refund.- A message missing
holdKeynow logs a warning on both paths instead of a silent no-op.
Added 12 tests covering all of the above; full workers/api (8901 passed)
and workers/batch (401 passed) suites clean; typecheck clean on all three
packages. Full detail in PR #2981’s review comment.