Skip to content
AnglinAI Docs

Bedrock Adoption for PDF Conversions (ZDR) — with Cost Controls

Why

A customer requires Zero Data Retention (ZDR) on PDF conversions. ZDR means the LLM provider does not persist prompt/response content after serving a request and does not train on it. Our only ZDR-capable posture today is Amazon Bedrock with PutAccountDataRetention mode: none + invocation logging off (already in place for the VPAT worker — see bedrock-vpat-setup.md).

Bedrock hosts Anthropic Claude and Amazon Nova, but not Google Gemini or OpenAI. So delivering ZDR on the PDF pipeline forces every Gemini/OpenAI call in that pipeline onto a Claude or Nova model on Bedrock.

This plan is the PDF-only, cost-control-focused execution of the broader bedrock-zdr-rollout-roadmap.md. It is deliberately built so the provider abstraction and cost-control machinery are product-agnostic and reused when ZDR expands to other products.

Decisions locked (2026-08-15)

DecisionChoiceConsequence
ScopeAll PDF conversions route through BedrockOne auditable ZDR story; Gemini fully removed from the PDF path
Cheap-tier modelAmazon Nova (lite/micro) replaces Gemini Flash-tier tasksZDR-eligible, cheaper than Gemini Flash on our volumes
High-fidelity pathClaude Sonnet on Bedrock (not Opus)Opus uses the Anthropic Files API, unsupported on Bedrock — tracked in #1978
Budget-breach behaviorHard kill-switch — fail closed (pause/queue)Never fall back to a non-ZDR provider; overspend is capped, not silently absorbed
Budget ceilingPropose from current spend — see Proposed budgetStart $500/mo with 50/80/100% alerts; re-index to volume when large customers onboard
Gemini fallback on killForbiddenFalling back to Gemini breaks ZDR and the “Gemini removed” guarantee

Non-negotiable invariant

A PDF conversion must never reach a non-ZDR provider. Every migrated call site fails closed (mirror the vpat-parse #1192 pattern: return an error / queue the job rather than silently using Anthropic-direct, Gemini, or OpenAI). This applies to error fallbacks, circuit-breaker escalation, and budget kill-switch behavior alike.

Current state (what we’re migrating)

Full call-site inventory is in the roadmap doc; the PDF-pipeline-specific picture:

Already ZDR-capable pattern

  • workers/vpat-parse — Bedrock Converse via aws4fetch + SigV4, fails closed on 409 when ZDR required but unavailable. Reference implementation for “fail closed.”

Claude-direct → Bedrock (easy — same models exist on Bedrock)

Main converters and quality gates: claude-converter, agentic-vision-converter (ClaudeVisionStrategy), unpdf-claude-converter, alt-text-suggester, cover-text-describer, image-region-recovery, vision-table-extractor, struct-table-extractor, form-hybrid-converter, premium-form-converter. Each constructs new Anthropic({ apiKey }) inline — mechanical swap to the factory.

Opus → Sonnet-on-Bedrock (#1978)

opus-page-converter (Files API — no Bedrock equivalent). ZDR jobs route high-fidelity requests to Sonnet on Bedrock instead. Tracked in #1978.

Gemini → Bedrock (Nova or Claude) — the provider change

Central factory workers/api/src/utils/gemini-client.ts fronts most of these; model IDs centralized in workers/api/src/config/models.ts.

StageFileCurrent Gemini model→ Bedrock target
Cost/complexity predictionconversion-predictor.tsgemini-3-flash-previewNova-lite
Alt-text critiquealt-text-critique.tsgemini-2.0-flash-liteNova-micro
Long descriptions (charts)long-description-generator.tsgemini-3-flash-previewNova-lite (Sonnet if quality regresses)
Visual layout scoringvisual-layout-scorer.tsgemini-3-flash-previewNova-lite
Vision equation auditvision-equation-audit.tsgemini-3-flash-previewNova-lite
Style mimic (hi-fi)pdf-style-mimic.tsgemini-2.5-proClaude Sonnet (Bedrock)
Image enhance / alt-textimage-enhancer.tsgemini-3-flash-previewNova-lite (config-driven)
Image OCRimage-ocr.tsgemini-3-flash-previewNova-lite
Segmented (diagram alt)segmented-converter.tsgemini-1.5-flashNova-lite
HTML remediator (alt)html-remediator.tsgemini-1.5-flashNova-lite
WCAG structured fixeswcag-orchestrator/wcag-impl.tsgemini-2.0-flash (@ai-sdk/google)Claude Haiku/Sonnet on Bedrock
Smart-cascade Gemini branchsmart-cascade-converter.tsgemini/gpt tiersNova / Sonnet

Nova is the default substitute for every Gemini Flash-tier task; escalate a specific task to Claude Sonnet only if the eval corpus shows a real quality regression. Re-baseline each task — the prompts were tuned for Gemini.

Architectural facts that help

  • workers/api runs on Lambda + EC2 Node, both of which carry an IAM role → Bedrock needs no static keys (use @anthropic-ai/bedrock-sdk, which mirrors messages.create() 1:1, and the Bedrock Converse API for Nova).
  • The Gemini factory is a single chokepoint; the cost table (workers/api/src/utils/llm-cost.ts) is centralized.
  • Anthropic has no factory today (25 inline new Anthropic() calls) — the main mechanical cost.

Provider abstraction (foundation — reuse #1652, do NOT build a competing factory)

Do not create a new anthropic-client.ts. The existing Bedrock epic #1651 already owns the provider abstraction: #1652 builds the @accessible/ai semantic-tier library — call sites request a capability tier (TEXT_CHEAP, TEXT_QUALITY, VISION_CHEAP, VISION_QUALITY, IMAGE_GEN), and the tier→(provider, modelId) mapping lives in one env-overridable registry. Provider migration becomes a config flip, not a code change. This plan rides on that library; it must not fork a parallel factory.

What this PDF-ZDR plan requires from / on top of the #1652 library:

  • The library exposes a uniform complete({ tier, messages, schema?, images?, policy? }) → { text|object, usage, modelUsed }, backed by Claude-on-Bedrock (@anthropic-ai/bedrock-sdk) and an Amazon Nova Converse path (SigV4 via IAM role). Direct new Anthropic() remains only as an explicit, non-ZDR break-glass.
  • ZDR policy — the PDF-specific addition (tracked in #1981): a requireZdr policy on the library that fails closed — never silently downgrades to a retaining provider (Anthropic-direct / Gemini / OpenAI) on error, circuit-breaker escalation, or budget kill. Mirror the vpat-parse #1192 409 pattern.
  • Backend selection: LLM_BACKEND (bedrock | anthropic) defaults to bedrock for the PDF path once migrated; PDF’s capability tiers resolve to Nova (cheap) / Claude Sonnet (quality/hi-fi) on Bedrock.
  • CI lint (in #1981) forbids new inline new Anthropic() / new GoogleGenAI() / new OpenAI() outside the library — prevents ZDR regressions.

The library is product-agnostic by design (#1652), so audit/vpat/org-chart/ course-map reuse it; this plan only adds the ZDR policy + PDF tier mappings.

Cost controls (the core of this plan)

Three independent layers, from real-time to authoritative. We do not wait on AWS billing latency for protection — the self-metered ledger is the real-time guardrail; AWS data reconciles it.

Layer 1 — Self-metered, per-call (REAL-TIME, primary)

Every Bedrock response returns usage (input/output/cache tokens). At each call site we already compute cost via llm-cost.ts and write a cost_ledger row (product, model, backend, tokens, estimated_cost_usd, operation_type). This is truly real-time (per request) and independent of AWS billing.

Work:

  • Add Bedrock model IDs + Bedrock pricing (Nova-lite/micro, Sonnet/Haiku on Bedrock, incl. Bedrock prompt-cache rates) to llm-cost.ts.
  • Stamp backend='bedrock' and keep operation_type per stage (migration to add any new operation_type values — the DB CHECK is the source of truth; see reference_cost_ledger_guards).
  • A cheap rollup (materialized view or 1-min cron) maintains bedrock_spend_rollup (today, month-to-date, by product/model). This is what the budget checks read — no full-table scans on the hot path.

Layer 2 — CloudWatch Bedrock metrics (NEAR-REAL-TIME, ZDR-safe cross-check)

Bedrock emits InputTokenCount / OutputTokenCount / Invocations as CloudWatch metrics — these are token counts, not prompt content, so they are emitted even with invocation logging off and do not violate ZDR. Poll them (1–5 min) to independently verify our self-metered totals and catch any un-instrumented call site (a call we forgot to ledger still shows up in the metric counts). Divergence between Layer 1 and Layer 2 → alert.

Layer 3 — AWS Budgets + Cost Explorer / CUR (AUTHORITATIVE, lagged)

  • AWS Budgets: a monthly cost budget scoped to the Bedrock service (and, once tagged, to the PDF cost-allocation tag), thresholds at 50/80/100%, SNS → ingest endpoint. Refreshes every ~8–12h — reconciliation, not protection.
  • Cost & Usage Report (CUR) or Cost Explorer API pulled daily to true-up the ledger against the actual invoice and correct our pricing table drift.

Budgets, alerts & the hard kill-switch

Mirror the existing GCP budget-killswitch pattern (gcp-budget-killswitch.md, gcp_budget_events + /admin/spend) on the AWS side:

  • New aws_budget_events table + an ingest endpoint for the AWS Budgets SNS notifications; surfaced in the /admin/spend panel next to GCP/Anthropic.
  • Alert thresholds via the existing spend_alert_settings + cost-monitors cron + dispatchAdminAlert (Telegram/email). Escalate at 50% / 80% / 100% of the monthly ceiling, plus a daily guardrail (see below).
  • Hard kill-switch (fail closed): a bedrock_pdf_conversions_enabled flag (KV, checked by the conversion dispatcher). When month-to-date self-metered spend crosses 100% (or the daily cap trips):
    1. Flip the flag off.
    2. New PDF conversions are queued/held with a clear status (paused_budget), not dispatched — no Gemini fallback.
    3. Fire a critical alert; require a manual break-glass re-enable (with a reason logged) or an approved ceiling bump.
    4. In-flight jobs finish; only new dispatch is gated.
  • Daily guardrail ($ per day, proposed $50): the real defense against a runaway (e.g., an agentic-vision loop stuck retrying at Sonnet rates, or abuse). Catches within hours what a monthly budget would miss for weeks. Reads Layer 1, so it trips in near-real-time.

Cost attribution & expansion-readiness

Because cost_ledger already carries product, model, backend, operation_type, the same budget/alert/kill-switch machinery generalizes: per-product budgets (PDF now; audit/vpat/etc. later) are just filtered rollups. Build the /admin/spend AWS panel and the kill-switch flag keyed by product from day one.

Proposed budget

Derived from cost_ledger (product = pdf-converter), last 120 days:

  • PDF LLM-only run-rate today: ~$28–35 / month. (The larger PDF line items — Marker ~$36 and Mathpix ~$20 over 120d — are non-LLM services and do not move to Bedrock.)
  • Gemini slice being replaced: ~$4.6 / month (gemini-pro-hifi + gemini-3-flash
    • legacy gemini-2.5-flash). Nova is cheaper than Gemini Flash, so the flash-tier tasks get cheaper; the one exception is pdf-style-mimic (Gemini Pro → Sonnet) which rises but is low-volume (~68 calls/120d).
  • Opus → Sonnet (#1978) should reduce the ~$26/120d Opus line.
  • Net effect at current volume: roughly flat, ~$30–40/month LLM on Bedrock.
  • Trend: August is already ~2–3× spring months — real growth.

Proposed starting ceiling: $500 / month, alerts at $250 / $400 / $500 (50/80/100%), hard kill-switch at 100%, plus a $50 / day runaway guardrail.

Rationale: a kill-switch exists to stop a runaway bug or abuse, not to cap normal growth. $500 is ~10–15× today’s LLM run-rate — deliberate headroom so the switch never trips on organic growth, while still capping a runaway at hundreds (not thousands) of dollars; the $50/day guardrail is the tight, fast trip.

Re-index on volume. This ceiling assumes current volume (~a few thousand pages/mo). A large customer changes everything: the USD proposal alone is 225,000 pages/yr (~18,750/mo) — ~40× current volume. Before onboarding any customer >5,000 pages/mo, switch the budget to volume-indexed ($ per 1,000 pages) and raise the ceiling accordingly. Track the $/page unit cost from the ledger so this is a one-number update.

Phased execution (PDF-scoped)

Phase 0 — Foundations (no behavior change)

  1. Verify the Bedrock ZDR account posture covers the AWS account Lambda/EC2 use (may differ from the vpat-parse account — ZDR is per-account). Re-run the PutAccountDataRetention mode:none + logging-off checks.
  2. Confirm each target model is ZDR-eligible under mode:none (the Fable-5 provider_data_share trap — pre-Fable Claude + Nova are fine).
  3. Add IAM bedrock:InvokeModel (+ inference-profile ARNs) to the Lambda execution role and EC2 instance role. No static keys.
  4. Add Bedrock model IDs + pricing to llm-cost.ts.
  5. Land the #1652 @accessible/ai library (or use it if already landed) and add the PDF-ZDR requireZdr fail-closed policy + CI lint guard on top of it (#1981). Do not build a separate anthropic-client.ts factory.

Phase 0 verification log (2026-08-15, #1981)

Account-side gate — verified with an operator identity (root/CLI) against the live conversion account. These confirm the account posture; they are not proof that the Lambda execution roles can invoke — those roles remain un-granted until the IAM change below is deployed.

CheckResult
AWS account for the PDF Lambdas696081601609 — same account as vpat-parse (ZDR posture applies)
data-retention modenone (set 2026-06-18) ✅
Model invocation loggingOFF (no config) ✅ — prompts never land in CloudWatch/S3
Nova Lite / Nova Micro under mode:noneinvoke OK ✅ (ZDR-eligible)
Claude Sonnet 4.6 (us.anthropic.claude-sonnet-4-6) under mode:noneinvoke OK ✅ (hi-fi target, #1978)
Claude Sonnet 5 (us.anthropic.claude-sonnet-5) under mode:noneinvoke OK ✅ (not caught in the Fable/Mythos trap)
Claude Haiku 4.5verified end-to-end by vpat-parse (bedrock-vpat-setup.md §Status) ✅

IAM (the one real gap): neither PDF Lambda role (accessible-pdf-production-api, …-api-ai-fix-stream) had bedrock:InvokeModel. Both roles are CDK-managed, so the grant was added in infra/cdk/lib/stacks/api-stack.ts (statement InvokeZdrEligibleBedrockModels), not via a manual put-role-policy that would drift on the next cdk deploy.

Two layers keep this fail-closed:

  1. Account posture (primary): data-retention mode: none makes any Fable/Mythos-tier model (which requires provider_data_share) return unavailable at invoke time.
  2. IAM scope (secondary): the grant enumerates the ZDR-eligible Claude families — claude-opus-*, claude-sonnet-*, claude-haiku-* (all pre-Fable) — plus amazon.nova-*, across the foundation-model ARNs (region-wildcarded so cross-region us. profiles resolve) and the account-scoped inference-profile ARNs. It deliberately does not list claude-fable-* / claude-mythos-*. A catch-all claude-* would have matched Fable’s ID (anthropic.claude-fable-5) and left the account setting as the only guard; enumerating the families denies Fable/Mythos at the IAM layer even if the account posture were ever loosened.

Not yet deployed (cdk deploy AccessiblePdfProd-Api pending) — merging does not apply the grant.

Still blocked on #1652 (the library doesn’t exist yet): the requireZdr fail-closed policy wiring, the CI lint guard against new inline new Anthropic()/new GoogleGenAI()/new OpenAI(), and the /health backend+ZDR field. These land with / on top of #1652 — doing a half lint-guard now (with a ~25-entry allowlist of existing sites) is churn #1652 would rework.

Not applicable — the on-prem Node box (10.1.1.4) is not EC2 and carries no IAM role. Its workers/api Bedrock calls will need either static keys (vpat-parse pattern) or a proxy through the Lambda/role’d path — decide in #1983 before the Node runtime flips to Bedrock. Only the Lambda path is IAM-covered by this change.

Phase 1 — Cost-control infrastructure FIRST (before flipping traffic)

Deliberately ahead of the traffic migration so we have guardrails before spend moves:

  1. bedrock_spend_rollup + Layer-1 real-time budget checks.
  2. aws_budget_events + AWS Budgets SNS ingest + /admin/spend AWS panel (product-keyed).
  3. Kill-switch flag + dispatcher gate (paused_budget job status) + break-glass.
  4. Daily-guardrail + 50/80/100% alerts via cost-monitors / dispatchAdminAlert.
  5. CloudWatch Bedrock-metrics poller (Layer 2) + divergence alert.

Phase 2 — Migrate Claude call sites (lowest risk)

  1. Replace inline new Anthropic() in the PDF converters/quality-gates with the #1652 library complete({ tier }) calls; one PR per cluster.
  2. Route high-fidelity to Sonnet-on-Bedrock (#1978); do not call opus-page-converter on ZDR jobs.
  3. Canary via a Lambda alias; verify output parity, latency (cold-start + inference-profile), and cost (ledger). Canary % → 100%.

Phase 3 — Migrate Gemini/OpenAI call sites (the provider change)

  1. Port each Gemini/OpenAI PDF call site to Nova (default) or Sonnet (hi-fi) per the mapping table; re-baseline quality against the eval corpus per task.
  2. Remove the Gemini branch from smart-cascade-converter for PDF.
  3. Gate rollout on no quality regression; watch high-parallelism alt-text cost.

Phase 4 — Enforce & verify

  1. Make LLM_BACKEND=bedrock the PDF default; direct Anthropic is break-glass only.
  2. /health reports active backend + ZDR posture; startup assertion fails loudly on misconfig.
  3. Reconcile ledger vs CUR for one full cycle; tune pricing table.
  4. Only after verification, update the public ZDR claim to cover PDF (today it’s VPAT-only).

Risks & open items

  • Per-account ZDR posture — verify the conversion account, don’t assume it matches vpat-parse (Phase 0).
  • Quality regression on Gemini→Nova/Claude re-prompts — budget eval time; the prompts were tuned for Gemini.
  • High-parallelism cost — alt-text can fan out to ~80 calls/page; confirm Nova pricing holds at that fan-out and that the daily guardrail is sized right.
  • Latency — Bedrock cold-start + inference-profile routing vs direct API; measure on the canary.
  • CUR/Cost Explorer lag — never rely on it for protection; it’s reconciliation only. Protection is Layer 1 + the kill-switch.
  • Nova vision parity — confirm Nova-lite/micro handle the vision tasks (layout scoring, image OCR) acceptably; fall back to Sonnet per-task if not.

Issue tracking

Filed under epic #1980 (ZDR for PDF conversions + cost-control guardrails):

  • #1981 — ZDR account posture + fail-closed provider policy (Phase 0 ZDR bits + lint)
  • #1982 — Cost-control infrastructure (rollup, AWS Budgets ingest, fail-closed kill-switch, daily guardrail, CloudWatch poller) — build before flipping traffic
  • #1983 — PDF cutover + verification + public ZDR claim (Phases 2–4)

Reused from the existing Bedrock epic #1651 — do not refile:

  • #1652 — @accessible/ai semantic-tier provider library (the abstraction)
  • #1655 — migrate workers/api vision call sites to Bedrock (benchmark-gated)
  • #1653 — vpat-parse Bedrock default (reference implementation)
  • #1978 — Sonnet-on-Bedrock for high fidelity (Opus Files-API gap)