AI Risk Register — Gevell

Scope: AI features in Gevell PDF (pdf.gevell.com), Gevell Audit (audit.gevell.com), and Gevell Report Card (acr.gevell.com) — the three products covered by the 2026-09 enterprise security assessment. Other Gevell products run on the same workers/api AI services; risks here apply to them equally except where noted.

Owner: Engineering lead Review cadence: Quarterly, and on any new AI feature, provider change, or model-family change. Next review: 2026-12-14

Companion document: nist-ai-rmf-alignment.md maps these risks to the NIST AI RMF functions. Non-AI security incidents live in ../../Policies/incident-risk-register.md, which uses the same scoring scale so the two registers can be read together.

Scoring

Matches the scale used by the security incident register.

Likelihood = probability of at least one occurrence in the next 3 years given current controls. Disaster = severity if it happens, with no additional controls beyond those listed.

L	Likelihood	D	Disaster
5	Near-certain (>80%)	5	Existential — business-ending or contract-ending
4	Likely (50–80%)	4	Severe — breach notification, customer loss, weeks of recovery
3	Possible (20–50%)	3	Serious — days of downtime or material cost, recoverable
2	Unlikely (5–20%)	2	Contained — hours of cleanup, no customer impact
1	Rare (<5%)	1	Nuisance

Residual is the score after the controls listed in each entry. An entry whose controls are only planned carries its inherent score until they ship.

Ranked summary

#	Risk	L	D	Residual	Status	Issue
1	PII/PHI in customer documents reaches a model with no content filter	5	3	15	Unmitigated — text filter withdrawn (#3463); needs image-channel redaction	#3463
2	Differential treatment in alt text for images of people	1	3	3	Measured + remediated	#3016
3	Malware in an uploaded file reaches the processing pipeline	3	3	9	Implemented; inert unless flag + bucket set	#3019
4	Prompt injection via document content steers a generated deliverable	3	3	9	Mitigated	—
5	AI call bypasses ZDR infrastructure and reaches a provider directly	2	4	8	Mitigated except one blocked call site	#3013 / #3014
6	Customer content retained or used for training by a provider	1	5	5	Mitigated	—
7	Admin-only experimental converter processes data outside ZDR	1	3	3	Accepted	#3052

1. PII/PHI in customer documents reaches a model with no content filter

L5 / D3 — residual 15. The highest-scoring entry, and the only one where likelihood is effectively certain: customers upload university advising worksheets, org charts, and catalogs that contain named students and employees, and every one of those documents is sent to a model in full. There is no content filter today. Likelihood is 5 because this is not an incident that might happen — it is the normal operation of the product.

Disaster is held at 3 rather than higher because the content is processed under Bedrock ZDR: it is not retained by the provider, not used for training, and not exposed to a third party. The realistic damage is contractual and reputational — a customer with a HIPAA or FERPA posture that requires pre-ingestion filtering cannot be served, and answering “no” to that questionnaire item costs deals.

Controls

Bedrock ZDR covers retention (entry 6), which bounds the impact but does not address ingestion.
The Disable AI services switch, settable per user and per tenant (middleware/ require-ai-enabled.ts takes either as disabling). This is the one control that actually keeps content away from a model, and it is all-or-nothing by design. Note it is not uniformly graceful: the converter degrades to deterministic processing, while several routes (advising-extract, tax-exemption, forms) decline the operation with aiDisabledByUser.
Withdrawn (#3015 → #3463): the ai_pii_filter setting (off / pii / phi) and its AWS Comprehend / Comprehend Medical scrubber. Removed from the UI; the code is inert behind AI_PII_FILTER_ENABLED (default off).

Why it was withdrawn rather than extended. The earlier read was that coverage was partial — wired into PDF conversion, missing from org-chart extraction, alt-text suggestion, HTML remediation and the rest — and that the fix was to extend it. That read flattered the control in one specific way: the conversion path it was wired into is the reflow cascade (smart-cascade-converter.ts), and that lane prompts the model with the page itself, as PDF bytes or a rendered PNG. The text the filter scrubbed is used there for routing and coverage checks. So on the largest covered path, redaction was removing identifiers from a channel the model was largely not reading.

It was not ineffective everywhere. Two text-only paths — unpdf-claude-converter.ts and the ACR pre-grader — had genuine redaction, and withdrawing the filter gives that up. It was never wired into the preserve (tag-in-place) lane at all.

The decision was that a control covering three call paths, surfaced in the UI as an account-wide “AI data protection” level, misdescribed its scope — and scope is what a customer acts on. Correcting that was judged worth more than keeping redaction on two paths nobody had switched on. Reducing this score meaningfully requires redaction covering the image channel; text-only redaction cannot move it far. See pii-filter-coverage.md.

Residual: L5 / D3 = 15, unchanged by the withdrawal. The filter’s removal does not raise this score, because the paths it actually protected were two text-only calls that no customer had enabled — in production ai_pii_filter was off for every profile and for the tenant. Nor does it lower it. The score was 15 before and is 15 now; what changed is that the register no longer reports a mitigation that was not operating on the bulk of traffic.

(An earlier revision of this entry scored the conversion path at L2 / D3 on the strength of the filter being wired into it. That was wrong — the cascade sends the page image, so the filter was not reducing exposure there. The single score above replaces it.)

2. Differential treatment in alt text for images of people

L3 / D3 — residual 9. The alt-text pipeline (alt-text-suggester.ts, embedded-image-describer.ts, image-description-pipeline.ts) uses a vision model to describe images inside customer documents. Vision models have documented tendencies to describe people differently by perceived race, gender, age, or disability — unsolicited demographic labelling, differential specificity, or stereotyped language.

This is the register’s sharpest irony and the reason it is not scored lower: the product exists to serve disabled users, and the failure mode is a screen-reader user receiving a worse or editorialising description of a person than a sighted reader would infer. Disaster is 3 rather than 4 — it is a quality and credibility failure rather than a breach — but for an accessibility vendor the credibility component is disproportionate.

Status: measured and remediated (#3016, 2026-09-14). This entry is no longer an estimate. Full methodology, results, and remediation in bias-evaluation.md; the harness is reproducible in scripts/bias-eval/.

What the evaluation found. Against a balanced 112-image FairFace benchmark (7 race groups × 2 genders), the concern in this entry was real but narrower than feared. The describer volunteered a skin-tone descriptor on 8.9% of images, and unevenly — 18.8% for Black and Indian subjects against 0% for Latino, Middle Eastern, and Southeast Asian subjects. Description length also skewed (~22% spread, Black subjects longest, Latino shortest). Explicit race labelling was rare (1/112) and stereotyped or evaluative language did not appear at all (0/112).

Controls

Underlying models (Anthropic Claude, Amazon Nova) carry the providers’ own bias evaluations and responsible-AI programs. This is a baseline, not evidence about our prompts or our inputs.
Implemented (#3016): an explicit neutrality instruction in the describer prompts (root cause) plus a demographic-label output filter (backstop).
Re-evaluation against the same 112 images: demographic labelling 0/112, race labelling 0/112, description-length spread roughly halved (69 → 36 chars). The neutrality rule also reduced an omission differential in the suggester path (66% → 25% “decorative”, more uniform).

Residual: L1 / D3 — 3. Scored down from L3 because the measured failure mode is now zero on the benchmark and the fix addresses the cause rather than only the symptom. Not zero-risk: this is a point-in-time audit of one benchmark, the filter is a backstop rather than a guarantee, and the result does not transfer automatically to a new vision model or a materially changed prompt — which is exactly when it must be re-run.

Assessment answer: see the attestation in bias-evaluation.md §7, which is written to be quoted directly.”

3. Malware in an uploaded file reaches the processing pipeline

L3 / D3 — residual 9. A malicious PDF is unlikely to compromise the model itself, but it transits our storage and is handled by parsing libraries (pdfjs, unpdf, Mathpix) that have their own CVE history. Scanning is now implemented, but it is feature-flagged, so the score stays at 9 until the flag and staging bucket are confirmed set in the deployed environment.

Controls

Parsing runs in Lambda/Workers sandboxes, not on a workstation.
Implemented (#3019): uploads are presigned to a native-S3 staging bucket rather than R2, GuardDuty Malware Protection scans them there, and the finding handler copies clean objects into R2 (scanning → uploaded) or fails infected ones. Nothing reaches R2 unscanned. Wired in routes/files.ts with the S3 staging bucket and GuardDuty provisioned in the CDK api-stack.

Gated off until deployment config is set. isMalwareScanEnabled() requires both UPLOAD_MALWARE_SCAN_ENABLED === 'true' and a configured UPLOAD_STAGING_BUCKET; otherwise uploads fall back to writing straight to R2. Requiring both is deliberate — a flag with no bucket is a misconfiguration rather than a reason to silently skip scanning — but it does mean this control is inert in any environment where those two variables are not set, and the register should not be read as saying uploads are scanned today. Confirm the deployed values before citing this control in an assessment.

Known limitation, to state plainly in assessments: GuardDuty is signature-based. It reliably catches known malware and will not catch a novel PDF exploit crafted against this pipeline. It is a meaningful control, not a guarantee.

4. Prompt injection via document content steers a generated deliverable

L3 / D3 — residual 9. A hostile document carries text instructing the model to alter the output — insert a link, misstate a conformance result, change a description.

The classic prompt-injection disaster — injected text reaching a model that holds tools or credentials — is not live for this architecture. Conversion-pipeline inference is stateless and tool-less, and customer documents are never opened in agent sessions with MCP servers attached. What remains is injection-to-output, which needs no tools. See entry 2 of the security incident register for the fuller treatment; this entry exists so the AI register is not silent on it.

Controls (implemented, verified in code)

Untrusted document content is wrapped in delimited blocks (<untrusted_context>, <untrusted_alt>, <untrusted_filename>) with an explicit system-prompt instruction to treat the contents as third-party data, never as instructions — alt-text-suggester.ts:90.
sanitizeUntrusted() strips closing tags so injected text cannot break out of its delimiter — alt-text-suggester.ts:221.
Output format constraints (structured JSON, bounded max_tokens) limit what a successful injection can express.

Red-team status: design-level pass complete (2026-09-14, #3021) — prompt-injection-redteam-2026-09-14.md works concrete injection payloads against each AI-facing feature (Fix with AI, alt text, WCAG pre-grader, triage agent) and traces them to the mitigation that catches them. Three of the four remove the high-severity outcomes at the code/architecture layer; the pre-grader is the weakest (no input delimiter) but is advisory and human-reviewed. A live run against a billable test environment is recommended and remains open.

5. AI call bypasses ZDR infrastructure and reaches a provider directly

L2 / D4 — residual 8. Before #3013, most Anthropic SDK calls went straight to api.anthropic.com, outside the Bedrock ZDR agreement. Disaster is 4: the ZDR guarantee is the central claim in the enterprise assessment, and a demonstrated bypass would undermine every other answer in it.

Controls (implemented)

createAnthropicClient(apiKey, bedrock) (workers/api/src/services/anthropic-client.ts) routes through Bedrock whenever AWS credentials are present. Call sites resolve the profile id with resolveModelId(model, bedrock) and send the resolved id, so the id that is billed is the id that was invoked. 31 modules across the assessed products are migrated (#3013, #3014).
Fail-closed by design. An unmapped model id is prefixed into Bedrock form rather than falling back to the direct API — a wrong id fails loudly at Bedrock instead of silently leaking traffic to a non-ZDR endpoint. The Gemini shim (bedrock-gemini-shim.ts, #2751) applies the same rule and throws when AWS credentials are absent.
Model ids are mapped exactly, never by substring, and every mapping is verified against the account’s real inference-profile inventory (docs/admin/inference_profiles.json) by a unit test, so a plausible-but-nonexistent profile fails in CI rather than at call time.
is_bedrock_model() (migration 302) matches us./eu./apac./global. profile ids, so Bedrock spend is attributable and reconcilable against CloudWatch — the detective control that would surface an unnoticed routing change. The routing regex and this predicate are kept deliberately identical; a divergence between them is how a routed call goes unbilled.

Residual driver — one production call site, and it is the awkward one. #3014 is substantially complete: opus-page-converter.ts is the only remaining service constructing the SDK directly. It is not an oversight. The service uploads PDFs through the Anthropic Files API (client.beta.files.*), which has no Bedrock equivalent, so it cannot be migrated without reworking it to inline document content. It sits inside Gevell PDF — an assessed product — which makes this a ZDR gap within scope rather than outside it. Likelihood stays at 2 because the path is a narrow, admin-triggered converter rather than the default pipeline, but it should be described accurately in assessments rather than reported as fully closed.

tools/segment-cli/ also constructs the SDK directly. It is a local developer CLI, not a deployed service, and does not process customer documents in production; migrating it to the factory is tracked separately as #3090.

Verification: routing is enforced in code and unit tests, and (2026-09-14) confirmed in production by a live smoke test — see “ZDR routing verification” below. Periodic CloudTrail re-verification remains scheduled per the measurement cadence.

6. Customer content retained or used for training by a provider

L1 / D5 — residual 5. The lowest likelihood and the highest disaster in the register: a provider retaining or training on customer documents would be contract-ending for the university accounts this business depends on, but it is contractually prohibited rather than merely discouraged.

Controls (implemented)

AWS Bedrock Zero Retained Data agreement covers all Claude and Nova inference.
Mathpix improve_mathpix: false at every call site (PR #3018) — opts out of QA-team data access and image persistence. Verified present in mathpix-pdf.ts, segmented-converter.ts, smart-cascade-converter.ts.
Gemini-shaped callers route through Bedrock by default and throw rather than fall back to Google’s non-ZDR API (#2751).

Residual is not zero because it depends on provider compliance, which we cannot directly observe — the control is contractual, and the detective control is entry 5’s spend attribution.

7. Admin-only experimental converter processes data outside ZDR

L1 / D3 — residual 3. Accepted.

opus-page-converter (the opus-page parser) calls the Anthropic API directly rather than Bedrock. It cannot use Bedrock: it uploads the PDF to the Files API and enables the Code Execution tool so the model can crop figures in a server-side sandbox, and neither feature exists on Bedrock. This is architectural, not a configuration or model-id problem — migrating it would be a rewrite that discards the quality characteristics the approach was chosen for.

Why the residual is 3 rather than a live gap

Never auto-selected. Every parser: 'auto' request — all customer traffic — is forced onto the high-fidelity Bedrock cascade. routes/convert.ts:829-832.
Admin-gated. Requesting opus-page or opus-harness without an @theaccessible.org / @gevell.com account returns 403 FORBIDDEN. routes/convert.ts:836-844.
Labelled at the gate. The code marks it “EXPERIMENTAL and non-ZDR (direct Anthropic API, not Bedrock) — admin accounts only, for testing. Never an auto default.”

Likelihood is 1 because customer data reaching it requires a staff member to deliberately upload a customer document and explicitly name the parser.

Decision: accepted (Larry, 2026-09-14, #3052). An admin-only, explicitly labelled, non-ZDR internal testing path is acceptable; it processes no customer data.

Assessment answer:

All customer document processing routes through Amazon Bedrock under Zero Retained Data. One experimental converter reaches the Anthropic API directly; it is unavailable to customers, restricted to staff accounts by an explicit authorization check, never selected by automatic routing, and used only for internal quality testing.

Revisit if the converter is promoted out of experimental, exposed beyond admin accounts, or auto-routing is ever changed to select it. Any of those invalidates the acceptance above.

GOVERN evidence status

nist-ai-rmf-alignment.md (#3020) cites docs/admin/ai/ai-development-policy.md as the evidence for five GOVERN controls — named feature owners, the approved-tool register, the incident reporting procedure, the quarterly review commitment, and the human-review requirement.

When this register was first drafted that file did not exist, and those rows were unevidenced. It has since been written and merged (ai-development-policy.md, #3066), so the document an assessor would ask for is now in the repository and all five controls are addressed in its Policy section.

The evidence is not yet complete, though, and the distinction matters for an assessment: the policy is explicitly a draft for adoption, and its own adoption checklist is unticked — no policy owner is recorded, the approved-tool register is not published, and the PR template has no AI-assistance field. Until that checklist is complete, treat those five GOVERN rows as evidenced by a draft policy that is not yet in force, not as satisfied controls, and describe the policy externally the way the document itself instructs — as a draft being implemented.

ZDR routing verification — method and evidence (entry 5)

Status: VERIFIED (2026-09-14). #3013 and #3014 are both closed — all in-scope production call sites route through the Bedrock ZDR factory (createAnthropicClient), with the single documented opus-page-converter exception (Files API, entry 7, #3052). A live production smoke test confirmed routing end-to-end on the HTTP API Lambda host:

POST https://api-pdf.theaccessible.org/api/pdf/alt-text — a ZDR-only route that calls bedrockCreds(c.env) → callClaudeBedrock → AnthropicBedrock (and returns 502 if the Lambda’s temporary credentials lack a session token) — returned 200 ({"alt":"","decorative":true,"confidence":0.95}, 1.4s). CloudWatch (request d395ae1f, log group /aws/lambda/accessible-pdf-production-api) shows the 200 with no anthropic_bedrock_fallback warning, confirming the call went through Bedrock rather than the direct Anthropic API. This followed the #3085 fix that wired AWS_SESSION_TOKEN into c.env on the Lambda host (deploy run 34893093288, Lambda updated 2026-09-14T20:40:56Z). The same c.env → bedrockCreds path backs the Audit (remediate) and Report Card (acr) routes, so they are covered by the same fix.

Full verification method (for periodic re-measurement — see cadence):

CloudTrail — Bedrock invocations. Confirm bedrock:InvokeModel / InvokeModelWithResponseStream events appear in CloudTrail for the assessed products’ AWS account over a representative window, and that the modelId values match the inference-profile ids in docs/admin/inference_profiles.json. Capture: event count, distinct modelIds, userIdentity (the Lambda/task role), and the time window.
Cost-ledger cross-check. is_bedrock_model() (migration 302) tags Bedrock spend in the cost ledger. Reconcile the ledger’s Bedrock-tagged AI calls for the same window against the CloudTrail invocation count — a ledger row with an AI operation but no corresponding Bedrock event is a bypass signal. Capture: ledger row count by backend, and any anthropic-backed rows that should have been bedrock.
Negative check — direct Anthropic traffic. Confirm there is no unexpected egress to api.anthropic.com from the production roles (VPC flow logs / egress inspection where available), except the documented, admin-only opus-page-converter path (entry 7). Any other direct-API traffic is a finding.
Fail-closed proof. Confirm in a non-prod run that removing AWS credentials makes the Anthropic and Gemini-shim paths throw rather than fall back to a direct API (the behaviour asserted by unit tests today), so the log evidence above reflects an enforced invariant, not a coincidence of configuration.

Evidence to attach to the assessment: the dated live smoke-test result above (200 + no fallback warning) plus, for a broader window, a CloudTrail export (or Athena query result), the reconciled cost-ledger summary, and a one-line attestation naming the window, the account (696081601609), and the single documented exception (entry 7, #3052). ZDR routing may now be described in assessments as “enforced in code, verified by unit tests, and confirmed in production by a live smoke test on 2026-09-14 (200 response with no direct-API fallback); periodic CloudTrail re-verification per the measurement cadence.”

Open measurement work

The register identifies and scores risks. The bias evaluation (entry 2) is complete — see bias-evaluation.md. Two of #3021’s measurement items remain outstanding, and the entries they cover stay provisional until they complete:

Work	Covers	Status	Issue
Adversarial red-team of each AI-facing feature	Entry 4	Done (design-level) — `prompt-injection-redteam-2026-09-14.md`; live run recommended once a billable test env exists	#3021
ZDR routing verified via CloudTrail / Bedrock invocation logs	Entry 5	Verified 2026-09-14 — #3013/#3014 closed; live prod smoke test (200 + no direct-API fallback) confirms routing; periodic CloudTrail re-check per cadence — see “ZDR routing verification” above	#3021

The design-level red-team found one code-level gap: the WCAG pre-grader (wcag-ai-quality-checker.ts) inlines untrusted headings/body without the delimiter-and-”treat as data” wrapper the other document-facing features use. Raised as a follow-up; not changed here.