Malware scan gate

Every file-ingest surface should route user-supplied bytes through the shared scan gate before they reach durable storage (R2). The gate stages the bytes in a native AWS S3 bucket that GuardDuty Malware Protection scans on arrival, and only promotes them to R2 on a clean verdict. This is epic #3262 (extraction: #3263); the original single-surface implementation was #3019.

How it works

client / API  ──►  S3 staging bucket  ──►  GuardDuty scans on upload
                     (pending_uploads row)          │
                                                     ▼
                       EventBridge "Object Scan Result"
                                                     │
                              lambda-scan-finding.ts → handleScanResult()
                                                     │
     NO_THREATS_FOUND ─► surface.admit()  → copy staging→R2, mark record ready
     THREATS_FOUND    ─► surface.reject()  → mark record failed (MALWARE_DETECTED)
     UNSUPPORTED/…    ─► surface.reject()  → mark record failed (SCAN_UNAVAILABLE)   ← fail closed

• workers/api/src/services/scan-gate.ts — the generic gate: two entry points, the finding handler, the pending-record lifecycle, and the shared timeout sweep. Knows nothing about any surface’s schema.
• workers/api/src/services/scan-surfaces.ts — per-surface registrations.
• pending_uploads table (migration 314) — one row per staged object, keyed by staging object key, surface-agnostic. The finding handler looks a row up by key, reads surface, and dispatches to that surface’s handler.
• Staging key: staging/<surface>/<ownerId>/<recordId>/<name>. The surface segment is how the finding handler routes with no reverse index.

Fail closed is the invariant. THREATS_FOUND, UNSUPPORTED, ACCESS_DENIED, FAILED, an unknown status, and a scan that never returns (timeout sweep) all reject the upload. Nothing reaches R2 without a NO_THREATS_FOUND verdict.

Recipe: add a scanned upload surface

1. Register the surface in scan-surfaces.ts:

   const acrSurface: ScanSurface = {
     name: 'acr',
     admit: async (ctx) => {
       // copy the clean bytes to durable storage and mark YOUR record ready.
       // MUST be idempotent + compare-and-set. Return false if the record
       // already advanced (lost the race to the sweep / a duplicate delivery).
       const bytes = await ctx.getStagedBytes();
       await ctx.storage.objects.put(ctx.pending.durableKey!, bytes, { ... });
       return markAcrJobReady(ctx.storage, ctx.pending.recordId); // compare-and-set
     },
     reject: async (ctx, failure) => {
       // mark YOUR record failed; never promote. Compare-and-set.
       await markAcrJobFailed(ctx.storage, ctx.pending.recordId, failure);
     },
   };
   // add to registerAllScanSurfaces()
   registerScanSurface(acrSurface);

2. Change the upload path to route through the gate:

   • Presigned direct-to-R2 today → swap the presign for presignForScan({ surface, ownerId, recordId, fileName, contentType, durableKey, metadata }) and return its uploadUrl. The client still does one PUT — to staging now.
   • Server-side formData() → objects.put today → replace the durable write with stageAndAwait({ ..., bytes }). It returns a pending handle, not a verdict (GuardDuty is async) — your record enters a scanning state and the finding handler completes it. Give the client a scanning state to poll; don’t return the result inline.

3. Add a scanning state to your record’s status enum + surface it in the UI so “uploaded” no longer means “ready”.

4. Verify fail-closed with an EICAR test object: upload it and confirm the record ends failed with MALWARE_DETECTED, never promoted.

That’s it — event routing, the pending row, the timeout sweep, and the fail-closed decision are all handled by the gate. You own only the two record-specific transitions.

Hosts and env

isMalwareScanEnabled() requires both UPLOAD_MALWARE_SCAN_ENABLED='true' and UPLOAD_STAGING_BUCKET; otherwise the gate no-ops and uploads go straight to R2 unscanned.

• Lambda host (api-pdf.gevell.com) — both set in infra/cdk/lib/stacks/api-stack.ts. The finding handler and the hourly cron (which runs the timeout sweep) also get UPLOAD_STAGING_BUCKET + staging-bucket grants there.
• Node host (api.gevell.com, 10.1.1.4) — set in .env.node-server. The staging bucket is native AWS S3, not R2: the host’s S3_* creds are R2 creds and cannot reach it. The Node host needs AWS credentials (AWS_ACCESS_KEY_ID/SECRET or an instance role) with PutObject/GetObject/DeleteObject on the staging bucket. Confirm that before setting UPLOAD_MALWARE_SCAN_ENABLED='true' on .4 — with the flag on and no AWS access, uploads there fail to stage.

The parity ratchet scan-env-parity.test.ts fails CI if a scan var is set on one host’s committed config and missing from the other (.env.node-server.example is the Node host’s committed representation).

Timeout sweep

A single sweep (sweepScanTimeouts, run hourly by lambda-cron.ts) fails closed any pending_uploads row still scanning past DEFAULT_SCAN_TIMEOUT_MS (5 min) — one sweep for every surface, so there is no per-surface drift. The files surface additionally has a fast-path timeout check on GET /api/files/:id/status for immediate UX feedback; both are compare-and-set and safe together.