Skip to content

Malware Scanning

Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. SPDX-License-Identifier: MIT-0

Documents arrive in this solution from outside it — uploaded through the web UI, posted to the Jobs API, or dropped in the input bucket by another system. By default every one of them is fetched by OCR, rendered to page images and sent to Amazon Bedrock, and it stays in the input bucket until the retention period expires, reachable by anyone who can presign it.

Malware scanning is an optional ingest gate over that path. With it enabled, Amazon GuardDuty Malware Protection for S3 scans each upload and the pipeline admits the document only once the verdict comes back clean. A document that is infected, or that GuardDuty cannot scan, reaches a distinct terminal status with the reason recorded on the document and an alert to the stack’s SNS topic.

It is off by default. Enabling it is a deliberate choice with a cost and a quota attached, both covered below.

GuardDuty runs two scan engines — one AWS builds and manages, plus a third-party vendor engine — fed by threat-intelligence indicators, YARA rules written by AWS security engineers, and heuristic and ML models. The managed engine’s signature list is updated every 15 minutes, so there is nothing to patch on your side. That currency is the main reason to prefer it over a self-managed ClamAV scanner.

⚠️ It is file-based static detection, not behavioural analysis. AWS states that the engine “doesn’t perform live behavioral analysis, where malware detonation monitors the sample as it executes”. Novel malware that no signature, YARA rule or model recognises will be reported clean. This gate raises the cost of delivering known malware through document ingest; it is not a guarantee that every admitted file is safe.

Two CloudFormation parameters, in the Malware Scanning group.

ParameterValuesMeaning
MalwareScanningDISABLED (default), GATE_ONLY, CREATE_PLANWhether to gate ingest, and who owns the GuardDuty plan
AllowUnscannableDocumentsfalse (default), trueCeiling on whether a document GuardDuty could not scan may still be processed

CREATE_PLAN creates the GuardDuty Malware Protection plan and the IAM role GuardDuty assumes, and gates on the result. It needs GuardDuty permissions at deploy time — a restricted CloudFormation service role may not have them — and it uses one of the 25 protected buckets allowed per account per Region. The delegated-deployment role shipped under iam-roles/cloudformation-management/IDP-Cloudformation-Service-Role.yaml carries an unconditional guardduty:* grant for this reason; a deployer who will never use CREATE_PLAN can remove that line safely — the service-role validator here demands one <service>:* per declared resource type, so dropping the grant and the resource together keeps the role in line.

GATE_ONLY creates no GuardDuty resources and gates on scan results produced by a plan somebody else manages, which is the shape to use where a central security account owns GuardDuty. The gate reads the verdict from EventBridge, which delivers it regardless of who created the plan, so the two modes run identical code. Point that plan at this stack’s input bucket.

The input bucket is encrypted with the stack’s customer-managed KMS key, so the role GuardDuty assumes is granted kms:Decrypt and kms:GenerateDataKey on that key, conditioned on kms:ViaService: s3.<region>.<url-suffix>. ⚠️ Removing that statement does not degrade scanning — it stops every document. GuardDuty answers ACCESS_DENIED / UNAUTHORIZED_TO_GET_OBJECT for an object it cannot decrypt, which is an unscannable verdict, and the default policy refuses those.

No change to the KMS key policy is needed to go with it. The key’s Enable IAM User Permissions statement delegates to IAM for in-account principals, and GuardDuty acts as this role, which is in-account. Do not add a guardduty service-principal statement to the key policy to “make this work” — that is not the principal being used.

In GATE_ONLY mode the externally managed plan’s role needs the same grant on this stack’s key, which is the one piece of cross-account setup that mode requires.

With scanning enabled, the upload event records the document and stops:

  1. An upload lands in the input bucket. EventBridge delivers Object Created.
  2. The queue-sender Lambda creates the tracking row with status MALWARE_SCANNING and does not send it to the processing queue. The document appears in the UI immediately, so an upload never looks lost.
  3. GuardDuty scans the object and publishes a GuardDuty Malware Protection Object Scan Result event. A rule scoped to this stack’s input bucket routes it back to the same Lambda.
  4. The verdict decides:
GuardDuty verdictDocument statusWhat happens
scanStatus=COMPLETED, NO_THREATS_FOUNDQUEUEDProcessed normally. The verdict is recorded on the document.
THREATS_FOUNDQUARANTINEDNot processed. Metric, alert, and the threat name recorded. The object stays in the bucket, tagged.
SKIPPED / FAILED (any reason)MALWARE_SCAN_FAILED, or QUEUED if unscannable documents are allowedNot processed by default. Metric and alert either way.

Admission requires the exact pair scanStatus=COMPLETED and scanResultStatus=NO_THREATS_FOUND. Everything else — a reason GuardDuty adds to the API in future, a malformed event, the two fields disagreeing — falls through to the unscannable branch. Reading “not THREATS_FOUND” as clean would turn every future API addition into an open gate.

Gating at ingest rather than as a workflow step

Section titled “Gating at ingest rather than as a workflow step”

The scan is not a Step Functions state, and that is deliberate. A state would have to wait for an asynchronous scan, holding one execution and one of the stack’s concurrency slots per uploaded file — a thousand parked executions for a thousand-document batch. Gating first also means a refused document’s bytes never reach OCR, page rendering or Bedrock at all, which is the property worth having.

The UI does not lose the step, though: the flow diagram draws Malware scan as its first step, built from the document’s own verdict — green for clean, red with the threat name for a detection, amber for a skipped scan.

The verdict is stored on the document and surfaced in three places:

  • A banner on a refused document, naming the threat or the skip reason and what to do next. For a refused document no workflow ran, so there is no execution to open — this banner is the whole explanation.
  • A Malware scan field in the document detail grid, on clean documents as well as refused ones. “The scan ran and found nothing” is the observation an audit needs, and without it a clean document is indistinguishable from an unscanned one.
  • The first step of the flow diagram, as above.

A document admitted because it could not be scanned reads Admitted unscanned, never Clean.

The Reprocess action is disabled for a refused document, with the reason in its tooltip, and the API refuses it too — reprocess re-queues a document directly rather than going through this gate.

It is also disabled while a document is still being scanned, and that case is worded as a wait rather than a refusal because it clears by itself. Reprocess does not wait for a verdict: it builds a fresh queued document and starts the workflow immediately, so allowing it on a held document would send the file to OCR and Bedrock with nothing having decided it is safe — during exactly the window the gate exists for. A second guard sits behind it: if a refusing verdict ever arrives to find the document already past the gate, the gate applies the refusal anyway rather than dismissing it as a repeat delivery, and raises <StackName>-malware-unapplied-refusal.

⚠️ There is no malware column in the document list, deliberately. The list has two backends: the date-range resolver returns the whole tracking-table item and would carry these attributes, but the faster GSI resolver can only return attributes in that index’s INCLUDE projection, which DynamoDB does not allow to be amended in place. A column that populated on one list path and was empty on the other would be worse than none, so the verdict lives on the document’s own page. The status column already distinguishes QUARANTINED and MALWARE_SCAN_FAILED on both paths.

Policy: what to do with an unscannable document

Section titled “Policy: what to do with an unscannable document”

GuardDuty returns no verdict for a password-protected file, an object over its quotas, an SSE-C encrypted object, or an internal error. Password-protected PDFs are a legitimate input to a document-processing system, so this is a real decision rather than an edge case.

Two settings, and both must agree before such a document is processed:

  • malware_scanning.on_unscannable in the configuration — block (default) or allow. Editable in the Visual Editor, versioned with the rest of the configuration, and applied from the version the document was queued under.
  • AllowUnscannableDocuments on the stack — false (default) or true. A ceiling: with it false, setting the configuration to allow has no effect and the gate logs that it ignored it.

The split exists because the configuration is editable by any Admin from the web UI. A security posture that one UI edit can lower is not much of a posture, so loosening it takes both a configuration change and a stack update.

malware_scanning.alert_on_block controls whether a refusal publishes to the SNS alerts topic. The CloudWatch metrics and the document’s own status are recorded either way — silencing a notification is not the same as deciding nothing happened.

A document GuardDuty reports as infected is always refused. That is not configurable, and the configuration schema offers no field for it.

Every gate metric lands in the stack’s own CloudWatch namespace (<StackName>) and on the Malware Scanning (ingest gate) widget of the stack dashboard. No count is quoted — that is the figure that went stale last time, and a test derives the set from the gate’s own _put_metric call sites and fails if any is missing from the widget:

MetricMeaning
MalwareScanHeldParked at the gate awaiting a verdict
MalwareScanVerdictAppliedA verdict was acted on, whichever way it went
MalwareScanCleanAdmitted after a clean scan
MalwareScanThreatsFoundDetection; document quarantined
MalwareScanUnscannableBlockedRefused for want of a verdict
MalwareScanUnscannableAllowedAdmitted without a verdict, under policy
MalwareScanVerdictBeforeIngestVerdict arrived before the upload event was recorded (including a re-upload of an existing key); retried
MalwareScanVerdictDuplicateRepeat delivery of a verdict already applied
MalwareScanVerdictSupersededVerdict for a version the key no longer holds; discarded
MalwareScanJobFinalizeFailedA refused /Jobs document could not be marked terminal in its job
MalwareScanVerdictUnappliedRefusalA refusing verdict reached a document that had already left the gate, so it was processed unscanned and the refusal applied late

Alarms, all publishing to the stack’s AlertsTopic:

  • <StackName>-malware-threats-found on any detection. One infected upload is an incident, so the threshold is zero. It has no OK action: “no malware in the last five minutes” is not news, and a recovery message after every detection teaches the recipient to ignore the pair.
  • <StackName>-malware-unscannable-documents on a run of refusals. Usually password-protected files or quota overruns — but a sudden run of them can also mean the GuardDuty role lost access to the input bucket’s KMS key, in which case every upload is being refused.
  • <StackName>-malware-verdict-dlq-depth on any verdict EventBridge could not deliver to the queue sender after every retry. A verdict that was delivered but failed in the queue sender every time goes to the queue sender’s own dead-letter queue instead (QueueSenderDLQAlarm). Unlike the two above, this is about the gate’s own plumbing rather than a document: nothing decided anything, and a document is stuck. Read the message off the dead-letter queue for the bucket, key and version. This one does announce recovery, because “the queue is empty again” is what you are waiting to hear.
  • <StackName>-malware-no-verdicts when documents are held at the gate and no verdict has been applied for an hour. ⚠️ This is the only signal for GuardDuty not answering at all — a plan deleted, a GATE_ONLY stack pointed at a plan that does not cover this bucket, GuardDuty disabled in the Region, or the plan’s role broken. Every other metric here is published when something is decided, so in that state ingest halts completely while all of them read zero: on a gate that fails closed, the loudest failure with the quietest signal. Detected by comparing documents held against verdicts applied rather than by watching one series, because you cannot alarm on an absence.
  • <StackName>-malware-unapplied-refusal when a refusing verdict arrives for a document that has already left the gate. Threshold zero: that document was processed with nothing having decided it was safe, so its OCR output and any Bedrock calls happened before the file was known to be infected. The refusal is still applied — the document ends QUARANTINED or MALWARE_SCAN_FAILED and an alert is published — but the processing already happened. Reprocess refuses a document that is still being scanned, so reaching this means some other path does not; the queue-sender log names the document and the status its row was in.
  • <StackName>-malware-job-finalize-failed when a refused document belonged to a /Jobs batch and the gate could not mark it terminal, or could not hand the finished batch on to be zipped. Threshold zero. ⚠️ This is the only silent stall the gate itself can cause: both code paths deliberately swallow the error, because the refusal is already recorded and raising would re-deliver the verdict to no effect — so the batch answers IN_PROGRESS indefinitely with this metric as the one signal. Every other metric here records a decision; this one records a decision that could not be written down.
  • <StackName>-malware-verdicts-unapplied on verdicts that reached the gate and were deliberately not applied — a superseded one (the key was re-uploaded mid-scan, which self-heals) plus one that outran its ingest event (which does not). A few superseded verdicts are normal where people replace files; a sustained rate means documents are accumulating at the gate.

Each refusal also publishes a message naming the object, the threat or skip reason, and the remedy.

GuardDuty Malware Protection for S3 includes a 12-month free tier of 1,000 requests and 1 GB scanned per month. Beyond that, $0.09 per GB plus $0.215 per 1,000 objects — AWS’s own worked example is 4,000 files totalling 350 GB for $32.36 per month. Object tagging, the S3 API calls GuardDuty makes, and EventBridge are billed separately and sit outside the free tier.

Non-adjustable quotas: 100 GB maximum object size, 100,000 extracted files per object, 100 levels of archive nesting, and 25 protected buckets per account per Region. That last one is the constraint to plan around if you run many stacks in one account and Region.

  • It is asynchronous. The object exists in the input bucket, and is reachable by anyone who can presign it, between the upload and the verdict. The gate stops the file reaching the pipeline; it does not stop it arriving.
  • Reprocessing a clean document does not rescan it. It was scanned on upload. Reprocessing a refused or still-scanning document is blocked outright, and in both reprocess paths — they share no code. The web UI and the REST API go through the reprocess resolver; idp-cli reprocess, documents.reprocess() and batches.reprocess() go through the SDK’s rerun processor, which runs client-side under the caller’s own credentials and reaches SQS itself. Guarding only the first would leave the gate bypassable from the command line, and --batch-id is the worse door because it re-queues a whole batch unattended. scripts/tests/test_reprocess_paths_agree.py reconciles the two lists, which cannot be a shared constant: the resolver imports idp_common directly and the SDK reaches it through a lazy path shim.
  • The staging bucket is not scanned. .zip uploads expanded by the test-set extractor and the batch pre-processor go through a different bucket, which is outside this gate. Protecting it as well would cost a second of the 25 bucket slots per stack.
  • No GuardDuty findings without a detector. If Malware Protection for S3 is enabled as an independent feature, with no GuardDuty detector in the Region, a detection produces no GuardDuty finding — only the EventBridge event, the object tag, and this stack’s own alert and UI surfaces. Enable GuardDuty itself if you want findings in Security Hub.
  • GuardDuty calls PutBucketNotification on the input bucket to assert that EventBridge notifications are on. This template already sets EventBridgeEnabled: true and declares no other notification configuration, so it is a no-op in practice — but it is a managed write to a CloudFormation-owned property and belongs on your list of drift sources.

⚠️ Setting MalwareScanning back to DISABLED does not release documents that are already held. Every malware resource is conditional on the feature being enabled, so disabling it removes the EventBridge rule that delivers verdicts, the dead-letter queue and all six alarms — while any row sitting at MALWARE_SCANNING stays exactly where it is. Because that status is non-terminal, the detail page keeps polling such a document indefinitely, a /Jobs batch holding one stays IN_PROGRESS, and the alarm that would have told you is the one you just deleted.

This matters because disabling is the obvious reaction to the gate stalling, which is the moment documents are most likely to be held.

So drain first, or clean up afterwards. Either way the held documents have to be re-submitted: delete them from the input bucket and upload them again, which creates fresh rows on the un-gated path. Reprocess will not do it — it is refused for a document that is still being scanned, precisely so that a held file cannot be pushed into the pipeline without a verdict. There is no automatic sweep; if the list is long, the document list filtered on MALWARE_SCANNING is the set to work through.

A phantom document named malware-protection-resource-validation-object. GuardDuty writes that object into the protected bucket to prove it can. The gate skips the key unconditionally, in both modes, so it never becomes a document. If you see one in the document list, the skip has regressed.

A document stuck in MALWARE_SCANNING. The verdict never arrived or could not be applied. Three causes, each with its own signal: the malware-verdict dead-letter queue (EventBridge could not deliver it at all), MalwareScanVerdictBeforeIngest (it outran the upload event and Lambda’s two retries, about three minutes in all, were exhausted, so the verdict is in the queue sender’s dead-letter queue) and MalwareScanVerdictSuperseded (it described a version the key no longer holds — normally self-healing, since the newer upload’s own verdict decides). Then the queue-sender logs, which name the cause per document. Check the protection plan’s status too: aws guardduty list-malware-protection-plans then get-malware-protection-plan. A WARNING carrying INSUFFICIENT_TEST_OBJECT_PERMISSIONS means the role policy is wrong.

A Jobs API batch containing a refused file still completes. The refused file is recorded as FAILED in the job’s file map, so the job reaches PARTIALLY_SUCCEEDED with a results.zip holding the successful files’ output — or, if every file was refused, FAILED with no zip, since there is nothing to put in one. Without this the batch would report IN_PROGRESS indefinitely, because the job tracker only hears about documents that ran a workflow.

The gate records the file’s status but does not build the zip itself. It runs in the queue-sender Lambda, which is sized and scoped for one invocation per upload — 128 MB, a 30-second timeout, and no object read or write on the output bucket — whereas zipping a batch’s outputs is what the job tracker’s 3 GB, 900-second timeout and 5 GB of ephemeral storage exist for. So when a refusal is the last file in its batch to go terminal, and no execution-status event is therefore coming, the gate asynchronously invokes the job tracker to finish the job. If that invoke fails it is logged and counted under MalwareScanJobFinalizeFailed rather than retried: the file statuses are already correct, so the worst case is the IN_PROGRESS stall that existed before, now attributable to a named metric. Any other ordering — a clean file in the batch finishing last — completes by the job tracker’s ordinary route with no delegation involved.

Re-uploading the same filename while a scan is in flight. Handled, and worth knowing how. The document row is keyed on the object key alone, so both uploads share one row — and without a discriminator v1’s clean verdict would satisfy the gate while the bucket held an unscanned v2. The gate therefore records which object version it is waiting on and pins its compare-and-set to it, so a verdict for a superseded upload is discarded (metric MalwareScanVerdictSuperseded) and the document waits for the verdict belonging to the bytes actually stored. The same guard covers the mirror case: a THREATS_FOUND verdict for a version since replaced by a clean file does not quarantine the clean one.

A version mismatch does not by itself say which side is out of date, because the verdict for a new upload can arrive before that upload’s own event has updated the document row, which then still describes the previous upload or the previous run. So on a mismatch the gate reads the object’s current version from the input bucket. A verdict for the version the bucket holds now is retried (metric MalwareScanVerdictBeforeIngest) until the upload has been recorded; only a verdict for a version the bucket no longer holds is discarded as superseded. If the bucket cannot be read, the verdict is retried rather than discarded.

The two can also cross the other way. The gate’s check that the document is still waiting can fail against the previous run’s row, and the upload’s own event can then finish updating the row before the gate reads it back, so the row it reads is waiting for exactly this verdict. The gate then handles the verdict again from the start, against the row the upload just wrote, and applies it. If the row keeps changing, it retries the verdict rather than discarding it.

How a retried verdict is retried. EventBridge invokes the queue sender asynchronously, so when the gate raises an error to retry a verdict, Lambda does the retrying: two retries, about one and then about two minutes after the first attempt (EventInvokeConfig on QueueSender in template.yaml). If all three attempts fail, the verdict goes to the queue sender’s dead-letter queue and QueueSenderDLQAlarm fires. The retry policy on the verdict rule is a different mechanism. It covers only EventBridge failing to hand the verdict to Lambda, because Lambda accepts an event before the function runs. Those failures go to the malware-verdict dead-letter queue. In practice the upload is recorded within a second of its verdict, so the three-minute window is ample.

Load testing. scripts/malware_gate_burst.py pushes a few hundred documents through the gate at once and asserts the properties concurrency breaks — every upload gets a row, nothing is left in MALWARE_SCANNING, no benign document is quarantined, and admission is not serialised. Deliberately not part of the nightly suite: admitting a few hundred documents starts a few hundred Step Functions executions and the Bedrock calls behind them. Run it for a release check or after changing the gate:

Terminal window
AWS_PROFILE=default python3 scripts/malware_gate_burst.py \
--stack-name my-idp-stack --count 300 --with-eicar

Testing it. Upload a file containing the EICAR test string — a harmless, industry-standard probe that scanners recognise, and the one GuardDuty’s own documentation uses in its THREATS_FOUND example. Expect QUARANTINED, no Step Functions execution, the red banner naming EICAR-Test-File, and the object still present and tagged GuardDutyMalwareScanStatus=THREATS_FOUND.