Malware Scanning
Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. SPDX-License-Identifier: MIT-0
Malware Scanning on Document Ingest
Section titled “Malware Scanning on Document Ingest”Documents arrive in this solution from outside it — uploaded through the web UI, posted to the Jobs API, or dropped in the input bucket by another system. By default every one of them is fetched by OCR, rendered to page images and sent to Amazon Bedrock, and it stays in the input bucket until the retention period expires, reachable by anyone who can presign it.
Malware scanning is an optional ingest gate over that path. With it enabled, Amazon GuardDuty Malware Protection for S3 scans each upload and the pipeline admits the document only once the verdict comes back clean. A document that is infected, or that GuardDuty cannot scan, reaches a distinct terminal status with the reason recorded on the document and an alert to the stack’s SNS topic.
It is off by default. Enabling it is a deliberate choice with a cost and a quota attached, both covered below.
What it does and does not detect
Section titled “What it does and does not detect”GuardDuty runs two scan engines — one AWS builds and manages, plus a third-party vendor engine — fed by threat-intelligence indicators, YARA rules written by AWS security engineers, and heuristic and ML models. The managed engine’s signature list is updated every 15 minutes, so there is nothing to patch on your side. That currency is the main reason to prefer it over a self-managed ClamAV scanner.
⚠️ It is file-based static detection, not behavioural analysis. AWS states that the engine “doesn’t perform live behavioral analysis, where malware detonation monitors the sample as it executes”. Novel malware that no signature, YARA rule or model recognises will be reported clean. This gate raises the cost of delivering known malware through document ingest; it is not a guarantee that every admitted file is safe.
Enabling it
Section titled “Enabling it”Two CloudFormation parameters, in the Malware Scanning group.
| Parameter | Values | Meaning |
|---|---|---|
MalwareScanning | DISABLED (default), GATE_ONLY, CREATE_PLAN | Whether to gate ingest, and who owns the GuardDuty plan |
AllowUnscannableDocuments | false (default), true | Ceiling on whether a document GuardDuty could not scan may still be processed |
CREATE_PLAN creates the GuardDuty Malware Protection plan and the IAM role
GuardDuty assumes, and gates on the result. It needs GuardDuty permissions at
deploy time — a restricted CloudFormation service role may not have them — and it
uses one of the 25 protected buckets allowed per account per Region. The
delegated-deployment role shipped under
iam-roles/cloudformation-management/IDP-Cloudformation-Service-Role.yaml
carries an unconditional guardduty:* grant for this reason; a deployer who
will never use CREATE_PLAN can remove that line safely — the service-role
validator here demands one <service>:* per declared resource type, so
dropping the grant and the resource together keeps the role in line.
GATE_ONLY creates no GuardDuty resources and gates on scan results produced by
a plan somebody else manages, which is the shape to use where a central security
account owns GuardDuty. The gate reads the verdict from EventBridge, which
delivers it regardless of who created the plan, so the two modes run identical
code. Point that plan at this stack’s input bucket.
The GuardDuty role needs KMS
Section titled “The GuardDuty role needs KMS”The input bucket is encrypted with the stack’s customer-managed KMS key, so the
role GuardDuty assumes is granted kms:Decrypt and kms:GenerateDataKey on that
key, conditioned on kms:ViaService: s3.<region>.<url-suffix>. ⚠️ Removing that
statement does not degrade scanning — it stops every document. GuardDuty
answers ACCESS_DENIED / UNAUTHORIZED_TO_GET_OBJECT for an object it cannot
decrypt, which is an unscannable verdict, and the default policy refuses those.
No change to the KMS key policy is needed to go with it. The key’s
Enable IAM User Permissions statement delegates to IAM for in-account
principals, and GuardDuty acts as this role, which is in-account. Do not add a
guardduty service-principal statement to the key policy to “make this work” —
that is not the principal being used.
In GATE_ONLY mode the externally managed plan’s role needs the same grant on
this stack’s key, which is the one piece of cross-account setup that mode
requires.
How a document moves through the gate
Section titled “How a document moves through the gate”With scanning enabled, the upload event records the document and stops:
- An upload lands in the input bucket. EventBridge delivers
Object Created. - The queue-sender Lambda creates the tracking row with status
MALWARE_SCANNINGand does not send it to the processing queue. The document appears in the UI immediately, so an upload never looks lost. - GuardDuty scans the object and publishes a
GuardDuty Malware Protection Object Scan Resultevent. A rule scoped to this stack’s input bucket routes it back to the same Lambda. - The verdict decides:
| GuardDuty verdict | Document status | What happens |
|---|---|---|
scanStatus=COMPLETED, NO_THREATS_FOUND | QUEUED | Processed normally. The verdict is recorded on the document. |
THREATS_FOUND | QUARANTINED | Not processed. Metric, alert, and the threat name recorded. The object stays in the bucket, tagged. |
SKIPPED / FAILED (any reason) | MALWARE_SCAN_FAILED, or QUEUED if unscannable documents are allowed | Not processed by default. Metric and alert either way. |
Admission requires the exact pair scanStatus=COMPLETED and
scanResultStatus=NO_THREATS_FOUND. Everything else — a reason GuardDuty adds to
the API in future, a malformed event, the two fields disagreeing — falls through
to the unscannable branch. Reading “not THREATS_FOUND” as clean would turn
every future API addition into an open gate.
Gating at ingest rather than as a workflow step
Section titled “Gating at ingest rather than as a workflow step”The scan is not a Step Functions state, and that is deliberate. A state would have to wait for an asynchronous scan, holding one execution and one of the stack’s concurrency slots per uploaded file — a thousand parked executions for a thousand-document batch. Gating first also means a refused document’s bytes never reach OCR, page rendering or Bedrock at all, which is the property worth having.
The UI does not lose the step, though: the flow diagram draws Malware scan as its first step, built from the document’s own verdict — green for clean, red with the threat name for a detection, amber for a skipped scan.
What you see in the UI
Section titled “What you see in the UI”The verdict is stored on the document and surfaced in three places:
- A banner on a refused document, naming the threat or the skip reason and what to do next. For a refused document no workflow ran, so there is no execution to open — this banner is the whole explanation.
- A
Malware scanfield in the document detail grid, on clean documents as well as refused ones. “The scan ran and found nothing” is the observation an audit needs, and without it a clean document is indistinguishable from an unscanned one. - The first step of the flow diagram, as above.
A document admitted because it could not be scanned reads Admitted unscanned, never Clean.
The Reprocess action is disabled for a refused document, with the reason in its tooltip, and the API refuses it too — reprocess re-queues a document directly rather than going through this gate.
It is also disabled while a document is still being scanned, and that case is
worded as a wait rather than a refusal because it clears by itself. Reprocess does
not wait for a verdict: it builds a fresh queued document and starts the workflow
immediately, so allowing it on a held document would send the file to OCR and
Bedrock with nothing having decided it is safe — during exactly the window the
gate exists for. A second guard sits behind it: if a refusing verdict ever arrives
to find the document already past the gate, the gate applies the refusal anyway
rather than dismissing it as a repeat delivery, and raises
<StackName>-malware-unapplied-refusal.
⚠️ There is no malware column in the document list, deliberately. The list has
two backends: the date-range resolver returns the whole tracking-table item and
would carry these attributes, but the faster GSI resolver can only return
attributes in that index’s INCLUDE projection, which DynamoDB does not allow to
be amended in place. A column that populated on one list path and was empty on the
other would be worse than none, so the verdict lives on the document’s own page.
The status column already distinguishes QUARANTINED and MALWARE_SCAN_FAILED on
both paths.
Policy: what to do with an unscannable document
Section titled “Policy: what to do with an unscannable document”GuardDuty returns no verdict for a password-protected file, an object over its quotas, an SSE-C encrypted object, or an internal error. Password-protected PDFs are a legitimate input to a document-processing system, so this is a real decision rather than an edge case.
Two settings, and both must agree before such a document is processed:
malware_scanning.on_unscannablein the configuration —block(default) orallow. Editable in the Visual Editor, versioned with the rest of the configuration, and applied from the version the document was queued under.AllowUnscannableDocumentson the stack —false(default) ortrue. A ceiling: with it false, setting the configuration toallowhas no effect and the gate logs that it ignored it.
The split exists because the configuration is editable by any Admin from the web UI. A security posture that one UI edit can lower is not much of a posture, so loosening it takes both a configuration change and a stack update.
malware_scanning.alert_on_block controls whether a refusal publishes to the SNS
alerts topic. The CloudWatch metrics and the document’s own status are recorded
either way — silencing a notification is not the same as deciding nothing
happened.
A document GuardDuty reports as infected is always refused. That is not configurable, and the configuration schema offers no field for it.
Monitoring
Section titled “Monitoring”Every gate metric lands in the stack’s own CloudWatch namespace
(<StackName>) and on the Malware Scanning (ingest gate) widget of the
stack dashboard. No count is quoted — that is the figure that went stale last
time, and a test derives the set from the gate’s own _put_metric call sites
and fails if any is missing from the widget:
| Metric | Meaning |
|---|---|
MalwareScanHeld | Parked at the gate awaiting a verdict |
MalwareScanVerdictApplied | A verdict was acted on, whichever way it went |
MalwareScanClean | Admitted after a clean scan |
MalwareScanThreatsFound | Detection; document quarantined |
MalwareScanUnscannableBlocked | Refused for want of a verdict |
MalwareScanUnscannableAllowed | Admitted without a verdict, under policy |
MalwareScanVerdictBeforeIngest | Verdict arrived before the upload event was recorded (including a re-upload of an existing key); retried |
MalwareScanVerdictDuplicate | Repeat delivery of a verdict already applied |
MalwareScanVerdictSuperseded | Verdict for a version the key no longer holds; discarded |
MalwareScanJobFinalizeFailed | A refused /Jobs document could not be marked terminal in its job |
MalwareScanVerdictUnappliedRefusal | A refusing verdict reached a document that had already left the gate, so it was processed unscanned and the refusal applied late |
Alarms, all publishing to the stack’s AlertsTopic:
<StackName>-malware-threats-foundon any detection. One infected upload is an incident, so the threshold is zero. It has no OK action: “no malware in the last five minutes” is not news, and a recovery message after every detection teaches the recipient to ignore the pair.<StackName>-malware-unscannable-documentson a run of refusals. Usually password-protected files or quota overruns — but a sudden run of them can also mean the GuardDuty role lost access to the input bucket’s KMS key, in which case every upload is being refused.<StackName>-malware-verdict-dlq-depthon any verdict EventBridge could not deliver to the queue sender after every retry. A verdict that was delivered but failed in the queue sender every time goes to the queue sender’s own dead-letter queue instead (QueueSenderDLQAlarm). Unlike the two above, this is about the gate’s own plumbing rather than a document: nothing decided anything, and a document is stuck. Read the message off the dead-letter queue for the bucket, key and version. This one does announce recovery, because “the queue is empty again” is what you are waiting to hear.<StackName>-malware-no-verdictswhen documents are held at the gate and no verdict has been applied for an hour. ⚠️ This is the only signal for GuardDuty not answering at all — a plan deleted, aGATE_ONLYstack pointed at a plan that does not cover this bucket, GuardDuty disabled in the Region, or the plan’s role broken. Every other metric here is published when something is decided, so in that state ingest halts completely while all of them read zero: on a gate that fails closed, the loudest failure with the quietest signal. Detected by comparing documents held against verdicts applied rather than by watching one series, because you cannot alarm on an absence.<StackName>-malware-unapplied-refusalwhen a refusing verdict arrives for a document that has already left the gate. Threshold zero: that document was processed with nothing having decided it was safe, so its OCR output and any Bedrock calls happened before the file was known to be infected. The refusal is still applied — the document endsQUARANTINEDorMALWARE_SCAN_FAILEDand an alert is published — but the processing already happened. Reprocess refuses a document that is still being scanned, so reaching this means some other path does not; the queue-sender log names the document and the status its row was in.<StackName>-malware-job-finalize-failedwhen a refused document belonged to a/Jobsbatch and the gate could not mark it terminal, or could not hand the finished batch on to be zipped. Threshold zero. ⚠️ This is the only silent stall the gate itself can cause: both code paths deliberately swallow the error, because the refusal is already recorded and raising would re-deliver the verdict to no effect — so the batch answersIN_PROGRESSindefinitely with this metric as the one signal. Every other metric here records a decision; this one records a decision that could not be written down.<StackName>-malware-verdicts-unappliedon verdicts that reached the gate and were deliberately not applied — a superseded one (the key was re-uploaded mid-scan, which self-heals) plus one that outran its ingest event (which does not). A few superseded verdicts are normal where people replace files; a sustained rate means documents are accumulating at the gate.
Each refusal also publishes a message naming the object, the threat or skip reason, and the remedy.
Cost and quotas
Section titled “Cost and quotas”GuardDuty Malware Protection for S3 includes a 12-month free tier of 1,000 requests and 1 GB scanned per month. Beyond that, $0.09 per GB plus $0.215 per 1,000 objects — AWS’s own worked example is 4,000 files totalling 350 GB for $32.36 per month. Object tagging, the S3 API calls GuardDuty makes, and EventBridge are billed separately and sit outside the free tier.
Non-adjustable quotas: 100 GB maximum object size, 100,000 extracted files per object, 100 levels of archive nesting, and 25 protected buckets per account per Region. That last one is the constraint to plan around if you run many stacks in one account and Region.
What this gate does not cover
Section titled “What this gate does not cover”- It is asynchronous. The object exists in the input bucket, and is reachable by anyone who can presign it, between the upload and the verdict. The gate stops the file reaching the pipeline; it does not stop it arriving.
- Reprocessing a clean document does not rescan it. It was scanned on upload.
Reprocessing a refused or still-scanning document is blocked outright, and in
both reprocess paths — they share no code. The web UI and the REST API go
through the reprocess resolver;
idp-cli reprocess,documents.reprocess()andbatches.reprocess()go through the SDK’s rerun processor, which runs client-side under the caller’s own credentials and reaches SQS itself. Guarding only the first would leave the gate bypassable from the command line, and--batch-idis the worse door because it re-queues a whole batch unattended.scripts/tests/test_reprocess_paths_agree.pyreconciles the two lists, which cannot be a shared constant: the resolver importsidp_commondirectly and the SDK reaches it through a lazy path shim. - The staging bucket is not scanned.
.zipuploads expanded by the test-set extractor and the batch pre-processor go through a different bucket, which is outside this gate. Protecting it as well would cost a second of the 25 bucket slots per stack. - No GuardDuty findings without a detector. If Malware Protection for S3 is enabled as an independent feature, with no GuardDuty detector in the Region, a detection produces no GuardDuty finding — only the EventBridge event, the object tag, and this stack’s own alert and UI surfaces. Enable GuardDuty itself if you want findings in Security Hub.
- GuardDuty calls
PutBucketNotificationon the input bucket to assert that EventBridge notifications are on. This template already setsEventBridgeEnabled: trueand declares no other notification configuration, so it is a no-op in practice — but it is a managed write to a CloudFormation-owned property and belongs on your list of drift sources.
Turning it off is not retroactive
Section titled “Turning it off is not retroactive”⚠️ Setting MalwareScanning back to DISABLED does not release documents that
are already held. Every malware resource is conditional on the feature being
enabled, so disabling it removes the EventBridge rule that delivers verdicts, the
dead-letter queue and all six alarms — while any row sitting at
MALWARE_SCANNING stays exactly where it is. Because that status is
non-terminal, the detail page keeps polling such a document indefinitely, a
/Jobs batch holding one stays IN_PROGRESS, and the alarm that would have told
you is the one you just deleted.
This matters because disabling is the obvious reaction to the gate stalling, which is the moment documents are most likely to be held.
So drain first, or clean up afterwards. Either way the held documents have to be
re-submitted: delete them from the input bucket and upload them again, which
creates fresh rows on the un-gated path. Reprocess will not do it — it is
refused for a document that is still being scanned, precisely so that a held file
cannot be pushed into the pipeline without a verdict. There is no automatic sweep;
if the list is long, the document list filtered on MALWARE_SCANNING is the set
to work through.
Operating notes
Section titled “Operating notes”A phantom document named malware-protection-resource-validation-object.
GuardDuty writes that object into the protected bucket to prove it can. The gate
skips the key unconditionally, in both modes, so it never becomes a document. If
you see one in the document list, the skip has regressed.
A document stuck in MALWARE_SCANNING. The verdict never arrived or could
not be applied. Three causes, each with its own signal: the malware-verdict
dead-letter queue (EventBridge could not deliver it at all),
MalwareScanVerdictBeforeIngest (it outran the upload event and Lambda’s two
retries, about three minutes in all, were exhausted, so the verdict is in the
queue sender’s dead-letter queue) and MalwareScanVerdictSuperseded (it described a version the key no
longer holds — normally self-healing, since the newer upload’s own verdict
decides). Then the queue-sender logs, which name the cause per document. Check
the protection plan’s status too: aws guardduty list-malware-protection-plans then get-malware-protection-plan. A WARNING
carrying INSUFFICIENT_TEST_OBJECT_PERMISSIONS means the role policy is wrong.
A Jobs API batch containing a refused file still completes. The refused file
is recorded as FAILED in the job’s file map, so the job reaches
PARTIALLY_SUCCEEDED with a results.zip holding the successful files’ output —
or, if every file was refused, FAILED with no zip, since there is nothing to
put in one. Without this the batch would report IN_PROGRESS indefinitely,
because the job tracker only hears about documents that ran a workflow.
The gate records the file’s status but does not build the zip itself. It runs
in the queue-sender Lambda, which is sized and scoped for one invocation per
upload — 128 MB, a 30-second timeout, and no object read or write on the output
bucket — whereas zipping a batch’s outputs is what the job tracker’s 3 GB, 900-second
timeout and 5 GB of ephemeral storage exist for. So when a refusal is the last
file in its batch to go terminal, and no execution-status event is therefore
coming, the gate asynchronously invokes the job tracker to finish the job. If that
invoke fails it is logged and counted under MalwareScanJobFinalizeFailed rather
than retried: the file statuses are already correct, so the worst case is the
IN_PROGRESS stall that existed before, now attributable to a named metric. Any
other ordering — a clean file in the batch finishing last — completes by the job
tracker’s ordinary route with no delegation involved.
Re-uploading the same filename while a scan is in flight. Handled, and worth
knowing how. The document row is keyed on the object key alone, so both uploads
share one row — and without a discriminator v1’s clean verdict would satisfy the
gate while the bucket held an unscanned v2. The gate therefore records which
object version it is waiting on and pins its compare-and-set to it, so a verdict
for a superseded upload is discarded (metric MalwareScanVerdictSuperseded) and
the document waits for the verdict belonging to the bytes actually stored. The
same guard covers the mirror case: a THREATS_FOUND verdict for a version since
replaced by a clean file does not quarantine the clean one.
A version mismatch does not by itself say which side is out of date, because the
verdict for a new upload can arrive before that upload’s own event has updated
the document row, which then still describes the previous upload or the previous
run. So on a mismatch the gate reads the object’s current version from the input
bucket. A verdict for the version the bucket holds now is retried (metric
MalwareScanVerdictBeforeIngest) until the upload has been recorded; only a
verdict for a version the bucket no longer holds is discarded as superseded. If
the bucket cannot be read, the verdict is retried rather than discarded.
The two can also cross the other way. The gate’s check that the document is still waiting can fail against the previous run’s row, and the upload’s own event can then finish updating the row before the gate reads it back, so the row it reads is waiting for exactly this verdict. The gate then handles the verdict again from the start, against the row the upload just wrote, and applies it. If the row keeps changing, it retries the verdict rather than discarding it.
How a retried verdict is retried. EventBridge invokes the queue sender
asynchronously, so when the gate raises an error to retry a verdict, Lambda does
the retrying: two retries, about one and then about two minutes after the first
attempt (EventInvokeConfig on QueueSender in template.yaml). If all three
attempts fail, the verdict goes to the queue sender’s dead-letter queue and
QueueSenderDLQAlarm fires. The retry policy on the verdict rule is a different
mechanism. It covers only EventBridge failing to hand the verdict to Lambda,
because Lambda accepts an event before the function runs. Those failures go to
the malware-verdict dead-letter queue. In practice the upload is recorded within
a second of its verdict, so the three-minute window is ample.
Load testing. scripts/malware_gate_burst.py pushes a few hundred documents
through the gate at once and asserts the properties concurrency breaks — every
upload gets a row, nothing is left in MALWARE_SCANNING, no benign document is
quarantined, and admission is not serialised. Deliberately not part of the
nightly suite: admitting a few hundred documents starts a few hundred Step
Functions executions and the Bedrock calls behind them. Run it for a release
check or after changing the gate:
AWS_PROFILE=default python3 scripts/malware_gate_burst.py \ --stack-name my-idp-stack --count 300 --with-eicarTesting it. Upload a file containing the EICAR test string — a harmless,
industry-standard probe that scanners recognise, and the one GuardDuty’s own
documentation uses in its THREATS_FOUND example. Expect QUARANTINED, no Step
Functions execution, the red banner naming EICAR-Test-File, and the object still
present and tagged GuardDutyMalwareScanStatus=THREATS_FOUND.