No File Left Behind

Does your organization manage extra large files via workflows using Amazon S3 that should be scanned for viruses or malware to meet compliance requirements and eliminate downstream risk but that can't be? 

Perhaps as a media company you want to ensure any video files received from third parties are infection free before storing or working with them, but didn't have a way to scan the files because their size. Or as a life sciences organization, you want to ensure all data sets you're ingesting and making available for use are free of ransomware, but couldn't because of the volume of data contained in a singe file.     

No more!  To meet customer demand, Antivirus for Amazon S3 now supports scanning files up to 5 TB in size. This is perfect for health care organizations, defense departments and any other industry that has really BIG files. Never skip scanning a file again due to size with our Extra Large File Scanning functionality.

The Technical Specifics

Most Antivirus for Amazon S3 customers scan their files under the Fargate disk cap of 200 GB. But those in industries that have files larger than 200 GB and require malware scanning need an alternative option that can manage more than that. To meet that need, Antivirus for Amazon S3 now supports scanning files up to 5 TB in size (Amazon S3's max file size). This is done by bypassing the internal disk limitations of Fargate leveraging an EC2 instance(s) for such files.

Extra large file scanning can be triggered by one of the scanning models we offer: event based scanning, retro scanning and even API based scanning (scan existing API only). When any of these scanning agents picks up a file that is too large to scan (too large is based on the disk size users assign under the Agent Settings or API Agent Settings within the console) and the Extra Large File Scanning toggle is on (snapshot from the console below), a Job is defined to be kicked off.

The job will be picked up within 10 minutes and kicked off. A temporary EC2 instance will be spun up with an EBS volume of the size defined in the Disk Size field. The EC2 will pick up the file and scan it. Because it is a "job", it is monitored under the Jobs page within Antivirus for Amazon S3. On the Jobs page you can monitor the job going through "Not Started" while waiting for the EC2 to start up, "Scanning", and "Completed". Each "large file" is treated as its own job. If you have 50 large files come in then 50 jobs will be kicked off for the duration it takes to scan each individual file.

Sample Scenarios and Scanning Outcomes

(Note: subtract 5 GB for overhead from disk sizes for agents and Extra Large File Scan size)