S3 Buckets — Evidence Collection and Log Analysis

S3 (Simple Storage Service) is the backbone of AWS storage. It's where CloudTrail logs land, where flow logs are stored, where application data lives, and where attackers look for sensitive information. Understanding S3 from an IR perspective means understanding both how attacks happen against S3, and how S3 itself becomes your central evidence repository.

S3 Access Controls — The Basics

S3 access is controlled at multiple layers. Every S3 bucket is private by default — no public access unless explicitly granted.

Access can be granted through:

• IAM policies (attaches to users/roles, grants access to specific buckets),

• Bucket policies (attaches to the bucket itself, can grant access to specific IAM accounts or make the bucket public)

• ACLs (older mechanism, avoid these for anything new).

  
  

The infamous 'public S3 bucket' misconfiguration happens when an administrator enables public access either through a bucket policy with Principal: '*' (wildcard), or by disabling the Block Public Access setting — which is the AWS-level override that prevents any accidental public exposure.

💡 IR Tip: In any investigation involving data exposure, check the S3 Block Public Access settings first — it's a single boolean setting at the account level and the bucket level. If either is off, public exposure is possible. AWS Console → S3 → Block Public Access settings for this account.

The Two Different S3 Log Types

• CloudTrail data events = API access (who called GetObject via SDK/CLI)

• S3 server access = HTTP/web access (presigned URLs, browser downloads)

⚠️ During exfiltration investigations — check BOTH log types. API exfiltration → CloudTrail. Presigned URL/web exfiltration → S3 server access logs.

This is one of the most commonly misunderstood things in AWS forensics. There are two completely separate logging mechanisms for S3 access, and they capture different things:

1. CloudTrail Data Events for S3

CloudTrail data events record API-level access to S3. This means any access through: AWS CLI (aws s3 cp), AWS SDK (boto3, AWS SDK for Java), Cross-account role access, Lambda functions accessing S3. These logs are in CloudTrail format — you get the full userIdentity, sourceIPAddress, and event details.

2. S3 Server Access Logs

S3 server access logs record HTTP-level access to S3. This means any access through: a web browser directly (using presigned URLs), HTTP GET requests to public bucket objects, any access that goes through the S3 HTTPS endpoint without AWS credentials. These logs look more like web server logs — you get the source IP, HTTP method, object key, response code, and bytes transferred.

💡 IR Tip: If you're investigating S3 data exfiltration and only looking at CloudTrail — you may miss exfiltration that happened via presigned URLs or public bucket access. Check both log types. They're stored separately and enabled separately.

---------------------------------------------------------------------------------------------------------

Transfer Acceleration

S3 Transfer Acceleration is an optional feature that routes S3 uploads and downloads through AWS CloudFront edge locations worldwide, rather than directly to the S3 bucket's region. This dramatically speeds up transfers for users far from the bucket's region.

From an IR perspective: if Transfer Acceleration is enabled, the source IPs in your S3 logs will be CloudFront edge node IPs — not the original requester's IP. The X-Forwarded-For header in server access logs will show the real source IP.

Using Athena to Search S3 Logs at Scale

▸ S3 Bucket → Glue Crawler → Glue Data Catalog → Athena → SQL Query

When you have weeks or months of CloudTrail logs in S3, you need a way to search them efficiently. AWS Athena is a serverless SQL query engine that reads directly from S3 — you pay per query, not per hour of infrastructure.

The pipeline:

(1) CloudTrail logs land in S3,

(2) AWS Glue Crawler scans the S3 bucket and creates a table schema in the Glue Data Catalog, (3) Athena uses that schema to run SQL queries against the raw S3 files. You don't need a database, a server, or ETL pipelines.

Example Athena query to find all ConsoleLogin events from a specific IP:

SELECT eventtime, useridentity.arn, sourceipaddress, responseelements FROM cloudtrail_logs WHERE eventname = 'ConsoleLogin' AND sourceipaddress = '91.200.14.77' ORDER BY eventtime DESC LIMIT 100;

Athena pricing: $5 per TB of data scanned. Partition your CloudTrail bucket by year/month/day to drastically reduce scan costs — a query scoped to one week might only scan a few GB instead of a year's worth.

⚡ Update (2024): CloudTrail Lake now provides a managed alternative to the S3+Glue+Athena pipeline. It includes a built-in query interface with pre-configured schemas. For new deployments, this is often simpler. For existing environments with S3-based trails, Athena remains the standard approach.

Setting Up Your Evidence S3 Bucket

Your forensic evidence bucket needs specific protections:

(1) Enable versioning — so evidence files can't be overwritten without keeping the history.

(2) Enable MFA Delete — requires MFA to permanently delete any object.

(3) Enable object lock (WORM) — Write Once Read Many, prevents any modification or deletion for a defined retention period.

(4) Enable server-side encryption — SSE-KMS with your own KMS key.

(5) Enable access logging — so you can see who has accessed your evidence bucket.

-----------------------------------------------------------------------------------------------------------

What's Next

Next article covers the automated detection stack — CloudTrail Insights, GuardDuty, and AWS Detective. These are the tools that sit on top of everything we've discussed and surface attacks automatically, so you're not manually querying CloudTrail for every threat.

--------------------------------------------Dean-----------------------------------------------------