Storage Classes and When They Make Sense
On the practical side, S3 has six primary storage classes, and choosing the wrong one is the single easiest way to overspend on AWS. I've audited accounts where 80% of objects were in S3 Standard but hadn't been accessed in over a year. That's straight-up burning money.
Here's the real decision framework, stripped of AWS marketing:
- The s3 Standard: data you access frequently. No minimum storage duration, no retrieval fees. Your default for active data.
- The s3 Standard-IA: data accessed less than once a month. Cheaper storage ($0.0125/GB vs $0.023/GB) but $0.01/GB retrieval fee. The 128KB minimum charge per object means small files cost the same as 128KB files.
- S3 One Zone-IA: same as Standard-IA but stored in one AZ. 20% cheaper. Fine for data you can regenerate — processed outputs, secondary copies, non-critical logs.
- The s3 Glacier Instant Retrieval: archive data you might need in milliseconds. Cheapest option for data accessed roughly once per quarter.
- The s3 Glacier Flexible Retrieval: data you can wait minutes to hours for. Expedited retrieval (1-5 minutes) costs extra; standard (3-5 hours) is cheap.
- S3 Glacier Deep Archive: cheapest storage at $0.00099/GB. 12-48 hour retrieval. Compliance archives, regulatory data you hope you'll never need.
Lifecycle Policies That Actually Save Money
The right lifecycle policy depends on your access pattern data, not guesswork. Before configuring anything, enable S3 Storage Lens or check S3 Analytics for your bucket. Let it run for 30 days. You need actual access frequency data.
# Terraform lifecycle configuration
resource "aws_s3_bucket_lifecycle_configuration" "main" {
bucket = aws_s3_bucket.data.id
rule {
id = "log-transitions"
status = "Enabled"
filter {
prefix = "logs/"
}
transition {
days = 30
storage_class = "STANDARD_IA"
}
transition {
days = 90
storage_class = "GLACIER_IR"
}
transition {
days = 365
storage_class = "DEEP_ARCHIVE"
}
expiration {
days = 2555 # 7 years for compliance
}
}
rule {
id = "abort-incomplete-multipart"
status = "Enabled"
abort_incomplete_multipart_upload {
days_after_initiation = 7
}
}
}That last rule — aborting incomplete multipart uploads — is one people forget. Incomplete multipart uploads accumulate silently and you pay Standard rates for those fragments. I've seen buckets with tens of GB of orphaned multipart parts.
The Small Object Trap
Looking closer, S3-IA classes charge a minimum of 128KB per object. If you have millions of 1KB metadata files, transitioning them to IA actually increases costs because each tiny file gets billed as 128KB. Filter these out of your lifecycle rules or aggregate them into larger objects before archiving.
Similarly, IA classes have a 30-day minimum storage duration charge. If you transition objects that get deleted within 30 days, you pay for the full 30 days anyway. Set your transition rule's day count high enough that the vast majority of objects have settled into their long-term access pattern.
Cost Modeling Spreadsheet Approach
I build a simple model with four columns: storage class, monthly storage cost per GB, retrieval cost per GB, and minimum duration. Then I plug in my actual data volume and access patterns.
For example: 10TB of log data, accessed twice in the first week, then maybe once a quarter. The math works out like this — S3 Standard for the first 30 days costs $230/month. Transitioning to Glacier Instant Retrieval after 30 days drops that to $40/month with $0.03/GB when you do need to retrieve something. Over a year, that's roughly $2,300 in Standard vs $510 in Glacier IR with occasional retrievals.
Intelligent Tiering: The Autopilot Option
In production, S3 Intelligent-Tiering automatically moves objects between access tiers based on usage. It's a monitoring fee of $0.0025 per 1,000 objects, and there are no retrieval fees when objects move between tiers.
It's genuinely useful when you can't predict access patterns. The downside is the per-object monitoring fee — at scale with billions of tiny objects, it adds up. For predictable access patterns, explicit lifecycle rules are cheaper. For unpredictable workloads or mixed-use buckets, Intelligent-Tiering saves you from guessing wrong.
One thing I've learned: don't enable Intelligent-Tiering and lifecycle rules on the same prefix. They'll conflict. Pick one strategy per prefix and stick with it.
S3 Analytics and Storage Lens
From our experience, S3 Storage Class Analysis watches your access patterns at the prefix level and recommends transitions after 30+ days of observation. Enable it on your largest buckets where a storage class change would actually save meaningful money.
S3 Storage Lens gives you an organization-wide view across all accounts and buckets: total storage, object counts, request patterns, and what percentage of your storage is in each class.
resource "aws_s3_bucket_analytics_configuration" "logs" {
bucket = aws_s3_bucket.main.id
name = "logs-analysis"
filter { prefix = "logs/" }
storage_class_analysis {
data_export {
destination {
s3_bucket_destination {
bucket_arn = aws_s3_bucket.analytics.arn
prefix = "s3-analytics/"
}
}
}
}
}Versioning and Cost Control
Versioning protects against accidental deletion but can silently balloon storage costs. Every overwrite creates a new version, and old versions remain indefinitely unless you configure lifecycle rules for noncurrent versions.
I audited a client's bucket that had versioning enabled for three years without noncurrent version cleanup. The current objects were 2TB. Noncurrent versions were 14TB. At Standard pricing, that's $322/month in storage for data nobody intended to keep.
Request-Level Cost Optimization
Storage cost gets all the attention, but request costs add up for high-throughput buckets. S3 Standard charges $0.005 per 1,000 GET requests. For a bucket serving 100 million GET requests per month, that's $500 just in request fees.
CloudFront in front of S3 reduces request costs because CloudFront-to-S3 requests are cheaper, and the cache hit ratio means fewer requests reach S3 at all. A 90% cache hit ratio drops the request cost to $50.
Cross-Region Replication Cost Traps
S3 Cross-Region Replication (CRR) copies objects to a bucket in another region for disaster recovery or compliance. The replication itself is free for the service, but you pay for: data transfer between regions ($0.02/GB), PUT requests in the destination bucket, and storage in the destination bucket. For a 10 TB dataset, the initial replication costs about $200 in transfer alone, plus ongoing costs for new and updated objects.
Same-Region Replication (SRR) avoids the data transfer cost but still charges for PUT requests and storage in the destination bucket. SRR is useful for copying data between accounts within the same region, or for maintaining a backup in a different storage class.
A cost trap I've seen repeatedly: teams enable replication for an entire bucket when they only need it for critical data. A bucket with 50 TB of logs and 500 GB of important data replicates all 50 TB. Use replication rules with prefix or tag filters to replicate only what matters.
resource "aws_s3_bucket_replication_configuration" "selective" {
role = aws_iam_role.replication.arn
bucket = aws_s3_bucket.primary.id
rule {
id = "critical-data-only"
status = "Enabled"
filter {
tag { key = "Replicate" value = "true" }
}
destination {
bucket = aws_s3_bucket.replica.arn
storage_class = "STANDARD_IA"
}
}
}Notice the destination storage class is set to STANDARD_IA instead of STANDARD. Since replicated data is typically for DR and accessed rarely, there's no reason to store it in the most expensive tier. This one setting can cut your replication storage costs by 45%.
Multipart Upload Optimization
For objects larger than 100 MB, use multipart upload. It breaks the upload into parts that transfer in parallel, dramatically improving upload speed on high-bandwidth connections. The AWS SDK handles this automatically when you use the high-level transfer APIs, but the default part size and concurrency settings aren't always optimal.
For most connections, 8 MB part sizes with 10 concurrent uploads works well. On very fast connections (10 Gbps+), increase parts to 64 MB and concurrency to 25. The goal is saturating your available bandwidth without overwhelming the client's memory.