S3 Storage Classes (Storage Tier) – cost, durability, and availability – can be changed manually or using lifecycle policies.
Link: https://aws.amazon.com/s3/storage-classes/
- Object Durability: The percentage over a one-year period that an object in S3 will not be lost.
- Object Availability: The percentage over a one-year period that an object in S3 will be accessible.
S3 Tiered Storage Classes
Storage classes can be modified on the fly.
- Lifecycle rules are added at a bucket level.
- Lifecycle automatically moves objects between the different storage tiers.
- You can add a rule to a bucket such as Standard -> Glacier -> Delete.
- Automatic transition of object between storage classes.
- Lifecycle rules can apply to buckets, prefixes, and tags. Also, they work with current or previous versions of an object.
- Object must stay at least 30 days in S3 Standard before moving to IA or one zone IA. But you can transfer objects directly to Glacier at any time.
- Standard: default
- 11 nines durability, 4 nines availability
- Replicated in +AZs (redundant), no retrieval fee but most expensive base fee
- Standard Infrequent Access (Standard-IA)
- Fast real-time access but less frequently, 3+ AZs replication
- 30-day and 128 KB minimum charge, object retrieval fee
- One Zone IA
- non-critical or reproducible data
- reduced availability (99.5%), 1 AZ, 30-day and 128 KB minimum charge
- Glacier
- long-term secure, durable archival, 3+ AZs
- retrieval types
- Expedited retrieval: 1-5 mins
- Standard retrieval: 3-5 hours
- Bulk retrieval: 5-12 hours with a large amount of data
- 90-day and 40KB minimum charge, object retrieval fee
- Glacier Deep Archive
- long-term cold backups, lowest cost
- Retrieval time: 12 hours
- 180-day and 40KB minimum charge, object retrieval fee
- Intelligent tiering: (for objects bigger than 128 kB)
- for unknown or unpredictable access patterns
- A minimum 30 days before transitioning to IA storage tiers.
- After 30 days, objects that are not accessed are moved to the IA tier.
S3 Versioning
When the versioning is enabled, any modification of an object generated a new version of the original object.
- Once enabled, S3 stores all versions of an object with previous permissions. It is a great backup tool.
- Version is per bucket and is disabled by default.
- Once the version is enabled on a bucket, it cannot be disabled – only suspended.
- AWS charges all versions of all objects.
- Older versions can be accessed using the object and a version ID.
- Specific versions can be deleted. (It is checked as “delete marker.”)
- Versioning integrates with Lifecycle rules.
- Versioning can use with MFA (Multi-Factor Authentication) Delete, which can provide additional security for accidental deletion or overwrite.
S3 Performance
- Limits: 3,500 PUT/POST/DELETE operations per second, 5,500 GET requests per second per prefix.
- Introduce key randomness (random prefix) to spread load across multiple partitions.
- Prefix is what is between the bucket name and the object name in URL (folders and subfolders).
- You can get better performance by spreading the reads across different prefixes.
When you use SSE-KMS for encryption, KMS has some limits (KMS has a region-specific quota).
- Use multipart uploads (recommended over 100MB and required over 5GB) to increase upload performance.
- Use S3 Byte-range fetches: It parallelizes downloads by specifying byte ranges or download the partial amount of file.
