[AWS] S3 Storage Classes and Versioning

S3 Storage Classes (Storage Tier) – cost, durability, and availability – can be changed manually or using lifecycle policies.

Link: https://aws.amazon.com/s3/storage-classes/

  • Object Durability: The percentage over a one-year period that an object in S3 will not be lost.
  • Object Availability: The percentage over a one-year period that an object in S3 will be accessible.

S3 Tiered Storage Classes

Object Storage Classes – Amazon S3

  • Standard: default
    • 11 nines durability, 4 nines availability
    • Replicated in 3+AZs (redundant)
    • No retrieval fee but most expensive base fee
    • For frequently accessed data
  • Standard Infrequent Access (Standard-IA)
    • Fast real-time access but less frequently (such as once a month)
    • 3+ AZs replication
    • 30-day and 128 KB minimum charge, object retrieval fee
  • One Zone IA
    • non-critical or reproducible data
    • reduced availability (99.5%), 1 AZ
    • 30-day and 128 KB minimum charge, object retrieval fee
  • Glacier
    • long-term secure, durable archival
    • 3+ AZs
    • retrieval types
      • Instant retrieval
        • It is ideal for data that is accessed once or twice per quarter, and that requires immediate access.
        • 90-day and 128 KB minimum charge, object retrieval fee
      • Flexible retrieval
        • minutes ~ 12 hours
        • Expedited: 1 ~ 5 minutes
        • Standard: 3 ~ 5 hours
        • 90-day, object retrieval fee
  • Glacier Deep Archive
    • long-term cold backups, lowest cost
    • 3+AZs
    • Retrieval time: ~12 hours
    • Bulk Retrieval time: ~ 48 hours
    • 180-day, object retrieval fee
  • Intelligent tiering: (for objects bigger than 128 kB)
    • for unknown or unpredictable access patterns
    • A minimum 30 days before transitioning to IA storage tiers.
    • After 30 days, objects that are not accessed are moved to the IA tier.

Lifecycle Management

Storage classes can be modified on the fly.

  • Lifecycle rules are added at a bucket level.
    • Lifecycle automatically moves objects between the different storage tiers.
    • You can add a rule to a bucket such as Standard -> Glacier -> Delete.
    • Automatic transition of object between storage classes.
    • Lifecycle rules can apply to buckets, prefixes, and tags. Also, they work with current or previous versions of an object.
    • Object must stay at least 30 days in S3 Standard before moving to IA or one zone IA. But you can transfer objects directly to Glacier at any time.

S3 Versioning

When the versioning is enabled, any modification of an object generated a new version of the original object.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/versioning-workflows.html

  • Once enabled, S3 stores all versions of an object with previous permissions. It is a great backup tool.
  • Version is per bucket and is disabled by default.
  • Once the version is enabled on a bucket, it cannot be disabled – only suspended.
  • AWS charges all versions of all objects.
  • Older versions can be accessed using the object and a version ID.
  • Specific versions can be deleted. (It is checked as “delete marker.”)
  • Versioning integrates with Lifecycle rules.
  • Versioning can use with MFA (Multi-Factor Authentication) Delete, which can provide additional security for accidental deletion or overwrite.

S3 Performance

  • Limits: 3,500 PUT/POST/DELETE operations per second, 5,500 GET requests per second per prefix.
  • Introduce key randomness (random prefix) to spread load across multiple partitions.
    • Prefix is what is between the bucket name and the object name in URL (folders and subfolders).
    • You can get better performance by spreading the reads across different prefixes.

When you use SSE-KMS for encryption, KMS has some limits (KMS has a region-specific quota).

  • Use multipart uploads (recommended over 100MB and required over 5GB) to increase upload performance.
  • Use S3 Byte-range fetches: It parallelizes downloads by specifying byte ranges or download the partial amount of file.

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s