[AWS] S3 Architecture

AWS S3 (Simple Storage Service) is a global object storage platform that can be used to store and retrieve any amount of data.

---

What is S3?

• S3 is an object storage.
• An object is saved as a key-value pair.
  • Key: the name of an object
  • Value: data, sequence of bytes
  • Version ID: used for versioning
  • Metadata: properties about the data
• Unlike a File Storage, you can not open a file and modify it.
  • You need to overwrite a file (object).
  • With versioning, you can keep the old object.

---

S3 Buckets

• Bucket names must be globally unique.
  • S3 is a universal namespace.
  • 3 ~ 63 characters in length
  • Naming rules
    • lowercase letters, numbers, dot and hyphen only
    • no uppercase or underscore
    • must start and end with a lowercase or number
    • can’t be formatted as an IP address

• Default 100 buckets per account, hard 1,000 bucket-limit via support request
  • Unlimited objects and unlimited total capacity in buckets
  • Each object’s size is 0 ~ 5TB

• Even though S3 is a global service, you need to select a specific region for a bucket.
  • Objects are physically in the specified region but can be accessed globally.

• Buckets are private by default.
  • You can make individual objects public by using Object ACLs.
  • You can make an entire bucket public using bucket policies.

---

Uploading Objects to S3

• Uploading an object: can be done using S3 Console, CLI, or SDKs

• Single PUT upload: ~ 5GB,
  • might cause the performance issue when the file is big
• Multipart upload: Each part – 5MB ~ 5GB, up to 10,000 parts.
  • Faster (parallel upload)
  • an individual upload can fail and can be retried.
  • Recommended over 100MB, required for bigger than 5GB.
• HTTP 200 response if the upload was successful

Accessing the Object

• Object URL
  • https://{bucket-name}.s3.amazonaws.com/{key}
  • ex) https://mybucket.s3.amazonaws.com/mypic.png
• S3 URI
  • s3://{bucket-name}/{key}
  • ex) s3://mybucket/mypic.png
• ARN (Amazon Resources Name)
  • arn:aws:s3:::{bucket-name}/{key}
  • ex) arn:aws:s3:::mybucket/mypic.png

---

Data Consistency

• Read after Write Consistency
  • for PUTs of new objects
  • You can read the file right after creating one
• Eventual Consistency
  • for overwrite PUTs and DELETEs
  • If you update or delete an existing file, it takes some time to propagate. You might access the old version.