[AWS] Elastic Block Store (EBS)

Elastic Block Store (EBS) is a network-attached storage service that creates and manages persistent volumes. It is designed for mission-critical workloads with HA (Highly Available) and scalability.

EBS Features

  • Volumes are persistent (removed and attached) and are replicated within a single AZ.
  • Scalability
    • You can dynamically increase capacity or change the volume type without no downtime or performance impact.
  • Highly Available
    • By default, EBS volumes are replicated within the AZ.
  • Termination Protection is turned off by default. You must turn it on.
  • On an EBS-backed instance, the root volume will be deleted by default when the instance is terminated.

EBS Volume Types

  • Cold HDD (sc1)
    • lowest cost, infrequent access such as file servers
    • can not be a boot volume
    • Performance
      • 500 GiB ~ 16 TiB,
      • Baseline Throughput: 12MiB/s per TiB, able to burst up to 80MiB/s per TiB
      • Maximum Throughput: ~ 250 MiB/s per volume
  • Throughput Optimized HDD (st1)
    • low cost, throughput intensive(streaming)
      • Frequently accessed workloads: big data, data warehouse, log processing, and ETL (extract, transform, and load).
    • can not be a boot volume
    • Performance
      • 500 GiB ~ 16 TiB,
      • Baseline Throughput: 40MiB/s per TiB
      • Maximum Throughput: ~ 500 MiB/s per volume

  • General Purpose SSD (gp2)
    • default for most workload, balanced price and performance
      • Good for boot volumes or development/test applications that are not latency sensitive.
    • Performance
      • 1GiB ~ 16 TiB,
      • 3 IOPS/GiB (100 ~ 16,000 IOPS),
      • Bursts up to 3,000 IOPS for volumes less than 1 TB
      • ~ 250MiB/s
  • General Purpose SSD (gp3)
    • A new update from gp2
      • 20% cheaper than gp2
    • Performance
      • Baseline of 3,000 IOPS for any volume size (1GB ~ 16 TB)
      • Up to 16,000 IOPS

  • Provisioned IOPS SSD (io1)
    • High performance but expensive
      • Used if you need more than 16,000 IOPS
      • Designed for I/O intensive applications such as databases, and latency-sensitive workloads
    • Performance
      • 4GiB ~ 16 TiB
      • 50 IOPS per GiB, Up to 64,000 IOPS
      • up to 99.9% of durability
      • ~ 1,000 MiB/s
  • Provisioned IOPS SSD (io2)
    • An upgrade from io1
      • Higher durability and more IOPS per GiB than io1 with the same price
    • Performance
      • 500 IOPS per GiB, Up to 64,000 IOPS
      • 99.999% of durability
  • io2 Block Express
    • It provides SAN (Storage Area Network) in the cloud for higher performance and lower latency (sub-millisecond).
    • It uses EBS Block Express architecture.
      • It can be used for large and critical high-performance applications such as big enterprise databases (Oracle or SQL Server).
    • Performance
      • Up to 64 TiB
      • Up to 256,000 IOP
      • S99.999% durability

IOPS vs. Throughput

Input/output operations per second (IOPS, pronounced eye-ops) is an input/output performance measurement used to characterize computer storage devices.

https://en.wikipedia.org/wiki/IOPS
IOPSThroughput
Ability to read and write data quicklyAbility to read/write large datasets
Measures the number of read/write operations per secondMeasures the number of bits read or written per second (MB/s)
Metric for low latency apps or quick transactional workloadsMetric for large I/O size
Choose Provisioned IOPS SSD (io1 or io2)Might choose throughput optimized HDD (st1)

EBS Snapshots

A snapshot is a point-in-time backup of an ESB Volume stored in S3.

  • To protect against AZ failure, EBS snapshots (to S3) can be used.
  • Snapshots are incremental. The initial snapshot is a full copy, and future ones only store the data changed since the last snapshot, which can reduce the storage cost.
  • You can take a snapshot while the instance is running except the root volume.
  • You can create an AMI from a snapshot.
  • EBS volumes are in the same AZ of the EC2 instance.
  • Use case 1: Move volumes between AZs.
    1. Take a snapshot
    2. Create an AMI from it
    3. Launch an instance using the AMI.
  • Use case 2: Move volumes between regions.
    1. Take a snapshot
    2. Create an AMI from it
    3. Copy the AMI to another region
    4. Launch a new instance using the AMI.
  • Snapshots can be copied between regions, shared, and automated using Data Lifecycle Manager (DLM).
  • Snapshots copied from an encrypted EBS volume will also be encrypted. Since CMK (Customer Master Key) is regional, a new CMK from the destination region will be needed for cross-region copies.
  • You can share snapshots with other accounts.
  • Fast Snapshot Restore (FSR) allows you to promptly restore fully provisioned EBS volumes from snapshots, regardless of the size of the volume or snapshot.

EBS Volume Encryption

Volume encryption uses EC2 host hardware to encrypt data at rest and in-transit between EBS and EC2 instances.

  • KMS (Key Management Service) generates a Data Encryption Key (DEK) from a Customer Master Key (CMK) in each region. A unique DEK encrypts each volume. Snapshots of that volume are encrypted with the same DEK.
  • Encrypted DEKs (stored in a volume) are decrypted by KMS using a CMK and given to the EC2 host.
  • A decrypted (text) DEK is stored in EC2 memory while it is active and used to encrypt/decrypt data. The plaintext DEK is discarded when an instance is rebooted. KMS must resend the plaintext DEK.
  • Case 1: Encrypting the EBS volume
    • Select an “Encrypt” option while creating an instance.
    • When you enable the encryption for EBS, you can override the default key and select symmetric customer-managed CMK. EBS does not support asymmetric CMKs.
    • You can encrypt data before saving them on ESB by calling KMS API.
  • Case 2: Encrypting an unencrypted EBS volume
    • You cannot directly enable encryption in an existing unencrypted volume.
    • Use the Operating System level encryption such as bit locker.
    • Take a snapshot -> create a copy with encryption
    • If you need to use the volume as a root volume, make an AMI with an encrypted volume and deploy the AMI.

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s