[AWS] Amazon SQS

Amazon Simple Queue Service (Amazon SQS) is a fully managed, highly available message queue service to create distributed/decoupled architecture.

Amazon Simple Queue Service (SQS)

A queue is a temporary repository for messages that are waiting for processing. Using SQS, you can send, store, and receive messages between software components at any volume, without losing messages or requiring other services to be available.

Messages are added to a queue and retrieved via polling. (Pull-based)

Short Polling

  • Default – WaitTimeSeconds setting is 0
  • Available messages are returned right away, might return 0 messages, cause a lot of API calls.

Long Polling

  • Wait for messages for a given interval (WaitTimeSeconds setting – 1 ~ 20 seconds)
  • Reduces API requests and will have fewer empty API responses

Configuration

  • When a message is polled, it is hidden and can be deleted when the processing is completed. Otherwise, after a VisibilityTimeout period, the message will return to the queue (auto return).
  • Queues can be configured with a maxReceiveCount.

Bad messages

  • Dead-Letter Queues: used to deal with a malformed message from consumers
  • To delete a message, you need the queue URL and the receipt handle.

Access and Security

  • SQS can encrypt message data using KMS.
  • To grant access to SQS queues to another AWS account, SQS resource-based policies can be used.
  • Lambda functions can be a good choice to handle requests based on messages – Scaling and fast response.

SQS Messages

  • SQS message can contain up to 256 KB of text in any format; it usually has a link to big data in S3.
  • Can be retained from 1 minute to 14 days in a queue (4 days default). Once the message retention limit is reached, the messages are automatically deleted.
  • The maximum VisibilityTimeout is 12 hours (30 seconds default).

Types of SQS Queues

SQS offers two types of message queues.

Standard Queues

  • Guaranteed delivery of each message at least once with best-effort ordering (ordering is not guaranteed, and there might be duplicate messages) and at-least-once delivery.
  • It supports a nearly unlimited throughput.
  • It supports multiple producers and multiple consumers.

FIFO queues

  • Guarantee that messages are sent and processed exactly once, in the exact order that they are sent.
  • It supports multiple producers but only supports multiple consumers through group IDs.
  • Messages are processed in order with respect to the group.
  • Limited throughput, 300 messages/sec.

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s