Amazon Simple Queue Service (Amazon SQS) is a fully managed, highly available message queue service to create distributed/decoupled architecture.
Amazon Simple Queue Service (SQS)
A queue is a temporary repository for messages that are waiting for processing. Using SQS, you can send, store, and receive messages between software components at any volume, without losing messages or requiring other services to be available.
Messages are added to a queue and retrieved via polling. (Pull-based)
- Default – WaitTimeSeconds setting is 0
- Available messages are returned right away, might return 0 messages, cause a lot of API calls.
- Wait for messages for a given interval (WaitTimeSeconds setting – 1 ~ 20 seconds)
- Reduces API requests and will have fewer empty API responses
- When a message is polled, it is hidden and can be deleted when the processing is completed. Otherwise, after a VisibilityTimeout period, the message will return to the queue (auto return).
- Queues can be configured with a maxReceiveCount.
- Dead-Letter Queues: used to deal with a malformed message from consumers
- To delete a message, you need the queue URL and the receipt handle.
Access and Security
- SQS can encrypt message data using KMS.
- To grant access to SQS queues to another AWS account, SQS resource-based policies can be used.
- Lambda functions can be a good choice to handle requests based on messages – Scaling and fast response.
- SQS message can contain up to 256 KB of text in any format; it usually has a link to big data in S3.
- Can be retained from 1 minute to 14 days in a queue (4 days default). Once the message retention limit is reached, the messages are automatically deleted.
- The maximum VisibilityTimeout is 12 hours (30 seconds default).
Types of SQS Queues
SQS offers two types of message queues.
- Guaranteed delivery of each message at least once with best-effort ordering (ordering is not guaranteed, and there might be duplicate messages) and at-least-once delivery.
- It supports a nearly unlimited throughput.
- It supports multiple producers and multiple consumers.
- Guarantee that messages are sent and processed exactly once, in the exact order that they are sent.
- It supports multiple producers but only supports multiple consumers through group IDs.
- Messages are processed in order with respect to the group.
- Limited throughput, 300 messages/sec.