Amazon Simple Queue Service (SQS) is a fully managed message queuing service. You can send, store, and receive messages between distributed software components at any volume, reliably. SQS helps to decouple, coordinate and scale microservices, distributed systems, and serverless applications in cloud. SQS was the first service on the AWS.
SQS solves the producer consumer problem efficiently. SQS standard queues offer maximum throughput, best-effort ordering, and at-least-once delivery. SQS FIFO queues guarantee that messages are processed exactly once, in the exact order, with limited throughput. Can access SQS from Management Console, SDK or SQS webservice APIs.
Processing Messages in Queue
You can call the ReceiveMessage action to receive the message.
After a consumer receives a message from a queue, the message remains in the queue until the consumer delete the message after processing it. A message is considered as in-flight after it's received by consumer, but not yet deleted from queue. For standard queues, there can be a maximum of 120K inflight messages per queue. For FIFO queues, max is 20K. If you reach limit, an OverLimit error message occurs for standard queues but not for FIFO queues.
To delete only specific messages, can use the DeleteMessage or DeleteMessageBatch actions. Can delete all messages in an SQS queue using the PurgeQueue action.
To prevent other consumers from processing the message again, SQS sets a visibility timeout, during which SQS prevents other components from receiving the message. For standard queues, the visibility timeout isn't a guarantee against receiving a message twice. FIFO queues allow the producer or consumer to attempt multiple retries using the same message deduplication ID or request attempt ID respectively.
You can change the timeout setting for the entire queue, or per message (using the ChangeMessageVisibility action) when receiving messages. You can terminate the visibility timeout for a specific message by calling ChangeMessageVisibility with VisibilityTimeout set to 0 seconds. This make the message immediately visible to other components.
Message Retention Period
SQS automatically deletes messages that have been in a queue for more than maximum message retention period. The default message retention period is 4 days. You can set the message retention period to a value from 60 seconds to 1,209,600 seconds (14 days) using the SetQueueAttributes action.
Short polling vs. long polling
Consumers poll the queue and pull messages from the queue. With regular polling (short polling), consumer requests return immediately with or without message (if no message is available) and is charged every time.
With long polling, consumer requests wait until a message is available or the long polling request times out. Maximum timeout is 20 seconds.To enable long polling, set ReceiveMessageWaitTimeSeconds is set to a value greater than zero. All AWS SDKs work with 20-second long polls by default.
SQS - Limits & Defaults
Messages can contain 1KB (1024 bytes) upto 256 KB of text in any format (including XML, JSON and unformatted text).
A single request can have from 1 to 10 messages, up to the maximum total payload of 256 KB.
Each queue starts with a default setting of 30 seconds for the visibility timeout. Maximum timeout is 12 hours.
SQS message retention period can be set to a value from 1 minute to 14 days. The default is 4 days.
With Amazon SQS Extended Client Library for Java, a referenced message payload in S3 can be as large as 2 GB.
Queue names are limited to 80 characters.
Queue names can use alphanumeric characters, hyphens (-), and underscores (_).
A message queue's name must be unique within an AWS account and region.
SQS - Important Points (Exam Tips)
Queue can be simultaneously used be many different components.
Can use in conjunction with SNS to "fan out" messages to multiple queues.
All messages have a global unique ID that SQS returns when the message is delivered to the message queue. The ID isn’t required to perform any further actions.
Time-in-queue value can be derived by subtracting SentTimestamp attribute value from current time.
For anonymous access, the SenderId attribute for a message will be the ip address.
The cost of SQS is calculated per request, plus data transfer charges for data transferred out of Amazon SQS. Every SQS action counts as a request. No charges if data is transferred to EC2 or to AWS Lambda within the same region.
API actions for sending, receiving, deleting, and changing visibility of messages from FIFO queues are charged at FIFO rates. Other API requests are charged at standard rates.
When using the SQS Extended Client Library to send payloads using S3, you incur S3 charges for any S3 storage you use. When using the AWS KMS to manage keys for SQS server-side encryption, you incur charges for calls from SQS to KMS.
Batch operations (SendMessageBatch, DeleteMessageBatch, and ChangeMessageVisibilityBatch) all cost the same as other SQS requests. By grouping messages into batches, you can thus reduce your costs.
Free tier: Customers can make 1 million Amazon SQS requests for free each month. After free limit, per 1 Million Requests, $0.40 for standard queue and $0.50 for FIFO queue.