Amazon Simple Queue Service (SQS) Overview

Amazon Simple Queue Service (SQS) is a fully managed message queuing service. You can send, store, and receive messages between distributed software components at any volume, reliably. SQS helps to decouple, coordinate and scale microservices, distributed systems, and serverless applications in cloud. SQS was the first service on the AWS.

SQS solves the producer consumer problem efficiently. SQS standard queues offer maximum throughput, best-effort ordering, and at-least-once delivery. SQS FIFO queues guarantee that messages are processed exactly once, in the exact order, with limited throughput. Can access SQS from Management Console, SDK or SQS webservice APIs.

 

Processing Messages in Queue

You can call the ReceiveMessage action to receive the message. 

After a consumer receives a message from a queue, the message remains in the queue until the consumer delete the message after processing it. A message is considered as in-flight after it's received by consumer, but not yet deleted from queue. For standard queues, there can be a maximum of 120K inflight messages per queue. For FIFO queues, max is 20K. If you reach limit, an OverLimit error message occurs for standard queues but not for FIFO queues.

To delete only specific messages, can use the DeleteMessage or DeleteMessageBatch actions. Can delete all messages in an SQS queue using the PurgeQueue action.

 

Visibility Timeout

To prevent other consumers from processing the message again, SQS sets a visibility timeout, during which SQS prevents other components from receiving the message. For standard queues, the visibility timeout isn't a guarantee against receiving a message twice. FIFO queues allow the producer or consumer to attempt multiple retries using the same message deduplication ID or request attempt ID respectively.

You can change the timeout setting for the entire queue, or per message (using the ChangeMessageVisibility action) when receiving messages. You can terminate the visibility timeout for a specific message by calling ChangeMessageVisibility with VisibilityTimeout set to 0 seconds. This make the message immediately visible to other components. 

 

Message Retention Period

SQS automatically deletes messages that have been in a queue for more than maximum message retention period. The default message retention period is 4 days. You can set the message retention period to a value from 60 seconds to 1,209,600 seconds (14 days) using the SetQueueAttributes action.  

 

Short polling vs. long polling

Consumers poll the queue and pull messages from the queue. With regular polling (short polling), consumer requests return immediately with or without message (if no message is available) and is charged every time.

With long polling, consumer requests wait until a message is available or the long polling request times out. Maximum timeout is 20 seconds.To enable long polling, set ReceiveMessageWaitTimeSeconds is set to a value greater than zero. All AWS SDKs work with 20-second long polls by default.

 

SQS - Limits & Defaults

  1. Messages can contain 1KB (1024 bytes) upto 256 KB of text in any format (including XML, JSON and unformatted text).

  2. A single request can have from 1 to 10 messages, up to the maximum total payload of 256 KB.

  3. Each queue starts with a default setting of 30 seconds for the visibility timeout. Maximum timeout is 12 hours.

  4. SQS message retention period can be set to a value from 1 minute to 14 days. The default is 4 days

  5.  With Amazon SQS Extended Client Library for Java, a referenced message payload in S3 can be as large as 2 GB

  6.  Queue names are limited to 80 characters.

  7. Queue names can use alphanumeric characters, hyphens (-), and underscores (_).

  8. A message queue's name must be unique within an AWS account and region

 

SQS - Important Points (Exam Tips)

  1. Queue can be simultaneously used be many different components.

  2. Can use in conjunction with SNS to "fan out" messages to multiple queues.

  3. All messages have a global unique ID that SQS returns when the message is delivered to the message queue. The ID isn’t required to perform any further actions.

  4. Time-in-queue value can be derived by subtracting SentTimestamp attribute value from current time. 

  5. For anonymous access, the SenderId attribute for a message will be the ip address.

 

SQS Pricing

The cost of SQS is calculated per request, plus data transfer charges for data transferred out of Amazon SQS. Every SQS action counts as a request. No charges if data is transferred to EC2 or to AWS Lambda within the same region. 

API actions for sending, receiving, deleting, and changing visibility of messages from FIFO queues are charged at FIFO rates.  Other API requests are charged at standard rates.

When using the SQS Extended Client Library to send payloads using S3, you incur S3 charges for any S3 storage you use. When using the AWS KMS to manage keys for SQS server-side encryption, you incur charges for calls from SQS to KMS.

Batch operations (SendMessageBatch, DeleteMessageBatch, and ChangeMessageVisibilityBatch) all cost the same as other SQS requests. By grouping messages into batches, you can thus reduce your costs.

Free tier: Customers can make 1 million Amazon SQS requests for free each month. After free limit, per 1 Million Requests, $0.40 for standard queue and $0.50 for FIFO queue.

Contact

Please first use the contact form or facebook page messaging to connect.

Offline Contact
We currently connect locally for discussions and sessions at Bangalore, India. Please follow us on our facebook page for details.
WhatsApp (Primary): (+91) 7411174113
Phone (Escalations): (+91) 7411174114

Business newsletter

Complete the form below, and we'll send you an e-mail every now and again with all the latest news.

About

CloudMaterials.com is my blog to share notes and learning materials on Cloud and Data Analytics. My current focus is on Amazon Web Services.

I like to write. I try to document what I learn and share with others. I believe that knowledge is useless unless you share it; the more you share, the more you learn.

Recent comments

Photo Stream

,