Amazon SQS vs. Other AWS Services

Amazon AWS vs. Amazon Kinesis Streams

Amazon SQS is a hosted queue for storing messages as they travel between distributed application components and helps you decouple these components. SQS provides common middleware constructs such as dead-letter queues and poison-pill management. It also provides a generic web services API and can be accessed by the AWS SDK. SQS supports both standard and FIFO queues.

Use SQS when you need each unique message to be consumed only once and for cases such as following:

1. Decoupling the components of an application

   1. You have a queue and want to track the successful completion of each item independently. After a configured visibility timeout, Amazon SQS deletes acknowledged messages and redelivers failed messages.

2. Configuring individual message delay

   1. You have a job queue and you need to schedule individual jobs with a delay. With SQS, you can configure individual messages to have a delay of up to 15 minutes.

3. Dynamically increasing concurrency or throughput at read time

   1. You have a queue and want to add more readers temposarily. With Kinesis Streams, you must provision enough shards upfront to scale up. SQS requires no pre-provisioning.

4. Scaling transparently

   1. SQS can scale transparently to handle the load without any explicit provisioning instructions.

Amazon Kinesis Streams allows real-time processing of streaming big data and the ability to read and replay records to multiple Amazon Kinesis Applications. The Amazon Kinesis Client Library (KCL) delivers all records for a given partition key to the same record processor, making it easier to build multiple applications that read from the same Amazon Kinesis stream.

Use Amazon Kinesis Streams when you need multiple consumers to be able to process each record and for use cases such as the following:

1. Routing related records to the same record processor

   1. You stream MapReduce. Actions such as counting and aggregation are simpler when all records for a given key are routed to the same record processor.

2. Allowing multiple applications to consume the same stream concurrently

   1. You have one application that updates a real-time dashboard and another that archives data to Amazon Redshift. You want both applications to consume data from the same stream concurrently and independently.