Amazon Web Services Messaging System: SNS vs SQS vs Kinesis

As modern application architecture are rapidly moving from trivial monolithic to server-less microservices and event-based systems which makes the application more flexible, fault-tolerant, modular, and easily maintainable apart from these benefits it also comes with the added overhead of managing the communication between various components this is where messaging systems come in to picture. Amazon Web Services (AWS) provides a well-equipped fully managed messaging stack’s as services like SNS, SQS, Kinesis which comes in handy in a very wide range of business scenarios. In this article, we will see details about each and the scenarios where they fit.

Table of Contents:

1. Simple Notification Service (SNS)
2. Simple Queue Service (SQS)
3. Kinesis
4. SNS + SQS
5. Difference’s between SNS, SQS, and Kinesis
6. Summary

1. Simple Notification Service (SNS)

SNS is a push notification service that is used to send event notifications as messages to a number of recipients. It is a flexible, fast, fully, and cost-effective service to send push notifications to mobile device users, email recipients, HTTP endpoints, and other distributed services, messages are grouped into a logical entity known as Topic to which various consumer subscribe and same copy of the message is sent to different message consumers. SNS is easy to use and manage and can be used in scenarios where a number of services/consumers need to be notified of an event.

SNS can be used to fan out mobile push notifications to iOS, Android, Fire OS, Windows, and Baidu-based devices it can also fan out Short Message Service( SMS)  to a number of countries and fan out email messages. It comes with a limitation of 10 emails per second.

Some common use cases would be, send event notifications for failure and other monitoring parameters, sending transactional email and SMS notifications, RSS feed, etc.

2. Simple Queue Service (SQS)

SQS is a fully managed distributed message queuing system.  Consumers have to poll SQS and pull messages. It provides the most basic way to perform inter-process communication and message delivery is guaranteed. A queue, can be used to send, store, and receive messages between software components at any volume, without losing messages or requiring other services to be always available.

SQS comes in two types of services STANDARD and FIFO. Standard messages are unordered while in FIFO messages can be consumed from the queue in first-in-first-out order. As it is a fully managed service there is no overhead of managing message-oriented middleware and it can be scaled very easily on demand.

SQS also provides the feature to persist messages for a period of 1 minute to 14 days which makes sure the message is consumed even if the receiver service is temporarily busy or down.

SQS maintains a dead letter queue where those messages are stored that were not consumed for some reason.

3. Kinesis

Kinesis is based on Apache Kafka, it is a fully managed, real-time Data Streaming and analytic service used to handle the very large stream of data, big data from a large number of sources, for example, Netflix uses Kinesis to handle Terabytes of data each day from events occurring from millions of connected IoT devices in real-time. The real-time data may include video, audio, application logs, website clickstreams, and IoT telemetry data for machine learning, analytics, and other applications.

The Kinesis stream is subdivided into shards and each shard can process a stream of data in isolation of other shards. These subdivisions provide a degree of load-balancing in the stream. Kinesis is very easy to set up and scale and minimizes the overhead of setting and maintaining Kafka clusters.

Kinesis data streams can easily scale to hundreds of data sources and process gigabytes of data per second.

Amazon Kinesis Data Firehose is used to reliably load streaming data into data lakes, data stores, and analytics tools.

Another thing is Amazon Kinesis Data Analytics, which is used to analyze streaming data, gain actionable insights, and respond to business and customer needs in real-time. It reduces the complexity of the building, managing, and integrating streaming applications with other AWS services. It comes with SQL support and provides a SQL editor to query on the data stream.

4. SNS + SQS

 By combining services SNS and SQS, we can achieve functionality similar to kinesis in a very simple and easy, scalable way. SNS provides real-time event notification push to the SQS queues and SQS retains the message and guarantees its delivery. This comes with a limitation as SNS can deliver messages to standard queues and not FIFO queues. Both SNS and SQS can be scaled automatically without any external instructions.

5. Difference between SNS, SQS, and Kinesis

Table 1 shows the major differences in SNS, SQS, and Kinesis.

Summary

We can see all three services provide an efficient way to handle event notifications, messaging between components, and real-time large data streams.

SNS manages event notification by push mechanisms in a fire and forgets manner which doesn’t guarantee delivery.

SQS provides support for messages to be consumed by the consumer through the use of queue and guarantees delivery by using the retention period.

Kinesis provides support for processing large data streams in real-time and it reduces the huge burden for setting and maintaining Kafka clusters.

   

About the Author

ByteScout Team of Writers ByteScout has a team of professional writers proficient in different technical topics. We select the best writers to cover interesting and trending topics for our readers. We love developers and we hope our articles help you learn about programming and programmers.