Companies like Uber, Twitter, and LinkedIn are known for their real-time updates. Live notifications from Uber rides, tweets from friends, and insightful advice from industry experts can be read and shared within a matter of seconds. As soon as the information is online, it is available to the world. Users are always responding positively to this easy and prompt availability of information and are always looking for such quality-of-life improvements.
To fulfill the increasing demands from consumers, many companies are shifting from traditional request and response frameworks to event driven architectures.
The former VP/distinguished engineer at AWS, Tim Bray, during his talk at the 2019 AWS Re: Invent, said: “This [event driven architecture] is the right way to build Amazon.com and any other way would be the wrong way.” Hopefully, this statement has piqued your interest regarding event driven architecture.
So, what is event driven architecture? Before we dive deeper into this topic, we should look at traditional request-response. Request and response is a synchronous call invoking a stream of services and expecting the call to return with a response until the next request can be made.
The example above displays a monolithic architecture. Now, in this setting, what if the backend service fails? The request from the client will gets lost. For commerce sites like Amazon, this would mean billions of dollars of sale loss. In a small company that handles lesser traffic, this monolithic architecture fits the use-case. However, once the company starts thinking about scaling in the future, this pattern presents huge bottlenecks.
Here, event driven architectures come into the picture. So, let’s dive deeper and understand the use-case and industry trend tools.
What Is an Event Driven Architecture?
Unlike in request-response systems, in an event driven architecture, a requestor sends an event (usually a message with a header and payload) to the distributed layer. Services that are listening to the particular topic can consume the message. This downstream service then can utilize this payload or further propagate this message to another service.
The initial sender of this event is called the publisher/producer and the receiving end is called the consumer/subscriber. This system allows multiple services to consume a single event as long as their channel is listening to the same topic.
Let’s look at the diagram above. The topic in this diagram is “slick-car”. The consumer (microservice) that’s listening to “slick-car” will consume the message that is being sent by the producer. The other consumer will not receive this message as they are listening to “classic car”. The distributed layer is a middleman that aggregates messages sent by the producer. If a consumer faces a problem receiving a message, the distributed layer can hold the message by queuing it and sending it over to the consumer when they are ready.
This is very similar to how the current mail system works. When the grandmother (producer) sends a package (message) to her grandchild (consumer), she has to specify the address (topic). When the package is sent, it first reaches to the post office (distributed layer). When arriving at the destination, if the grandchild’s family is available, the package can be delivered immediately. Otherwise, the post office can hold the package until they are ready to receive it.
This holding mechanism ensures that messages are never lost and are queued to be delivered as soon as the consumer is ready. This is the primary difference between an event driven architecture and a request-response architecture.
Why Use an Event Driven Architecture?
There are many advantages to adapting event driven architectures. But, notably, there are four benefits — real-time output, easy scalability, and fault tolerable and high availability.
Real-time Output
When an event is trigger by the client, the data sent from this event is usually in the JSON or Avro format. When this data is sent to the distribution layer, the message is serialized to a byte format. Since the distribution layer does not need to know about the content and its only job is to pass the message to the respective consumer. This serialization process makes the delivery extremely fast, resulting in the higher throughput of data flows. This enables real-time updates.
Easy Scalability
As a company migrates its infrastructure from a monolithic design to an event driven architecture, the primary goal is to partialize these services into microservices. Event driven architectures handle requests that get sent from service to service by having a centralized place for the messages to queue and be stored.
This makes wiring different microservices easier, and enhances development time, which means higher velocity to scale faster.
Fault Tolerable and High Availability
One of the disadvantage of using a request-response architecture is that there is a single point of failure. If a transient service goes down, the entire flow can be disrupted.
This is where event driven architectures shine the most. This is because distributed layers can have multiple servers dedicated to holding messages and can horizontally scale as needed. Then, even if one system goes down, other servers can pick up and relay messages.
Moreover, with the queuing mechanism, messages can be queued when the consumer service goes down and sent as soon as consumers start listening to the messages again.
How Do I Get Started?
These are notable tools you can use in the production environment to start your event driven architecture journey.
RabbitMQ
RabbitMQ is an open source product developed by Pivotal software. One notable thing about RabbitMQ is its ability to prioritize messages so that consumers can consume the specific message sooner.
You can find out more about setting RabbitMQ up on their website.
Kafka
Kafka is another open source framework that utilizes streaming-processing. It is known to have high throughput and low latency, which handles real-time data feeds effectively.
You can learn more about setting up Kafka on its website.
Conclusion
Event driven architectures are gaining high popularity for a good reason. Scaling while creating a decoupled microservice is a paradigm that many enterprises constantly aim to achieve, and event driven architectures fulfill that aspects. With innovative tools like Kafka and RabbitMQ, event driven architectures are becoming more resilient, versatile, and reliable to serve many business needs.
This powerful tool does comes with a price: a high learning curve and a challenging initial setup. However, once the structure is in place, the advantages are obvious as data flows in shorter latency and in higher throughput. So, why not event driven architecture?
Thank you for taking your time to read this post!
You can find me on LinkedIn.
