Understanding Asynchronous Messaging For Microservices
One of the possible consequences of adopting the microservice architecture is that the communication between components takes place over the network. For all that, the microservice environment is a distributed system. To fulfill the promises of microservices, like being able to scale, operate, and evolve each service, the communication must transpire in a loosely coupled and reliable manner.
A general way to loosely couple services is to unmask an API using the REST architecture. REST APIs are founded on the architecture of the web and enable loose coupling between communicating parties. It is an excellent way to decouple interfaces from concrete implementations, and to inform clients about the next line of action, by using links and link relations.
Although REST APIs are more common and helpful in microservice design, REST APIs are generally designed with synchronous communications, where a response is needed. A request from an end-user client can spark off a complex communication channel within your service landscape, which can successfully incur coupling between the services at runtime. And for this reason, we have mitigation patterns like a circuit breakers.
If losing-coupling is crucial, especially in a system that needs high resilience and has variability in terms of scale, an alternative is asynchronous messaging.
Asynchronous messaging is a key approach for integrating autonomous systems or assembling a set of loosely coupled systems that can operate, scale and evolve with autonomy and flexibility.
Single-Receiver Message-Based Communication
Message-based asynchronous communication with a single receiver implies an end-to-end communication that renders a message to exactly one of the users that’s reading from the channel, and the message processing is done just once.
Still, there are special cases. For example, in a cloud system that attempts to automatically recover from failures, the same message could resend multiple times. Owing to network or other failures, the client has to be able to repeatedly send messages, and the server has to implement an action to be unchanged to process a specific message just once.
Single-receiver message-based communication is perfectly suited for sending asynchronous commands from one microservice to another. But one limitation is that when you begin sending message-based communication (either with commands or events), you should steer clear of combining message-based communication with synchronous HTTP communication.
Multiple- Receiver Message-Based Communication
As a more flexible strategy, you might also try to resort to a publish/subscribe mechanism so that messages from the sender will be accessible to another subscriber microservice or external applications.
Thus, it favors an open/ closed principle in the sending service. While using a publish/subscribe communication, you might want to use an event bus interface to publish events to any subscriber.
Advantages of Asynchronous Messaging
Let us examine a few advantages of asynchronous communication:
- In a system that comprises asynchronous messaging, the server does not need to be functional all the time.
- Messages that are pushed into the message queue can be batch processed in the future.
- In place of a single instance of the tracing server processing the message queue, you could induce multiple instances to reduce the load.
- If you are using the modern version of a messaging queue, there is a possibility to get to use a replay feature. This is helpful in re-processing a message that gave an error message initially.
- Asynchronous communication is best suited for systems that need eventual consistency.
Drawbacks of Asynchronous Communication
Asynchronous communication can’t be applied if there is a limitation on the response time of a specific request i.e. if the processing has to be in real-time.
There are many possibilities for asynchronous integration. Some popular ones are
Kafka, RabbitMQ, Google Pub/Sub, Amazon Services, ActiveMQ, Azure Services.
Let us look into Kafka for instance as a messaging system.
Kafka is a distributed, duplicated commit log. There is no concept of queue with Kafka, given that it is majorly used as a messaging system.
Queues have been tantamount to messaging system for quite some time. Let us analyze the terms distributed, duplicated commit log:
Kafka being deployed as a cluster of nodes, for both fault tolerance and scale, hence “distributed”.
“Duplicated” because messages are duplicated across multiple nodes (servers).
Commit Log because messages are saved in segmented, append only logs which are called Topics. This idea is the most revered aspect of Kafka.
Kafka uses a pull model and which is justifiable because of its segmented nature. Consumers request messages in batches from a specified offset. Kafka permits long-pooling, which averts tight loops when there is no message past the offset. Also, the message order is provided in a partitions with no contending consumers. This enables users to take advantage of the message batching for effective delivery and higher throughput.
Also Read: Blockchain in Financial Services
Kafka’s model is a dumb broker/ smart consumer model which does not try to track which messages are read by which consumers. The messages are retained only for a set time period.
In the above article, we discussed asynchronous messaging and its various advantages and disadvantages.
It has seen rising popularity within middleware systems but also with edge client systems. In due course, users have invented protocols and standards to perform asynchronous messaging such that different systems can be integrated.