In current times, systems are designed as a collection of small, independent, and loosely coupled services. This development architecture is referred to as the Microservices architecture style. Developing applications as microservices enables development teams to work independently and focus on business logic instead of object interactions and their side effects. This provides flexibility, agility, and maintainability, as opposed to the monolithic architecture, where changes to one feature may impact other features.
As we move towards the microservices architecture style, our system gets distributed into multiple components, each having a specific business function. These components interact with each other through synchronous or asynchronous mechanisms to exchange data/events. This interservice communication presents a new set of challenges for the development team related to data consistency.
To solve such problems in the microservices architecture, multiple patterns are introduced, one of which is the Outbox pattern.
Outbox pattern
According to this pattern, the data changes required another operation/service are recorded in a temporary storage called as outbox. The stored data is then consumed by the service and hence ensures eventual consistency and reliability.
Let’s try to understand this with a real-world problem.
Problem statement
Assume we are building an e-commerce website where users can visit the catalog and place an order. Once an order is placed, the customer is informed about the order status through notifications like email, SMS, or push notifications.
In the above scenario, it is unreliable to send a message in the middle of a transaction. That the transaction will commit isn’t guaranteed. In a similar vein, there is no assurance that a service will not crash before sending a message if it does so after committing the transaction.
Messages must also be sent to the message broker in the same sequence that the service sent them. Although that is outside the purview of this pattern, they must typically be provided to each customer in the same order.
Solution
The message must first be stored in the database by the service sending it, as part of the transaction that changes the business entities. The messages are subsequently sent to the message broker by means of another channel.
As depicted in the above diagram, we have pulled out the process of sending messages to the broker from the transaction process. When a business transaction occurs within a microservice, it makes the necessary changes to a database transaction. We store all events in the outbox table in the same database as the microservice.
Another service (Event relay) continually monitors the outbox table and sends stored events to a specified message broker. Following on, services that have subscribed to the message broker are notified of the occurrences and respond according to their business rules.
Pros & Cons
Pros
- Messages are guaranteed to be sent if and only if the database transaction commits
- Messages are sent to the message broker in the order they were sent by the application
Cons
- The Message relay might publish a message more than once. As a result, a message consumer must be idempotent, perhaps by tracking the IDs of the messages that it has already processed. Fortunately, since message Consumers usually need to be idempotent (because a message broker can deliver messages more than once) this is typically not a problem.
References
- https://microservices.io/patterns/data/transactional-outbox.html