Member-only story
Transactional Outbox: A Simplified Approach
Transactional Outbox
I was reading the “Microservices Patterns” book by Chris Richardson and came across this pattern called “Transactional Outbox”, described also on the author’s website. It allows you to solve a case when you want to write a record to the database and send a related message to the consumers via a message broker, while keeping those two actions transactional.
The problem this pattern tries to solve is as follows. Suppose you persist an Order entity in your order-service microservice, and then send an OrderCreated message to another microservice (execution-service) that has to execute this Order. If these two actions are not executed in an atomic transaction, there’s a possibility that you save an Order to the database but fail to send it if the message broker is down.
If you first send the OrderCreated message and then save the Order to the database, it could happen that the database will be down or fails to accept the Order due to some constraint violation. Then the Order will not be saved, but its processing will be started, since we’ve already sent OrderCreated message to the broker.
We can use a distributed transaction (two-phase commit) to solve this. It would guarantee exactly-once delivery by joining multiple different resources (database and message broker) in a single atomic transaction. But this technique is heavy, and doesn’t work well in a high load and high throughput application.
So the author proposes to use the Transactional Outbox pattern, described as follows:
- order-service saves the Order to the database;
- in the same transaction, order-service saves the OrderCreated message related to this Order into another table of the same database (OUTBOX); since transaction is the same, these two inserts constitute an atomic action;
- the message-relay (working in a different transaction) reads the OrderCreated message from the OUTBOX table and sends it to the message broker;
- the message-relay removes the record from the OUTBOX table.
It’s easy to see that this approach has at-least-one guarantee: if we fail to remove the message from the OUTBOX table due to the failure of the database…
