Almost two years ago, in my Dual Write post, I introduced a few patterns for handling dual writes and said I might explore them in more detail later. That took a while, but here we are. If you haven’t read that post, it covers the problem and why simply reordering the writes or retrying in memory doesn’t solve it.

The idea behind these patterns is simple: when we can’t make two writes to independent systems atomic, we make one follow from a durable record of the other. In the e-commerce example from the previous post, the search service indexes a listing after its database record has been saved, or after an event to create it has been durably published. That distinction matters for the listen-to-yourself pattern, as we’ll see later. Either way, the search service needs a reliable way to learn about the listing so buyers can find it.

At Hasaki, when a warehouse doesn’t have enough stock to fulfill an order, the system creates an internal transfer request to move inventory from another location. The existing flow publishes one Kafka message per order, and each message triggers a separate transfer request.

During peak hours, a single warehouse can generate hundreds of these messages in minutes. Each message creates a small transfer request. The warehouse team ends up processing dozens of small requests for the same destination when a single consolidated batch would do.

When working on backend systems, we often follow a common pattern for handling incoming data. A server receives data from a message broker or an API, processes it according to the business rules, and inserts or updates it in the database. It then sends the processed data to the next server through a message broker or an API. This pattern is called dual write, sometimes also called multi-write or sync-write.