Event-Driven Architecture

Service Communication showed why Kestrel Energy's meter-api should not call the billing system KBS for every smart-meter reading, and why readings should travel as messages instead of through shared tables on db01. This page goes one step further: what happens when services announce what happened and let others decide what to do with it. Event sourcing and CQRS, which are also called "event-driven", are on Data in Distributed Systems.

Events, commands and messages

An event, in software architecture, is a record of something that has happened in a system, published so that other parts can react to it. It is named in the past tense, describes a fact that can no longer be changed, and is sent without knowing who will receive it: ReadingReceived, InvoiceSent, ContractEnded. (In IT service management an event is any change of state that matters for managing a service, such as a monitoring alert; the idea is related, but the word is used more narrowly here.)

A command is different: it asks one specific receiver to do something, and may be refused: CalculateInvoice, SendReminder. Both travel as messages through the same brokers, but they create different dependencies:

What event-driven architecture means

Event-driven architecture is an architectural style in which services communicate mainly by publishing events about changes in their own domain and reacting to events from others, instead of calling each other directly. The publisher finishes its own work and announces it; whatever follows is decided by the subscribers.

The term covers several patterns that are often confused. Martin Fowler's article "What do you mean by 'Event-Driven'?" (2017) separates four: event notification, event-carried state transfer, event sourcing and CQRS. The first two are about how services communicate and are explained here; the last two are about how a service stores its data. They are independent: you can use events between services without event sourcing inside any of them.

The same idea exists at smaller scales. Event-driven programming inside one program reacts to clicks and key presses, and the Observer pattern (see Behavioral Patterns) lets objects subscribe to changes in another object. The architectural version adds a broker, the network, and everything that can go wrong between them.

Publish-subscribe

Publish-subscribe (pub/sub) is a messaging pattern in which publishers send messages to a topic on a broker, and the broker delivers a copy to every subscriber of that topic; publishers and subscribers do not know about each other. Microsoft's Azure Architecture Center describes its purpose as "Enable an application to announce events to multiple consumers asynchronously, without coupling senders to receivers." Unlike the Observer pattern, where the subject keeps a list of its observers, the publisher here only knows the topic.

A small in-memory broker shows the effect. meter-api publishes ReadingReceived to a readings topic; billing and the portal's usage cache subscribe. Later a third team adds an anomaly check:

from collections import defaultdict


class Broker:
    """An in-memory stand-in for a message broker with topics."""

    def __init__(self):
        self.subscribers = defaultdict(list)

    def subscribe(self, topic, name, handler):
        self.subscribers[topic].append((name, handler))

    def publish(self, topic, event):
        # A real broker gives every subscriber its own copy, asynchronously.
        for name, handler in self.subscribers[topic]:
            print(f"  -> {name}: {handler(event)}")


broker = Broker()
broker.subscribe("readings", "billing (KBS)",
                 lambda e: f"book {e['kwh']} kWh for {e['ean'][-4:]}")
broker.subscribe("readings", "usage cache",
                 lambda e: f"refresh graph for {e['ean'][-4:]}")

reading = {"ean": "871687400000000017", "interval": "02:15", "kwh": 0.42}
print("meter-api publishes ReadingReceived")
broker.publish("readings", reading)

# A new team subscribes. meter-api does not change at all.
broker.subscribe("readings", "anomaly check",
                 lambda e: "unusual" if e["kwh"] > 5 else "normal")
print("meter-api publishes ReadingReceived")
broker.publish("readings", {**reading, "interval": "02:30", "kwh": 6.1})

meter-api publishes ReadingReceived
  -> billing (KBS): book 0.42 kWh for 0017
  -> usage cache: refresh graph for 0017
meter-api publishes ReadingReceived
  -> billing (KBS): book 6.1 kWh for 0017
  -> usage cache: refresh graph for 0017
  -> anomaly check: unusual

The third subscriber needed no change and no release of meter-api. That is the main attraction of pub/sub: adding a consumer is cheap, and the publisher's code stays small. A real broker delivers each copy separately and asynchronously, so a slow anomaly check does not delay billing, and each subscriber has its own queue that it reads at its own pace.

Event notification and event-carried state transfer

How much should an event contain? Fowler names two extremes.

Event notification

Event notification is an event-driven style in which an event only says that something happened and identifies it, and receivers that need details ask the publisher for them. A notification for a new reading carries the meter and the interval, not the value. Events stay small and the publisher keeps full control of its data, but every subscriber calls back, so the publisher must be up and able to handle that load, and some temporal coupling returns.

Event-carried state transfer

In event-carried state transfer the event carries all the data that receivers are expected to need, so they can keep their own copy and never have to call the publisher. The cost is larger events, duplicated data, and copies that are briefly out of date. For a minute of Kestrel readings with three subscribers:

import json

EVENTS = 1_000        # readings published in one minute
SUBSCRIBERS = 3       # billing, usage cache, anomaly check

notification = {"type": "ReadingReceived", "ean": "871687400000000017",
                "interval": "2026-10-06T02:15"}
carried = {**notification, "kwh": 0.42, "quality": "measured",
           "contract": "CT-20417"}

for name, event, calls_back in (("event notification", notification, True),
                                ("event-carried state", carried, False)):
    size = len(json.dumps(event))
    callbacks = EVENTS * SUBSCRIBERS if calls_back else 0
    print(f"{name:20} {size:3} bytes per event, "
          f"{callbacks:5,} calls back to meter-api")

event notification    88 bytes per event, 3,000 calls back to meter-api
event-carried state  148 bytes per event,     0 calls back to meter-api

The larger event saves 3,000 requests a minute to meter-api, and the subscribers keep working when meter-api is down. Kestrel's plan for readings (ADR-0007) uses event-carried state transfer and the CloudEvents format, a specification of the Cloud Native Computing Foundation (CNCF) for describing events in a common way (version 1.0.2 as of October 2026). Its required attributes are id, source, specversion and type; the payload goes in data:

{
  "specversion": "1.0",
  "type": "com.example.kestrel.reading.received",
  "source": "/meter-api",
  "id": "9f1c2d7e-5b3a-4e8f-a0c4-6d2e8b1f7a35",
  "time": "2026-10-06T00:30:04Z",
  "subject": "871687400000000017",
  "datacontenttype": "application/json",
  "data": {
    "ean": "871687400000000017",
    "intervalStart": "2026-10-06T02:15:00+02:00",
    "kwh": 0.42,
    "quality": "measured"
  }
}

Include what subscribers need to do their work, not the whole database row "just in case": every field you publish is a field someone will depend on.

Publishing reliably: the transactional outbox

meter-api has to do two things for each reading: store it in its own database and publish the event. Doing them one after the other is called a dual write (see Data in Distributed Systems), and it fails in both orders. Store first, and a crash or a broker outage before the publish loses the event: billing never hears about a reading that is in the database. Publish first, and a failed database write announces a reading that does not exist. A distributed transaction across the database and the broker would solve it in theory, but most brokers do not support one, and it would tie meter-api's availability to both.

The transactional outbox pattern solves it with one local transaction. The service writes the business change and the outgoing event, as a row in an outbox table in the same database, in a single transaction; a separate relay process then reads new outbox rows, publishes them to the broker and marks them as sent. microservices.io states the problem as "How to atomically update the database and send messages to a message broker?"

The example first shows the dual write losing an event, then two readings stored through the outbox, with a relay that crashes after publishing the first event but before marking it as sent:

import json
import sqlite3

db = sqlite3.connect(":memory:")
db.executescript("""
CREATE TABLE readings (ean TEXT, interval_start TEXT, kwh REAL,
                       PRIMARY KEY (ean, interval_start));
CREATE TABLE outbox (id INTEGER PRIMARY KEY, type TEXT, payload TEXT,
                     published INTEGER NOT NULL DEFAULT 0);
""")
broker = []    # stands in for the message broker
EAN = "871687400000000017"


def store_then_publish(interval, kwh):
    """Dual write: the database and the broker, one after the other."""
    with db:
        db.execute("INSERT INTO readings VALUES (?, ?, ?)", (EAN, interval, kwh))
    raise ConnectionError("broker unreachable")   # the publish fails


def store_with_outbox(interval, kwh):
    """The reading and its event commit in ONE local transaction."""
    with db:
        db.execute("INSERT INTO readings VALUES (?, ?, ?)", (EAN, interval, kwh))
        payload = json.dumps({"ean": EAN, "interval": interval, "kwh": kwh})
        db.execute("INSERT INTO outbox (type, payload) VALUES (?, ?)",
                   ("ReadingReceived", payload))


def relay(crash_after_first=False):
    """Publish unpublished outbox rows in order, then mark them."""
    rows = db.execute("SELECT id, type, payload FROM outbox "
                      "WHERE published = 0 ORDER BY id").fetchall()
    for row_id, kind, payload in rows:
        broker.append(f"{kind} {payload}")
        if crash_after_first:
            print(f"relay crashed after publishing row {row_id}")
            return
        with db:
            db.execute("UPDATE outbox SET published = 1 WHERE id = ?", (row_id,))


try:
    store_then_publish("2026-10-06T02:00", 0.40)
except ConnectionError as error:
    print("dual write:", error, "- reading stored, event lost")

store_with_outbox("2026-10-06T02:15", 0.42)
store_with_outbox("2026-10-06T02:30", 0.38)
relay(crash_after_first=True)
relay()                                  # after a restart
for message in broker:
    print("broker got", message)

dual write: broker unreachable - reading stored, event lost
relay crashed after publishing row 1
broker got ReadingReceived {"ean": "871687400000000017", "interval": "2026-10-06T02:15", "kwh": 0.42}
broker got ReadingReceived {"ean": "871687400000000017", "interval": "2026-10-06T02:15", "kwh": 0.42}
broker got ReadingReceived {"ean": "871687400000000017", "interval": "2026-10-06T02:30", "kwh": 0.38}

Nothing is lost, but the first event reached the broker twice: after the restart the relay found the row still unmarked and published it again. That is inherent to the pattern, and microservices.io warns about it: "The Message relay might publish a message more than once." Consumers must therefore be idempotent consumers, which KBS is, keyed on the meter and interval.

There are two common ways to build the relay:

• Polling publisher: query the outbox table every second or so for unpublished rows, as above. Simple, and enough for most systems; delete or archive published rows regularly so the table stays small.
• Transaction log tailing, also called change data capture: read the database's transaction log and publish every new outbox row from it, with tools such as Debezium for PostgreSQL. Lower delay and no polling load, but one more component that needs database privileges.

Kestrel proposes this in ADR-0007, "Use a message broker between meter-api and KBS", with the outbox as the way readings would be published. The record is still under review: Ravi Kumar and Joost Bakker decide, after consulting Hamid Rahimi and Yara Haddad. The format of such a record is on Documenting Architecture.

Living with eventual consistency

Once KBS keeps its own copy of the readings, there are moments when meter-api has a reading that KBS has not processed yet. The data is eventually consistent: in Werner Vogels' words, "if no new updates are made to the object, eventually all accesses will return the last updated value." That is not a bug to hide but a property to design for:

• Agree on how stale is acceptable. For billing, a reading may arrive up to 30 minutes late; Kestrel monitors this as a freshness service level objective and alerts on the age of the oldest unprocessed message.
• Show it to people. The portal says "usage up to 02:30" instead of pretending its figures are live.
• Keep strong consistency where it matters. A customer's bank account change is handled synchronously by the one service that owns it, not spread by events and hoped for.

Sagas: transactions across services

Within one service, a database transaction makes several changes all-or-nothing. Across services with their own databases there is no such transaction. A saga is a way to carry out a business process that spans several services as a sequence of local transactions, one per service, where each step is committed on its own and a failure later on is handled by running compensating actions that undo the effects of the steps already completed. The idea dates from a 1987 database paper by Hector Garcia-Molina and Kenneth Salem; microservices.io describes the modern use.

At Kestrel, a customer moving house is a saga: end the contract at the old address, record the final meter reading, send the final invoice, start the contract at the new address. If the last step fails because the new address is not connected yet, the earlier steps cannot simply be rolled back, because they were committed long ago, and the final invoice may already be in the customer's mailbox. They have to be compensated, in reverse order:

A saga can be coordinated in two ways.

Orchestration

In orchestration, one component, the orchestrator, tells each participating service which step to perform, waits for the result, and decides on the next step or the compensations. The whole process is visible in one place. A minimal orchestrator for the move:

from dataclasses import dataclass


class StepFailed(Exception):
    pass


@dataclass
class Step:
    name: str
    compensation: str | None = None   # None: nothing to undo
    fails_with: str | None = None     # set to simulate a failure

    def execute(self):
        if self.fails_with:
            raise StepFailed(self.fails_with)


def run_saga(steps):
    done = []
    for number, step in enumerate(steps, start=1):
        try:
            step.execute()
        except StepFailed as reason:
            print(f"step {number}: {step.name}: FAILED ({reason})")
            for earlier in reversed(done):      # undo in reverse order
                if earlier.compensation:
                    print(f"  compensate: {earlier.compensation}")
            return "move canceled, customer stays at the old address"
        print(f"step {number}: {step.name}: done")
        done.append(step)
    return "move completed"


move_house = [
    Step("end contract at Kerkstraat 12", "reopen contract at Kerkstraat 12"),
    Step("record final meter reading"),               # a fact: nothing to undo
    Step("send final invoice", "void final invoice with a credit note"),
    Step("start contract at Molenweg 3", fails_with="address not connected yet"),
]
print(run_saga(move_house))

step 1: end contract at Kerkstraat 12: done
step 2: record final meter reading: done
step 3: send final invoice: done
step 4: start contract at Molenweg 3: FAILED (address not connected yet)
  compensate: void final invoice with a credit note
  compensate: reopen contract at Kerkstraat 12
move canceled, customer stays at the old address

In Java, an interface makes the contract of a step explicit: every step can execute, and may override a default compensation that does nothing. Each step is a record that implements it:

import java.util.ArrayDeque;
import java.util.Deque;
import java.util.List;

public class Main {
    public static void main(String[] args) {
        List<SagaStep> moveHouse = List.of(
            new EndContract("Kerkstraat 12"),
            new FinalReading(),
            new FinalInvoice(),
            new StartContract("Molenweg 3"));
        System.out.println(run(moveHouse));
    }

    static String run(List<SagaStep> steps) {
        Deque<SagaStep> done = new ArrayDeque<>();
        int number = 1;
        for (SagaStep step : steps) {
            try {
                step.execute();
            } catch (IllegalStateException e) {
                System.out.println("step " + number + ": " + step.name()
                    + ": FAILED (" + e.getMessage() + ")");
                while (!done.isEmpty()) {
                    done.pop().compensate();      // newest first
                }
                return "move canceled, customer stays at the old address";
            }
            System.out.println("step " + number++ + ": " + step.name() + ": done");
            done.push(step);
        }
        return "move completed";
    }
}

interface SagaStep {
    String name();

    void execute();

    default void compensate() {
        // most steps have something to undo; facts such as a reading do not
    }
}

record EndContract(String address) implements SagaStep {
    public String name() { return "end contract at " + address; }
    public void execute() { }
    public void compensate() {
        System.out.println("  compensate: reopen contract at " + address);
    }
}

record FinalReading() implements SagaStep {
    public String name() { return "record final meter reading"; }
    public void execute() { }
}

record FinalInvoice() implements SagaStep {
    public String name() { return "send final invoice"; }
    public void execute() { }
    public void compensate() {
        System.out.println("  compensate: void final invoice with a credit note");
    }
}

record StartContract(String address) implements SagaStep {
    public String name() { return "start contract at " + address; }
    public void execute() {
        throw new IllegalStateException("address not connected yet");
    }
}

step 1: end contract at Kerkstraat 12: done
step 2: record final meter reading: done
step 3: send final invoice: done
step 4: start contract at Molenweg 3: FAILED (address not connected yet)
  compensate: void final invoice with a credit note
  compensate: reopen contract at Kerkstraat 12
move canceled, customer stays at the old address

In a real system each execute() sends a command to a service and waits for a reply message, and the orchestrator stores its progress in a database, so that it can continue after a restart instead of forgetting a half-finished move. Workflow engines exist for exactly this.

Choreography

In choreography there is no coordinator: each service listens for the events of the previous step, does its own part and publishes an event that triggers the next. The contract service publishes ContractEnded; metering reacts with a final reading and publishes FinalReadingRecorded; billing sends the final invoice; and when NewContractFailed appears, each service undoes its own part.

When events are the wrong tool

Event-driven architecture trades simplicity for independence. That trade is worth it for some interactions and not for others:

• The caller needs an answer now. A customer checking a price waits for it; publish an event and the answer arrives somewhere else, later. Use a synchronous call.
• The flow must be easy to follow. With events, "what happens when a reading arrives?" can only be answered by searching every subscriber. Document the events and their subscribers, and propagate trace context in message headers so one reading can be followed through all of them.
• Strong consistency is required. If two changes must always be seen together, keep them in one service and one transaction.
• Nobody will run the broker. A message broker is critical infrastructure. A small system with one team and one database is often better served by a modular monolith that calls functions and commits transactions.