Cloud-Native Architecture

Moving an application to a cloud provider does not make it cloud-native. A virtual machine in the cloud that needs a manual restart after every update, keeps sessions in memory and writes logs to a local disk has the same weaknesses it had in the data center, now with a monthly bill. This page is about the design properties that let an application benefit from the cloud's strengths: fast provisioning, horizontal scaling, replaceable instances and services that someone else runs. The cloud building blocks themselves (compute, containers, serverless, managed databases) are covered in the Cloud and Containers & Kubernetes topics.

What cloud-native means

Cloud-native describes applications and practices designed to run in dynamic, automated environments such as public, private or hybrid clouds: built from loosely coupled parts that can be deployed, scaled and replaced independently and automatically, and that are resilient, manageable and observable. The Cloud Native Computing Foundation (CNCF) maintains a definition; version 1.1, approved in February 2024, starts with practices rather than technology:

"Cloud native practices empower organizations to develop, build, and deploy workloads in computing environments (public, private, hybrid cloud) to meet their organizational needs at scale in a programmatic and repeatable manner. It is characterized by loosely coupled systems that interoperate in a manner that is secure, resilient, manageable, sustainable, and observable."

It then lists technologies that cloud-native systems "typically consist of some combination of": containers, service meshes, multi-tenancy, microservices, immutable infrastructure, serverless and declarative APIs, adding "this list is non-exhaustive". The older version 1.0 text from 2018, which begins "Cloud native technologies empower organizations to build and run scalable applications…", is still widely quoted but has been replaced.

At Kestrel Energy, the customer portal already runs on cloud virtual machines at a public cloud provider and is the natural candidate. The billing system KBS (Kestrel Billing System) runs on Kestrel's own servers in a rented colocation data center and does not need to move at all to benefit from several of the practices below. Cloud-native is a direction, not a switch.

The twelve-factor app

A twelve-factor app is an application built according to twelve principles for software-as-a-service applications, published at 12factor.net by Adam Wiggins and colleagues at the platform company Heroku around 2011, which make it portable between environments, easy to deploy continuously and able to scale out without redesign. More than a decade later, it is still the most concrete checklist for cloud-native application design. The twelve factors, with what each means for meter-api, Kestrel's service that receives smart-meter readings:

Factor XI connects to the observability topic: an application that writes structured lines to standard output lets the platform collect them, so no instance holds the only copy of its logs (see Logs). The sections below go deeper into the factors that shape architecture most.

Configuration and backing services

Factor III asks for a strict separation of configuration from code: everything that differs between deploys, such as database addresses, feature switches and batch sizes, comes from the environment the process runs in, typically environment variables. The same build then runs unchanged in test, acceptance and production. In Kubernetes these values come from a ConfigMap; on a virtual machine from a systemd unit or the deployment tool.

Read and validate all settings once, at startup. A missing or mistyped value should stop the process immediately with a clear message, not surface as an error during the billing export at 02:30:

import os
from urllib.parse import urlsplit

# In production these come from the deployment (a ConfigMap, a unit file, ...);
# the example sets them itself so that it is repeatable.
os.environ["DATABASE_URL"] = "postgresql://meter_api@db01.example.internal:5432/metering"
os.environ["BROKER_URL"] = "amqp://meter_api@broker.example.internal:5672/kestrel"
os.environ["READING_BATCH_SIZE"] = "500"

class ConfigError(Exception):
    pass

def load_config(env):
    """Read and validate all settings once, at startup: fail fast, not at 02:30."""
    missing = [k for k in ("DATABASE_URL", "BROKER_URL") if k not in env]
    if missing:
        raise ConfigError(f"missing settings: {', '.join(missing)}")
    batch = int(env.get("READING_BATCH_SIZE", "200"))
    if not 1 <= batch <= 5_000:
        raise ConfigError(f"READING_BATCH_SIZE out of range: {batch}")
    return {
        "database": urlsplit(env["DATABASE_URL"]),
        "broker": urlsplit(env["BROKER_URL"]),
        "batch_size": batch,
        "log_level": env.get("LOG_LEVEL", "INFO"),
    }

config = load_config(os.environ)
for name in ("database", "broker"):
    url = config[name]
    print(f"{name:9} {url.scheme}://{url.hostname}:{url.port}{url.path}"
          f"  user={url.username}  password in URL: {url.password is not None}")
print(f"batch size {config['batch_size']}, log level {config['log_level']}")

try:
    load_config({"DATABASE_URL": "postgresql://db01/metering"})
except ConfigError as error:
    print("startup refused:", error)

database  postgresql://db01.example.internal:5432/metering  user=meter_api  password in URL: False
broker    amqp://broker.example.internal:5672/kestrel  user=meter_api  password in URL: False
batch size 500, log level INFO
startup refused: missing settings: BROKER_URL

Secrets are not ordinary config

Notice that neither URL contains a password. The twelve-factor text treats credentials as config and puts them in the environment too, but it was written before secrets managers were common, and environment variables leak easily: into crash reports, debug pages, child processes and the output of commands that show a process's environment. Today's practice: non-secret settings in environment variables, secrets in a secrets manager, fetched at startup or mounted as a file with tight permissions (see Secrets in Automation).

Backing services

A backing service is, in 12factor.net's words, "any service the app consumes over the network as part of its normal operation": a database, a message broker, a cache, an email relay or an external API. Factor IV says to treat each one as an attached resource, identified by a URL or address in the configuration, so that it can be replaced without changing code. The text's own example: a deploy "should be able to swap out a local MySQL database with one managed by a third party (such as Amazon RDS) without any changes to the app's code." For Kestrel, moving meter-api's data from db01 to a managed PostgreSQL service then becomes a change of DATABASE_URL plus a data migration, not a code change.

Stateless services

A stateless service is a service whose instances keep nothing in memory or on local disk between requests that another instance would need, so that any instance can handle any request and instances can be added or removed at any time. The state still exists; it lives in backing services built to keep it safe. Factor VI is blunt: "Twelve-factor processes are stateless and share-nothing", and "Any data that needs to persist must be stored in a stateful backing service, typically a database." This is the application side of the stateless design that makes horizontal scaling and failover work. (HTTP itself is a stateless protocol; a stateless service is the same idea one level up.)

What goes wrong without it is easy to show. Two portal instances behind a round-robin load balancer, first with the login session in each instance's memory, then in a shared session store:

from itertools import cycle

class Instance:
    def __init__(self, name, store=None):
        self.name = name
        self.store = store if store is not None else {}   # own memory, or shared

    def handle(self, customer, action):
        if action == "login":
            self.store[customer] = "logged in"
            return "welcome"
        return "statement" if self.store.get(customer) else "please log in again"

def run(instances):
    balancer = cycle(instances)        # round robin, as a load balancer would
    for action in ("login", "view", "view", "view"):
        instance = next(balancer)
        print(f"  {action:5} -> {instance.name}: {instance.handle('K-1042', action)}")

print("state in each instance's memory:")
run([Instance("web01"), Instance("web02")])

print("state in a shared session store (a backing service):")
sessions = {}
run([Instance("web01", sessions), Instance("web02", sessions)])

state in each instance's memory:
  login -> web01: welcome
  view  -> web02: please log in again
  view  -> web01: statement
  view  -> web02: please log in again
state in a shared session store (a backing service):
  login -> web01: welcome
  view  -> web02: statement
  view  -> web01: statement
  view  -> web02: statement

The tempting fix for the first case is "sticky sessions": the load balancer sends each user to the same instance. The twelve-factor text rejects it ("Sticky sessions are a violation of twelve-factor and should never be used or relied upon"), and for a practical reason: when that instance is replaced, scaled in or crashes, its users lose their sessions anyway. Stateless does not forbid caches in memory; it forbids depending on them. A cached tariff table that a new instance can rebuild is fine.

Disposability

Disposability is the property of an application process that it can be started or stopped at a moment's notice without harm: it starts quickly, shuts down gracefully when asked, and survives being killed abruptly. It is factor IX, and 12factor.net asks that a process takes "a few seconds from the time the launch command is executed until the process is up and ready to receive requests or jobs."

Disposability matters every day, not only during failures:

• Scaling. meter-api receives about 444 readings per second on average, in bursts at :00, :15, :30 and :45. An autoscaler that adds an instance when the burst starts helps only if the instance is ready within seconds. With a startup time of 90 seconds, the new copy arrives after the burst is over.
• Deployments. Rolling and canary deployments replace instances one by one; slow starts make every release slow.
• The platform's own work. Container platforms and cloud providers move instances when hosts are patched or fail.

For graceful shutdown, 12factor.net says: "Processes shut down gracefully when they receive a SIGTERM signal from the process manager." Docker sends SIGTERM and kills the process with SIGKILL after a grace period (see stopping, signals and PID 1), and Kubernetes works the same way with a configurable grace period. A worker that consumes readings stops taking new work, finishes the batch in hand and exits:

import os
import signal

queue = [f"reading-{n}" for n in range(1, 8)]   # work waiting in this process
stopping = False

def on_sigterm(signum, frame):
    global stopping
    print("SIGTERM received: finishing the current batch, taking no new work")
    stopping = True

signal.signal(signal.SIGTERM, on_sigterm)

batch_no = 0
while queue and not stopping:
    batch, queue = queue[:3], queue[3:]
    batch_no += 1
    if batch_no == 2:
        os.kill(os.getpid(), signal.SIGTERM)    # the platform stops us mid-batch
    print(f"batch {batch_no}: stored {', '.join(batch)}")

print(f"{len(queue)} reading(s) left unacknowledged for the broker to redeliver; exit 0")

batch 1: stored reading-1, reading-2, reading-3
SIGTERM received: finishing the current batch, taking no new work
batch 2: stored reading-4, reading-5, reading-6
1 reading(s) left unacknowledged for the broker to redeliver; exit 0

The handler only sets a flag; the loop checks it between batches, so a batch is never cut in half. The last reading was never acknowledged to the broker, so the broker delivers it to another instance. The twelve-factor text adds that processes "should also be robust against sudden death": a SIGKILL or a power failure gives no warning at all. That is why the reading is acknowledged only after it is stored, and why consumers must handle a redelivered message (see Service Communication).

Dev/prod parity

Dev/prod parity is keeping development, test and production environments as similar as possible, in time between writing and deploying code, in the people involved and in the tools and backing services used, so that problems show up before production. Factor X names three gaps: the time gap ("A developer may work on code that takes days, weeks, or even months to go into production"), the personnel gap ("Developers write code, ops engineers deploy it") and the tools gap, for example "SQLite" in development where production uses another database.

Kestrel's developers run meter-api's tests against SQLite because it needs no setup. Last spring a query that SQLite accepted failed on PostgreSQL in acceptance, and a type difference went unnoticed until production. The twelve-factor text is explicit: "The twelve-factor developer resists the urge to use different backing services between development and production, even when adapters theoretically abstract away any differences." Containers make running the real thing locally cheap. A Compose file for meter-api's development setup:

# compose.yaml: meter-api with its backing services, including the broker proposed in ADR-0007
services:
  meter-api:
    build: .
    environment:
      DATABASE_URL: postgresql://meter_api@db:5432/metering
      BROKER_URL: amqp://meter_api@broker:5672/kestrel
      DB_PASSWORD: ${DEV_DB_PASSWORD}   # development only; production uses the secrets manager
    ports:
      - "127.0.0.1:8080:8080"
    depends_on:
      - db
      - broker
  db:
    image: postgres:16                  # same major version as db01
    environment:
      POSTGRES_USER: meter_api
      POSTGRES_PASSWORD: ${DEV_DB_PASSWORD}
      POSTGRES_DB: metering
  broker:
    image: rabbitmq:4.3

The other two gaps are about process, not tools: small, frequent deployments close the time gap (see DevOps), and developers who deploy and watch their own code close the personnel gap.

Serverless as an architectural style

Serverless computing runs code as functions that the provider starts on demand, scales automatically and bills per invocation. The Cloud topic explains how functions, triggers, cold starts and limits work. Architecturally, a system built from functions is an extreme form of the patterns on this page: every function is stateless and disposable by design, and all state lives in managed backing services.

That has architectural consequences beyond the platform:

• Many small deployment units. Twenty functions are twenty things to version, monitor and secure. If they call each other synchronously and must be released together, they form a distributed monolith (see Architecture Anti-Patterns).
• Good fit: event-driven, spiky or irregular work, such as generating an annual-statement PDF when a customer asks for one, or processing a file when it lands in storage.
• Poor fit: steady high load, long-running work, or chatty steps that pass large amounts of data between functions. A frequently cited 2023 case: a team at Amazon Prime Video reported cutting the cost of one monitoring service by 90% by moving it from serverless functions and workflow orchestration to a single process on containers. That was one service with heavy data passing between steps, not a verdict on serverless or microservices in general.

For Kestrel, the PDF generation is a good candidate. meter-api, with a constant stream of readings and a database connection per instance, is better served by a few long-running instances.

Managed services as building blocks

A managed service is a component such as a database, message queue, cache or identity provider that a provider installs, operates, patches, backs up and scales for you, so that you use it through an API or connection string instead of running it yourself. A managed database is the best-known example. Cloud-native designs lean on them heavily, because running a highly available broker or database well takes skills and on-call time that most teams would rather spend on their own product.

Lock-in is a trade-off to manage, not a reason to refuse every managed service. Services with a standard interface (PostgreSQL, AMQP or Kafka protocols, S3-compatible storage) are easier to leave than proprietary ones, and treating them as backing services behind a URL keeps the application code portable. Ravi Kumar's rule of thumb for the portal team: use managed services for commodity components (database, broker, cache, identity), and spend the team's own effort on what is specific to Kestrel.