Docker Compose

Describing a multi-container application in a compose.yaml file: services, networks, volumes, environment and secrets, dependencies and health checks, and the docker compose commands.

What Compose is

Running Kestrel Energy's portal by hand takes several commands: a network, a volume, a database container, the meter-api, the portal, each with the right options in the right order. Typed by hand, that is slow and nobody does it the same way twice. Docker Compose is the Docker tool that reads a description of a multi-container application from a YAML file and creates, starts, stops and removes all of it with one command. The file is declarative: it states what should exist, and Compose works out what to create or change.

Compose runs on one host, talking to that host's Docker Engine. It is the standard tool for development set-ups, test environments in CI pipelines, and small production applications on a single server.

Which Compose?

The name covers three generations, and old material mixes them up:

  • Compose v1 (2014) was a separate Python program, docker-compose with a hyphen. It stopped receiving updates in 2023.
  • Compose v2 (announced 2020) is a rewrite in Go that plugs into the Docker command line: docker compose with a space.
  • Compose v5 (2025) is, in Docker's words, functionally identical to v2; the major version jumped to 5 to end the confusion with the old file-format versions 2 and 3, and it adds an official Go library. As of October 2026 the current release is v5.6.

The recorded sessions on this page come from a test machine with Docker Engine 28.2 and Compose v2.36.2; the commands and the messages are the same in Compose v5.

The compose file

A compose file is the YAML file that describes an application for Compose: its services (containers), networks, volumes, secrets and configuration. The preferred name is compose.yaml; Compose also reads compose.yml and, for backward compatibility, docker-compose.yaml and docker-compose.yml. It looks for the file in the current folder, so you run the commands from the folder that holds it.

Here is the whole Kestrel application: the portal, the meter-api, PostgreSQL and the nightly billing export. 

name: kestrel

services:
  portal:
    image: registry.example.com/kestrel/portal:1.4.2
    ports:
      - "127.0.0.1:8080:8080"
    environment:
      METER_API_URL: http://meter-api:8000
    depends_on:
      meter-api:
        condition: service_healthy
    restart: unless-stopped

  meter-api:
    image: registry.example.com/kestrel/meter-api:2.3.0
    environment:
      DATABASE_HOST: db
      DATABASE_PASSWORD_FILE: /run/secrets/db_password
    secrets:
      - db_password
    healthcheck:
      test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/healthz')"]
      interval: 10s
      timeout: 3s
      retries: 5
      start_period: 10s
    depends_on:
      db:
        condition: service_healthy
    restart: unless-stopped

  db:
    image: postgres:17
    environment:
      POSTGRES_DB: kestrel
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password
    secrets:
      - db_password
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres -d kestrel"]
      interval: 5s
      timeout: 3s
      retries: 10
    restart: unless-stopped

  billing-export:
    image: registry.example.com/kestrel/billing-export:1.0.7
    profiles: ["jobs"]
    depends_on:
      db:
        condition: service_healthy

volumes:
  pgdata:

secrets:
  db_password:
    file: ./secrets/db_password.txt

Reading it from top to bottom:

  • name sets the project name (see below). There is no version: line: that element is obsolete, and Compose only prints a warning when it finds one.
  • Under services, each key is one service with its image and the same settings you would otherwise pass to docker run: ports, environment, volumes, restart.
  • Only the portal publishes a port, and only on 127.0.0.1. The portal reaches the meter-api at http://meter-api:8000: service name and container port.
  • volumes: pgdata: at the bottom declares a named volume (see Volumes and Bind Mounts); secrets declares the password file.
  • The healthcheck blocks and the depends_on conditions control the start order, explained further down.
The Compose project kestrel on its network kestrel_default. The portal depends on the meter-api, the meter-api depends on db, and billing-export, which only runs with the profile jobs, also depends on db. The portal publishes 127.0.0.1:8080, the db stores its data in the volume pgdata, and the secret db_password is mounted into the meter-api and db. project kestrel · network kestrel_default portal meter-api db billing-export profile jobs 127.0.0.1:8080 secret db_password volume pgdata arrows: depends_on, waits until the target is healthy
The Kestrel compose file as a picture: four services, one network, one volume, one secret, and the start order.

Projects and services

A Compose service is one entry under services: a description of a container (image, settings, mounts) that Compose runs, normally as one container and optionally as several identical replicas. A Compose project is everything Compose creates from one compose file, grouped under one project name. The name comes from name: in the file, from -p on the command line, or else from the folder name.

The project name is the prefix of everything Compose creates, which is how two copies of the same application (say kestrel-test and kestrel-acc) can run on one host without colliding:

Object Name pattern Example (project kestrel)
Default network <project>_default kestrel_default
Named volume <project>_<volume> kestrel_pgdata
Container <project>-<service>-<number> kestrel-meter-api-1

Every project gets its own user-defined network, and every service is reachable on it by its service name. That is why the compose file needs no networking settings at all for the three services to find each other. A service such as the database that has no ports entry is reachable only from the other containers in the project.

Start order and health checks

depends_on is the service setting that lists which other services must be started first, and (in its long form) what "ready" means for each of them. Compose starts services in dependency order and stops them in reverse order. The conditions are:

Condition Compose waits until the dependency …
service_started has been started (the default, and what the short list syntax depends_on: [db] means)
service_healthy reports "healthy" from its healthcheck
service_completed_successfully has run to completion with exit code 0 (for set-up jobs)

"Started" is not "ready". PostgreSQL needs a few seconds after its container starts before it accepts connections, and a meter-api that connects in that window crashes or logs errors. A healthcheck (the Compose equivalent of the HEALTHCHECK instruction) runs a test command inside the container at an interval; with condition: service_healthy Compose holds the dependent service back until that test passes. Two more options belong to the long form: restart: true restarts the dependent service after Compose updates the dependency, and required: false only warns when the dependency is missing.

The test run used a simplified version of the application: the meter-api with a health check, and the portal waiting for it to be healthy. The file puts a password directly in the environment to keep it short; do not copy that part (see configuration and secrets). 

name: kestrel

services:
  meter-api:
    image: kestrel-meter-api:1.1
    environment:
      PORT: "8000"
      DB_PASSWORD: example-only
    healthcheck:
      test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"]
      interval: 5s
      timeout: 3s
      retries: 3
      start_period: 5s
    volumes:
      - meter-data:/data

  portal:
    image: kestrel-portal:multi
    ports:
      - "127.0.0.1:18082:80"
    depends_on:
      meter-api:
        condition: service_healthy

volumes:
  meter-data:
$ docker compose -p kestrel up -d
 Network kestrel_default  Creating
 Network kestrel_default  Created
 Volume "kestrel_meter-data"  Creating
 Volume "kestrel_meter-data"  Created
 Container kestrel-meter-api-1  Creating
 Container kestrel-meter-api-1  Created
 Container kestrel-portal-1  Creating
 Container kestrel-portal-1  Created
 Container kestrel-meter-api-1  Starting
 Container kestrel-meter-api-1  Started
 Container kestrel-meter-api-1  Waiting
 Container kestrel-meter-api-1  Healthy
 Container kestrel-portal-1  Starting
 Container kestrel-portal-1  Started

Compose creates the network and the volume, creates both containers, starts the meter-api, and then waits until its health check reports healthy before it starts the portal.

When the dependency never gets healthy

In a second run, the health check tested the wrong port (9999 instead of 8000), so it could never pass: 

$ docker compose -p kestrel-unhealthy up -d
 Network kestrel-unhealthy_default  Creating
 Network kestrel-unhealthy_default  Created
 Volume "kestrel-unhealthy_meter-data"  Creating
 Volume "kestrel-unhealthy_meter-data"  Created
 Container kestrel-unhealthy-meter-api-1  Creating
 Container kestrel-unhealthy-meter-api-1  Created
 Container kestrel-unhealthy-portal-1  Creating
 Container kestrel-unhealthy-portal-1  Created
 Container kestrel-unhealthy-meter-api-1  Starting
 Container kestrel-unhealthy-meter-api-1  Started
 Container kestrel-unhealthy-meter-api-1  Waiting
 Container kestrel-unhealthy-meter-api-1  Error
dependency failed to start: container kestrel-unhealthy-meter-api-1 is unhealthy
$ docker compose -p kestrel-unhealthy ps -a
NAME                            IMAGE                   COMMAND                  SERVICE     CREATED          STATUS                      PORTS
kestrel-unhealthy-meter-api-1   kestrel-meter-api:1.1   "python app.py"          meter-api   16 seconds ago   Up 16 seconds (unhealthy)   8000/tcp
kestrel-unhealthy-portal-1      kestrel-portal:multi    "/docker-entrypoint.…"   portal      16 seconds ago   Created

The meter-api itself runs fine; only its check fails, three times in a row (retries: 3), which marks it unhealthy. The portal stays Created and never starts, and up exits with 1. The health check's own output, in docker inspect, shows why (see Debugging Containers). A wrong health check fails a deployment just as surely as a broken service.

Working out the start order

Compose sorts services so that each one starts after everything it depends on: a topological sort. Python's standard library has one, which also shows what happens with a circular dependency (Compose refuses such a file): 

from graphlib import CycleError, TopologicalSorter

depends_on = {
    "portal": ["meter-api"],
    "meter-api": ["db"],
    "billing-export": ["db"],          # only with --profile jobs
    "db": [],
}
print("start order:", " -> ".join(TopologicalSorter(depends_on).static_order()))

sorter = TopologicalSorter(depends_on)
sorter.prepare()
step = 1
while sorter.is_active():
    ready = sorted(sorter.get_ready())
    print(f"step {step}: start {', '.join(ready)}")
    sorter.done(*ready)
    step += 1

depends_on["db"] = ["portal"]          # a mistake: db now waits for portal
try:
    list(TopologicalSorter(depends_on).static_order())
except CycleError as error:
    print("cycle:", " -> ".join(error.args[1]))
start order: db -> meter-api -> billing-export -> portal
step 1: start db
step 2: start billing-export, meter-api
step 3: start portal
cycle: portal -> db -> meter-api -> portal

Services in the same step have no dependency on each other, so Compose may start them in parallel. In the cycle, each service is needed by the next one, so none of them can ever go first.

Configuration and secrets

Settings reach a container in three ways, and they are easy to confuse:

  • environment: sets environment variables in the container, directly in the compose file.
  • env_file: reads environment variables for the container from a file. When both set the same variable, environment: wins.
  • A file named .env next to compose.yaml is read by Compose itself, to fill in ${VARIABLE} placeholders in the compose file, such as image: registry.example.com/kestrel/portal:${PORTAL_TAG}. Its values do not reach the containers unless the file passes them on.

None of these is a good place for a password. Environment variables show up in docker inspect, in the output of docker compose config, in crash reports and in the process list, and a compose file with a real password ends up in Git sooner or later (see what never belongs in Git). Compose secrets mount a file instead: the db_password secret above appears inside the meter-api and db containers as the file /run/secrets/db_password. Applications read the password from that file; the official postgres image supports this through POSTGRES_PASSWORD_FILE. The secret file itself stays out of Git, in .gitignore, just like .env.

A small review script catches the most common problems before a compose file is merged. Python's standard library cannot read YAML, so this one works on the file already loaded as a dictionary: 

compose = {
    "version": "3.8",
    "services": {
        "portal": {"image": "registry.example.com/kestrel/portal:latest",
                   "ports": ["8080:8080"]},
        "meter-api": {"image": "registry.example.com/kestrel/meter-api:2.3.0",
                      "environment": {"DB_PASSWORD": "example-only"},
                      "depends_on": ["db"]},
        "db": {"image": "postgres:17", "ports": ["5432:5432"]},
    },
}

if "version" in compose:
    print("top level: 'version' is obsolete, remove it")
for name, svc in compose["services"].items():
    image = svc.get("image", "")
    if image.endswith(":latest") or ":" not in image.split("/")[-1]:
        print(f"{name}: pin a version instead of {image}")
    for port in svc.get("ports", []):
        if port.count(":") == 1:
            print(f"{name}: port {port} is published on all interfaces")
    for key in svc.get("environment", {}):
        if "PASSWORD" in key or "SECRET" in key:
            print(f"{name}: {key} in plain text, use a secret file")
    if isinstance(svc.get("depends_on"), list):
        print(f"{name}: depends_on waits only for 'started', not 'healthy'")
top level: 'version' is obsolete, remove it
portal: pin a version instead of registry.example.com/kestrel/portal:latest
portal: port 8080:8080 is published on all interfaces
meter-api: DB_PASSWORD in plain text, use a secret file
meter-api: depends_on waits only for 'started', not 'healthy'
db: port 5432:5432 is published on all interfaces

The published database port deserves the most attention: on a server, "5432:5432" offers PostgreSQL to the whole network, and a host firewall does not stop it (see Container Networking).

Everyday commands

Every command works on the project in the current folder (or the one named with -p):

Command What it does
docker compose config Validates the file and prints the full, normalized model with all defaults and variables filled in
docker compose pull Downloads the images of all services
docker compose up -d Creates and starts everything in the background; run again after a change, it recreates only the services whose configuration or image changed
docker compose ps Lists the project's containers with status and ports
docker compose logs -f meter-api Follows the logs of one service (or all, without a name)
docker compose exec db psql -U postgres Runs a command in a running service container
docker compose stop / start Stops or starts the containers, keeping them
docker compose down Stops and removes the containers and the network; named volumes stay
docker compose down -v The same, and also removes the volumes: the data is gone

docker compose config is the first command to run when something behaves unexpectedly: it shows what Compose really understood. For this file it expands the short port syntax and fills in the generated names: 

$ docker compose -p kestrel config
name: kestrel
services:
  meter-api:
…
  portal:
    depends_on:
      meter-api:
        condition: service_healthy
        required: true
    image: kestrel-portal:multi
    networks:
      default: null
    ports:
      - mode: ingress
        host_ip: 127.0.0.1
        target: 80
        published: "18082"
        protocol: tcp
networks:
  default:
    name: kestrel_default
volumes:
  meter-data:
    name: kestrel_meter-data

The running project, a request to the portal, and the logs:

$ docker compose -p kestrel ps
$ curl -s http://127.0.0.1:18082/
$ docker compose -p kestrel logs --tail 1 meter-api
NAME                  IMAGE                   COMMAND                  SERVICE     CREATED         STATUS                   PORTS
kestrel-meter-api-1   kestrel-meter-api:1.1   "python app.py"          meter-api   6 seconds ago   Up 5 seconds (healthy)   8000/tcp
kestrel-portal-1      kestrel-portal:multi    "/docker-entrypoint.…"   portal      6 seconds ago   Up Less than a second    127.0.0.1:18082->80/tcp
<!doctype html><title>Kestrel Energy portal</title><h1>Kestrel Energy</h1><p>Your meter readings, without the paperwork.</p>
meter-api-1  | 127.0.0.1 - - [06/Oct/2026 12:22:58] "GET /health HTTP/1.1" 200 -

The meter-api shows 8000/tcp without an arrow: the port exists, but only inside the project network. The log line is the health check calling the service from inside its own container. From the portal container, the meter-api is reachable by service name: 

$ docker compose -p kestrel exec portal wget -qO- http://meter-api:8000/readings
[{"meter": "KE-0001", "kwh": 4123.7}, {"meter": "KE-0002", "kwh": 977.1}, {"meter": "KE-0003", "kwh": 15.0}]

Stopping one service, and then removing the whole project:

$ docker compose -p kestrel stop meter-api
$ docker compose -p kestrel ps -a
 Container kestrel-meter-api-1  Stopping
 Container kestrel-meter-api-1  Stopped
NAME                  IMAGE                   COMMAND                  SERVICE     CREATED          STATUS                                PORTS
kestrel-meter-api-1   kestrel-meter-api:1.1   "python app.py"          meter-api   26 seconds ago   Exited (137) Less than a second ago
kestrel-portal-1      kestrel-portal:multi    "/docker-entrypoint.…"   portal      26 seconds ago   Up 19 seconds                         127.0.0.1:18082->80/tcp

Exit code 137 after a plain stop is worth a second look: this simplified meter-api runs as process 1 in its container and ignores the polite stop signal (SIGTERM), so Docker killed it after the 10-second grace period. A well-behaved service exits with 0 or 143; see Running Containers. 

$ docker compose -p kestrel down -v
 Container kestrel-portal-1  Stopping
 Container kestrel-portal-1  Stopped
 Container kestrel-portal-1  Removing
 Container kestrel-portal-1  Removed
 Container kestrel-meter-api-1  Stopping
 Container kestrel-meter-api-1  Stopped
 Container kestrel-meter-api-1  Removing
 Container kestrel-meter-api-1  Removed
 Network kestrel_default  Removing
 Volume kestrel_meter-data  Removing
 Volume kestrel_meter-data  Removed
 Network kestrel_default  Removed

Compose stops the portal first (it depends on the meter-api), then the meter-api, and with -v the volume goes as well.

Profiles and one-off jobs

Not every service should start with up. The billing export runs once a night, so it has profiles: ["jobs"]: services with a profile start only when that profile is activated. Joost Bakker runs the export by hand, as a one-off container that is removed when it ends: 

$ docker compose --profile jobs run --rm billing-export --date 2026-10-05

run starts one new container of the service with the arguments you give (here replacing the image's default command arguments), and it starts the dependencies first, so the database is healthy before the export begins. On a single server, a cron job or systemd timer can run this command every night; in Kubernetes the same job becomes a CronJob.

Outdated advice you will still find

Old habit Current practice (October 2026)
docker-compose up, installing Compose with pip docker compose up, the plugin that comes with Docker; v1 is unmaintained
version: "3.8" at the top Leave it out; Compose always uses the current Compose Specification and only warns about the line
docker-compose.yml compose.yaml (old names still work)
links: between services Not needed: service names resolve on the project network
depends_on plus a "wait-for-it" script in the image condition: service_healthy with a healthcheck
Passwords under environment: secrets: mounted as files, read through *_FILE variables

When Compose is not enough

Compose manages containers on one host. If that host fails, the application is down until someone starts it elsewhere; Compose does not move containers between machines, balance load over several hosts or roll out a new version without a short interruption. For that you need an orchestrator such as Kubernetes (see why orchestration and How a Kubernetes Cluster Works). Compose remains the quickest way to run the same application on a developer's laptop and in a CI test job, and many small applications run happily on one well-backed-up server with Compose and a restart policy.