Deployment Pipelines and GitOps

Environments

Between a merged pull request and real users there are usually several places where the software runs. A deployment environment is one such target, with its own infrastructure, configuration, data and access rules. A common set:

In the Netherlands and Belgium this sequence is often called OTAP or DTAP; the Release and Deployment Management page describes it from the service-management side. Not every team needs four: a small internal tool may go from pull request checks straight to production behind a feature flag. What matters is that each environment has a clear purpose, and that staging resembles production closely enough for its results to mean something.

Build once, promote the same artifact

The pipeline builds the application once, from one commit, into one versioned build artifact, for example a container image tagged with the commit SHA (the Secure Hash Algorithm hash that names a commit). Environment promotion then moves that exact artifact from one environment to the next: what was tested in staging is, byte for byte, what reaches production. Only configuration differs per environment, supplied from outside the artifact: the database address, feature flags, log level, secrets.

Rebuilding per environment looks harmless but breaks the guarantee: a rebuild a day later can pick up a newer base image or dependency, and production then runs something no one tested. Tag artifacts with the commit SHA (or a version number that maps to one), keep them in a registry, and promote by changing which tag an environment runs.

Approvals and GitHub environments

Continuous delivery keeps every change ready for production, with a human decision before the last step. A deployment approval is that decision built into the pipeline: the deployment job pauses until an authorized person approves it, and the approval is recorded with the run. In GitHub Actions, approvals belong to environments, configured under the repository's Settings → Environments. Each environment can have:

• Required reviewers: people or teams who must approve before a job that targets the environment starts. A setting can stop the person who triggered the run from approving their own deployment.
• A wait timer: a delay of a set number of minutes before the job may start, for example to leave time to cancel.
• Deployment branches and tags: only runs from, say, main may deploy to production.
• Environment secrets and variables: a job only receives them after the protection rules pass; restrict deployment branches to main so that a run from a feature branch cannot get the production credentials at all.

A job selects its environment with the environment key:

# .github/workflows/deploy.yml
name: deploy

on:
  push:
    branches: [main]

permissions:
  contents: read

jobs:
  build:
    runs-on: ubuntu-24.04
    permissions:
      contents: read
      packages: write          # push the image to GitHub Packages
    steps:
      - uses: actions/checkout@v7
        with:
          persist-credentials: false
      - run: ./scripts/build-and-push.sh "ghcr.io/example-corp/inventory-api:${GITHUB_SHA}"
        env:
          REGISTRY_TOKEN: ${{ secrets.GITHUB_TOKEN }}

  deploy-staging:
    needs: build
    runs-on: ubuntu-24.04
    environment: staging
    permissions:
      contents: read
      id-token: write          # OIDC: exchange a short-lived token for cloud credentials
    steps:
      - uses: actions/checkout@v7
        with:
          persist-credentials: false
      - run: ./scripts/deploy.sh staging "ghcr.io/example-corp/inventory-api:${GITHUB_SHA}"
      - run: ./scripts/smoke-test.sh https://staging.example.com

  deploy-production:
    needs: deploy-staging
    runs-on: ubuntu-24.04
    environment:
      name: production         # required reviewers are set on this environment
      url: https://inventory.example.com
    concurrency: production    # never two production deployments at once
    permissions:
      contents: read
      id-token: write
    steps:
      - uses: actions/checkout@v7
        with:
          persist-credentials: false
      - run: ./scripts/deploy.sh production "ghcr.io/example-corp/inventory-api:${GITHUB_SHA}"
        env:
          NOTIFY_WEBHOOK: ${{ secrets.DEPLOY_NOTIFY_WEBHOOK }}   # an environment secret
      - run: ./scripts/smoke-test.sh https://inventory.example.com

When the run reaches deploy-production, it stops with the status "waiting" and notifies the reviewers. The same image (tagged with GITHUB_SHA) goes to both environments. Neither deploy job stores cloud keys: with id-token: write the job can request an OpenID Connect (OIDC) token and exchange it for short-lived cloud credentials, as explained on the GitHub Actions page. The rules themselves (who approves, and the deployment branches, which for production should be only main) live in the repository settings, not in this file, so a pull request cannot weaken them.

Deployment strategies and rollback

How the new version replaces the old one in an environment is the deployment strategy: a rolling deployment, a blue-green deployment or a canary release, possibly with feature flags; the DevOps and CI/CD page explains each. In a pipeline, the strategy is usually a parameter of the deploy step (or of the platform it calls), and a good pipeline checks health after each step and stops by itself when error rates rise.

When a release goes wrong there are two ways forward:

• Roll back: deploy the previous, known-good artifact again. Because artifacts are versioned and kept, this is quick and needs no new build. Blue-green makes it almost instant: switch traffic back.
• Roll forward: fix the problem in a new commit and push it through the pipeline. Sensible when the fix is small and the pipeline is fast, and sometimes the only option, for example when a database migration cannot be reversed.

Databases are where rollback gets hard: the old version of the code may not understand a column the new version added. Teams avoid the trap by making schema changes backward compatible ("expand, then contract": add the new column, deploy code that uses it, and remove the old one only in a later release), so the previous version can always run against the current schema. Decide the rollback route before deploying; the IT service management (ITSM) name for that is a back-out plan.

Push-based and pull-based deployment

Someone has to make the target environment run the new version. There are two models:

• Push-based deployment: the pipeline connects to the target and changes it, for example by running kubectl apply, an Ansible playbook or a cloud command-line (CLI) tool. The workflow above is push-based. It is simple and familiar, but the CI system needs credentials that can change production, and the target only changes when a pipeline runs, so manual changes made in between go unnoticed.
• Pull-based deployment: an agent running inside the target environment watches a source (a Git repository or a registry) and pulls changes in. The CI system never needs credentials for production; it only writes to Git. Because the agent keeps watching, it also notices when the environment no longer matches the source.

GitOps

GitOps is an operating model in which the desired state of a system is declared in Git and software agents continuously make the running system match it. Git becomes the single place where changes happen: to deploy, you merge a pull request; to see what runs in production, you read the repository; to undo, you revert a commit. The OpenGitOps project (opengitops.dev) defines four principles (version 1.0.0):

1. Declarative: the desired state of the system is expressed declaratively (what should exist, not the steps to get there; see declarative configuration).
2. Versioned and immutable: the desired state is stored in a way that keeps it immutable and versioned, with a complete history.
3. Pulled automatically: software agents pull the desired state from the source automatically.
4. Continuously reconciled: software agents continuously observe the actual state and try to bring it to the desired state.

Git is the usual store for principle 2 (hence the name), but the principles themselves do not require it. GitOps is used mostly with Kubernetes, whose own controllers already work declaratively.

The reconciliation loop

Principle 4 is a reconciliation loop: observe the actual state, compare it with the desired state, act on the difference, wait, and repeat, forever. It is the same idea that Kubernetes controllers use inside a cluster, applied one level up, with Git as the source of the desired state.

The small program below is the heart of that loop in a few lines: given a desired and an actual state (here: which version of each service should run with how many replicas), it works out what to create, update and delete.

desired = {  # from Git
    "inventory-api": {"image": "inventory-api:3f9c2a1", "replicas": 3},
    "label-printer": {"image": "label-printer:8e41d07", "replicas": 1},
}
actual = {  # observed in the cluster
    "inventory-api": {"image": "inventory-api:3f9c2a1", "replicas": 2},
    "old-reports": {"image": "old-reports:1a2b3c4", "replicas": 1},
}


def reconcile(desired, actual):
    actions = []
    for name in sorted(desired.keys() | actual.keys()):
        want, have = desired.get(name), actual.get(name)
        if have is None:
            actions.append(f"create {name}: {want}")
        elif want is None:
            actions.append(f"delete {name} (not in Git)")
        elif want != have:
            changes = {key: f"{have[key]} -> {want[key]}"
                       for key in want if want[key] != have[key]}
            actions.append(f"update {name}: {changes}")
    return actions


for action in reconcile(desired, actual) or ["in sync, nothing to do"]:
    print(action)

update inventory-api: {'replicas': '2 -> 3'}
create label-printer: {'image': 'label-printer:8e41d07', 'replicas': 1}
delete old-reports (not in Git)

Real agents do the same with Kubernetes objects. Whether they actually delete objects that are no longer in Git ("prune") and undo manual changes ("self-heal") is a setting: many teams start with reporting only and switch on automatic correction once they trust the setup.

Argo CD

Argo CD is a GitOps continuous-delivery tool for Kubernetes, part of the Argo project, which graduated in the Cloud Native Computing Foundation (CNCF) in December 2022 (Argo CD 3.x is current as of 2026). It runs in a cluster, has a web interface that shows each application's sync and health status, and is configured with Kubernetes objects of its own. An Application says which path in which repository should run where:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: inventory-api
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://git.example.com/platform/deploy-config.git
    targetRevision: main
    path: apps/inventory-api/production
  destination:
    server: https://kubernetes.default.svc   # the cluster Argo CD runs in
    namespace: inventory
  syncPolicy:
    automated:
      prune: true      # delete objects that were removed from Git
      selfHeal: true   # undo changes made directly in the cluster

Flux

Flux is a GitOps toolkit for Kubernetes that graduated in the CNCF in November 2022 (Flux v2 has been generally available since July 2023). Instead of one application, it is a set of controllers, each with its own object type: a GitRepository tells the source controller what to fetch and how often, and a Kustomization tells the kustomize controller which path to apply from it:

apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: deploy-config
  namespace: flux-system
spec:
  interval: 1m
  url: https://git.example.com/platform/deploy-config.git
  ref:
    branch: main
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: inventory-api
  namespace: flux-system
spec:
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: deploy-config
  path: ./apps/inventory-api/production
  prune: true

Both tools do the same job; Argo CD is often chosen for its interface and multi-cluster overview, Flux for its small, composable controllers that are driven entirely by Kubernetes objects. In both, the usual pattern uses two repositories: the application repository, whose CI pipeline tests the code and pushes an image, and a configuration repository with the manifests per environment. Promotion becomes a commit in the configuration repository that changes the image tag for an environment, often opened as a pull request by the pipeline or by an image-update automation, and reviewed like any other change.

Drift

Configuration drift is the difference between what is declared and what really runs, usually caused by someone "quickly" changing something by hand. With push-based deployment, drift goes unnoticed until the next deployment silently overwrites it, and with it a change someone may have been relying on. A GitOps agent sees drift on its next pass: it Argo CD, for example, marks the application "OutOfSync" and, with self-heal on, puts the declared state back. That also means an emergency fix made directly in the cluster will be undone within minutes. The fix must go through Git, or automation must be paused deliberately for that application.

Secrets in GitOps

If everything lives in Git, where do passwords go? Never into the repository in plain text, and Kubernetes Secret objects are only base64-encoded, not encrypted. The common answers:

• Encrypted in Git: Sealed Secrets (only a controller in the cluster can decrypt) or SOPS (Secrets OPerationS, which encrypts the values in a YAML file with age, PGP or cloud KMS keys).
• Referenced from Git: the External Secrets Operator stores only a reference in Git; the controller fetches the value from a secrets manager at run time. Nothing secret is ever committed.

Infrastructure as code in pipelines

The same discipline applies to the infrastructure under the application. With infrastructure as code the usual pipeline for Terraform (or OpenTofu) is:

1. On a pull request: terraform fmt -check and terraform validate, then terraform plan; the plan's summary is posted on the pull request so reviewers see exactly which resources will be created, changed or destroyed.
2. On merge: terraform apply, behind a protected environment with approval for production. Applying a saved plan file (terraform plan -out=tfplan, then terraform apply tfplan) guarantees that what is applied is what was reviewed; if the state has changed since the plan was made (for example because another apply ran), Terraform refuses the stale plan.

terraform plan -out=tfplan      # on the pull request: what would change?
terraform show -no-color tfplan  # readable plan for the review
terraform apply tfplan           # after merge and approval: apply exactly this plan

Two rules matter more here than anywhere: only one pipeline run may apply to the same state at a time (Terraform's state locking, plus a concurrency group in the workflow), and the pipeline's cloud credentials should be short-lived OIDC credentials, with read-only access for the plan job and write access only for the apply job. The infrastructure as code workflow and state are explained on the Cloud pages.

Pipelines and change enablement

Organizations that follow ITIL control production changes through change enablement. That can look like the opposite of continuous delivery, but the two fit together well when the pipeline does the paperwork:

• Automated evidence. Every deployment already leaves a record: the pull request with its review and discussion, the commit SHA, the test and scan results, who approved the environment, when it ran, and whether the smoke tests passed. That is better evidence than a hand-filled form, and it cannot be forgotten.
• Standard changes. A change that is low-risk, frequent and follows a proven procedure can be pre-authorized as a standard change. A deployment that passed all automated gates of a well-run pipeline is a strong candidate: the pipeline is the proven procedure.
• Normal changes for the rest. Big or risky changes (a database migration that cannot be rolled back, a new region) still go to a change authority; the pipeline can open the change record automatically and wait for its approval, just like a required reviewer.

Many tools for IT service management integrate with CI/CD platforms for exactly this: a deployment job creates or updates the change record and stops until it is approved. The result is a faster process with a better audit trail, not a weaker one.