Infrastructure as Code in Pipelines

Why changes go through a pipeline

On a laptop, tofu apply runs with whatever code, provider versions and credentials happen to be there. Two people can apply different versions of the same configuration on the same afternoon, and nobody else sees the plan. A pipeline turns that into one controlled path: every change is a pull request, every plan is visible, the same tool versions run every time, and the powerful credentials exist only inside the pipeline.

The basic pattern is already described for Terraform on Deployment Pipelines and GitOps and for the cloud in general on DevOps and CI/CD. This page goes into the details that make it safe: the flags, the workflow file, the credentials, drift detection, and what to do with Ansible.

The flow: plan in review, apply after approval

A plan made on a pull request is a speculative plan: it shows what would change if this branch were applied now, but it is never applied itself. Its job is review. By the time the branch is merged, other changes may have landed on main, so the pipeline plans again on main.

That second plan is saved with -out=tfplan, shown to the approver, and applied with tofu apply tfplan. This is plan-apply separation: making the plan and carrying it out are two steps, with a human decision in between and a guarantee that nothing else is carried out. If the state changed after the plan was saved, OpenTofu refuses to apply it. Applying the same saved plan twice shows that check:

╷
│ Error: Saved plan is stale
│
│ The given plan file can no longer be applied because the state was changed
│ by another operation after the plan was created.
╵

Terraform behaves the same way. Never "fix" a stale plan by running apply without a plan file in the pipeline: that makes a fresh plan that nobody has seen and applies it at once.

Running OpenTofu without a human

Interactive habits do not work in a pipeline: there is nobody to answer a prompt, colors turn into escape codes in logs, and a script needs to know whether there are changes without parsing text. These flags handle that (Terraform has the same ones):

Normally tofu plan exits with 0 whether or not there are changes. With -detailed-exitcode the detailed exit code carries the answer: 0 means no changes, 1 means an error, and 2 means the plan succeeded and there are changes. A script can branch on it like on any other exit status. The Kestrel portal configuration, planned for the test environment (already up to date) and for production (not yet):

tofu fmt -check -recursive
tofu validate -no-color
tofu plan -input=false -no-color -detailed-exitcode -var environment=test -out=tfplan
tofu plan -input=false -detailed-exitcode -var environment=prod > /dev/null; echo "exit code: $?"
tofu plan -input=false -detailed-exitcode -var environment=test > /dev/null; echo "exit code: $?"

Success! The configuration is valid.
terraform_data.release: Refreshing state... [id=e750094c-58ff-cf63-01f9-a5eb48342a0e]
random_id.deployment: Refreshing state... [id=CoDo_w]
random_password.db: Refreshing state... [id=none]
local_file.upstream: Refreshing state... [id=24f7367f646f81ac74d14656ca66aea60f98c78f]

No changes. Your infrastructure matches the configuration.
exit code: 2
exit code: 0

tofu fmt -check printed nothing (all files formatted). In a script, remember that set -e stops at exit code 2 as if it were an error; capture the code first, as the workflow below does.

Plans for tools: JSON

Policy tools, scanners and pipeline scripts should read the plan as JSON, never scrape the text. tofu show -json tfplan turns a saved plan into one JSON document; its resource_changes list has the actions per resource:

tofu plan -input=false -no-color -detailed-exitcode -var environment=prod -out=tfplan
tofu show -json tfplan | python3 -c 'import json,sys; p=json.load(sys.stdin); print([(c["address"], c["change"]["actions"]) for c in p["resource_changes"]])'

…
Saved the plan to: tfplan

To perform exactly these actions, run the following command to apply:
    tofu apply "tfplan"
[('local_file.upstream', ['delete', 'create']), ('random_id.deployment', ['no-op']), ('random_password.db', ['no-op']), ('terraform_data.release', ['update'])]

["delete", "create"] is a replacement, ["update"] an in-place change and ["no-op"] nothing. tofu plan -json instead streams one JSON object per line while the plan runs, which suits tools that show progress; OpenTofu 1.12 added -json-into=FILE to write that stream to a file while the normal text still goes to the log. A pipeline step can use the actions to add its own rules, for example a second approval whenever something that holds data would be destroyed:

# The actions per resource, as `tofu show -json tfplan` reported them
# for the Kestrel portal configuration (environment=prod)
changes = [
    ("local_file.upstream", ["delete", "create"]),
    ("random_id.deployment", ["no-op"]),
    ("random_password.db", ["no-op"]),
    ("terraform_data.release", ["update"]),
]

# Types whose destruction needs a second approval (data lives there)
PROTECTED = ("random_password.", "aws_db_instance.")

add = change = destroy = 0
needs_second_approval = []
for address, actions in changes:
    if actions == ["no-op"]:
        continue
    if "create" in actions:
        add += 1
    if actions == ["update"]:
        change += 1
    if "delete" in actions:
        destroy += 1
        if address.startswith(PROTECTED):
            needs_second_approval.append(address)
    kind = "replace" if len(actions) == 2 else actions[0]
    print(f"{kind:8} {address}")

print(f"Plan: {add} to add, {change} to change, {destroy} to destroy.")
print("Second approval needed for:", needs_second_approval or "nothing")

replace  local_file.upstream
update   terraform_data.release
Plan: 1 to add, 1 to change, 1 to destroy.
Second approval needed for: nothing

A replacement counts once as "to add" and once as "to destroy", the same way OpenTofu's own summary line counts it. Policy engines such as Open Policy Agent do this kind of check in a more general way; see Testing and Policy as Code.

A GitHub Actions workflow

Ravi Kumar's workflow for the kestrel-infra repository puts the pieces together for GitHub Actions. It runs OpenTofu through the community-maintained opentofu/setup-opentofu action; for Terraform, HashiCorp's hashicorp/setup-terraform plays the same role. Production state lives in an S3 bucket with native locking; the AWS account ID and role names are examples.

.github/workflows/infra.yml

name: infra

on:
  pull_request:
    paths: ["portal/**"]
  push:
    branches: [main]
    paths: ["portal/**"]

permissions:
  contents: read

jobs:
  plan:
    if: github.event_name == 'pull_request'
    runs-on: ubuntu-24.04
    permissions:
      contents: read
      id-token: write          # may request an OIDC token
    defaults:
      run:
        working-directory: portal
    steps:
      # Pin every action to the full commit SHA of the release you reviewed
      - uses: actions/checkout@<commit-sha>
      - uses: opentofu/setup-opentofu@<commit-sha>   # v2
        with:
          tofu_version: 1.13.1
          tofu_wrapper: false   # keep tofu's real exit codes (the wrapper turns 2 into 0)
      - uses: aws-actions/configure-aws-credentials@<commit-sha>
        with:
          role-to-assume: arn:aws:iam::111122223333:role/kestrel-tofu-plan
          aws-region: eu-west-1
      - run: tofu fmt -check -recursive
      - run: tofu init -input=false -lockfile=readonly
      - run: tofu validate -no-color
      - name: Speculative plan
        run: |
          tofu plan -input=false -no-color -lock=false -out=tfplan
          {
            echo '```'
            tofu show -no-color tfplan
            echo '```'
          } >> "$GITHUB_STEP_SUMMARY"

  apply:
    if: github.event_name == 'push'
    runs-on: ubuntu-24.04
    environment: production    # required reviewer: the release manager
    concurrency:
      group: tofu-prod
      cancel-in-progress: false
    permissions:
      contents: read
      id-token: write
    defaults:
      run:
        working-directory: portal
    steps:
      - uses: actions/checkout@<commit-sha>
      - uses: opentofu/setup-opentofu@<commit-sha>   # v2
        with:
          tofu_version: 1.13.1
          tofu_wrapper: false   # keep tofu's real exit codes (the wrapper turns 2 into 0)
      - uses: aws-actions/configure-aws-credentials@<commit-sha>
        with:
          role-to-assume: arn:aws:iam::111122223333:role/kestrel-tofu-apply-prod
          aws-region: eu-west-1
      - run: tofu init -input=false -lockfile=readonly
      - run: tofu plan -input=false -no-color -lock-timeout=5m -out=tfplan
      - run: tofu apply -input=false -no-color -lock-timeout=5m tfplan

• Two jobs, two roles. The plan job runs for every pull request with a role that can read, not change; the apply job runs only on main with the production role.
• environment: production pauses the apply job until a required reviewer approves it (see deployment approval). At Kestrel that is Yara Haddad, the release manager; the approval and the run log become part of the change record.
• concurrency queues a second apply behind the first instead of running both. The state lock would stop the second one too, but with an error; cancel-in-progress: false makes sure a running apply is never killed by a newer push.
• -lockfile=readonly makes init fail if the committed .terraform.lock.hcl does not match the configuration, so the pipeline never picks different provider versions than the ones reviewed.
• -lock=false on the speculative plan lets many pull requests plan at the same time without queuing behind each other or behind an apply. It is acceptable only because that plan is never applied; never use it on a plan or apply that changes anything.
• The plan in the job summary puts the readable plan next to the run, where reviewers find it. Teams that want it as a pull request comment add a step with pull-requests: write permission; keep comments to the text plan, never the JSON.

Here the apply job plans and applies in one go, after the approval. That keeps the example short, but the approver approves the job, not a specific plan. A stricter variant splits it in two: a job without an environment saves tfplan as an artifact and shows it, and the approved job downloads that file and only runs tofu apply tfplan. If anything changed in between, the "Saved plan is stale" check stops it.

Least-privilege credentials

An infrastructure pipeline holds the keys to everything it manages, which makes its credentials the most valuable secret in the organization. Three rules follow from least privilege:

1. No stored cloud keys. The workflow requests a short-lived token through OpenID Connect (OIDC), which the cloud exchanges for credentials valid for about an hour (see workload identity federation). Nothing can leak from the repository's secrets, and nothing needs rotating.
2. Separate plan and apply roles. A plan needs to read every managed resource and the state, but must not change resources. In practice "read-only" has two exceptions: the plan role can read the state, which can contain secrets (so protect who can run plans), and a locking plan needs write access to the lock file in the state bucket.
3. One apply role per environment, trusted only for that environment. The cloud's trust policy checks the token's subject claim, so the production role accepts only tokens from jobs that run in the protected production environment.

For Kestrel's repository, the subject claims that the roles trust look like this:

Drift detection

Drift is any difference between the code and reality, usually from a manual change during an incident (see configuration drift). The pipeline only notices it when someone plans. Drift detection is a scheduled job that plans against each environment, for example every night, and raises an alarm when the plan is not empty. A plan exiting with code 2 means either drift or code that was merged but never applied; both need a human to look.

In a test run, a line was added by hand to a file that OpenTofu manages. The next plan wanted to put the file back, and tofu plan -detailed-exitcode exited with code 2 (recorded with OpenTofu 1.13):

echo '    server 192.0.2.13:8080;  # added by hand' >> out/portal-upstream.conf
tofu plan -var environment=test -detailed-exitcode > /dev/null

tofu plan -refresh-only shows only what changed outside OpenTofu, under the heading "Objects have changed outside of OpenTofu", without mixing in changes from the code; that output is on Changing Infrastructure Safely. A drift job as a GitHub Actions job:

  drift:
    if: github.event_name == 'schedule'
    runs-on: ubuntu-24.04
    permissions:
      contents: read
      id-token: write
    defaults:
      run:
        working-directory: portal
    steps:
      - uses: actions/checkout@<commit-sha>
      - uses: opentofu/setup-opentofu@<commit-sha>   # v2
        with:
          tofu_version: 1.13.1
          tofu_wrapper: false   # keep tofu's real exit codes (the wrapper turns 2 into 0)
      - uses: aws-actions/configure-aws-credentials@<commit-sha>
        with:
          role-to-assume: arn:aws:iam::111122223333:role/kestrel-tofu-plan
          aws-region: eu-west-1
      - run: tofu init -input=false -lockfile=readonly
      - name: Plan against production
        run: |
          set +e
          tofu plan -input=false -no-color -detailed-exitcode -lock-timeout=5m
          code=$?
          set -e
          if [ "$code" -eq 2 ]; then
            echo "::error::Production differs from main: drift or unapplied changes"
          fi
          exit "$code"

The job needs schedule: [{cron: "0 5 * * *"}] under on: (GitHub runs schedules in UTC). GitHub Actions runs run: steps with bash -e, which is why the step switches -e off while it captures the exit code. The setup step sets tofu_wrapper: false: the action's default wrapper reports exit code 2 as success, which would hide every drift. A failed drift job should land where someone acts on it — at Kestrel, an incident ticket for the NOC — not in a mailbox nobody reads.

Atlantis and managed platforms

Not every team builds this from workflow files. Atlantis is a self-hosted application that listens to pull request webhooks, runs plan for every changed project, posts the output as a comment, and runs apply when an authorized reviewer comments atlantis apply; it locks each project while a pull request holds an unapplied plan. Atlantis is open source (Apache 2.0), a CNCF Sandbox project, at v0.48.0 as of October 2026. It works with both Terraform and OpenTofu.

Hosted platforms do the same as a service. HashiCorp's HCP Terraform runs speculative plans on pull requests, applies after approval, stores state and evaluates policies. As of October 2026 its legacy free plan has ended (31 March 2026); the free tier now covers up to 500 managed resources with unlimited users. Commercial alternatives exist for both Terraform and OpenTofu; compare their prices per resource or run before you commit.

A different model again is pull-based: a controller in a Kubernetes cluster watches Git and keeps reconciling, the way GitOps works for applications (see push and pull deployment). For infrastructure that is less common, but Crossplane follows this model (see Other Infrastructure as Code Tools).

Ansible in a pipeline

Configuration management follows the same shape: check on the pull request, run after merge, with approval for production.

• Pull request: ansible-playbook --syntax-check, ansible-lint and Molecule; then a dry run against the test inventory with ansible-playbook site.yml --check --diff, so the review shows the file changes that would be made. Remember that check mode skips command and shell tasks, so it is a forecast, not a guarantee (see Playbooks and Idempotency).
• After merge: the real run, first against test, then — after approval — against production, often with --limit to one web server before the rest.
• Secrets: the Ansible Vault password comes from the pipeline's secret store into a file only for that job, and the SSH key for the managed nodes is a deploy key used only by the pipeline (see Secrets in Automation).
• Recap as a gate: the exit code is 2 when a host failed and 4 when a host was unreachable, so the pipeline stops on its own; publish the PLAY RECAP in the job summary.

Ansible runs from a pipeline need network access to the managed nodes, which hosted CI runners usually do not have. A self-hosted runner inside the management network, or a controller such as AWX or Red Hat Ansible Automation Platform, is the usual answer.

Pipelines and change management

A production apply is a change, and the pipeline produces most of the change record by itself: the pull request says what and why, the plan says exactly which resources change, the approval says who agreed, and the run log says when it happened and how it ended. Many organizations therefore treat small, well-tested infrastructure changes as standard changes, and reserve a full change review for plans that replace or destroy resources (see change types and CI/CD and change enablement).

The mistakes that come up most often in reviews:

• Applying a fresh plan after merge instead of the reviewed or approved one.
• -auto-approve in a job without an approval gate in front of it.
• A plan job that holds write credentials.
• Two pipelines (or a pipeline and a person) applying to the same state.
• Plan files or JSON with secrets kept as public artifacts.
• A drift job whose failures nobody reads.