State: How Terraform Remembers

What the state file holds

The Cloud topic introduces Terraform state: a JSON file in which the tool records which real object belongs to which block in the code. This page goes further: where that file should live, who may read it, and what to do when it goes wrong. Everything here applies to Terraform and OpenTofu alike unless the text says otherwise; the commands are the same with terraform instead of tofu.

Ravi Kumar, the portal tech lead at Kestrel Energy, manages the portal's configuration files and a generated database password with OpenTofu. After a few applies, the working directory looks like this:

$ cd /home/amara/kestrel-infra/portal
$ ls
$ head -12 terraform.tfstate

main.tf
out
outputs.tf
templates
terraform.tfstate
terraform.tfstate.backup
variables.tf
versions.tf
{
  "version": 4,
  "terraform_version": "1.13.1",
  "serial": 5,
  "lineage": "78dad6fb-d82a-c52c-e674-ac52c60b24c0",
  "outputs": {
    "db_password": {
      "value": "cyJYJeiYc9v95H3k8plU",
      "type": "string",
      "sensitive": true
    },
    "deployment_id": {

The top of the file already tells a lot:

• version is the format of the state file; terraform_version is the version of the tool that last wrote it (OpenTofu keeps the field name for compatibility).
• serial goes up by one with every write. lineage is a random ID given to the state when it was first created; it stops you from accidentally pushing the state of one configuration over another one.
• outputs holds the output values — including the one marked "sensitive": true, in plain text.
• Further down, resources lists every managed object: its address (random_password.db), the provider that manages it, its ID in the real world and every attribute the provider reported.

terraform.tfstate.backup is the previous version, written just before each change. Both files belong in .gitignore.

Three views of the same infrastructure

Every plan involves three descriptions. The configuration says what should exist. The state says what the tool created last time and how it looked then. The real infrastructure is what exists now. During a plan the tool first refreshes: it asks the provider for the current attributes of every object in the state. Then it compares the result with the configuration.

This is why a resource that someone deletes from the state (but not from the real world) is suddenly "new" to the tool: it will try to create it again, and that often fails because the name is taken. It is also why an object that someone creates by hand is invisible until it is imported. The state is the tool's memory, nothing more.

Backends: where the state lives

A state backend decides where the state is stored and how it is locked. Without a backend block the local backend is used: the terraform.tfstate file in the working directory. That is fine for learning and for a single person's experiments, and nothing else: the file lives on one laptop, is lost with it, and a colleague who runs the same code from a fresh clone starts with empty state and tries to create everything again.

Common backends are s3 (Amazon S3 and S3-compatible object stores), azurerm (an Azure Storage blob container), gcs (Google Cloud Storage), pg (a PostgreSQL database), http (any server that implements a simple REST protocol) and, for Terraform, HCP Terraform. The production state of the Kestrel portal lives in a private, versioned bucket. The block below is illustrative:

backend.tf

terraform {
  backend "s3" {
    bucket       = "kestrel-tofu-state"
    key          = "portal/prod.tfstate"
    region       = "eu-central-1"
    encrypt      = true
    use_lockfile = true
  }
}

• key is the path of this configuration's state inside the bucket. Each configuration and each environment gets its own key (see Environments: Dev, Test and Production).
• encrypt = true asks S3 for server-side encryption at rest. That protects the disks, not the content: anyone allowed to read the object still reads plain text.
• use_lockfile = true turns on locking with a small lock object next to the state. It became generally available in Terraform 1.11 (which deprecated the separate DynamoDB table older tutorials use) and arrived in OpenTofu 1.10.
• No access keys appear in the block. The backend takes credentials from the environment — a signed-in session, or the pipeline's short-lived OpenID Connect credentials.

After adding or changing a backend block, run tofu init again. When state already exists locally, init notices the move and offers to copy it to the new backend (-migrate-state); answer yes only after checking that the target key is the right one.

Remote state and reading other states

Remote state is state kept in a shared backend instead of a local file, so that every person and every pipeline run works from the same copy. A second meaning shows up in code: one configuration can read the outputs of another with the terraform_remote_state data source — for example, the portal configuration reading the network IDs that the network configuration exports.

That convenience has a catch. To read the outputs, the reader needs read access to the whole state object, including every secret in it. Where possible, publish the values a neighbor needs somewhere less sensitive (a parameter store, DNS, a data source that looks the network up by its tags) and keep read access to state for the people and pipelines that really apply that configuration.

State locking

State locking stops two runs from changing the same state at the same time. Every command that may write the state first takes a lock; a second run that finds the lock taken stops instead of reading a half-written state. Without it, two applies could each create a server and only one of them would be remembered — the other becomes an orphan that nobody manages. Not every backend supports locking; the ones above do.

Hamid Rahimi, the database administrator, runs a configuration in which a slow resource stands in for a database migration on db01 (here a time_sleep of 25 seconds). While his apply is still running, a second apply starts in another terminal:

$ tofu apply -auto-approve -var release=2026.10.2

╷
│ Error: Error acquiring the state lock
│
│ Error message: resource temporarily unavailable
│ Lock Info:
│   ID:        59a466c2-1a7b-c2e1-5aa0-b69f4af768c3
│   Path:      terraform.tfstate
│   Operation: OperationTypeApply
│   Who:       amara@control01
│   Version:   1.13.1
│   Created:   2026-10-06 16:00:45.221332488 +0000 UTC
│   Info:
│
│
│ OpenTofu acquires a state lock to protect the state from being written
│ by multiple users at the same time. Please resolve the issue above and try
│ again. For most commands, you can disable locking with the "-lock=false"
│ flag, but this is not recommended.
╵

The second run exits with status 1 and changes nothing. The Lock Info box is what you need to decide what to do: the lock ID, the operation that holds it, who (user and host) and since when. With the local backend the lock is an operating system file lock on terraform.tfstate, plus a small .terraform.tfstate.lock.info file that describes it:

$ cat .terraform.tfstate.lock.info

{"ID":"59a466c2-1a7b-c2e1-5aa0-b69f4af768c3","Operation":"OperationTypeApply","Info":"","Who":"amara@control01","Version":"1.13.1","Created":"2026-10-06T16:00:45.221332488Z","Path":"terraform.tfstate"}

Waiting is often the right answer. -lock-timeout makes a command retry for a while before giving up, which is useful in pipelines where two jobs may briefly overlap:

$ tofu plan -lock-timeout=5s

Acquiring state lock. This may take a few moments...
╷
│ Error: Error acquiring the state lock
…

When a run dies holding the lock

A run can be killed halfway — a canceled pipeline job, a closed laptop, kill -9. In a test run the first apply was killed while it showed Creating.... The lock-info file stayed behind, but the operating system lock died with the process, so the next command simply worked:

$ tofu state list
$ tofu plan -var release=2026.10.4

time_sleep.db_migration
time_sleep.db_migration: Refreshing state... [id=2026-10-06T16:01:44Z]

OpenTofu used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement

OpenTofu will perform the following actions:

  # time_sleep.db_migration must be replaced
-/+ resource "time_sleep" "db_migration" {
      ~ id              = "2026-10-06T16:01:44Z" -> (known after apply)
      ~ triggers        = { # forces replacement
          ~ "release" = "2026.10.3" -> "2026.10.4"
        }
        # (1 unchanged attribute hidden)
    }

Plan: 1 to add, 0 to change, 1 to destroy.
…

Look closely: the state still says release 2026.10.3, although the killed apply had already destroyed that object. A killed run can leave state and reality out of step. The next plan's refresh notices and repairs what it can — here by planning a replacement — which is why you re-plan after a crash instead of touching the state.

Remote backends behave differently: there the lock is a record in the backend, and a crashed run can leave it behind. The command for that case is tofu force-unlock LOCK_ID, used only after you have made sure the run that held the lock is really gone. Troubleshooting Infrastructure as Code walks through it. It does not apply to local state, as a test run showed while another process was still applying:

$ tofu force-unlock f2b6c5e0-ac2b-79de-02c7-19ae9e91be10

╷
│ Error: State locked by another local process
│
│ Local state cannot be unlocked by another process.
╵

Sensitive values are still in the state

A sensitive value is a variable, output or resource attribute marked as secret (with sensitive = true, or by the provider). The tool then hides it in plans and normal output, showing (sensitive value) or <sensitive> instead. That is all it does. The OpenTofu documentation is blunt: sensitive values are still recorded in the state, and anyone who can read the state can read them.

The portal's database password is generated by a random_password resource and exposed as a sensitive output. On screen it is hidden; in the file it is not:

$ grep -n '"result"' terraform.tfstate

103:            "result": "cyJYJeiYc9v95H3k8plU",
117:                "value": "result"

The same is true for private keys from tls_private_key, database passwords passed into a cloud database resource, and connection strings that a provider returns. Saved plan files contain the values too. Treat state (and plan files) as secrets:

• Never commit *.tfstate, *.tfstate.backup or plan files to Git (see what never belongs in Git), and never mail or paste them into a chat or ticket.
• Restrict read access to the state bucket or container to the people and pipeline roles that apply that configuration — least privilege, not "everyone in IT, because it is only state".
• Turn on versioning on the bucket, so a broken write can be rolled back, and access logging, so reads can be audited.

Keeping secrets out: ephemeral values

Newer versions give you ways to stop the secret from reaching the state at all. They need Terraform 1.10/1.11 or OpenTofu 1.11, and a provider that implements them.

An ephemeral resource (declared with an ephemeral block instead of resource) produces a value that exists only during one run — reading a secret from a secrets manager, or generating a password — and is never stored in state or plan. Ephemeral values may only flow into places that are not stored either: other ephemeral resources, ephemeral variables and outputs, locals, provider configuration, provisioners and connection blocks, and write-only attributes. Using one anywhere else is an error.

A write-only attribute is a resource argument that the provider sends to the real system but never stores: in state and plan it is always null. Because the tool cannot see the old value, such arguments come with a companion version argument that you raise when the secret should be sent again. An illustrative example with the AWS provider and hashicorp/random (whose ephemeral random_password appeared in the 3.7 series):

ephemeral "random_password" "portal_db" {
  length  = 24
  special = false
}

resource "aws_secretsmanager_secret_version" "portal_db" {
  secret_id                = aws_secretsmanager_secret.portal_db.id
  secret_string_wo         = ephemeral.random_password.portal_db.result
  secret_string_wo_version = 1
}

The password is generated during the apply, written into the secrets manager, and forgotten. The state records only that version 1 was written. To rotate it, raise secret_string_wo_version to 2. Which resources offer write-only arguments (they end in _wo by convention) is up to each provider; check its documentation. Secrets in Automation covers where such secrets should live.

State encryption in OpenTofu

State encryption is OpenTofu's client-side encryption of state and plan files (since OpenTofu 1.7): the file is encrypted before it leaves your machine or pipeline, so the backend — and anyone who can read the bucket — only sees ciphertext. Terraform itself does not encrypt state; it relies on the backend's encryption and access control.

The configuration has two parts. A key provider supplies the key: pbkdf2 derives one from a passphrase (at least 16 characters), and aws_kms, gcp_kms, azure_vault, openbao and external fetch one from a key management service or a program. A method encrypts with it: aes_gcm (AES in Galois/Counter Mode, symmetric encryption that also detects tampering). Joost Bakker's billing configuration already has plaintext state with a generated password in it. The target configuration:

encryption.tf

variable "state_passphrase" {
  type      = string
  sensitive = true
}

terraform {
  encryption {
    key_provider "pbkdf2" "kestrel" {
      passphrase = var.state_passphrase
    }

    method "aes_gcm" "kestrel" {
      keys = key_provider.pbkdf2.kestrel
    }

    state {
      method = method.aes_gcm.kestrel
    }

    plan {
      method = method.aes_gcm.kestrel
    }
  }
}

The passphrase comes from an environment variable (TF_VAR_state_passphrase), never from a file in Git. Adding this block to a configuration that already has plaintext state fails, with a confusing message:

$ export TF_VAR_state_passphrase=example-only-passphrase
$ tofu plan

╷
│ Error: Error acquiring the state lock
│
│ Error message: failed to write backup file: encountered unencrypted payload
│ without unencrypted method configured
│
│ OpenTofu acquires a state lock to protect the state from being written
│ by multiple users at the same time. Please resolve the issue above and try
│ again. For most commands, you can disable locking with the "-lock=false"
│ flag, but this is not recommended.
╵

Nobody else holds a lock here. In OpenTofu 1.13.1 problems with reading encrypted (or not-yet-encrypted) state surface inside the state-lock error; the real cause is the line after Error message:. The existing state is plaintext, and the configuration does not allow reading plaintext.

Migrating existing state

The fix is a one-off migration: add the unencrypted method as a fallback, so OpenTofu may read the old plaintext state once and write it back encrypted.

variable "state_passphrase" {
  type      = string
  sensitive = true
}

terraform {
  encryption {
    key_provider "pbkdf2" "kestrel" {
      passphrase = var.state_passphrase
    }

    method "aes_gcm" "kestrel" {
      keys = key_provider.pbkdf2.kestrel
    }

    # Only for the migration run: allows reading the old plaintext state.
    method "unencrypted" "migrate" {}

    state {
      method = method.aes_gcm.kestrel

      fallback {
        method = method.unencrypted.migrate
      }
    }

    plan {
      method = method.aes_gcm.kestrel

      fallback {
        method = method.unencrypted.migrate
      }
    }
  }
}

$ tofu apply -auto-approve
$ head -c 400 terraform.tfstate
$ grep -c 'OTG5' terraform.tfstate

╷
│ Warning: Unencrypted method configured
│
│   on encryption.tf line 7, in terraform:
│    7:   encryption {
│
│ Method unencrypted is present in configuration. This is a security risk and
│ should only be enabled during migrations.
╵
random_password.billing_db: Refreshing state... [id=none]

No changes. Your infrastructure matches the configuration.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

billing_db_password = <sensitive>
{"serial":1,"lineage":"31d4568a-73ca-4c57-0f84-54b8bbc040c3","meta":{"key_provider.pbkdf2.kestrel":"eyJzYWx0IjoiY3hGb1FoV0FmYS9MR1krNzJwbFpHdUpkNXZObXBBQVFaRWZyblVkS0cvWT0iLCJpdGVyYXRpb25zIjo2MDAwMDAsImhhc2hfZnVuY3Rpb24iOiJzaGE1MTIiLCJrZXlfbGVuZ3RoIjozMn0="},"encrypted_data":"TLDb034b2O1GjKzcWOtqNglW9WkApwoPHpxK4k59HDolFX412RK49pG1ldNnUJDxRqwL3w98A0X1b7FPWRZA8wztZaT2uOO0KBr/NJlWy1tXnRT7ArPxxn2RjWs
0

The apply changed no infrastructure, but it rewrote the state. The file is still JSON, with only a serial, a lineage, a meta field and an opaque encrypted_data blob. The meta value is not secret: decoded from Base64 it holds the key derivation settings ({"salt":"…","iterations":600000,"hash_function":"sha512","key_length":32}, the defaults). The password that started with OTG5 no longer appears anywhere (grep -c counts 0 lines).

Then remove the unencrypted method and the fallbacks again, so plaintext state is refused from now on, and check that everything still works:

$ tofu plan
$ tofu output -raw billing_db_password

Acquiring state lock. This may take a few moments...
random_password.billing_db: Refreshing state... [id=none]

No changes. Your infrastructure matches the configuration.
OTG5OCmXqvWUARTkpAmJhCaT

Encryption protects the file, not the person who holds the key: with the right passphrase, tofu output still prints the secret. With a wrong passphrase, the authentication check of AES-GCM fails — again wrapped in a state-lock error:

$ export TF_VAR_state_passphrase=wrong-passphrase-123
$ tofu plan

Acquiring state lock. This may take a few moments...
╷
│ Error: Error acquiring the state lock
│
│ Error message: failed to write backup file: decryption failed for all
│ provided methods
│ attempted decryption failed for state: decryption failed: cipher: message
│ authentication failed
…

In a pipeline the whole encryption configuration can also come from the TF_ENCRYPTION environment variable instead of a file; OpenTofu merges it with the code. Setting enforced = true inside the state and plan blocks makes OpenTofu refuse to write unencrypted files even if someone later adds an unencrypted method.

State commands

The tofu state (or terraform state) subcommands inspect and change the state without touching the real infrastructure:

state mv and state rm change the state immediately, from one person's terminal, without a plan and without review. The moved, removed and import blocks described in Changing Infrastructure Safely do the same through a normal plan that a colleague can read in a pull request. Use the commands when there is no other way, and take a backup first:

$ tofu state pull > backup-2026-10-06.tfstate

That backup contains every secret in the state: keep it somewhere protected and delete it when the work is done.

Protecting state: a checklist

• Remote, with locking, one state per configuration and per environment.
• Private and versioned storage, with access logging; read access only for the people and pipeline roles that apply that configuration.
• No state, backups or plan files in Git, mail, tickets or AI chats.
• Secrets kept out where possible with ephemeral values and write-only attributes; with OpenTofu, state and plan encryption with a key from a key management service.
• One writer at a time: locking, plus a pipeline that runs one apply per state (see Infrastructure as Code in Pipelines).
• Changes through code: moved, removed and import blocks rather than hand-run state commands.