Variables, Facts and Templates

Where variables are defined

The playbook on Playbooks and Idempotency uses {{ nginx_worker_connections }}, {{ portal_port }} and {{ kestrel_env }} but never says what they are. That is deliberate: the tasks describe how a web server is configured, and variables say with which values for each host. The same playbook then serves test and production, and two servers that should differ in one setting differ only in one line of a variable file. A variable in Ansible is a name for a value, as in any programming language; what is special is the number of places it can come from.

Kestrel's Ansible directory:

ansible/
├── ansible.cfg
├── inventory/
│   └── hosts.ini
├── group_vars/
│   ├── all.yml             kestrel_root, kestrel_env, portal_port
│   ├── web.yml             nginx_worker_connections: 1024
│   └── db/
│       ├── vars.yml        database settings
│       └── vault.yml       encrypted secrets (Ansible Vault)
├── host_vars/
│   └── web02.yml           nginx_worker_connections: 2048
├── templates/
│   └── portal-upstream.conf.j2
└── site.yml

group_vars

group_vars is the directory whose files hold the variables for a group of hosts: Ansible loads group_vars/web.yml for every host in the group web, and group_vars/all.yml for every host. Instead of one file, a group can have a directory (group_vars/db/) whose files are all loaded; that is how Kestrel keeps the encrypted secrets for db01 in a separate vault.yml (see Secrets in Automation). group_vars/all.yml holds the values every host shares:

# Every managed host is simulated by a folder below hosts/.
kestrel_root: "{{ playbook_dir }}/hosts/{{ inventory_hostname }}"
kestrel_env: test
portal_port: 8080

(kestrel_root exists only because the recordings on these pages run on one test machine; on real servers it would be an empty string or /.)

host_vars

host_vars is the matching directory for single hosts: host_vars/web02.yml applies only to web02. web02 has more memory than web01, so it gets more nginx worker connections:

$ cd /home/amara/kestrel-infra/ansible
$ cat group_vars/web.yml host_vars/web02.yml

nginx_worker_connections: 1024
nginx_worker_connections: 2048

Ansible looks for group_vars/ and host_vars/ next to the inventory file and next to the playbook; Kestrel keeps them next to the playbooks. Variables can also be written in the inventory itself (as ansible_host is), but keep the inventory for "which hosts and how to reach them" and the variable directories for configuration.

Variable precedence

Variable precedence is the fixed order Ansible uses to decide which value wins when the same variable is defined in more than one place. The docs sum it up: Ansible "gives precedence to variables that were defined more recently, more actively, and with more explicit scope". A small playbook shows where nginx_worker_connections comes from:

- name: Where does nginx_worker_connections come from?
  hosts: web
  gather_facts: false

  tasks:
    - name: Show the value per host
      ansible.builtin.debug:
        var: nginx_worker_connections

$ ansible-playbook precedence.yml

PLAY [Where does nginx_worker_connections come from?] **************************

TASK [Show the value per host] *************************************************
ok: [web01] => {
    "nginx_worker_connections": 1024
}
ok: [web02] => {
    "nginx_worker_connections": 2048
}

PLAY RECAP *********************************************************************
web01                      : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
web02                      : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

web01 has only the group value; for web02 the host value wins over the group value. ansible-inventory --host shows the inventory and variable-file values a host ends up with, which is the first thing to check when a value is not what you expect:

$ ansible-inventory --host web02

{
    "ansible_connection": "local",
    "ansible_host": "192.0.2.12",
    "kestrel_env": "test",
    "kestrel_root": "{{ playbook_dir }}/hosts/{{ inventory_hostname }}",
    "nginx_worker_connections": 2048,
    "portal_port": 8080
}

kestrel_root is still the raw template: Ansible renders templates when a value is used, not when it is loaded.

The full order

The Ansible documentation lists 22 levels, from lowest to highest. You will rarely need more than a handful, but when a value is mysterious, this is the list to check:

1. command-line values such as -u deploy (these set connection options, not variables)
2. role defaults (defaults/main.yml in a role)
3. group variables in an inventory file or script
4. inventory group_vars/all
5. playbook group_vars/all
6. inventory group_vars/*
7. playbook group_vars/*
8. host variables in an inventory file or script
9. inventory host_vars/*
10. playbook host_vars/*
11. host facts and cached set_fact values
12. play vars
13. play vars_prompt
14. play vars_files
15. role vars (vars/main.yml in a role)
16. block vars
17. task vars
18. include_vars
19. registered variables and set_fact
20. role (and include_role) parameters
21. include parameters
22. extra vars (-e), which always win

Precedence decides which whole value wins. Ansible does not merge dictionaries by default: a host that redefines one key of a dictionary loses the others. (A configuration setting can switch to merging, but the Ansible documentation warns against it: content written for one setting breaks under the other, and roles from others all assume the default.) This model walks a few of the levels for each host:

levels = [  # a few of Ansible's levels, lowest precedence first
    ("role defaults", {"nginx_worker_connections": 768}),
    ("group_vars/all", {"portal": {"port": 8080, "workers": 4, "tls": True}}),
    ("group_vars/web", {"nginx_worker_connections": 1024}),
    ("host_vars/web02", {"nginx_worker_connections": 2048, "portal": {"workers": 2}}),
]


def variables_for(host, extra_vars=None):
    """Walk the levels from low to high; a later definition replaces an earlier one."""
    values, origin = {}, {}
    for level, definitions in levels + [("extra vars", extra_vars or {})]:
        if level.startswith("host_vars/") and level != "host_vars/" + host:
            continue  # another host's file
        for name, value in definitions.items():
            values[name] = value  # replaced, never merged (also for dictionaries)
            origin[name] = level
    return values, origin


for host, extra in [("web01", None), ("web02", None),
                    ("web02", {"nginx_worker_connections": "512"})]:
    values, origin = variables_for(host, extra)
    print(host + (" with -e" if extra else ""))
    for name in sorted(values):
        print(f"  {name} = {values[name]!r}  ({origin[name]})")

web01
  nginx_worker_connections = 1024  (group_vars/web)
  portal = {'port': 8080, 'workers': 4, 'tls': True}  (group_vars/all)
web02
  nginx_worker_connections = 2048  (host_vars/web02)
  portal = {'workers': 2}  (host_vars/web02)
web02 with -e
  nginx_worker_connections = '512'  (extra vars)
  portal = {'workers': 2}  (host_vars/web02)

web02's portal dictionary has lost port and tls. To combine values on purpose, keep the defaults and the overrides under different names and merge them explicitly where you use them, with the combine filter: {{ portal_defaults | combine(portal_overrides) }}.

Extra vars

Extra vars are variables given on the command line with -e (--extra-vars), as key=value pairs, as JSON or YAML, or from a file with -e @file.yml. They have the highest precedence of all, so they override even role variables. That makes them right for one-off values (a release number from a pipeline) and wrong for configuration that should live in the repository. One real detail: key=value always passes a string:

$ ansible-playbook precedence.yml -e nginx_worker_connections=512
$ ansible-playbook precedence.yml -e '{"nginx_worker_connections": 512}' --limit web01

…
ok: [web01] => {
    "nginx_worker_connections": "512"
}
ok: [web02] => {
    "nginx_worker_connections": "512"
}
…
ok: [web01] => {
    "nginx_worker_connections": 512
}
…

"512" in quotes is text; 512 is a number. For a template that writes it into a file the difference is invisible, but a comparison such as nginx_worker_connections > 1000 fails on a string. Pass typed values as JSON, or convert where you use them (| int).

Facts

Ansible facts are the details Ansible discovers about a managed node: operating system and version, host name, network interfaces and addresses, memory, CPUs, mounted file systems, the Python version and much more. At the start of every play, unless the play says gather_facts: false, Ansible runs the setup module on each host and stores the result in the ansible_facts dictionary. You can run it ad hoc, with a filter to keep the output short:

$ ansible db01 -m setup -a "filter=ansible_distribution*"

db01 | SUCCESS => {
    "ansible_facts": {
        "ansible_distribution": "Debian",
        "ansible_distribution_file_parsed": true,
        "ansible_distribution_file_path": "/etc/os-release",
        "ansible_distribution_file_variety": "Debian",
        "ansible_distribution_major_version": "12",
        "ansible_distribution_minor_version": "11",
        "ansible_distribution_release": "bookworm",
        "ansible_distribution_version": "12.11"
    },
    "changed": false
}

Facts describe the machine where the module ran. In the test setup all three "hosts" use the local connection, so they report the same machine: ansible all -m setup -a "filter=ansible_hostname" returned "ansible_hostname": "control01" for web01, web02 and db01 alike. Against real servers each reports its own.

In playbooks, read facts from the ansible_facts dictionary, without the ansible_ prefix. This play gathers only the distribution facts and writes them into a generated file on db01:

- name: Use facts in a generated file
  hosts: db01
  gather_facts: false
  vars:
    services:
      portal: { port: 8080, public: true }
      meter-api: { port: 8443, public: true }
      billing: { port: 9100, public: false }

  tasks:
    - name: Gather only the distribution facts
      ansible.builtin.setup:
        filter:
          - ansible_distribution*

    - name: Write a service inventory file
      ansible.builtin.copy:
        dest: "{{ kestrel_root }}/etc/kestrel/services.yml"
        mode: "0644"
        content: |
          # Generated for {{ inventory_hostname }} ({{ ansible_facts['distribution'] }} {{ ansible_facts['distribution_version'] }})
          {{ services | to_nice_yaml(indent=2) }}

$ ansible-playbook facts.yml
$ cat hosts/db01/etc/kestrel/services.yml

PLAY [Use facts in a generated file] *******************************************

TASK [Gather only the distribution facts] **************************************
ok: [db01]

TASK [Write a service inventory file] ******************************************
changed: [db01]

PLAY RECAP *********************************************************************
db01                       : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

# Generated for db01 (Debian 12.11)
billing:
  port: 9100
  public: false
meter-api:
  port: 8443
  public: true
portal:
  port: 8080
  public: true

Gathering all facts costs a few seconds per host. Plays that do not use them set gather_facts: false, as Kestrel's plays do; plays that need a few call setup with a filter, as above.

Registered variables and set_fact

A registered variable holds the result of a task, saved with the register: keyword so that later tasks can use it: the command's rc (return code), stdout and stdout_lines, stderr, and the changed and failed flags. The pre-deployment check on the playbooks page registers the contents of the upstream file:

- name: Read the upstream file
  ansible.builtin.command: cat {{ kestrel_root }}/etc/nginx/conf.d/portal-upstream.conf
  register: upstream
  changed_when: false

- name: Show what was registered
  ansible.builtin.debug:
    var: upstream.stdout_lines

TASK [Show what was registered] ************************************************
ok: [web01] => {
    "upstream.stdout_lines": [
        "# Ansible managed",
        "upstream kestrel_portal {",
        "    server 192.0.2.11:8080;",
        "    server 192.0.2.12:8080;",
        "}"
    ]
}
…

A registered variable belongs to the host it ran on and lasts for the rest of the playbook run. set_fact creates a host variable from an expression, for values you compute once and use in several tasks:

- name: Work out how many portal backends there are
  ansible.builtin.set_fact:
    portal_backend_count: "{{ upstream.stdout_lines | select('search', 'server ') | list | length }}"

Both sit high in the precedence order: a registered result or a set_fact overrides values from group_vars and host_vars for the rest of the run. Choose names that cannot collide with configuration variables.

Magic variables

A magic variable is a variable that Ansible itself sets, describing the run rather than the machine. You cannot set them yourself; you read them. The useful ones:

groups and hostvars together let one host's configuration depend on the others — the inventory becomes the source of truth. This task on db01 lists the web servers' addresses:

- name: Web server addresses from the inventory
  ansible.builtin.debug:
    msg: "{{ groups['web'] | map('extract', hostvars, 'ansible_host') | join(', ') }}"

ok: [db01] => {
    "msg": "192.0.2.11, 192.0.2.12"
}

Templates

The ansible.builtin.template module renders a Jinja2 template on the control node, with all the host's variables, facts and magic variables available, and writes the result to the managed node — idempotently: if the rendered text equals the file that is there, nothing changes. Templates are the way to generate whole configuration files, the same idea as the configuration templates used for network devices. Kestrel's nginx upstream file, templates/portal-upstream.conf.j2:

# {{ ansible_managed }}
upstream kestrel_portal {
{% for host in groups['web'] %}
    server {{ hostvars[host].ansible_host }}:{{ portal_port }};
{% endfor %}
}

Rendered on web02 by the site.yml run:

# Ansible managed
upstream kestrel_portal {
    server 192.0.2.11:8080;
    server 192.0.2.12:8080;
}

• {{ … }} inserts a value, {% … %} runs a statement such as a loop or condition, and {# … #} is a comment.
• {{ ansible_managed }} renders a marker text ("Ansible managed" in ansible-core 2.19) that tells the next admin who opens the file not to edit it by hand.
• The loop over groups['web'] means the file follows the inventory: a third web server appears in the upstream automatically on the next run.

For files where a mistake locks you out, let the module check the result before it replaces the real file. validate runs a command on the rendered temporary file (%s) and only installs it if the command succeeds:

- name: Allow the deploy account to reload the portal
  ansible.builtin.template:
    src: kestrel-deploy.sudoers.j2
    dest: /etc/sudoers.d/kestrel-deploy
    mode: "0440"
    validate: visudo -cf %s
  become: true

ansible-core 2.19 overhauled templating. Two changes matter in daily work. Values keep their types (a number stays a number), and only strings from trusted sources — your playbooks, roles and variable files — are rendered as templates: if a command's output or a file read from a host happens to contain {{ … }}, it stays plain text. Templates inside templates are no longer rendered either. This closes a class of security holes, and it breaks some older tricks that built templates dynamically.

Filters

A Jinja2 filter transforms a value inside a template expression, written after a pipe: {{ value | filter }}, with arguments in parentheses and any number of filters chained. Jinja2 brings general filters (default, join, map, select, length); Ansible adds many for infrastructure work (to_nice_yaml, to_json, combine, dict2items, mandatory, regular-expression and path filters), and collections add more. This playbook tries a few on Kestrel's service list:

- name: Jinja2 filters on Kestrel data
  hosts: db01
  gather_facts: false
  vars:
    services:
      portal: { port: 8080, public: true }
      meter-api: { port: 8443, public: true }
      billing: { port: 9100, public: false }

  tasks:
    - name: Fall back to a default
      ansible.builtin.debug:
        msg: "Backup window: {{ backup_window | default('01:00-03:00') }}"

    - name: Render a structure as YAML
      ansible.builtin.debug:
        msg: "{{ services | to_nice_yaml }}"

    - name: Public services only
      ansible.builtin.debug:
        msg: "{{ services | dict2items | selectattr('value.public') | map(attribute='key') | list }}"

    - name: Web server addresses from the inventory
      ansible.builtin.debug:
        msg: "{{ groups['web'] | map('extract', hostvars, 'ansible_host') | join(', ') }}"

    - name: Use a variable that was never defined
      ansible.builtin.debug:
        msg: "Backups go to {{ backup_target }}"

$ ansible-playbook filters.yml

PLAY [Jinja2 filters on Kestrel data] ******************************************

TASK [Fall back to a default] **************************************************
ok: [db01] => {
    "msg": "Backup window: 01:00-03:00"
}

TASK [Render a structure as YAML] **********************************************
ok: [db01] => {
    "msg": "billing:\n    port: 9100\n    public: false\nmeter-api:\n    port: 8443\n    public: true\nportal:\n    port: 8080\n    public: true\n"
}

TASK [Public services only] ****************************************************
ok: [db01] => {
    "msg": [
        "portal",
        "meter-api"
    ]
}

TASK [Web server addresses from the inventory] *********************************
ok: [db01] => {
    "msg": "192.0.2.11, 192.0.2.12"
}

TASK [Use a variable that was never defined] ***********************************
[ERROR]: Task failed: Finalization of task args for 'ansible.builtin.debug' failed: Error while resolving value for 'msg': 'backup_target' is undefined

Task failed.
Origin: /home/amara/kestrel-infra/ansible/filters.yml:27:7
…
Error while resolving value for 'msg': 'backup_target' is undefined
Origin: /home/amara/kestrel-infra/ansible/filters.yml:29:14

27     - name: Use a variable that was never defined
28       ansible.builtin.debug:
29         msg: "Backups go to {{ backup_target }}"
                ^ column 14

fatal: [db01]: FAILED! => {"msg": "Task failed: Finalization of task args for 'ansible.builtin.debug' failed: Error while resolving value for 'msg': 'backup_target' is undefined"}

PLAY RECAP *********************************************************************
db01                       : ok=4    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   

• default('01:00-03:00') supplies a value when the variable is undefined, which makes it optional.
• to_nice_yaml sorts the keys; in a debug message the line breaks show as \n, but written to a file (as in the facts example) it is normal YAML.
• dict2items turns a dictionary into a list of key/value pairs, so selectattr and map can filter it.
• An undefined variable without a default stops the task. The 2.19 error is a chain of "caused by" origins (shortened here) that ends at the exact line and column of {{ backup_target }}. That failure is a feature: a configuration file with an empty value in it is worse than a run that stops. Use mandatory ({{ backup_target | mandatory }}) to make the intent explicit, and default only where an empty or fallback value is really fine.

Keeping variables manageable

• Prefix variables with their purpose or role (nginx_worker_connections, portal_port), so names from different roles cannot collide and a reader can guess where a variable is defined.
• Normal values in defaults, differences in group_vars. Environment differences (test versus production) belong in group variables or separate inventories per environment, not in conditions spread through the tasks.
• Use ansible_facts['…'], never the deprecated top-level fact names, in new code.
• Check before you guess. ansible-inventory --host <host> for inventory and variable files, a debug task with var: for the value at a given point in the play.
• No secrets in plain variable files. Passwords and keys go into an encrypted file and are referred to by name (see Secrets in Automation).