Inventory and ad-hoc commands

By the end you'll know

How to put groups inside groups, and see the result with ansible-inventory
How to give a group, or one server, its own settings, and which one wins
How to pick exactly the servers a command runs on, with patterns and --limit
How --become works, and why becoming a non-root user needs one extra package
How Ansible works on several servers at once, and how to slow it down
When to use command and when to use shell
How to find the module for a job, and its options, with ansible-doc

Before you start

You need: Lesson 1 done, with the three servers running. vagrant status should show all three running; if not, vagrant up, then vagrant ssh-config > ssh_config.
Get the new files: in ansible-practice, git pull. You'll have a longer inventory.ini, and two new folders, group_vars and host_vars.
Where commands run: your own computer's terminal, in ansible-practice.

The inventory is a map

Lesson 1's inventory was a list: two groups, three servers. Real inventories carry more. Which servers do what, which belong together, and what's different about each one: the web servers listen on a port, the database server has a database port, one server is special. Think of the inventory as the map, and ad-hoc commands as directions. The better the map, the shorter the directions: "every web server" instead of "web1 and web2 and web3 and...", and "its port" instead of looking it up for each one.

Groups of groups

Here's the new inventory.ini:

INI

# Which servers Ansible manages, in groups.
[web]
web1
web2 http_port=8080

[db]
db1

# A group of groups: every server that runs the bookshop.
[bookshop:children]
web
db

Two new things. [bookshop:children] makes a group whose members are other groups: every web server and every database server is in bookshop. Add a third web server to [web], and it's in bookshop too, with nothing else to change. And http_port=8080 after web2 is a setting for that one server, which the next section explains.

ansible-inventory --graph shows how Ansible reads it:

BASH

ansible-inventory --graph

OUTPUT

@all:
  |--@ungrouped:
  |--@bookshop:
  |  |--@web:
  |  |  |--web1
  |  |  |--web2
  |  |--@db:
  |  |  |--db1

@ marks a group. Every server is in all, and ungrouped holds any server listed outside a group: none here. The new group works like any other:

BASH

ansible bookshop -m ping -o

OUTPUT

web2 | SUCCESS => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python3.12"},"changed": false,"ping": "pong"}
db1 | SUCCESS => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python3.12"},"changed": false,"ping": "pong"}
web1 | SUCCESS => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python3.12"},"changed": false,"ping": "pong"}

Settings for groups, and for servers

Variables are named settings. Ansible looks for them in files next to the inventory, by name:

group_vars/all.yml applies to every server.
group_vars/web.yml applies to every server in the web group, and so on for any group.
host_vars/db1.yml applies to db1 only.

OUTPUT

==> group_vars/all.yml <==
# Settings for every server.
admin_email: ops@bookshop.example

==> group_vars/web.yml <==
# Settings for the web servers.
http_port: 80
site_name: bookshop

==> host_vars/db1.yml <==
# Settings for db1 only.
db_port: 5432

They're YAML: name: value, one per line, the same format as Kubernetes files and GitHub Actions. Add --vars to the graph, and every server shows what it ends up with:

BASH

ansible-inventory --graph --vars

OUTPUT

@all:
  |--@ungrouped:
  |--@bookshop:
  |  |--@web:
  |  |  |--web1
  |  |  |  |--{admin_email = ops@bookshop.example}
  |  |  |  |--{http_port = 80}
  |  |  |  |--{site_name = bookshop}
  |  |  |--web2
  |  |  |  |--{admin_email = ops@bookshop.example}
  |  |  |  |--{http_port = 8080}
  |  |  |  |--{site_name = bookshop}
  |  |  |--{http_port = 80}
  |  |  |--{site_name = bookshop}
  |  |--@db:
  |  |  |--db1
  |  |  |  |--{admin_email = ops@bookshop.example}
  |  |  |  |--{db_port = 5432}
  |--{admin_email = ops@bookshop.example}

Read it per server. Every one gets admin_email, from all. The web servers get site_name and http_port from web. db1 gets its own db_port. And web2's http_port is 8080, not the group's 80.

The more specific setting wins. A setting for one server beats a setting for its group, which beats a setting for all. That's what lets a group share sensible defaults while one server differs. The debug module prints a variable for each server, to check:

BASH

ansible web -m debug -a "var=http_port"

OUTPUT

web1 | SUCCESS => {
    "http_port": 80
}
web2 | SUCCESS => {
    "http_port": 8080
}

debug can also print a message built from variables. {{ ... }} inserts a value, the template language Lesson 5 is about:

BASH

ansible all -m debug -a "msg='{{ inventory_hostname }} is in {{ group_names }}, mail {{ admin_email }}'"

OUTPUT

web1 | SUCCESS => {
    "msg": "web1 is in ['bookshop', 'web'], mail ops@bookshop.example"
}
web2 | SUCCESS => {
    "msg": "web2 is in ['bookshop', 'web'], mail ops@bookshop.example"
}
db1 | SUCCESS => {
    "msg": "db1 is in ['bookshop', 'db'], mail ops@bookshop.example"
}

inventory_hostname and group_names are variables Ansible provides itself: the server's name in the inventory, and the groups it's in. And a variable nobody set:

BASH

ansible db -m debug -a "var=http_port"

OUTPUT

db1 | SUCCESS => {
    "http_port": "VARIABLE IS NOT DEFINED!"
}

db1 isn't a web server, so it has no http_port. In a playbook, using an undefined variable stops the run with an error, which is usually what you want: better than a config file with an empty port. Finally, ansible-inventory --host gives one server's settings, all merged:

BASH

ansible-inventory --host web2

OUTPUT

{
    "admin_email": "ops@bookshop.example",
    "http_port": 8080,
    "site_name": "bookshop"
}

When a server behaves differently from the rest and you don't know why, this is the first thing to run.

Choosing servers: patterns

The first word after ansible is a pattern: which servers to run on. A group name is the simplest. They combine:

BASH

ansible "web:db" --list-hosts
ansible "all:!db" --list-hosts
ansible "bookshop:&web" --list-hosts

OUTPUT

  hosts (3):
    web1
    web2
    db1
  hosts (2):
    web1
    web2
  hosts (2):
    web1
    web2

web:db: in web or db. Both groups together.
all:!db: everything except db. Useful for "every server but the database".
bookshop:&web: in bookshop and in web. Servers in both.

Quote patterns, as here: ! and & mean something to the shell. Names can be matched too:

BASH

ansible "web1*" --list-hosts
ansible "web[1]" --list-hosts

OUTPUT

  hosts (1):
    web1
  hosts (1):
    web2

* matches any characters, so web1* would also match a future web10. web[1] picks the group's member at position 1, counting from 0, so it's web2, not web1: a classic surprise.

--limit narrows any pattern further, and is how you usually aim at one or two servers. It matters more in playbooks, which name their own groups:

BASH

ansible all --limit db1 -m ping -o
ansible all --limit "web1,db1" -m ping -o

OUTPUT

db1 | SUCCESS => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python3.12"},"changed": false,"ping": "pong"}

db1 | SUCCESS => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python3.12"},"changed": false,"ping": "pong"}
web1 | SUCCESS => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python3.12"},"changed": false,"ping": "pong"}

Before running anything that changes servers, check the pattern with --list-hosts. It runs nothing, and tells you exactly which servers would be touched.

Becoming root, and becoming someone else

Ansible logs in as vagrant. --become switches to root for the command, through sudo:

BASH

ansible db1 -m command -a "whoami"
ansible db1 -m command -a "whoami" --become

OUTPUT

db1 | CHANGED | rc=0 >>
vagrant

db1 | CHANGED | rc=0 >>
root

Without it, anything that needs root fails, with the same error you'd get by hand:

BASH

ansible db1 -m command -a "cat /etc/shadow"
ansible db1 -m command -a "head -1 /etc/shadow" --become

OUTPUT

db1 | FAILED | rc=1 >>
cat: /etc/shadow: Permission deniednon-zero return code

db1 | CHANGED | rc=0 >>
root:*:20306:0:99999:7:::

(non-zero return code is Ansible's note on the end of the error, run together with it.) Use --become only for what needs it, the way you'd use sudo: it's the same principle as Lesson 11 of the SQL track, least privilege.

--become-user becomes someone other than root, like a service's own user. Try nobody, a user with almost no rights:

BASH

ansible db1 -m command -a "whoami" --become --become-user nobody

OUTPUT

db1 | FAILED | rc=-1 >>
Failed to set permissions on the temporary files Ansible needs to create when becoming an unprivileged user (rc: 1, err: chmod: invalid mode: ‘A+user:nobody:rx:allow’
Try 'chmod --help' for more information.
}). For information on working around this, see https://docs.ansible.com/ansible-core/2.18/playbook_guide/playbooks_privilege_escalation.html#risks-of-becoming-an-unprivileged-user

This is the most common surprise in Ansible, so it's worth understanding. To run a module, Ansible copies it to a temporary file as vagrant, then runs it as nobody. But nobody isn't allowed to read vagrant's files. Ansible has to grant that one user access to that one file, and the safe way is an access control list (ACL): an extra permission for one named user, beyond the owner, group and others from Linux Lesson 5. On Linux, that needs the setfacl command, from the acl package, which these minimal servers don't have. So Ansible fell back to a chmod style other systems use, which Linux's chmod doesn't understand. Install acl everywhere:

BASH

ansible all -m apt -a "name=acl state=present" --become -o

OUTPUT

web2 | CHANGED => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python3.12"},"cache_update_time": 176125
web1 | CHANGED => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python3.12"},"cache_update_time": 176125
db1 | CHANGED => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python3.12"},"cache_update_time": 1761257

(Lines cut short with cut -c1-120.) And try again:

BASH

ansible db1 -m command -a "whoami" --become --become-user nobody

OUTPUT

[WARNING]: Unable to use /nonexistent/.ansible/tmp as temporary directory,
failing back to system: [Errno 13] Permission denied: '/nonexistent'
db1 | CHANGED | rc=0 >>
nobody

nobody. It works. The warning is harmless: nobody's home folder is literally called /nonexistent, so Ansible used the system's temporary folder instead. Every playbook in this track installs acl early, for this reason.

Doing more at once: forks

Ansible works on several servers at the same time. How many is set by forks: 5 by default. Time a 2-second command on all three servers, then one at a time:

BASH

time ansible all -m command -a "sleep 2" -o
time ansible all -m command -a "sleep 2" -o --forks 1

OUTPUT

db1 | CHANGED | rc=0 | (stdout) 
web1 | CHANGED | rc=0 | (stdout) 
web2 | CHANGED | rc=0 | (stdout) 

real	0m2.869s

web1 | CHANGED | rc=0 | (stdout) 
web2 | CHANGED | rc=0 | (stdout) 
db1 | CHANGED | rc=0 | (stdout) 

real	0m7.913s

2.9 seconds, all three at once; 7.9 seconds, one after another. With 100 servers, the default 5 at a time would take about 40 seconds for this; --forks 20 would take about 10. More forks means more SSH connections from your computer at once, so it's a balance. And sometimes one at a time is exactly what you want: the order is predictable, and if something goes wrong, it goes wrong on one server, not five. Lesson 9 does that properly, for patching.

command and shell

command runs a program directly, with its arguments, and no shell in between. shell runs the line through /bin/sh, like typing it in a terminal. The difference shows with a pipe:

BASH

ansible web1 -m command -a "grep PRETTY /etc/os-release | cut -d= -f2"

OUTPUT

web1 | FAILED | rc=1 >>
grep: invalid argument ‘=’ for ‘--directories’
Valid arguments are:
  - ‘read’
  - ‘recurse’
  - ‘skip’
Usage: grep [OPTION]... PATTERNS [FILE]...
Try 'grep --help' for more information.non-zero return code

The error has nothing to do with pipes, and that's the lesson. Without a shell, | is just another word, so grep was given everything: PRETTY, /etc/os-release, |, cut, -d= and -f2. And -d happens to be one of grep's own options, short for --directories, which doesn't accept =. When an error makes no sense, check what the program was actually given. With shell:

BASH

ansible web1 -m shell -a "grep PRETTY /etc/os-release | cut -d= -f2"

OUTPUT

web1 | CHANGED | rc=0 >>
"Ubuntu 24.04.3 LTS"

What about a variable, like $HOME?

BASH

ansible web1 -m command -a "echo \$HOME"
ansible web1 -m shell -a "echo \$HOME"

OUTPUT

web1 | CHANGED | rc=0 >>
/home/vagrant

web1 | CHANGED | rc=0 >>
/home/vagrant

The same. (The \ stops your shell replacing $HOME before Ansible sees it.) Since ansible-core 2.16, command fills in environment variables itself. Pipes, redirects like >, && and ; still need shell.

So why not always use shell? Because a shell interprets everything it's given, which is SQL injection, Lesson 12 of the SQL track, all over again: put a value from a variable into a shell line, and a ; in that value runs a second command. command can't be tricked that way. Use command by default, shell only for pipes and redirects, and best of all, a module that does the job without either.

Finding the right module

There are thousands of modules. ansible-doc lists and explains them, offline:

BASH

ansible-doc -l | wc -l

OUTPUT

Over ten thousand, counting every collection Homebrew installed. The everyday ones are in ansible.builtin, which comes with Ansible itself: apt for packages, copy and template for files, file for permissions and folders, service and systemd for services, and user for accounts. Their full names start ansible.builtin., like ansible.builtin.user; the short name works too, as you've been using. ansible-doc -s shows a module's options as a snippet:

BASH

ansible-doc -s ansible.builtin.user | head -24

OUTPUT

- name: Manage user accounts
  user:
      append:                # If `true', add the user to the groups specified in
                             # `groups'. If
                             # `false', user will
                             # only be added to
                             # the groups
                             # specified in
                             # `groups', removing
                             # them from all other
                             # groups.
      authorization:         # Sets the authorization of the user. Can set
...

(It wraps oddly when it isn't printing to a terminal; ansible-doc ansible.builtin.user on its own, in a terminal, gives the full, readable page.) Put it to use. A deploy user on the web servers, twice:

BASH

ansible web -m user -a "name=deploy state=present shell=/bin/bash" --become -o
ansible web -m user -a "name=deploy state=present shell=/bin/bash" --become -o

OUTPUT

web1 | CHANGED => {..."changed": true,"comment": "","create_home": true,"group": 1001,"home": "/home/deploy","name": "deploy","shell": "/bin/bash","state": "present","system": false,"uid": 1001}
web2 | CHANGED => {..."changed": true,"comment": "","create_home": true,"group": 1001,"home": "/home/deploy","name": "deploy","shell": "/bin/bash","state": "present","system": false,"uid": 1001}

web1 | SUCCESS => {..."append": false,"changed": false,"comment": "","group": 1001,"home": "/home/deploy","move_home": false,"name": "deploy","shell": "/bin/bash","state": "present","uid": 1001}
web2 | SUCCESS => {..."append": false,"changed": false,"comment": "","group": 1001,"home": "/home/deploy","move_home": false,"name": "deploy","shell": "/bin/bash","state": "present","uid": 1001}

Created, with a home folder and the shell asked for; then already there. Linux Lesson 5 did this with useradd, which fails if the user exists. user just checks. Confirm, and remove it again:

BASH

ansible web -m command -a "id deploy"
ansible web -m user -a "name=deploy state=absent remove=true" --become -o

OUTPUT

web2 | CHANGED | rc=0 >>
uid=1001(deploy) gid=1001(deploy) groups=1001(deploy)
web1 | CHANGED | rc=0 >>
uid=1001(deploy) gid=1001(deploy) groups=1001(deploy)

web2 | CHANGED => {..."changed": true,"force": false,"name": "deploy","remove": true,"state": "absent","stderr": "userdel: deploy mail spool (/var/mail/deploy) not found\n",...}
web1 | CHANGED => {..."changed": true,"force": false,"name": "deploy","remove": true,"state": "absent","stderr": "userdel: deploy mail spool (/var/mail/deploy) not found\n",...}

Gone, home folder and all, from remove=true. The stderr note is harmless: userdel looked for a mailbox to delete, and there wasn't one. (... marks where repeated parts of each line are left out.)

In real systems

Inventories are usually generated, not typed. In a cloud, servers come and go, so an inventory plugin asks the cloud for its list each time, and groups servers by their tags: everything tagged role=web becomes the web group. The idea is the same as ssh_config here: let the thing that made the servers say where they are.
group_vars holds most of a real setup. Settings for each environment, like a staging group and a production group with different addresses and sizes, live there, and the playbooks stay the same for both.
Ad-hoc commands are for looking and one-off fixes. "Which servers have this package?", "restart this service on these two". Anything you'll do twice belongs in a playbook, saved and reviewed, which is Lesson 3.

Spot the problem

Work out what's wrong before you read the explanation.

The command meant for one server. Someone wants to restart nginx on web2 only, and types ansible web[2] -m service -a "name=nginx state=restarted" --become. There are three web servers, web1, web2 and web3. Which one restarts? web3. In a pattern, [2] is the group's member at position 2, counting from 0, so it's the third server, not "the one called web2". On a busy server, that's an outage on the wrong machine. Two habits would have caught it: name the server, ansible web2 ..., rather than counting; and check the pattern first with --list-hosts, which would have printed web3.

Keep this

To	Run
See the inventory as a tree	`ansible-inventory --graph`, with `--vars` for settings
One server's merged settings	`ansible-inventory --host web2`
Print a variable on each server	`ansible web -m debug -a "var=http_port"`
Which servers a pattern picks	`ansible "all:!db" --list-hosts`
Either group, both, all but	`web:db`, `bookshop:&web`, `all:!db`
Narrow to some servers	`--limit "web1,db1"`
Run as root	`--become`
Run as another user	`--become --become-user nobody` (needs `acl`)
One server at a time	`--forks 1`
Pipes and redirects	`-m shell -a "grep PRETTY /etc/os-release \| cut -d= -f2"`
Find a module, read its options	`ansible-doc -l`, `ansible-doc ansible.builtin.user`
Make sure a user exists	`ansible web -m user -a "name=deploy state=present" --become`

Where you are now

You can map your servers into groups, give them settings, aim at exactly the right ones, act as the right user, and find the module for a job. Every command so far has been a one-off, typed and forgotten. Lesson 3 writes them down: your first playbook, with a dry run before anything changes.

Check yourself

web2 has http_port=8080 in the inventory, and group_vars/web.yml says 80. Which does web2 get?
What does the pattern all:!db pick?
ansible web[1] runs on web2, not web1. Why?
--become-user nobody failed with a chmod error. What fixed it?
command was given grep PRETTY /etc/os-release | cut -d= -f2, and grep complained about --directories. Why?
Why use command rather than shell when either would work?

Key terms

Inventory: the servers Ansible manages, in groups. Groups can contain groups, with :children.
Variable: a named setting, for all servers, a group, or one server. The most specific one wins.
group_vars and host_vars: folders of variable files, named after the group or server they apply to.
Pattern: which servers to run on: a name, a group, or groups combined with :, :& and :!.
--limit: narrows a pattern further, for one run.
become: running as another user, usually root, through sudo.
ACL: an access control list: extra permissions for named users, set with setfacl from the acl package.
Forks: how many servers Ansible works on at once. 5 by default.

Go deeper

The Ansible documentation: How to build your inventory (opens in a new tab), and Patterns: targeting hosts and groups (opens in a new tab).
Try: add a [staging] group containing web2 only, and give it site_name: bookshop-staging in group_vars/staging.yml. What does ansible-inventory --host web2 say now, and why? (When two groups at the same level both set something, the order of names decides. Lesson 4 covers it.)
Try: ansible-doc ansible.builtin.copy, and find the option that makes a backup of a file before replacing it.