Updating all your servers with Ansible

From time to time, there's a security patch or other update that's critical to apply ASAP to all your servers. If you use Ansible to automate infrastructure work, then updates are painless—even across dozens, hundreds, or thousands of instances! I've written about this a little bit in the past, in relation to protecting against the shellshock vulnerability, but that was specific to one package.

I have an inventory script that pulls together all the servers I manage for personal projects (including the server running this website), and organizes them by OS, so I can run commands like ansible [os] command. Then that enables me to run commands like:

# Upgrade all the Ubuntu servers.
ansible ubuntu -m apt -a "upgrade=yes update_cache=yes" -b

# Upgrade all the Debian servers.
ansible debian -m apt -a "upgrade=yes update_cache=yes" -b

# Upgrade all the CentOS servers.
ansible centos -m yum -a "name=* state=latest" -b

# Upgrade all the Fedora servers.
ansible fedora -m dnf -a "name=* state=latest" -b

Then I can reboot all servers with ansible all -a "reboot" -b.

I've also built more intelligent playbooks for this purpose, allowing me to do rolling updates (e.g. don't reboot all servers at once—just do half, then the other half), monitor the progress with wait_for and connection: local... but I'll leave that exercise to the reader, since these kind of playbooks are usually more specific to your infrastructure (hint: google it).

Comments

In one sense, yes. However, for Debian/Ubuntu servers, that wouldn't update the apt cache, and would lead to failures frequently. Also, by splitting things out, you can also add other bits that would be helpful for different configurations. The best thing to do would be to use a playbook that runs over all hosts (maybe multiple plays, or in one play with conditionals for different OS environments) and updates everything, and additionally restarts the servers as required.

For example, here's a playbook I used to update some older Ubuntu 12.04 instances to Ubuntu 14.04: https://www.jeffgeerling.com/blog/2017/ansible-playbook-upgrade-all-ubuntu-1204-lts-hosts-1404-or-1604 (note: I'm not recommending running major OS upgrades—in the case of this example, I had to or risk having a bunch of servers on a no-longer-supported release!).

Hi,

Can I use ansible to upgrade HP Proliant server hardware firmware, SolarFlare network cards drivers? What other things can Ansible be used in a Data Center?
Thank you

You can do everything you can do with a SSH connection

Hi Jeff,

how do you manage the cases where a package to upgrade implies some interactive steps (e.g. grub)?

Thanks

what if something goes wrong with the applications running on it? can you test it before?

The second comment should state "Debian" instead of "Ubuntu" ;-)

Hi Jeff, what does the '-s' at the end of the reboot statement do? I've tried running this and Ansible doesn't recognise it. Should it be '-b'? Thanks for the great work by the way!!