ansible

Changing a deeply-nested dict variable in an Ansible playbook

I recently had to build an Ansible playbook that takes in a massive inventory structure (read from a YAML file), modifies a specific key in that file, then dumps the file back to disk. There are some other ways that may be more efficient standalone (e.g. using a separate Python/PHP/Ruby/etc. script and a good YAML library), but since I had to do a number of other things in this Ansible playbook, I thought it would keep it simple if I could also modify the key inside the playbook.

I was scratching my head for a while, because while I knew that I could use the dict | combine() filter to merge two dicts together (this is a feature that was introduced in Ansible 2.0), I hadn't done so for a deeply-nested dict.

Adding strings to an array in Ansible

From time to time, I need to dynamically build a list of strings (or a list of other things) using Ansible's set_fact module.

Since set_fact is a module like any other, you can use a with_items loop to loop over an existing list, and pull out a value from that list to add to another list.

For example, today I needed to retrieve a list of all the AWS EC2 security groups in a region, then loop through them, building a list of all the security group names. Here's the playbook I used:

dockrun oneshot — quick local environments for testing infrastructure

Since I work among a ton of different Linux distros and environments in my day-to-day work, I have a lot of tooling set up that's mostly-OS-agnostic. I found myself in need of a quick barebones CentOS 7 VM to play around in or troubleshoot an issue. Or I needed to run Ubuntu 16.04 and Ubuntu 14.04 side by side and run the same command in each, checking for differences. Or I needed to bring up Fedora. Or Debian.

I used to use my Vagrant boxes for VirtualBox to boot a full VM, then vagrant ssh in. But that took at least 15-20 seconds—assuming I already had the box downloaded on my computer!

Bash logic structures and conditionals (if, case, loops, etc.) in Travis CI

Travis CI's documentation often mentions the fact that it can call out to shell scripts in your repository, and recommends anything more complicated than a command or two (maybe including a pipe or something) be placed in a separate shell script.

But there are times when it's a lot more convenient to just keep the Travis CI-specific logic inside my repositories' .travis.yml file.

As it turns out, YAML is well-suited to, basically, inlining shell scripts. YAML's literal scalar indicator (a pipe, or |) allows you to indicate a block of content where newlines should be preserved, though whitespace before and after the line will be trimmed.

So if you have a statement like:

if [ "${variable}" == "something" ]; then
  do_something_here
fi

You can represent that in YAML via:

How to fix SSH errors when using Ansible with newer OSes like Ubuntu 16.04

Recently, as I've been building more and more servers running Ubuntu 16.04, I've hit the following errors:

PLAY [host] ************************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************
fatal: [1.2.3.4]: UNREACHABLE! => {"changed": false, "msg": "SSH Error: data could not be sent to remote host "1.2.3.4". Make sure this host can be reached over ssh", "unreachable": true}

or:

/bin/sh: 1: /usr/bin/python: not found

The former error seems to happen when you're running a playbook on an Ubuntu 16.04 host (with gather_facts: yes), while the latter happens if you're using a minimal distribution that doesn't include Python at all. The problem, in both cases, is that Python 2.x is not installed on the server, and there are two different fixes:

Cloning private GitHub repositories with Ansible on a remote server through SSH

One of Ansible's strengths is the fact that its 'agentless' architecture uses SSH for control of remote servers. And one classic problem in remote Git administration is authentication; if you're cloning a private Git repository that requires authentication, how can you do this while also protecting your own private SSH key (by not copying it to the remote server)?

As an example, here's a task that clones a private repository to a particular folder:

- name: Clone a private repository into /opt.
  git:
    repo: [email protected]:geerlingguy/private-repo.git
    version: master
    dest: /opt/private-repo
    accept_hostkey: yes
  # ssh-agent doesn't allow key to pass through remote sudo commands.
  become: no

If you run this task, you'll probably end up with something like:

Fix for Ansible hanging when used with Docker and TTY

For almost all my Ansible roles on Ansible Galaxy, I have a comprehensive suite of tests that run against all supported OSes on Travis CI, and the only way that's possible is using Docker containers (one container for each OS/test combination).

For the past year or so, I've been struggling with some of the test suites having strange issues when I use docker exec --tty (which passes through Ansible's pretty coloration) along with Ansible playbooks running inside Docker containers in Travis CI. It seems that certain services, when restarted on OSes running sysvinit (like Ubuntu 14.04 and CentOS 6), cause ansible-playbook to hang indefinitely, resulting in a build failure:

MidCamp 2017 Presentation - Drupal VM for Drupal 8 Development

MidCamp is one of my favorite Drupal events—it hits the sweet spot (at least for me) in terms of diversity, topics, and camp size. I was ecstatic when one of my session submissions was accepted, and just finished presenting Developing for Drupal 8 with Drupal VM.

Drupal VM presentation slide

You can see slides from the presentation here: Drupal VM for Drupal 8 Development, but without the full video there are a lot of gaps (especially on slides where there's just a giant emoji!). Luckily, Kevin Thull of Blue Drop Shop is hard at work recording all the sessions and posting them to YouTube. He's already processed the video from my session, and it's available below:

Soup to Nuts: Using Drupal VM to build local and prod

Update, January 2019: I would like to point out that using Drupal VM to build production servers is not officially supported, and though it may work pretty well, you are own your own if you do use it in that capacity. Please see Drop 'official-ish' support for using Drupal VM to run production servers for more details. What follows was mostly a tech demo for a MidCamp session, and has only been used by a small fraction of the Drupal VM userbase.

In preparing for my session Developing for Drupal 8 with Drupal VM at MidCamp later this month, I wanted to build out an example of a canonical "this is the way I'd do it" Drupal 8 site using nothing but Drupal VM and Composer. And I wanted to build both my local development environment and a production environment on DigitalOcean, all using the Ansible automation playbooks built into Drupal VM.

Ansible for DevOps is $0.99 for Black Friday/Cyber Monday 2016!

I decided to discount Ansible for DevOps on LeanPub to $0.99 for Black Friday and Cyber Monday. The book has already been purchased by over 8,000 people on Amazon, LeanPub, and the iBooks Store, and is the only book available that covers the latest version of Ansible—and is continuously updated!

I've written a lot about the process of self-publishing, in case you're interested. I'm hoping to keep updating Ansible for DevOps every quarter or so indefinitely, to make it the best resource now—and as long as possible—for learning infrastructure automation!