Changing a deeply-nested dict variable in an Ansible playbook

I recently had to build an Ansible playbook that takes in a massive inventory structure (read from a YAML file), modifies a specific key in that file, then dumps the file back to disk. There are some other ways that may be more efficient standalone (e.g. using a separate Python/PHP/Ruby/etc. script and a good YAML library), but since I had to do a number of other things in this Ansible playbook, I thought it would keep it simple if I could also modify the key inside the playbook.

I was scratching my head for a while, because while I knew that I could use the dict | combine() filter to merge two dicts together (this is a feature that was introduced in Ansible 2.0), I hadn't done so for a deeply-nested dict.

After banging my head on a test playbook for a while, I finally resorted to the #ansible IRC room and asked if anyone else knew how to do it. I posted this Pastebin demo playbook, and within a few minutes, both halberom and sivel responded with some ideas. Sivel suggested I structure the object correctly inside the combine() filter, and also helped me do everything in one task using the combine() filter's recursive=True option (see docs on the combine() filter for more details).

So in the end, I had a demonstration playbook like:

---
- hosts: localhost
  connection: local
  gather_facts: no

  vars:
    animals:
      mammals:
        humans:
          eyes: many-colored
          legs: two
      birds:
        cardinals:
          eyes: black
          feathers: blue # <-- I want to change this to 'red'.

  tasks:
    - name: Change animals.birds.cardinals.feathers to "red".
      set_fact:
        animals: "{{ animals|combine({'birds': {'cardinals': {'feathers': 'red'}}}, recursive=True) }}"

    - name: Echo the updated animals var to the screen.
      debug: var=animals

From that point, it was just a matter of writing the updated vars structure to a .yml file, which is as easy as:

- name: Write the updated animals dict to a YAML file.
  template:
    src: animals.yml.j2
    dest: /path/to/animals.yml

Then, inside the template animals.yml.j2:

---
animals:
  {{ animals | to_nice_yaml(width=80, indent=2) | indent(2) }}

After running the complete playbook, I ended up with an updated animals.yml YAML file formatted like:

---
animals:
  birds:
    cardinals:
      eyes: black
      feathers: red
  mammals:
    humans:
      eyes: many-colored
      legs: two

(Note that when you use Ansible, or really any tool, for automatically generating a YAML file, you might not get as much control over things like whitespace, comments, etc.)

I've attached an example playbook (which reads in animals.yml, modifies a nested dict variable, then writes out the updated file) to this blog post: Download animals-ansible-change-nested-dict-example.zip and try it for yourself!

Comments

This is now the 3rd time I've had to refer back to this blog post :)

Would you prefer using combine() over something like https://github.com/leapfrogonline/ansible-merge-vars ? I've finally outgrown simple group_vars.

Usually, yes; it's another dependency that needs installation, and for most of the use cases I've had, it's only marginally nicer to use. However, if I had to do a lot of merging, or a very hairy merge, a plugin would probably be a lot more efficient, especially considering I could break into real Python code to do the work (less annoying than working with the Python-in-Jinja2-in-Ansible YAML!).

Nice thing about this: exactly the same code works for adding a new key/value pair too - e.g. { habitat: nest }, or more practically, say you have a dictionary of PHP settings called php.ini - and you need a way to configure the sendmail_path for mailhog, but only on the dev server…

Awesome! Was about to break my fingers dealing with the same problem =O
Thanks for sharing this!