Changing a deeply-nested dict variable in an Ansible playbook

I recently had to build an Ansible playbook that takes in a massive inventory structure (read from a YAML file), modifies a specific key in that file, then dumps the file back to disk. There are some other ways that may be more efficient standalone (e.g. using a separate Python/PHP/Ruby/etc. script and a good YAML library), but since I had to do a number of other things in this Ansible playbook, I thought it would keep it simple if I could also modify the key inside the playbook.

I was scratching my head for a while, because while I knew that I could use the dict | combine() filter to merge two dicts together (this is a feature that was introduced in Ansible 2.0), I hadn't done so for a deeply-nested dict.

After banging my head on a test playbook for a while, I finally resorted to the #ansible IRC room and asked if anyone else knew how to do it. I posted this Pastebin demo playbook, and within a few minutes, both halberom and sivel responded with some ideas. Sivel suggested I structure the object correctly inside the combine() filter, and also helped me do everything in one task using the combine() filter's recursive=True option (see docs on the combine() filter for more details).

So in the end, I had a demonstration playbook like:

---
- hosts: localhost
  connection: local
  gather_facts: no

  vars:
    animals:
      mammals:
        humans:
          eyes: many-colored
          legs: two
      birds:
        cardinals:
          eyes: black
          feathers: blue # <-- I want to change this to 'red'.

  tasks:
    - name: Change animals.birds.cardinals.feathers to "red".
      set_fact:
        animals: "{{ animals|combine({'birds': {'cardinals': {'feathers': 'red'}}}, recursive=True) }}"

    - name: Echo the updated animals var to the screen.
      debug: var=animals

From that point, it was just a matter of writing the updated vars structure to a .yml file, which is as easy as:

- name: Write the updated animals dict to a YAML file.
  template:
    src: animals.yml.j2
    dest: /path/to/animals.yml

Then, inside the template animals.yml.j2:

---
animals:
  {{ animals | to_nice_yaml(width=80, indent=2) | indent(2) }}

After running the complete playbook, I ended up with an updated animals.yml YAML file formatted like:

---
animals:
  birds:
    cardinals:
      eyes: black
      feathers: red
  mammals:
    humans:
      eyes: many-colored
      legs: two

(Note that when you use Ansible, or really any tool, for automatically generating a YAML file, you might not get as much control over things like whitespace, comments, etc.)

I've attached an example playbook (which reads in animals.yml, modifies a nested dict variable, then writes out the updated file) to this blog post: Download animals-ansible-change-nested-dict-example.zip and try it for yourself!

Comments

This is now the 3rd time I've had to refer back to this blog post :)

Would you prefer using combine() over something like https://github.com/leapfrogonline/ansible-merge-vars ? I've finally outgrown simple group_vars.

Usually, yes; it's another dependency that needs installation, and for most of the use cases I've had, it's only marginally nicer to use. However, if I had to do a lot of merging, or a very hairy merge, a plugin would probably be a lot more efficient, especially considering I could break into real Python code to do the work (less annoying than working with the Python-in-Jinja2-in-Ansible YAML!).

Nice thing about this: exactly the same code works for adding a new key/value pair too - e.g. { habitat: nest }, or more practically, say you have a dictionary of PHP settings called php.ini - and you need a way to configure the sendmail_path for mailhog, but only on the dev server…

Awesome! Was about to break my fingers dealing with the same problem =O
Thanks for sharing this!

Why is that templating needed ?
simpe copy should also do or not ?:
- name: Copy using the 'content' for inline data
copy:
content: "{{ animals | to_nice_yaml(width=80, indent=2) | indent(2) }}"
dest: /path/to/animals.yml

I needed a way to remove a dict item (the information here explains how to add or modify, but not remove deeply nested dictionary items)... So here's an addendum to tell you how to remove items from deeply nested dictionaries: https://medium.com/@jmazzite/removing-a-deeply-nested-dict-item-in-ansib...