Changing a deeply-nested dict variable in an Ansible playbook

I recently had to build an Ansible playbook that takes in a massive inventory structure (read from a YAML file), modifies a specific key in that file, then dumps the file back to disk. There are some other ways that may be more efficient standalone (e.g. using a separate Python/PHP/Ruby/etc. script and a good YAML library), but since I had to do a number of other things in this Ansible playbook, I thought it would keep it simple if I could also modify the key inside the playbook.

I was scratching my head for a while, because while I knew that I could use the dict | combine() filter to merge two dicts together (this is a feature that was introduced in Ansible 2.0), I hadn't done so for a deeply-nested dict.

After banging my head on a test playbook for a while, I finally resorted to the #ansible IRC room and asked if anyone else knew how to do it. I posted this Pastebin demo playbook, and within a few minutes, both halberom and sivel responded with some ideas. Sivel suggested I structure the object correctly inside the combine() filter, and also helped me do everything in one task using the combine() filter's recursive=True option (see docs on the combine() filter for more details).

So in the end, I had a demonstration playbook like:

---
- hosts: localhost
  connection: local
  gather_facts: no

  vars:
    animals:
      mammals:
        humans:
          eyes: many-colored
          legs: two
      birds:
        cardinals:
          eyes: black
          feathers: blue # <-- I want to change this to 'red'.

  tasks:
    - name: Change animals.birds.cardinals.feathers to "red".
      set_fact:
        animals: "{{ animals|combine({'birds': {'cardinals': {'feathers': 'red'}}}, recursive=True) }}"

    - name: Echo the updated animals var to the screen.
      debug: var=animals

From that point, it was just a matter of writing the updated vars structure to a .yml file, which is as easy as:

- name: Write the updated animals dict to a YAML file.
  template:
    src: animals.yml.j2
    dest: /path/to/animals.yml

Then, inside the template animals.yml.j2:

---
animals:
  {{ animals | to_nice_yaml(width=80, indent=2) | indent(2) }}

After running the complete playbook, I ended up with an updated animals.yml YAML file formatted like:

---
animals:
  birds:
    cardinals:
      eyes: black
      feathers: red
  mammals:
    humans:
      eyes: many-colored
      legs: two

(Note that when you use Ansible, or really any tool, for automatically generating a YAML file, you might not get as much control over things like whitespace, comments, etc.)

I've attached an example playbook (which reads in animals.yml, modifies a nested dict variable, then writes out the updated file) to this blog post: Download animals-ansible-change-nested-dict-example.zip and try it for yourself!

Further reading

ansible
yaml
tutorial
dict
python
jinja2
irc

Comments

This is now the 3rd time I've had to refer back to this blog post :)

Usually, yes; it's another dependency that needs installation, and for most of the use cases I've had, it's only marginally nicer to use. However, if I had to do a lot of merging, or a very hairy merge, a plugin would probably be a lot more efficient, especially considering I could break into real Python code to do the work (less annoying than working with the Python-in-Jinja2-in-Ansible YAML!).

Nice thing about this: exactly the same code works for adding a new key/value pair too - e.g. { habitat: nest }, or more practically, say you have a dictionary of PHP settings called php.ini - and you need a way to configure the sendmail_path for mailhog, but only on the dev server…

Awesome! Was about to break my fingers dealing with the same problem =O
Thanks for sharing this!

Why is that templating needed ?
simpe copy should also do or not ?:
- name: Copy using the 'content' for inline data
copy:
content: "{{ animals | to_nice_yaml(width=80, indent=2) | indent(2) }}"
dest: /path/to/animals.yml

If there was a key: value pair appearing multiple time in a deeply nested hash without knowing where it would be, is there a way I can walk the hash and create a list of all key: value pair occurrences?

Thank you very much for this :) Hard to find information on performing such specific actions in ansible sometimes.

Any idea how I can add a missing or undefined entry but not overwrite existing values? This way I’d like to achieve fallback values.

Thanks for posting this. I could not find the answer on ansible.com / stack overflow / reddit.

If birds and mammals is an array element then how to proceed further. I am struck with this. Please help me out

Thank you so much for posting this. I can't count the number of times that I've referenced this, and will continue to reference this.

What if you want to change the eyes on both humans and cardinals to blue in the same task? Is there a way to loop through the dict and use a wild card or regex for the key names?

Thank you Jeff.

But just noticed that when there is a list items inside the dictionary, it will be replaced with new dict. I have tried with `list_merge=` options but didnt really help.

Any guess ?