Ansible best practices: using project-local collections and roles

Note for Tower/AWX users: Currently, Tower requires role and collection requirements to be split out into different files; see Tower: Ansible Galaxy Support. Hopefully Tower will be able to support the requirements layout I outline in this post soon!

Since collections will be a major new part of every Ansible user's experience in the coming months, I thought I'd write a little about what I consider an Ansible best practice: that is, always using project-relative collection and role paths, so you can have multiple independent Ansible projects that track their own dependencies according to the needs of the project.

Early on in my Ansible usage, I would use a global roles path, and install all the roles I used (whether private or on Ansible Galaxy) into that path, and I would rarely have a playbook or project-specific role or use a different playbook-local version of the role.

Over time, I found that many playbooks benefitted from having all role dependencies managed independently. That meant adding a requirements.yml file for that playbook, (usually, but not always) defining each role's version or Git commit hash, and adding an ansible.cfg file in the playbook project's root directory so Ansible would know to only load roles from that playbook's roles directory.

With collections, I'm expanding that best practice, and generally recommend against installing collections into a global location (like the default, which is ~/.ansible/collections or /usr/share/ansible/collections).

For any new project I start, I add a ansible.cfg with the following (at minimum):

[defaults]

# Chick-Fil-A would like a word...
nocows = True

# Installs collections into [current dir]/ansible_collections/namespace/collection_name
collections_paths = ./

# Installs roles into [current dir]/roles/namespace.rolename
roles_path = ./roles

Then I manage all my playbook's dependencies in a requirements.yml file like this one:

---
roles:
  - name: geerlingguy.java
    version: 1.9.7

collections:
  - name: community.kubernetes
    version: 0.9.0

Assuming this PR is merged prior to Ansible 2.10's release, you can then install all the playbook project's dependencies with:

ansible-galaxy install -r requirements.yml

Then your playbooks will use the correct requirements, and won't affect any other playbooks and Ansible projects.

This also optimizes things because instead of Ansible having to scan through the likely hundreds (thousands?) of role and collection directories that I would need to have in global paths to support the hundreds of projects I work with, it only needs to scan the few that are required for the given project I'm working on.

A final major benefit is you can then commit the roles and collections to your project code repository, and then the dependencies are part of the deployable project, which gives two major benefits:

  1. Other people (e.g. coworkers, or even CI systems) which work with the project would necessarily have the same revision as everyone else as part of the project VCS, and updates to any dependencies would be checked in via source control.
  2. You could still run the playbook if GitHub, Ansible Galaxy, or any other system required to download the dependencies were offline, since you wouldn't need to run ansible-galaxy at all (except to update dependencies when you want to do so).

Comments

Hey Jeff,

I just wanted to clarify for anyone using Ansible Tower:

Tower supports project-specific Ansible collections in job runs. If you specify a collections requirements file in the SCM at collections/requirements.yml, Tower will install collections in that file in the implicit project sync before a job run.

It works the same way for roles, you would need to specify a roles requirements file in the SCM at roles/requirements.yml.

https://docs.ansible.com/ansible-tower/latest/html/userguide/projects.html#ansible-galaxy-support

Thanks for the clarification; I didn't realize when writing this post that Tower doesn't support mixed roles and collections in one requirements file. This is a limitation in the current version of Tower, and not Ansible's core itself, so if you don't rely on Tower/AWX the original best practice still applies.

The most important part, however, is making sure you install collections and roles in a project-specific path—and it looks like Tower does that, which is nice. Otherwise you could run one playbook, and get version xyz of one dependency, then run another project, and it overwrites that to version abc, and breaks the first playbook!

I understand your points. But I have to disagree with some of your stated benefits. In my opinion it is ugly or borderline unacceptable to pull outside dependencies into a repository.

It is ugly and unacceptable, and I never do this in any of my normal software projects (PHP, Python, etc.)—however, since the ansible-galaxy CLI is severely lacking in ability to do dependency management basics (like equivalents to composer install (install all the versions in a lockfile / freeze file—galaxy won't overwrite an old version if it differs from the requirements.yml file unless you use --force which is more fragile since it will reinstall everything), composer update (or npm, gem, etc.)), it is a lot easier to track the deps locally so you can verify things after you have to run a ansible-galaxy install --force to update dependencies.

AWX seems to do this nicely (thanks for the previous comment!), so that is enough for me. I can avoid polluting my repositories ;)

I wish that ansible-galaxy could do dependencies properly. This will be extremely important in the future with the changes in Ansible 2.10 and I am very disappointed that the choice was done before ansible-galaxy has features to properly manage all those collections.

I am afraid it will be a horrible mess - a mess that the users might have to deal with for years to come.

Are there tickets open for getting `ansible-galaxy` to start doing that kind of thing, being more like Composer? If so, can you provide links?

Thanks for your article Jeff. Based on this structure, how would you recommend people use multiple collections that may require a reference to the same role in each collection? Would the role be cloned twice, into each collection? It's something I was hoping to avoid, but maybe it shouldn't be done. Thanks for any pointers. I've just been getting into the use of collections over the last few days.

Another benefit to a local, project-relative roles path: you can hack on the roles in this path to make local changes for faster development cycle than the normal commit-push-test cycle if using ansible galaxy.