travis ci

I just spent the past 6 hours migrating some of my open source projects from Travis CI to GitHub Actions, and I thought I'd pause for a bit (12 hours into this project, probably 15-20 more to go) to jot down a few thoughts.

I am not one to look a gift horse in the mouth. For almost a decade, Travis CI made it possible for me to build—and maintain, for years—hundreds of open source projects.

I have built projects for Raspberry Pi, PHP, Python, Drupal, Ansible, Kubernetes, macOS, iOS, Android, Docker, Arduino, and more. And almost every single project I built got immediate integration with Travis CI.

Without that testing, and the ability to run tests on a schedule, I would have abandoned most of these projects. But with the testing, I'm able to keep up with build failures induced by bit rot over the years and review PRs more easily.

What went wrong with Travis CI?

From the outset, Travis CI was built to integrate with GitHub repositories and offer free open source CI. At one time it was showered with praise on Hacker News and elsewhere for its culture and ethos.

I maintain a large number of Ansible Galaxy roles, and publish hundreds of new releases every year. If the process weren't fully automated, there would be no way I could keep up with it. For Galaxy roles, the process of tagging and publishing a new release is very simple, because Ansible Galaxy ties the role strongly to GitHub's release system. All that's needed is a webhook in your .travis.yml file (if using Travis CI):

For collections, Ansible Galaxy actually hosts an artifact—a .tar.gz file containing the collection contents. This offers some benefits that I won't get into here, but also a challenge: someone has to build and upload that artifact... and that takes more than one or two lines added to a .travis.yml file.

Until recently, I had been publishing collection releases manually. The process went something like:

After the announcement on September 26 that Ansible will be adopting molecule and ansible-lint as official 'Ansible by Red Hat' projects, I started moving more of my public Ansible projects over to Molecule-based tests instead of using the homegrown Docker-based Ansible testing rig I'd been using for a few years.

There was also a bit of motivation from readers of Ansible for DevOps, many of whom have asked for a new section on Molecule specifically!

In this blog post, I'll walk you through how to use Molecule, and how I converted all my existing roles (which were using a different testing system) to use Molecule and Ansible Lint-based tests.

I use Ansible Vault to securely store the project's secrets (e.g. API keys, default passwords, private keys, etc.) in the git repository for many of my infrastructure projects. I also like to make sure I cover everything possible in automated tests/CI, using either Jenkins or Travis CI (usually).

But this presents a conundrum: if some of your variables are encrypted with an Ansible Vault secret/passphrase, and that secret should be itself store securely... how can you avoid storing it in your CI system, where you might not be able to guarantee it's security?

The method I usually use for this case is including the Vault-encrypted vars at playbook runtime, using include_vars:

Travis CI's documentation often mentions the fact that it can call out to shell scripts in your repository, and recommends anything more complicated than a command or two (maybe including a pipe or something) be placed in a separate shell script.

But there are times when it's a lot more convenient to just keep the Travis CI-specific logic inside my repositories' .travis.yml file.

As it turns out, YAML is well-suited to, basically, inlining shell scripts. YAML's literal scalar indicator (a pipe, or |) allows you to indicate a block of content where newlines should be preserved, though whitespace before and after the line will be trimmed.

So if you have a statement like:

You can represent that in YAML via:

For almost all my Ansible roles on Ansible Galaxy, I have a comprehensive suite of tests that run against all supported OSes on Travis CI, and the only way that's possible is using Docker containers (one container for each OS/test combination).

For the past year or so, I've been struggling with some of the test suites having strange issues when I use docker exec --tty (which passes through Ansible's pretty coloration) along with Ansible playbooks running inside Docker containers in Travis CI. It seems that certain services, when restarted on OSes running sysvinit (like Ubuntu 14.04 and CentOS 6), cause ansible-playbook to hang indefinitely, resulting in a build failure:

The following post is an excerpt from chapter 11 in my book Ansible for DevOps. The example used is an Ansible role that installs Java—since the role is supposed to work across CentOS 6 and 7, Fedora 24, Ubuntu 12.04, 14.04, and 16.04, and Debian 8, I use Docker to run an end-to-end functional test on each of those Linux distributions. See an example test run in Travis CI, and the Travis file that describes the build.

Note: I do the same thing currently (as of 2019), but now I'm using Molecule to tie everything together; see Testing your Ansible roles with Molecule.