github

Getting colorized output from Molecule and Ansible on GitHub Actions for CI

For many new Ansible-based projects, I build my tests in Molecule, so I can easily run them locally or in CI. I also started using GitHub Actions for many of my new Ansible projects, just because it's so easy to get started and integrate with GitHub repositories.

I'm actually going to talk about this strategy in my next Ansible 101 live stream, covering Testing Ansible playbooks with Molecule and GitHub Actions CI, but I also wanted to highlight one thing that helps me when reviewing or observing playbook and molecule output, and that's color.

By default, in an interactive terminal session, Ansible colorizes its output so failures get 'red' color, good things / ok gets 'green', and changes get 'yellow-ish'. Also, warnings get a magenta color, which flags them well so you can go and fix them as soon as possible (that's one core principle I advocate to make your playbooks maintainable and scalable).

Enabling a stale issue bot on my GitHub repositories

For the past few years, the number of issues and PRs across all my GitHub repositories has gone from a steady stream to an ongoing deluge. There are currently over 1,500 open issues across my 194 GitHub repositories, and there's no way I can keep up with all of them.

Initially, I went through each issue in each project's issue queue on a monthly basis (mind you, this was—and is still—done on nights and weekends in my spare time). That slipped to a quarterly task... and has now slipped to only happening for higher-profile projects once or twice a year.

Probot Head from GitHub Probot project

The Kubernetes Collection for Ansible

Opera-bull with Ansible bull looking on

The Ansible community has long been a victim of its own success. Since I got started with Ansible in 2013, the growth in the number of Ansible modules and plugins has been astronomical. That's what happens when you build a very simple but powerful tool—easy enough for anyone to extend into any automation use case.

When I started, I remember writing in Ansible for DevOps about 'hundreds' of modules—at the time, mostly covering Linux administration use cases. Today there are many thousands, covering Linux and Windows server administration, network automation, security automation, and even stranger use cases.

Jan-Piet Mens summed it up succinctly in a blog post last year, titled I care about Ansible:

In my opinion they’re being inundated.

Running a Github Actions workflow on schedule and other events

One thing that was not obvious when I was setting up GitHub Actions on the Ansible Kubernetes Collection repository was how to have a 'CI' workflow run both on pull requests and on a schedule. I like to have scheduled runs for most of my projects, so I can see if something starts failing because an underlying dependency changes and breaks my tests.

The documentation for on.schedule just has an example with the workflow running on a schedule. For example:

on:
  schedule:
    # * is a special character in YAML so you have to quote this string
    - cron:  '*/15 * * * *'

Separately, there's documentation for triggering a workflow on events like a 'push' or a 'pull_request':

Saying 'No' to burnout as an open source maintainer

There's been a ton of writing about OSS stewardship, sustainability, funding, etc. in the past year, along with story after story of burnout. In this time, I've become very strict in my open source maintainership:

Unless it's generating income, it's for me and I'm not going to spend more than a couple hours a month looking at it—if that.

There are a number of projects that I maintain, which I'm not actively using on money-generating projects. I don't normally touch or even look at the issue queues on these projects until a CI test fails, or unless someone who contributes to my Patreon or GitHub supporters—or who I know from previous contributions—pings me directly about them. Every now and then I'll run through the list of PRs and merge a bugfix or docs fix here and there, but that only happens maybe once per repository per year.

Get a list of all a user's public GitHub repositories using GitHub API and jq

I recently needed to do a quick audit on all my Ansible roles, and the easiest way (since almost every one is on GitHub, and that's the main source of truth I use) was to grab a list of all my GitHub repositories. However, it can be a little tricky if you have hundreds of repos. I'm guessing most people don't have this problem, but whether you do or not, the easiest way to get all of any given user's repositories using the GitHub v3 API is to run the following command:

curl "https://api.github.com/users/geerlingguy/repos?per_page=100&page=1" | jq -r '.[] | .name'

Example output:

Idempotently adding an SSH key for a host to known_hosts file with bash

I noticed on one of the CI servers I'm running that the .ssh/known_hosts file had ballooned up to over 1,000,000 lines!

Looking into the root cause (I tailed the file until I could track down a few jobs that ran every minute), I found that there was the following line in a setup script:

ssh-keyscan -t rsa github.com >> /var/lib/jenkins/.ssh/known_hosts

"This can't be good!" I told myself, and I decided to add a condition to make it idempotent (that is, able to be run once or one million times but only affecting change the first time it's run—basically, a way to change something only if the change is required):

if ! grep -q "^github.com" /var/lib/jenkins/.ssh/known_hosts; then
  ssh-keyscan -t rsa github.com >> /var/lib/jenkins/.ssh/known_hosts
fi

Now the host key for github.com is only scanned once the first time that script runs, and it is only stored in known_hosts one time for the host github.com... instead of millions of times!

How can I get my PR merged into your open source project?

Recently I received an email from an IT student asking the following: I recently submitted a pull request to one of your open source projects on GitHub. What can I do to get this pull request merged? The answer below may sound somewhat like a cop-out, or harsh (especially considering it was to a starry-eyed student trying to dip his or her toes into the waters of open source software contribution)... but I've found that honesty is the best policy, and the best way I can maintain good OSS software is to guard my (limited) time for OSS work vigilantly, and try to not allow sentiment force the merge of any kind of code, no matter how simple/small the change. Here is my reply:

Thanks for the email! I maintain over 100 different open source projects on GitHub, all in my spare time (which can be hard to come by with 3 kids, a full time job at Acquia, and a few other hobbies!). I spend a few hours per quarter on any given project. Some of the more popular projects have dozens of issues, PRs, and new comments that need to be read through to figure out what I need to these few hours on.

Patching or using a forked version of an Ansible Galaxy role

I maintain a lot of Ansible Galaxy roles. I probably have a problem, but I won't admit it, so I'll probably keep adding more roles :)

One thing I see quite often is someone submitting a simple Pull Request for one of my roles on GitHub, then checking in here and there asking if I have had a chance to merge it yet. I'm guessing people who end up doing this might not know about one of the best features of Ansible Galaxy (and more generally, open source!): you can fork the role and maintain your changes in the fork, and it's pretty easy to do.

I just had to do it for one project I'm working on. I am using the rvm_io.ruby role to install specific versions of Ruby on some servers. But there seems to have been a breaking change to the upstream packages RVM uses, summarized in this GitHub issue. I found a pretty simple fix (removing one array item from a variable), and submitted this PR.

Git gives 'ERROR: Repository not found.' when URL is correct and SSH key is used

I had a fun problem that made me spin my wheels an hour or so today. I was having no issue cloning a remote repository a number of times in the morning while debugging a Jenkins build job that runs a git clone + Docker image build and push operation.

Suddenly, when I was doing some final testing, I started to get the following:

git clone git@github.com:geerlingguy/my-project.git                            
Cloning into 'my-project'...
ERROR: Repository not found.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

I know that I had the repository's SSH key loaded (via eval "$(ssh-agent -s)" && ssh-add ~/.ssh/deploy-key), and if I unloaded the key, I would instead get: