github

Get a list of all a user's public GitHub repositories using GitHub API and jq

I recently needed to do a quick audit on all my Ansible roles, and the easiest way (since almost every one is on GitHub, and that's the main source of truth I use) was to grab a list of all my GitHub repositories. However, it can be a little tricky if you have hundreds of repos. I'm guessing most people don't have this problem, but whether you do or not, the easiest way to get all of any given user's repositories using the GitHub v3 API is to run the following command:

curl "https://api.github.com/users/geerlingguy/repos?per_page=100&page=1" | jq -r '.[] | .name'

Example output:

Idempotently adding an SSH key for a host to known_hosts file with bash

I noticed on one of the CI servers I'm running that the .ssh/known_hosts file had ballooned up to over 1,000,000 lines!

Looking into the root cause (I tailed the file until I could track down a few jobs that ran every minute), I found that there was the following line in a setup script:

ssh-keyscan -t rsa github.com >> /var/lib/jenkins/.ssh/known_hosts

"This can't be good!" I told myself, and I decided to add a condition to make it idempotent (that is, able to be run once or one million times but only affecting change the first time it's run—basically, a way to change something only if the change is required):

if ! grep -q "^github.com" /var/lib/jenkins/.ssh/known_hosts; then
  ssh-keyscan -t rsa github.com >> /var/lib/jenkins/.ssh/known_hosts
fi

Now the host key for github.com is only scanned once the first time that script runs, and it is only stored in known_hosts one time for the host github.com... instead of millions of times!

How can I get my PR merged into your open source project?

Recently I received an email from an IT student asking the following: I recently submitted a pull request to one of your open source projects on GitHub. What can I do to get this pull request merged? The answer below may sound somewhat like a cop-out, or harsh (especially considering it was to a starry-eyed student trying to dip his or her toes into the waters of open source software contribution)... but I've found that honesty is the best policy, and the best way I can maintain good OSS software is to guard my (limited) time for OSS work vigilantly, and try to not allow sentiment force the merge of any kind of code, no matter how simple/small the change. Here is my reply:

Thanks for the email! I maintain over 100 different open source projects on GitHub, all in my spare time (which can be hard to come by with 3 kids, a full time job at Acquia, and a few other hobbies!). I spend a few hours per quarter on any given project. Some of the more popular projects have dozens of issues, PRs, and new comments that need to be read through to figure out what I need to these few hours on.

Patching or using a forked version of an Ansible Galaxy role

I maintain a lot of Ansible Galaxy roles. I probably have a problem, but I won't admit it, so I'll probably keep adding more roles :)

One thing I see quite often is someone submitting a simple Pull Request for one of my roles on GitHub, then checking in here and there asking if I have had a chance to merge it yet. I'm guessing people who end up doing this might not know about one of the best features of Ansible Galaxy (and more generally, open source!): you can fork the role and maintain your changes in the fork, and it's pretty easy to do.

I just had to do it for one project I'm working on. I am using the rvm_io.ruby role to install specific versions of Ruby on some servers. But there seems to have been a breaking change to the upstream packages RVM uses, summarized in this GitHub issue. I found a pretty simple fix (removing one array item from a variable), and submitted this PR.

Git gives 'ERROR: Repository not found.' when URL is correct and SSH key is used

I had a fun problem that made me spin my wheels an hour or so today. I was having no issue cloning a remote repository a number of times in the morning while debugging a Jenkins build job that runs a git clone + Docker image build and push operation.

Suddenly, when I was doing some final testing, I started to get the following:

git clone git@github.com:geerlingguy/my-project.git                            
Cloning into 'my-project'...
ERROR: Repository not found.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

I know that I had the repository's SSH key loaded (via eval "$(ssh-agent -s)" && ssh-add ~/.ssh/deploy-key), and if I unloaded the key, I would instead get:

Cloning private GitHub repositories with Ansible on a remote server through SSH

One of Ansible's strengths is the fact that its 'agentless' architecture uses SSH for control of remote servers. And one classic problem in remote Git administration is authentication; if you're cloning a private Git repository that requires authentication, how can you do this while also protecting your own private SSH key (by not copying it to the remote server)?

As an example, here's a task that clones a private repository to a particular folder:

- name: Clone a private repository into /opt.
  git:
    repo: git@github.com:geerlingguy/private-repo.git
    version: master
    dest: /opt/private-repo
    accept_hostkey: yes
  # ssh-agent doesn't allow key to pass through remote sudo commands.
  become: no

If you run this task, you'll probably end up with something like:

Why I close PRs (OSS project maintainer notes)

GitHub project notifications geerlingguy/drupal-vm PRs

I maintain many open source projects on GitHub and elsewhere (over 160 as of this writing). I have merged and/or closed thousands of Pull Requests (PRs) and patches in the past few years, and would like to summarize here many of the reasons I don't merge many PRs.

A few of my projects have co-maintainers, but most are just me. The bus factor is low, but I offset that by granting very open licenses and encouraging forks. I also devote a set amount of time (averaging 5-10 hours/week) to my OSS project maintenance, and have a personal budget of around $1,000/year to devote to infrastructure to support my projects (that's more than most for-profit companies who use my projects devote to OSS, sadly).

How I test Ansible configuration on 7 different OSes with Docker

The following post is an excerpt from chapter 11 in my book Ansible for DevOps. The example used is an Ansible role that installs Java—since the role is supposed to work across CentOS 6 and 7, Fedora 24, Ubuntu 12.04, 14.04, and 16.04, and Debian 8, I use Docker to run an end-to-end functional test on each of those Linux distributions. See an example test run in Travis CI, and the Travis file that describes the build.

Ansible Java role - Travis CI Docker-based test results

Major improvements to Drupal VM - PHP 7, MariaDB, Multi-OS

Drupal VM - Vagrant and Ansible Virtual Machine for Drupal Development

For the past couple years, I've been building Drupal VM to be an extremely-tunable, highly-performant, super-simple development environment. Since MidCamp earlier this year, the project has really taken off, with almost 200 stars on GitHub and a ton of great contributions and ideas for improvement (some implemented, others rejected).

In the time since I wrote Developing for Drupal with Vagrant and VMs, I've focused on meeting all my defined criteria for the perfect local development environment. And now, I'm able to say that I use Drupal VM when developing all my projects—as it is now flexible and fast enough to emulate any production environment I use for various Drupal projects.

Pages

Subscribe to RSS - github