shell

Idempotently adding an SSH key for a host to known_hosts file with bash

I noticed on one of the CI servers I'm running that the .ssh/known_hosts file had ballooned up to over 1,000,000 lines!

Looking into the root cause (I tailed the file until I could track down a few jobs that ran every minute), I found that there was the following line in a setup script:

ssh-keyscan -t rsa github.com >> /var/lib/jenkins/.ssh/known_hosts

"This can't be good!" I told myself, and I decided to add a condition to make it idempotent (that is, able to be run once or one million times but only affecting change the first time it's run—basically, a way to change something only if the change is required):

if ! grep -q "^github.com" /var/lib/jenkins/.ssh/known_hosts; then
  ssh-keyscan -t rsa github.com >> /var/lib/jenkins/.ssh/known_hosts
fi

Now the host key for github.com is only scanned once the first time that script runs, and it is only stored in known_hosts one time for the host github.com... instead of millions of times!

Bash logic structures and conditionals (if, case, loops, etc.) in Travis CI

Travis CI's documentation often mentions the fact that it can call out to shell scripts in your repository, and recommends anything more complicated than a command or two (maybe including a pipe or something) be placed in a separate shell script.

But there are times when it's a lot more convenient to just keep the Travis CI-specific logic inside my repositories' .travis.yml file.

As it turns out, YAML is well-suited to, basically, inlining shell scripts. YAML's literal scalar indicator (a pipe, or |) allows you to indicate a block of content where newlines should be preserved, though whitespace before and after the line will be trimmed.

So if you have a statement like:

if [ "${variable}" == "something" ]; then
  do_something_here
fi

You can represent that in YAML via:

Preventing yourself from accidentally breaking production with Drush

For all the sites I maintain, I have at least a local and production environment. Some projects warrant a dev, qa, etc. as well, but for the purposes of this post, let's just assume you often run drush commands on local or development environments during development, and eventually run a similar command on production during a deployment.

What happens if, at some point, you are churning through some Drush commands, using aliases (e.g. drush @site.local break-all-the-things to break things for testing), and you accidentally enter @site.prod instead of @site.local? Or what if you were doing something potentially disastrous, like deleting a database table locally so you can test a module install file, using drush sqlq to run a query?

$ drush @site.prod break-all-the-things -y
Everything is broken!                                    [sadpanda]

A brief history of SSH and remote access

This post is an excerpt from Chapter 11: Server Security and Ansible, in Ansible for DevOps.

In the beginning, computers were the size of large conference rooms. A punch card reader would merrily accept pieces of paper that instructed the computer to do something, and then a printer would etch the results into another piece of paper. Thousands of mechanical parts worked harmoniously (when they did work) to compute relatively simple commands.

As time progressed, computers became somewhat smaller, and interactive terminals became more user-friendly, but they were still wired directly into the computer being used. Mainframes came to the fore in the 1960s, originally used via typewriter and teletype interfaces, then via keyboards and small text displays. As networked computing became more mainstream in the 1970s and 1980s, remote terminal access was used to interact with the large central computers.

Quickly resetting a local MySQL database from the command line [Updated]

[Update: And, as quickly as I finished writing this post, I thought to myself, "surely, this would be a good thing to have drush do out-of-the-box. And... it already does, making my work on this shell script null and void. I present to you: drush sql-drop! Oh, well.]

When I'm creating or updating an installation profile/distribution for Drupal, I need to reinstall Drupal over and over again. Doing this requires a few simple steps: drop/recreate the database (or drop all db tables), then drush site-install (shortcut: si) with the proper arguments to install the site again.

In the past, I've often had Sequel Pro running in the background on my Mac, and I'd select all the database tables, right-click, choose 'Delete Tables', then have to click again on a button to confirm the deletion. This took maybe 10-20 seconds, depending on whether I already had Sequel Pro running, and how good my mouse muscles were working.

Arrow and Command Keys Not working in Ubuntu 10.04 for non-root Account

For some time, I was having trouble getting the arrow keys to function correctly in my terminal sessions when logging into one of my remote Linode servers running Ubuntu 10.04. Whenever I pressed an arrow key, instead of moving the cursor or going up and down the command history, I would get a string of gibberish like [[A^[[B^[[D^[[C. Not very helpful!

So, after some searching, I found that the cause for this is an incorrect shell environment being set in the passwd file. To fix this problem, simply edit the /etc/passwd file and change the final string (after the last :) to /bin/bash (it is set to /bin/sh if you create a user via the command line/useradd):

$ sudo nano /etc/passwd

Change this:
<username>:x:1000:1000::/home/<username>/:/bin/sh

to this:
<username>:x:1000:1000::/home/<username>/:/bin/bash

...and then save the file, log out, and log back in. Problem solved!