bash

Updating a Kubernetes Deployment and waiting for it to roll out in a shell script

For some Kubernetes cluster operations (e.g. deploying an update to a small microservice or app), I need a quick and dirty way to:

  1. Build and push a Docker image to a private registry.
  2. Update a Kubernetes Deployment to use this new image version.
  3. Wait for the Deployment rollout to complete.
  4. Run some post-rollout operations (e.g. clear caches, run an update, etc.).

There are a thousand and one ways to do all this, and many are a bit more formal than this, but sometimes you just need a shell script you can run from your CI server to do it all. And it's not too hard, nor complex, to do it this way:

The best way to get the full PHP version string

Recently, to automate building, tagging, and pushing my geerlingguy/php-apache Docker Hub image (see this issue), I needed to find a way to reliably determine the PHP major.minor.release version string. You'd think this would be simple.

Well, using Docker, I would run the image and then try:

# php --version
PHP 7.3.0-1+0~20181206202713.23+stretch~1.gbp076afd (cli) (built: Dec  6 2018 20:27:14) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.3.0-dev, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.3.0-1+0~20181206202713.23+stretch~1.gbp076afd, Copyright (c) 1999-2018, by Zend Technologies

That's great; it outputs the version right at the start. But there are a few problems here:

Idempotently adding an SSH key for a host to known_hosts file with bash

I noticed on one of the CI servers I'm running that the .ssh/known_hosts file had ballooned up to over 1,000,000 lines!

Looking into the root cause (I tailed the file until I could track down a few jobs that ran every minute), I found that there was the following line in a setup script:

ssh-keyscan -t rsa github.com >> /var/lib/jenkins/.ssh/known_hosts

"This can't be good!" I told myself, and I decided to add a condition to make it idempotent (that is, able to be run once or one million times but only affecting change the first time it's run—basically, a way to change something only if the change is required):

if ! grep -q "^github.com" /var/lib/jenkins/.ssh/known_hosts; then
  ssh-keyscan -t rsa github.com >> /var/lib/jenkins/.ssh/known_hosts
fi

Now the host key for github.com is only scanned once the first time that script runs, and it is only stored in known_hosts one time for the host github.com... instead of millions of times!

Parsing YAML files on the command line using Ruby

I have been working on an infrastructure project that uses YAML files for all inventory and configuration management, and for the most part, if you're using tools like Ansible, CloudFormation, etc., then you don't ever have to worry about the actual parsing of a YAML file, and keys and values in the file are readily accessible since these tools parse them and get them into a readable structure for you.

But there's often little bits of glue code, or infrastructure build/cleanup jobs, where you need to grab one specific value out of a YAML file, and all you have readily available is bash. Luckily for me, I also have Ruby available in the particular environment where I needed to parse the YAML file, so doing this was as easy as:

  1. Defining a little bit of Ruby code which would load the YAML file, then grab a value out of it.
  2. Running that Ruby code using ruby -e (-e for 'evaluate this code), operating on the contents of a YAML file.

And here's how that looked, in my case:

Bash logic structures and conditionals (if, case, loops, etc.) in Travis CI

Travis CI's documentation often mentions the fact that it can call out to shell scripts in your repository, and recommends anything more complicated than a command or two (maybe including a pipe or something) be placed in a separate shell script.

But there are times when it's a lot more convenient to just keep the Travis CI-specific logic inside my repositories' .travis.yml file.

As it turns out, YAML is well-suited to, basically, inlining shell scripts. YAML's literal scalar indicator (a pipe, or |) allows you to indicate a block of content where newlines should be preserved, though whitespace before and after the line will be trimmed.

So if you have a statement like:

if [ "${variable}" == "something" ]; then
  do_something_here
fi

You can represent that in YAML via:

Using Ubuntu Bash in Windows Creators' Update with Vagrant

When Microsoft announced the Windows Subsystem for Linux, now seemingly rebranded as Bash on ubuntu on Windows, I was excited at the possibility of having Drupal VM (and other similarly command-line-friendly open source projects) work better in a Windows environment. But unfortunately, the Anniversary update's version of WSL/Ubuntu Bash was half-baked, and there were a lot of little issues trying to get anything cohesive done between the Windows and Ubuntu Bash environments (even with cbwin).

Then, a year or so later, Microsoft finally announced that tons of improvements (including upgrading Ubuntu in the WSL from 14.04 to 16.04!) would be included in the 'Creators Update' to Windows 10, dropping tomorrow, April 11.

Remove ALL your local Vagrant Boxes via this bash command

Assuming you have only one box per provider, this command will delete ALL Vagrant boxes you currently have on your system:

$ vagrant box list | cut -f 1 -d ' ' | xargs -L 1 vagrant box remove -f

This command does the following:

  1. vagrant box list: Prints out a list of all installed vagrant boxes (with two columns—box name or path, and meta info)
  2. cut -f 1 -d ' ': Cuts the list and takes out just the first column (using spaces to delimit the columns)
  3. xargs -L 1 vagrant box remove -f: Use xargs to run one command per line, running the command vagrant box remove -f [box name from list/cut].

You can use xargs' -t option to output the commands being run just before they're executed. And if you have multiple boxes per provider, or if you have multiple versions of the same box, you'll likely need to modify the command a bit.

Using Ansible through Windows 10's Subsystem for Linux

Ever since I heard about the new 'Beta' Windows Subsystem for Linux, which basically installs an Ubuntu LTS release inside of Windows 10 (currently 14.04), I've been meaning to give it a spin, and see if it can be a worthy replacement for Cygwin, Git shell, Cmder, etc. And what I was most interested in was whether I could finally point people to a more stable and friendly way of using Ansible on a Windows workstation.

In the past, there was the option of running Ansible inside Cygwin (and this is still the best way to try getting Ansible working in an older Windows environment), but this always felt kludgy to me, and I hated having to recommend either that or forcing Windows users to do a full Linux VM installation just to run Ansible commands. I finally updated my PC laptop to the latest Windows 10 Anniversary Update, and installed the Windows Subsystem for Linux, and lo and behold, Ansible works!

Fastest way to reset a host key when rebuilding servers on the same IP or hostname frequently

I build and rebuild servers quite often, and when I want to jump into the server to check a config setting (when I'm not using Ansible, that is...), I need to log in via SSH. It's best practice to let SSH verify the host key every time you connect to make sure you're not getting MITMed or anything else is going on.

However, any time you rebuild a server from a new image/OS install, the host key should be new, and this will result in the following message the next time you try to log in:

Preventing yourself from accidentally breaking production with Drush

For all the sites I maintain, I have at least a local and production environment. Some projects warrant a dev, qa, etc. as well, but for the purposes of this post, let's just assume you often run drush commands on local or development environments during development, and eventually run a similar command on production during a deployment.

What happens if, at some point, you are churning through some Drush commands, using aliases (e.g. drush @site.local break-all-the-things to break things for testing), and you accidentally enter @site.prod instead of @site.local? Or what if you were doing something potentially disastrous, like deleting a database table locally so you can test a module install file, using drush sqlq to run a query?

$ drush @site.prod break-all-the-things -y
Everything is broken!                                    [sadpanda]

Pages

Subscribe to RSS - bash