Everything I know about Kubernetes I learned from a cluster of Raspberry Pis

I realized I haven't posted about my DrupalCon Seattle 2019 session titled Everything I know about Kubernetes I learned from a cluster of Raspberry Pis, so I thought I'd remedy that. First, here's a video of the recorded session:

The original Raspberry Pi Dramble Cluster
The original Pi Dramble 6-node cluster, running the LAMP stack.

I started running the Raspberry Pi Dramble in 2014, after I realized I could automate the setup of everything in a LAMP stack on a set of Raspberry Pi 2s using Ansible (one Pi for an HTTP load balancer/reverse proxy, two for PHP app backends, and two for MySQL redundancy. Kubernetes was the logical next step, so I moved things towards Kubernetes in 2017, but running Kubernetes was a lesson in pain due to the Pi's limited memory (1 GB maximum) at the time.

When the Raspberry Pi 4 came around, I acquired some 2 GB models as quickly as I could, and redeployed onto them. Gone were the restrictions that were causing Kubernetes' API to be flaky with the older Pis, and now the Pi 4 cluster is extremely reliable. Early on, cooling was an issue, but the recent firmware update has made that less problematic. I now power the cluster using the official PoE HAT, which means I only have one plug for each Pi, and everything fits nicely into a small travel case (if I want to bring the cluster with me anywhere).

Raspberry Pi Dramble 4 Kubernetes edition cluster
The current Pi Dramble, running Kubernetes and sporting BlinkStick Nanos.

I even got it all to run off a 10,000 mAh battery pack with a bunch of USB splitters... but it did not stay powered long, and kept giving low power warnings—so I'll have to consider other options for a highly-mobile four-node Kubernetes bare-metal cluster.

I've continually updated the cluster so it is tested in a Docker-based Kubernetes environment, a Vagrant-based Kubernetes local development environment, and of course, the Pi environment. The latter environment causes much consternation, as many common container images are not maintained in an armv6 or linux/arm format, which is required when running on the 32-bit ARM OS the Pi uses, Raspbian. But the Pi Dramble abides, and it quietly goes on, serving up traffic for https://www.pidramble.com through the years.

The official Pi Dramble Wiki has all the instructions for building your own Pi Kubernetes cluster, with links to buy all the parts I have, along with every step to get it running using open source Ansible roles to install Kubernetes and Docker for ARM, then configure a new four-node cluster.

Comments

Thanks for the hint on that little blinky thing called BlinkStickNanos.

If you're in the market, I've greatly enjoyed my blink(1) by ThingM: https://blink1.thingm.com/

This is very cool. I’ve been thinking about doing something like this and this post got me excited to try it.

Do you have any recommendations for hosting the pi rack? I imagine you’d need to co-locate?

I've often wondered why people use things like RPs for work like this rather than a multi-core desktop (or server) machine with a bunch of VMs. These days such machines can be found for real cheap, and they wouldn't present the issues you mentioned in your article. Sure, you can't put them in a brief case (unless you're using a laptop) and they might take more power than a cluster of RPs. But, if time is money, then I don't understand the attraction of RP-like boards for purposes other than special projects that will ultimately run on such boards.

Mostly fun :)

Agreed. It's a great visual technique. Keep up the great work. Big fan.

I've been curious to do something like this with the latest Raspberry Pi 4's as well -- but AFAIK power requirements are now an issue. Thanks for your power tips (including using the POE hat)