cluster

Monitoring Kubernetes cluster utilization and capacity (the poor man's way)

If you're running Kubernetes clusters at scale, it pays to have good monitoring in place. Typical tools I use in production like Prometheus and Alertmanager are extremely useful in monitoring critical metrics, like "is my cluster almost out of CPU or Memory?"

But I also have a number of smaller clusters—some of them like my Raspberry Pi Dramble have very little in the way of resources available for hosting monitoring internally. But I still want to be able to say, at any given moment, "how much CPU or RAM is available inside the cluster? Can I fit more Pods in the cluster?"

So without further ado, I'm now using the following script, which is slightly adapted from a script found in the Kubernetes issue Need simple kubectl command to see cluster resource usage:

Usage is pretty easy, just make sure you have your kubeconfig configured so kubectl commands are working on the cluster, then run:

Kubernetes' Complexity

Over the past month, I started rebuilding the Raspberry Pi Dramble project using Kubernetes instead of installing and configuring the LEMP stack directly on nodes via Ansible (track GitHub issues here). Along the way, I've hit tons of minor issues with the installation, and I wanted to document some of the things I think turn people away from Kubernetes early in the learning process. Kubernetes is definitely not the answer to all application hosting problems, but it is a great fit for some, and it would be a shame for someone who could really benefit from Kubernetes to be stumped and turn to some other solution that costs more in time, money, or maintenance!

Raspberry Pi Dramble cluster running Kubernetes with Green LEDs

Highly available Drupal on a Raspberry Pi Cluster - phptek 2016 session

Raspberry Pi Dramble Cluster with Mini Raspberry Pi Zero Cluster

Another year, another field trip for the Pi Dramble—my 5-Raspberry-Pi cluster! I presented a session titled Highly available Drupal on a Raspberry Pi Cluster at php[tek] 2016, which just so happens to have moved to my hometown, St. Louis, MO this year!

For this presentation, I remembered to record the audio using a lav mic plugged into my iPhone, as well as iShowU to record what was on my screen. Sadly, I didn't have a secondary camera to capture the Pi Dramble itself, but you can glance at all the other 'Let's build a Pi Cluster' videos if you want to see it in action!

Here's a video recording of the presentation:

How to Build Your Own Raspberry Pi Cluster ('Bramble')

Rasbperry Pi Dramble

One of the first questions I'm asked by those who see the Dramble is, "How do I build my own?" Since I've been asked the question many times, I put together a detailed parts list, and maintain it on the Dramble's project wiki on GitHub: Raspberry Pis and Accessories.

For a little over $400, you can have the exact same setup, with six Raspberry Pi 2s, a network switch, a rack inside which you can mount the Pis, microSD cards for storage, a 6-port USB power supply, and all the required cables and storage!

Raspberry Pi RGB LED boards

Lessons Learned building the Raspberry Pi Dramble

Raspberry Pi Dramble Bramble Cluster

Edit: Many people have been asking for more technical detail, benchmarks, etc. There is much more information available on the Raspberry Pi Dramble Wiki (e.g. Power Consumption, microSD card benchmarks, etc.), if you're interested.

After the Raspberry Pi 2 model B was released, I decided the Pi was finally a fast enough computing platform (with its 4-core 900 MHz ARMv7 architecture) with enough memory (1 GB per Pi) to actually use for web infrastructure. If not in a production environment (I would definitely avoid putting the Pi into a role as a high-load 24x7x365 server), then for development and/or testing purposes. And for some fun!

Introducing the Dramble - Raspberry Pi 2 cluster running Drupal 8

Dramble - 6 Raspberry Pi 2 model Bs running Drupal 8 on a cluster
Version 0.9.3 of the Dramble—running Drupal 8 on 6 Raspberry Pis

I've been tinkering with computers since I was a kid, but in the past ten or so years, mainstream computing has become more and more locked down, enclosed, lightweight, and, well, polished. I even wrote a blog post about how, nowadays, most computers are amazing. Long gone are the days when I had to worry about line voltage, IRQ settings, diagnosing bad capacitors, and replacing 40-pin cables that went bad!

But I'm always tempted back into my earlier years of more hardware-oriented hacking when I pull out one of my Raspberry Pi B+/A+ or Arduino Unos. These devices are as raw of modern computers as you can get—requiring you to actual touch the silicone chips and pins to be able to even use the devices. I've been building a temperature monitoring network that's based around a Node.js/Express app using Pis and Arduinos placed around my house. I've also been working a lot lately on a project that incorporates three of my current favorite technologies: The Raspberry Pi 2 model B (just announced earlier this month), Ansible, and Drupal!

In short, I'm building a cluster of Raspberry Pis, and designating it a 'Dramble'—a 'bramble' of Raspberry Pis running Drupal 8.