cluster

Raspberry Pi Cluster Episode 2 - Setting up the Cluster

This post is based on one of the videos in my series on Raspberry Pi Clustering, and I'm posting the video + transcript to my blog so you can follow along even if you don't enjoy sitting through a video :)

.embed-container { position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; } .embed-container iframe, .embed-container object, .embed-container embed { position: absolute; top: 0; left: 0; width: 100%; height: 100%; }

In the first episode, I talked about how and why I build Raspberry Pi clusters.

I mentioned my Raspberry Pi Dramble cluster, and how it's evolved over the past five years.

Raspberry Pi Cluster Episode 1 - Introduction to Clusters

I will be posting a few videos discussing cluster computing with the Raspberry Pi in the next few weeks, and I'm going to post the video + transcript to my blog so you can follow along even if you don't enjoy sitting through a video :)

.embed-container { position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; } .embed-container iframe, .embed-container object, .embed-container embed { position: absolute; top: 0; left: 0; width: 100%; height: 100%; }

This is a Raspberry Pi Compute Module.

7 Raspberry Pi Compute Modules in a stack

And this is a stack of 7 Raspberry Pi Compute Modules.

Mcrouter Operator - demonstration of K8s Operator SDK usage with Ansible

It wouldn't surprise me if you've never heard of Mcrouter. Described by Facebook as "a memcached protocol router for scaling memcached deployments", it's not the kind of software that everyone needs.

There are many scenarios where a key-value cache is necessary, and in probably 90% of them, running a single Redis or Memcached instance would adequately serve the application's needs. There are more exotic use cases, though, where you need better horizontal scaling and consistency.

Monitoring Kubernetes cluster utilization and capacity (the poor man's way)

If you're running Kubernetes clusters at scale, it pays to have good monitoring in place. Typical tools I use in production like Prometheus and Alertmanager are extremely useful in monitoring critical metrics, like "is my cluster almost out of CPU or Memory?"

But I also have a number of smaller clusters—some of them like my Raspberry Pi Dramble have very little in the way of resources available for hosting monitoring internally. But I still want to be able to say, at any given moment, "how much CPU or RAM is available inside the cluster? Can I fit more Pods in the cluster?"

So without further ado, I'm now using the following script, which is slightly adapted from a script found in the Kubernetes issue Need simple kubectl command to see cluster resource usage:

Usage is pretty easy, just make sure you have your kubeconfig configured so kubectl commands are working on the cluster, then run:

Kubernetes' Complexity

Over the past month, I started rebuilding the Raspberry Pi Dramble project using Kubernetes instead of installing and configuring the LEMP stack directly on nodes via Ansible (track GitHub issues here). Along the way, I've hit tons of minor issues with the installation, and I wanted to document some of the things I think turn people away from Kubernetes early in the learning process. Kubernetes is definitely not the answer to all application hosting problems, but it is a great fit for some, and it would be a shame for someone who could really benefit from Kubernetes to be stumped and turn to some other solution that costs more in time, money, or maintenance!

Raspberry Pi Dramble cluster running Kubernetes with Green LEDs

Highly available Drupal on a Raspberry Pi Cluster - phptek 2016 session

Raspberry Pi Dramble Cluster with Mini Raspberry Pi Zero Cluster

Another year, another field trip for the Pi Dramble—my 5-Raspberry-Pi cluster! I presented a session titled Highly available Drupal on a Raspberry Pi Cluster at php[tek] 2016, which just so happens to have moved to my hometown, St. Louis, MO this year!

For this presentation, I remembered to record the audio using a lav mic plugged into my iPhone, as well as iShowU to record what was on my screen. Sadly, I didn't have a secondary camera to capture the Pi Dramble itself, but you can glance at all the other 'Let's build a Pi Cluster' videos if you want to see it in action!

Here's a video recording of the presentation:

How to Build Your Own Raspberry Pi Cluster ('Bramble')

Rasbperry Pi Dramble

One of the first questions I'm asked by those who see the Dramble is, "How do I build my own?" Since I've been asked the question many times, I put together a detailed parts list, and maintain it on the Dramble's project wiki on GitHub: Raspberry Pis and Accessories.

For a little over $400, you can have the exact same setup, with six Raspberry Pi 2s, a network switch, a rack inside which you can mount the Pis, microSD cards for storage, a 6-port USB power supply, and all the required cables and storage!

Raspberry Pi RGB LED boards

Lessons Learned building the Raspberry Pi Dramble

Raspberry Pi Dramble Bramble Cluster

Edit: Many people have been asking for more technical detail, benchmarks, etc. There is much more information available on the Raspberry Pi Dramble Wiki (e.g. Power Consumption, microSD card benchmarks, etc.), if you're interested.

After the Raspberry Pi 2 model B was released, I decided the Pi was finally a fast enough computing platform (with its 4-core 900 MHz ARMv7 architecture) with enough memory (1 GB per Pi) to actually use for web infrastructure. If not in a production environment (I would definitely avoid putting the Pi into a role as a high-load 24x7x365 server), then for development and/or testing purposes. And for some fun!