cluster

Hosting this website on a farm - or anywhere

tl;dr: This website is currently being hosted off-grid, on a cluster of Raspberry Pis, via 4G LTE—or at some points through the same tunnel via WiFi if signal strength gets too low. Here's the GitHub repo for the project.

Note: The website was down for a few hours this morning, as shortly after this post I started getting a 40-50 Mbps flood of POST requests (over 6 million in a 30 minute time frame)... and yeah, no way the little Pi cluster could handle that. Thanks, Internet. It's back up through Cloudflare now, and I'll post more on this 'fun' experience later.

A couple weeks ago, after months of preparation, I took my 4-node Turing Pi 2 cluster (see my earlier review) to my cousin's farm, and ran this website (JeffGeerling.com) on it, live on the Internet—completely disconnected from grid power or hard-wired Internet.

Turing Pi 2: 4 Raspberry Pi nodes on a mini ITX board

Last year I spent a bit of time building a Kubernetes cluster with the original Turing Pi. It was fun, and interesting, but ultimately the performance of the Compute Module 3+ it was designed around led me to running my homelab off some newer Pi 4 model B computers, which are at least twice as fast for almost everything I run on them.

Turing Pi 2

So this year, I was excited when the folks at Turing Pi sent me a Turing Pi 2 to test drive. And the board arrived just in time for Patrick Kennedy from ServeTheHome to challenge me to a cluster build-off at Supercomputing '21! Check out his ARM cluster build here.

Why build a Raspberry Pi Cluster?

Raspberry Pi Cluster next to a banana for scale

After I posted my Raspberry Pi Blade server video last week, lots of commenters asked what you'd do with a Pi cluster. Many asked out of curiosity, while others seemed to shudder at the very idea of a Pi cluster, because obviously a cheap PC would perform better... right?

Before we go any further, I'd say probably 90 percent of my readers shouldn't build a Pi cluster.

But some of you should. Why?

Well, the first thing I have to clear up is what a Pi cluster isn't.

Note: This blog post corresponds to my YouTube video of the same name: Why would you build a Raspberry Pi Cluster?. Go watch the video on YouTube if you'd rather watch the video instead of reading this post!

Raspberry Pi Cluster Episode 6 - Turing Pi Review

A few months ago, in the 'before times', I noticed this post on Hacker News mentioning the Turing Pi, a 'Plug & Play Raspberry Pi Cluster' that sits on your desk.

It caught my attention because I've been running my own old-fashioned 'Raspberry Pi Dramble' cluster since 2015.

Raspberry Pi Dramble Cluster with Sticker - 2019 PoE Edition

So today, I'm wrapping up my Raspberry Pi Cluster series with my thoughts about the Turing Pi that I used to build a 7-node Kubernetes cluster.

Video version of this post

This blog post has a companion video embedded below:

Raspberry Pi Cluster Episode 5 - Benchmarking the Turing Pi

At this point, I've showed you how you can use the Turing Pi as a Kubernetes cluster to run different things. I barely scratched the surface of what's possible with Kubernetes, but I'm planning on doing another series exploring Kubernetes itself later this year. Subscribe to my YouTube channel if you want to see it!

In this post, I'm going to talk about the Turing Pi's performance. I'll compare it to a more traditional Raspberry Pi cluster, my Pi Dramble, and talk about important considerations for your cluster, like what kind of storage you should use, or whether you should run a 32-bit or 64-bit Pi operating system.

As with all the other work I've done on this cluster, I've been documenting it all in my open source Turing Pi Cluster project on GitHub.

Video version of this post

This blog post has a companion video embedded below:

Raspberry Pi Cluster Episode 2 - Setting up the Cluster

This post is based on one of the videos in my series on Raspberry Pi Clustering, and I'm posting the video + transcript to my blog so you can follow along even if you don't enjoy sitting through a video :)

.embed-container { position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; } .embed-container iframe, .embed-container object, .embed-container embed { position: absolute; top: 0; left: 0; width: 100%; height: 100%; }

In the first episode, I talked about how and why I build Raspberry Pi clusters.

I mentioned my Raspberry Pi Dramble cluster, and how it's evolved over the past five years.

Raspberry Pi Cluster Episode 1 - Introduction to Clusters

I will be posting a few videos discussing cluster computing with the Raspberry Pi in the next few weeks, and I'm going to post the video + transcript to my blog so you can follow along even if you don't enjoy sitting through a video :)

.embed-container { position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; } .embed-container iframe, .embed-container object, .embed-container embed { position: absolute; top: 0; left: 0; width: 100%; height: 100%; }

This is a Raspberry Pi Compute Module.

7 Raspberry Pi Compute Modules in a stack

And this is a stack of 7 Raspberry Pi Compute Modules.

Mcrouter Operator - demonstration of K8s Operator SDK usage with Ansible

It wouldn't surprise me if you've never heard of Mcrouter. Described by Facebook as "a memcached protocol router for scaling memcached deployments", it's not the kind of software that everyone needs.

There are many scenarios where a key-value cache is necessary, and in probably 90% of them, running a single Redis or Memcached instance would adequately serve the application's needs. There are more exotic use cases, though, where you need better horizontal scaling and consistency.

Monitoring Kubernetes cluster utilization and capacity (the poor man's way)

If you're running Kubernetes clusters at scale, it pays to have good monitoring in place. Typical tools I use in production like Prometheus and Alertmanager are extremely useful in monitoring critical metrics, like "is my cluster almost out of CPU or Memory?"

But I also have a number of smaller clusters—some of them like my Raspberry Pi Dramble have very little in the way of resources available for hosting monitoring internally. But I still want to be able to say, at any given moment, "how much CPU or RAM is available inside the cluster? Can I fit more Pods in the cluster?"

So without further ado, I'm now using the following script, which is slightly adapted from a script found in the Kubernetes issue Need simple kubectl command to see cluster resource usage:

Usage is pretty easy, just make sure you have your kubeconfig configured so kubectl commands are working on the cluster, then run: