compile

Raspberry Pi OS isn't really built to be a server OS; the main goals are stability and support for educational content. But that doesn't mean people like me don't use and abuse it to do just about anything.

In my case, I've been doing a lot of network testing lately—first with an Intel I340-T4 PCIe interface for 4.15 Gbps of networking, and more recently (yesterday, in fact!) with a Rosewill 2.5 GbE PCIe NIC.

And since the Pi's BCM2711 SoC is somewhat limited, it can't seem to pump through many Gbps of bandwidth without hitting IRQ limits, and queueing up packets.

In the case of the 2.5G NIC, I was seeing it max out around 1.92 Gpbs, and I just wouldn't accept that (at least not for a raw benchmark). Running atop, I noticed that during testing, the IRQ interrupts would max out at 99% on one CPU core—and it seems like it may be impossible to distribute interrupts across all four cores on the BCM2711.

After doing a video testing different external GPUs on a Raspberry Pi last week, I realized two things:

1. Compiling the Linux kernel on a Raspberry Pi is slow. It took 54 minutes, and I ended up doing it 7 times during the course of testing for that video.
2. If you ever want to figure out a better way to do something, write a blog post or create a video showing the less optimal way of doing it.

To the second point, about every fifth comment was telling me to cross-compile Linux on a faster machine instead of doing it on the Pi itself. For example:

And on the Pi Forums, it seems like nobody worth their salt compiles the kernel on the Pi either, so I figured—since I'm probably going to have to do it again another thousand times in my life—I might as well put together a guide for how to do it on a Mac.