networking

Two Tiny Dual-Gigabit Raspberry Pi CM4 Routers

Since I started testing various PCI Express cards on the Raspberry Pi Compute Module 4, I've been excited to see what new kinds of custom networking devices people would come up with.

Well, after months of delays due to part shortages, both DFRobot and Seeed Studios have come out with their 2-port Gigabit router board designs, and I was happy to receive a sample of each for testing:

DFRobot and Seeed Studios Router Boards with Dual Gigabit Ethernet

The boards are tiny, and even with the Compute Module 4 installed, they are incredibly small—take a look at the entire assembled DFRobot unit, complete with a Raspberry Pi attached:

DFRobot CM4 IoT Router Board with Raspberry Pi CM4 and Quarter

Setting up a Raspberry Pi with 2 Network Interfaces as a very simple router

I needed a very basic 'Internet sharing' router setup with one of my Raspberry Pis, and I thought I'd document the setup process here in case I need to do it again.

I should note that for more complex use cases, or where you really need to worry about security and performance, you should use something like OpenWRT, pfSense, or VyOS—or just buy a decent out-of-the-box router!

Seeed Studios Raspberry Pi Compute Module 4 Router Board

But I needed a super-simple router setup for some testing (seriously... look at the picture—the thing's about to fall off my desk!), and I had two network interfaces on a Raspberry Pi running the 64-bit build of Raspberry Pi OS. These instructions work on that OS, as well as Debian, Ubuntu, and derivative distros.

Taking control of the Pi PoE HAT's overly-aggressive fan

I am starting to rack up more Pis (quite literally) using the official Pi PoE HAT to save on cabling.

The one thing I hate most about those little HATs is the fact the fans spin up around 40°C, and then turn off a few seconds later, once the temperature is back down to 39 or so, all day long.

I'd be happy to let my Pis idle around 50-60°C, and only have the little whiny fans come on beyond those temperatures. Even under moderate load, the Pi rarely goes above 55°C in my basement, where there's adequate natural convection, so the fans would only really be necessary under heavy load.

Testing 2.5 Gbps Ethernet on the Raspberry Pi CM4

Rosewill 2.5 Gbps Ethernet adapter PCIe 1x card

I got this Rosewill RC-20001 PCIe 2.5 Gbps Network Adapter working on the Raspberry Pi Compute Module 4.

Right after I got the card working, though, I tested it in an external powered PCI Express riser, and that test released the card's magic smoke. Oops.

Here's a dramatic re-enactment that's actually pretty accurate to what it looked like in real life:

PCIe card lets out magic smoke

Luckily, buying a replacment wasn't too bad, since the card is less than $20. But to get it to work on my spiffy new ten gigabit network, I also had to buy a new SFP+ transceiver that was compatible with 1, 2.5, 5, and 10 Gbps data rates, and that cost $60!

Setting 9000 MTU (Jumbo Frames) on Raspberry Pi OS

Raspberry Pi OS isn't really built to be a server OS; the main goals are stability and support for educational content. But that doesn't mean people like me don't use and abuse it to do just about anything.

In my case, I've been doing a lot of network testing lately—first with an Intel I340-T4 PCIe interface for 4.15 Gbps of networking, and more recently (yesterday, in fact!) with a Rosewill 2.5 GbE PCIe NIC.

And since the Pi's BCM2711 SoC is somewhat limited, it can't seem to pump through many Gbps of bandwidth without hitting IRQ limits, and queueing up packets.

In the case of the 2.5G NIC, I was seeing it max out around 1.92 Gpbs, and I just wouldn't accept that (at least not for a raw benchmark). Running atop, I noticed that during testing, the IRQ interrupts would max out at 99% on one CPU core—and it seems like it may be impossible to distribute interrupts across all four cores on the BCM2711.

5 Gbps Ethernet on the Raspberry Pi Compute Module 4

tl;dr: I successfully got the Intel I340-T4 4x Gigabit NIC working on the Raspberry Pi Compute Module 4, and combining all the interfaces (including the internal Pi interface), I could get up to 3.06 Gbps maximum sustained throughput.

Update: I was able to boost things a bit to get 4.15 Gbps! Check out my video here: 4+ Gbps Ethernet on the Raspberry Pi Compute Module 4.

After my failure to light up a monitor with my first attempt at getting a GPU working with the Pi, I figured I'd try something a little more down-to-earth this time.

And to that end, I present to you this four-interface gigabit network card from Intel, the venerable I340-T4:

Intel I340-T4 NIC for PCI Express x4

Debugging networking issues with multi-node Kubernetes on VirtualBox

Since this is the third time I've burned more than a few hours on this particular problem, I thought I'd finally write up a blog post. Hopefully I find this post in the future, the fourth time I run into the problem.

What problem is that? Well, when I build a new Kubernetes cluster with multiple nodes in VirtualBox (usually orchestrated with Vagrant and Ansible, using my geerlingguy.kubernetes role), I get everything running. kubectl works fine, all pods (including CoreDNS, Flannel or Calico, kube-apiserver, the scheduler) report Running, and everything in the cluster seems right. But there are lots of strange networking issues.

Sometimes internal DNS queries work. Most of the time not. I can't ping other pods by their IP address. Some of the debugging I do includes:

Raspberry Pi (or another device) suddenly not getting a DHCP address?

Tonight, after I made a couple changes to my wired in-house Gigabit network (I recently added a few Cat6 runs after moving my main Wireless router—in this case an AirPort Extreme base station), I noticed the Raspberry Pi webserver that was hosting www.pidramble.com wasn't reachable over the network, and Server Check.in started reporting an outage.

I have that particular device set using a DHCP Reservation based on it's MAC address, and it's been working like a champ for over a year. So something was strange, since I hadn't made any networking configuration changes on the Pi itself in a few months, nor had I unplugged it at all in the past month.

Setup a FOSCAM WiFi camera directly connected to a Mac via Ethernet

I have two FOSCAM WiFi IP cameras set up in my house (in addition to an outdoor Arlo I use for security purposes), and generally all three of these cameras give a reliable connection and work without much intervention.

Recently, however, one of the FOSCAM cameras decided to stop connecting to the wireless network. I needed to get access to the camera via the wired interface, but I didn't want to have to bring the camera all the way to where my wired networking drops were located, because it's in a bit of an inconvenient area of the basement in the midst of some cleanup we're doing.

So I plugged the FOSCAM directly into my Mac's USB 3.0 ethernet adapter, and got blinky lights... but couldn't connect to the camera because it's wired connection is configured to use DHCP by default, and there's no Bonjour/zeroconf configuration.

To get around this and directly connect to the camera, I did the following:

SSH into a Raspberry Pi with only a network cable using OS X's 'Internet Sharing'

Recently, I found myself in a situation where I had to connect to a Raspberry Pi to set it up for a presentation, but I did not have:

  • A keyboard and/or other input device to use to type anything into the Pi
  • An HDMI cable to connect the Pi to a display so I could view anything on the Pi
  • A microSD card reader so I could modify the contents of the Pi's microSD card

Because of this, none of the standard methods of setting a static IP address, reconfiguring the Pi's WiFi configuration, or logging in on the Pi itself to find it's IP address or set things up so I could connect over a local network would work.

I remembered that Mac OS X handily includes an 'Internet Sharing' feature, which sets up a bridged network interface so your Mac is effectively a router and DHCP server to any devices connected to the shared interface.