raspberry pi

Using 4G LTE wireless modems on a Raspberry Pi

For a recent project, I needed to add cellular connectivity to a Raspberry Pi (actually, an entire cluster... but that's a story for a future time!).

Raspberry Pi 4 model B with 4G LTE wireless Quectel modem and antenna and USB adapter

I figured I'd document the process in this blog post so people who follow in my footsteps don't need to spend quite as much time researching. This post is the culmination of 40+ hours of reading, testing, and head-scratching.

There doesn't seem to be any good central resource for "4G LTE and Linux" out there, just a thousand posts about the ABC's of getting an Internet connection working through a 4G modem—but with precious little explanation about why or how it works. (Or why someone should care about random terms like PPP, ECM, QMI, or MBIM, or why someone would choose qmi_wwan over cdc_ether, or ... I could go on).

Hopefully you can learn something from my notes. Or point out places where I'm glaringly wrong :)

SSH and HTTP to a Raspberry Pi behind CG-NAT

For a project I'm working on, I'll have a Raspberry Pi sitting behind a 4G LTE modem:

Raspberry Pi 4 with 4G LTE modem and antenna on desk

This modem is on AT&T's network, but regardless of the provider, unless you're willing to pay hundreds or thousands of dollars a month for a SIM with a public IP address, the Internet connection will be running behind CG-NAT.

What this means is there's no publicly routable address for the Pi—you can't access it from the public Internet, since it's only visible inside the cell network's private network.

There are a few different ways people have traditionally dealt with accessing devices running through CG-NAT connections:

  1. Using a VPN
  2. Using a one-off tool like ngrok
  3. Using reverse tunnels, often via SSH

And after weighing the pros and cons, I decided to go with option 3, since—for my needs—I want to have two ports open back to the Raspberry Pi:

Autofocus on a Pi - ArduCam's new 16MP camera

ArduCam with other Raspberry Pi Cameras - v2 HQ and Autofocus 16MP

ArduCam recently completed a successful crowdfunding campaign for a 16 megapixel Raspberry Pi camera with built-in autofocus.

The camera is on a board with the same footprint as the Pi Camera V2, but it has a Sony IMX519 image sensor with twice the resolution (16 Mpix vs 8 Mpix) and a larger image sensor (1/2.53" vs 1/4"), a slightly nicer lens, and the headline feature: a built-in autofocus motor.

Autofocus performance

Getting right into the meat of it: autofocus works, with some caveats.

First, the good. Autofocus is quick to acquire focus in many situations, especially in well-lit environments with one main subject. Using ArduCam's fork of libcamera-still or libcamera-vid, you only need to pass in --autofocus and the camera will snap into focus immediately.

Raspberry Pi holds its own against low-cost ARM NAS

Earlier this year, I pitted the $549 ASUSTOR Lockerstor 4 NAS against a homebrew $350 Raspberry Pi CM4 NAS, and came to the (rather obvious) conclusion that the Lockerstor was better in almost every regard.

Jeff Geerling holding Raspberry Pi Radxa Taco NAS board and ASUSTOR Drivestor 4 Pro

Well, ASUSTOR introduced a new lower-cost NAS, the $329 Drivestor 4 Pro (model AS3304T—pictured above), and sent me one to review against the Raspberry Pi, since it make for a better matchup—both have 4-core ARM CPUs and a more limited PCI Express Gen 2 bus at their heart.

Around the same time, Radxa also sent me their new Taco—a less-than-$100 Raspberry Pi Compute Module 4 carrier board with 5x SATA ports, 1 Gbps and 2.5 Gbps Ethernet, an M.2 NVMe slot, and an M.2 A+E key slot. (The Taco will soon be available as part of a kit with a CM4 and case for around $200.)

The specs evenly matched, at least on paper:

Check your driver! Faster Linux 2.5G Networking with Realtek RTL8125B

Since the Raspberry Pi Compute Module 4 was introduced last year, I've been testing a variety of PCI Express NICs with it. One of the main types of NIC I'm interested in is cheap 2.5 Gigabit Ethernet adapters.

2.5 Gigabits is about the highest reasonable bandwidth you can get through the PCI Express Gen 2.0 x1 lane on the Raspberry Pi, and it's also a lot more accessible than 10 Gigabit networking, especially for home users who might already have Cat5e runs that they are loathe to swap out for Cat6 or better cabling.

In my testing, besides discovering that not all 10 Gbps SFP+ transceivers are created equal, I found out that when it comes to performance, the Linux driver you're using matters—a lot.

My Backup Plan

I've had a number of people ask about my backup strategy—how I ensure the 6 TB of video project files and a few TB of other files stays intact over time.

3-2-1 backup plan

Over the past year, since I got more serious about my growing YouTube channel's success, I decided to document and automate as much of my backups as possible, following a 3-2-1 backup plan:

  • 3 Copies of all my data
  • 2 Copies on different storage media
  • 1 Offsite copy

The culmination of that work is this GitHub repository: my-backup-plan.

The first thing I needed to do was take a data inventory—all the files important enough for me to worry about fell into six main categories:

6 backup categories

Install Python 3.9 on Raspberry Pi OS or Debian 10 (for Ansible or other uses)

I've started getting a lot of bug reports on my repos to the effect of "Ansible won't install on my Raspberry Pi anymore". Accompanying it is a debug message like one of the following:

$ python3 -m pip install ansible
No matching distribution found for ansible-core<2.13,>=2.12.0 (from ansible)

# Alternatively:
ERROR: No matching distribution found for ansible-core<2.13,>=2.12.0

The problem is ansible-core 2.12 has a new hard requirement for Python 3.8 or newer. And ansible-core 2.12 is included in Ansible 5.0.0, which was recently released. Raspberry Pi OS, which was based on Debian 10 ("Buster") until recently, includes Python 3.7, which is too old to satisfy Ansible's installation requirements.

There was recently a fix that makes it so Ansible 5.x won't get installed on these older systems, but who wants to get stuck on old unsupported Ansible versions?

There are three options:

Turing Pi 2: 4 Raspberry Pi nodes on a mini ITX board

Last year I spent a bit of time building a Kubernetes cluster with the original Turing Pi. It was fun, and interesting, but ultimately the performance of the Compute Module 3+ it was designed around led me to running my homelab off some newer Pi 4 model B computers, which are at least twice as fast for almost everything I run on them.

Turing Pi 2

So this year, I was excited when the folks at Turing Pi sent me a Turing Pi 2 to test drive. And the board arrived just in time for Patrick Kennedy from ServeTheHome to challenge me to a cluster build-off at Supercomputing '21! Check out his ARM cluster build here.

2021 Open Source Pay-it-Forward Pi Giveaway

This year, I wanted to solve two problems:

  1. Open source projects and maintainers often get no reward (even a simple word of thanks!) for their efforts maintaining the tools we rely on every day.
  2. I have a box full of really awesome Raspberry Pi Compute Module 4 boards and products like the CutiePi, a PiBox mini 2, and a MirkoPC!

To solve both problems, I'm doing a giveaway—to enter to win one of any of the pictured items below (and maybe a few others I can find lurking in my office), just donate or say thank you to any open source project or maintainer, then submit your entry.

OSSThanks Giveaway items

The drawing will be at random and should be held next Friday, so please make sure to fill out the entry form by then!