Check your driver! Faster Linux 2.5G Networking with Realtek RTL8125B

Since the Raspberry Pi Compute Module 4 was introduced last year, I've been testing a variety of PCI Express NICs with it. One of the main types of NIC I'm interested in is cheap 2.5 Gigabit Ethernet adapters.

2.5 Gigabits is about the highest reasonable bandwidth you can get through the PCI Express Gen 2.0 x1 lane on the Raspberry Pi, and it's also a lot more accessible than 10 Gigabit networking, especially for home users who might already have Cat5e runs that they are loathe to swap out for Cat6 or better cabling.

In my testing, besides discovering that not all 10 Gbps SFP+ transceivers are created equal, I found out that when it comes to performance, the Linux driver you're using matters—a lot.

And as a stark illustration of this point, I recently pitted an ASUSTOR Drivestor 4 Pro NAS against a Raspberry Pi 'Taco' NAS. Both of these devices use the popular Realtek RTL8125B chip, and both of them were reporting underwhelming iperf3 bandwidth results:

iperf3 bandwidth - kernel driver

The maximum throughput was eerily similar, just under 1.9 Gbps. And worse, I noticed when using devices through a PCI Express switch on the Pi, like a SATA RAID array, the network throughput took a hit, down to 1.51 Gbps on the Pi.

I wondered if the driver might've seen some improvements in later revisions of the Linux kernel, so I recompiled the kernel using the 5.15.x source:

menuconfig - Realtek 2.5G network driver in Linux kernel source tree

That made no difference—in fact, the results started to have more jitter in them, so I decided that avenue wasn't worth pursuing further.

So next, I downloaded the driver directly from Realtek's website, and installed version 9.007.01:

$ sudo apt-get install -y raspberrypi-kernel-headers
$ tar vjxf r8125-9.007.01.tar.bz2
$ cd r8125-9.007.01/
$ sudo ./autorun.sh

I re-ran my benchmarks, and wouldn't you know? I'm able to get the full 2.35 Gbps of real-world throughput through the card—regardless of other traffic on the PCIe bus!

iperf3 bandwidth - realtek driver

So the moral of the story seems to be: if you're not seeing the performance you expect, see if the vendor's driver is better than what's in the kernel tree.

Many vendors take an upstream-first approach, where they make sure any driver optimizations make their way to the kernel quickly... but not all. I'm not sure how the discrepancy is so big, though, in this case. It looks like 8125B support was only added a year and a half ago, and only a few chip-specific commits have been added since.

I have to wonder if maybe since far fewer Linux users have 2.5G-capable networks, these kinds of performance issues aren't discussed as much?

I confirmed with ASUSTOR that their current Drivestor 4 Pro software is running the driver from the kernel, and not the Realtek vendor driver. No word yet on if that could change—but if so, it may be able to unlock more performance from their value-lineup of 2.5G NASes!

For all my debugging details, check out my GitHub issue NAS Comparison - ASUSTOR Drivestor 4 Pro vs Pi CM4.

Comments

Interesting! The RTL 8125B chip seems to be quite common on PC mainboards as well - I just noticed that I'm using the same chip in my recently build SFF PC (Asus ROG Strix B560I). The firmware is really quite old: rtl8125b-2_0.0.2 07/13/20 (kernel 5.15.8)

However, all my other devices have 1GbE or less, so it doesn't matter anyway (at least for the forseeable future, I'm confident that the driver will be updated eventually).

I got an AMD 5 mini computer with dual 2g and 5G WiFi. The 5G connection came up and choosing it is much faster. Same cable modem from cable company install 2 yes ago...is it possible it has 5G on it or does 2G run faster to a 5G receiver?