Network interface routing priority on a Raspberry Pi

52Pi Raspberry Pi Compute Module 4 Router Board

As I start using Raspberry Pis for more and more network routing activities—especially as the Compute Module 4 routers based on Debian, OpenWRT, and VyOS have started appearing—I've been struggling with one particular problem: how can I set routing priorities for network interfaces?

Now, this is a bit of a loaded question. You could dive right into routing tables and start adding and deleting routes from the kernel. You could mess with subnets, modify firewalls, and futz with iptables.

But in my case, my need was simple: I wanted to test the speed of a specific interface, either from one computer to another, or over the Internet (e.g. via speedtest-cli).

The problem is, even if you try limiting an application to a specific IP address (each network interface has its own), the Linux kernel will choose whatever network route it deems the best.

But here's the problem: if I have eth0, connected to the Internet via a fast gigabit connection, and usb0, connected over slow 4G, the route will always be delivered through eth0.

Can't change the metric

And that's because by default, at least on Debian / Raspberry Pi OS (which uses dhcpcd), the routing table looks like this:

pi@turing-node-1:~ $ ip route list
default via 10.0.100.1 dev eth0 proto dhcp src 10.0.100.250 metric 202 
default via 192.168.225.1 dev usb0 proto dhcp src 192.168.225.57 metric 203 mtu 1360 
10.0.100.0/24 dev eth0 proto dhcp scope link src 10.0.100.250 metric 202 
192.168.225.0/24 dev usb0 proto dhcp scope link src 192.168.225.57 metric 203 mtu 1360

The metric (which is 202 for eth0, and 203 for usb0) indicates to Linux that the lowest interface (eth0) should be used for any routing to IPs within its scope—which if you run route -n, you'll find the netmask (labeled Genmask here) is 0.0.0.0 for both interfaces.

pi@turing-node-1:~ $ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.0.100.1      0.0.0.0         UG    202    0        0 eth0
0.0.0.0         192.168.225.1   0.0.0.0         UG    203    0        0 usb0
10.0.100.0      0.0.0.0         255.255.255.0   U     202    0        0 eth0
192.168.225.0   0.0.0.0         255.255.255.0   U     203    0        0 usb0

So in practice, this means that any traffic out to the Internet will be routed through eth0. And traditionally, you could use the ifmetric command to change the metric on the fly:

$ sudo apt install ifmetric
$ sudo ifmetric eth0 220

But that doesn't work:

pi@turing-node-1:~ $ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.0.100.1      0.0.0.0         UG    202    0        0 eth0
0.0.0.0         192.168.225.1   0.0.0.0         UG    203    0        0 usb0
10.0.100.0      0.0.0.0         255.255.255.0   U     202    0        0 eth0
192.168.225.0   0.0.0.0         255.255.255.0   U     203    0        0 usb0

dhcpcd priorities

Because the system uses dhcpcd, you have to override metrics in dhcpcd's config, then restart the service.

The rules for dhcpcd metric calculations seems to be:

  • Metrics are used to prefer an interface over another one, lowest wins.
  • dhcpcd will supply a default metric of 200 + if_nametoindex(3).
  • An extra 100 will be added for wireless interfaces.

Therefore, it seems that since usb0 comes after eth0 alphabetically, it loses and becomes the 203 to eth0's 202.

Basically:

# Edit dhcpcd's configuration.
$ sudo nano /etc/dhcpcd.conf

# Add the following to the end of the file and save it:
interface usb0
metric 0

# Restart dhcpcd.
$ sudo systemctl restart dhcpcd

# Et voila!
$ ip route list
default via 192.168.225.1 dev usb0 proto dhcp src 192.168.225.57 mtu 1360 
default via 10.0.100.1 dev eth0 proto dhcp src 10.0.100.250 metric 202 
10.0.100.0/24 dev eth0 proto dhcp scope link src 10.0.100.250 metric 202 
192.168.225.0/24 dev usb0 proto dhcp scope link src 192.168.225.57 mtu 1360

At this point, I can guarantee my Internet packets will go out through the usb0 interface, so I can run my benchmarks through it—even without disconnecting or otherwise disabling eth0. This is very helpful when benchmarking a Pi headless without serial console access!

Reverting the changes to dhcpcd and restarting the service got the order back to how it was before.

Thanks especially to NetBeez's post Linux for Network Engineers: How to Set Route Priorities, which, after many hours of DuckDuckGo'ing got me to this answer.

Comments

Thank you! I have a RPi VPN and my ISP is sometimes crap, so I was trying to go through Wifi via my phone hot-spot as a stop-gap when that happens, but could never get it to route properly. I will give this a try - looks like exactly what I was needing!!!

As far as I remember this route setting has to be repeated when connection drops or DHCP lease is renewed. So maybe that is the cause it didn't work for you.

I tried this edit to prioritize wifi over ethernet, but after restarting dhcpcd I immediately lost connectivity to my RPi from my VNC program, and I could not restore it. Neither from the wireless network, nor through the router eth network.

Did I do something wrong? I simply copied

interface wlan0
metric 0

at the end of my dhcpcd.conf file, and saved it, like you said

Yes, DHCPCD IS broken in spite of the fact that it faithfully supports an obscure RFC that breaks for every other dhcp implementation I tested (Windows, OS X, Android, iPhone, Amazon IoT devices). I got a wild hair one night and spent from 9pm to 5am the next morning taking packet and traces to determine what exactly it was doing "wrong".

There IS a solution. Don't use it, even if it is default.
Remove DHCPCD and use the ISC dhcp client. It IS available in the Debian, Raspberry Pi OS repositories (and most other distros).

Thanks for publishing this article. I ran into these kind of problems using two WiFi interfaces, where only one was connected to the internet. I figured out the metric issue and solved it by setting interface priority in the network config, but if the device was out of range for a while, the routes were reconfigured after returning to the network.
In the (way older) desktop/server linux distributions i have worked with, it was possible to set a system wide default gateway.
Things have changed and i'm glad i found this article to resolve things quickly.

Useful info. I'm working on an additional tweak: Automatically setting up a rule and tables using dhcpcd hook scripts. At the very list, it should set up a default route when the source address is that of the non-default interface. This is useful when running speedtest-cli to test the "slow" ISP.

Thank you for a great article.
I have a Raspberry PI 4 that I'm using as a Plex server and would like to change the WAN between the eth0 and wlan0.
In my case this is between a Starlink wireless client connection and a hard wire Cellular based hub.
As currently defined routers at 192.168.1.x and 0.x. the 0.x is the hardwired line to the cellular hub.
I've not made any changes currently and this is what my route looks like.

pi@raspberrypi:~ $ ip route list
default via 192.168.0.1 dev eth0 src 192.168.0.111 metric 202
default via 192.168.0.1 dev wlan0 src 192.168.1.111 metric 303
192.168.0.0/24 dev eth0 proto dhcp scope link src 192.168.0.111 metric 202
192.168.1.0/24 dev wlan0 proto dhcp scope link src 192.168.1.111 metric 303

pi@raspberrypi:~ $ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 192.168.0.1 0.0.0.0 UG 202 0 0 eth0
default 192.168.0.1 0.0.0.0 UG 303 0 0 wlan0
192.168.0.0 0.0.0.0 255.255.255.0 U 202 0 0 eth0
192.168.1.0 0.0.0.0 255.255.255.0 U 303 0 0 wlan0

I'm looking for a Fail over WAN switching between eth0 and wlan0
Any thoughts on an easy way to do this ?

thank you!!!! you just solved an painful problem i was having with my truck house backup network configuration. was using ifmetric and crontab as a temp fix because i just couldnt find where to set the priorities at the interface level. so many internet sources say my only option is ifmetric