performance

Raspberry Pi USB Boot - UASP, TRIM, and performance

In the past few weeks, I reviewed USB drive performance on the Raspberry Pi 4, and the importance of UASP support for USB drive performance.

Both posts generated great discussion, and there were three things I wanted to cover in this follow-up, namely:

  1. Which drives support UASP
  2. Real-world performance benchmarks
  3. TRIM support

For reference, here are all the products I'm testing in this post (product links are to their Amazon product page, starting from top middle, clockwise):

USB Performance testing - SATA SSD, NVMe, and Flash drives

Testing how long it takes Chromium to open, load a web page, and quit on Debian

Something I've long been meaning to benchmark, but never really got around to, is benchmarking the amount of time it takes on a Raspberry Pi to open a browser, load a page, and quit.

This is a relatively decent thing to benchmark, compared to other raw performance metrics, because it's something that probably 99% of Raspberry Pi users who use it with a GUI will do, with some frequency (well, probably loading more than one page before quitting, but still...).

So I asked on Twitter:

Enabling TRIM on an external SSD on a Raspberry Pi

I've been doing a lot of benchmarking and testing with the Raspberry Pi 4 and SSDs connected via USB. I explored UASP Support, which USB SSDs are the fastest, and I'm now booting my Pis from USB SSDs.

Anyways, one thing that I have wondered about—and some people have asked me about—is TRIM support.

I'm working on a new video for my YouTube channel that will go into some more detail on which of the drives I tested support TRIM, but while I was researching for that video, I also found that TRIM support in Linux is not as simple as it seems at first glance—it's definitely not plug-and-play, in my experience.

While internal microSD cards seem to support TRIM out of the box, none of the external USB drives I tested supported it out of the box. They all needed a little help!

The fastest USB storage options for Raspberry Pi

For years, I've been maintaining benchmarks for microSD cards on the Raspberry Pi, but I only spent a little time testing external USB storage, due to historic limitations with the Pi's USB 2.0 bus.

But the Pi 4 cleared away the limitations with a full-speed USB 3.0 bus offering much better performance, so I've done a lot of testing with USB boot, and with all the USB SSDs I had at my disposal. You can see some of those results in this blog post and video on booting a Pi 4 via USB.

After posting my tests concerning UASP support in USB SATA adapters, I got an email from Rob Logan mentioning the performance of some other types of drives he had with him. And he even offered to ship a few drives to me for comparisons!

There's also a video that accompanies this blog post, for the more visually-inclined:

The Pi 4 Compute Module might support NVMe storage

There is a companion video to this post: Is fast NVMe storage coming to the Raspberry Pi?.

A couple days ago, Tom's Hardware posted an article stating NVMe support might be coming to the Raspberry Pi Compute Module 4.

On the first episode of The Pi Cast, Eben Upton, the CEO of Raspberry Pi, said "microSD will always be the baseline for storage", but "it's fairly likely we'll support NVMe soon on the Compute Module 4, to some degree, using single-lane PCI Express." (Skip to about 11 minutes into the video for the NVMe discussion).

He also said NVMe support is not without cost, since there's an extra connector silicon required. And with the System on a Chip used in the Pi 4, there's also a tradeoff involved: There's only one PCIe 1x lane, and it's currently used for the Pi 4's USB 3.0. If you want to add NVMe support, you'd have to drop the USB 3.0 ports.

UASP makes Raspberry Pi 4 disk IO 50% faster

You can view a video related to this blog post here: Does UASP make the Raspberry Pi faster?.

A couple weeks ago, I did some testing with my Raspberry Pi 4 and external USB SSD drives. I found a USB 3.0 SSD was ten times faster than the fastest microSD card I tested.

In the comments on the video associated with that post, Brad Manske mentioned something I never even thought about. He noticed that I had linked to an Inateck USB 3.0 SATA case that didn't have UASP.

What's UASP, you might ask?

I'm booting my Raspberry Pi 4 from a USB SSD

September 2020 Update: USB boot is out of beta! Check out this video for simplified instructions. All you need to do now is run sudo apt-get dist-upgrade -y, then reboot, then your firmware should be up to date. Now, flash any USB drive with the latest Raspberry Pi OS, plug it into your Pi (unplugging any microSD card), and you're off to the races!

Recently, the Raspberry Pi Foundation announced a USB boot beta for the Raspberry Pi 4. For a very long time, the top complaint I've had with the Raspberry Pi is limited I/O speed (especially for the main boot volume). And on older Pis, with the maximum external disk speed limited especially by the USB 2.0 bus—which was shared with the network adapter, limiting its bandwidth further—even USB booting didn't make things amazing.

Revisiting Docker for Mac's performance with NFS volumes

tl;dr: Docker's default bind mount performance for projects requiring lots of I/O on macOS is abysmal. It's acceptable (but still very slow) if you use the cached or delegated option. But it's actually fairly performant using the barely-documented NFS option!

July 2020 Update: Docker for Mac may soon offer built-in Mutagen sync via the :delegated sync option, and I did some benchmarking here. Hopefully that feature makes it to the standard Docker for Mac version soon.

Ever since Docker for Mac was released, shared volume performance has been a major pain point. It was painfully slow, and the community finally got a cached mode that offered a 20-30x speedup for common disk access patterns around 2017. Since then, the File system performance improvements issue has been a common place to gripe about the lack of improvements to the underlying osxfs filesystem.

Limiting disk iops on a larger Munin server using rrdcached

I've long used Munin for basic resource monitoring on a huge variety of servers. It's simple, reliable, easy to configure, and besides the fact that it uses Perl for plugins, there's not much against it!

Last week, I got a notice from my 'low end box' VPS provider that my Munin server—which is aggregating data from about 50 other servers—had high IOPS and would be shut down if I didn't get it back into an allowed threshold. Most low end VPSes run things like static HTML websites, so disk IO is very low on average. I checked my Munin instance, and sure enough, it was constantly churning through around 50 iops. For a low end server, this can cause high iowait for other tenants of the same server, so I can understand why hosting providers don't want applications on their shared servers doing a lot of constant disk I/O.

Using iotop, I could see the munin-update processes were spending a lot of time writing to disk. And munin's own diskstats_iops plugin showed the same:

Raspberry Pi microSD follow-up, SD Association fools me twice?

 ____________________________________________
/ Fool me once, shame on you. Fool me twice, \
\ prepare to die. (Klingon Proverb)          /
 --------------------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

(Excerpt from Ansible for DevOps, chapter 12.)

The fallout from this year's microSD card performance comparison has turned into quite a rabbit hole; first I found that new 'A1' and 'A2' classifications were supposed to offer better performance than the not-Application-Performance-class-rated cards I have been testing. Then I found that A2 rated cards offer no better performance for the Raspberry Pi—in fact they didn't even perform half as well as they were supposed to, for 4K random reads and writes, on any hardware I have in my possession.