I'm booting my Raspberry Pi 4 from a USB SSD

Recently, the Raspberry Pi Foundation announced a USB boot beta for the Raspberry Pi 4. For a very long time, the top complaint I've had with the Raspberry Pi is limited I/O speed (especially for the main boot volume). And on older Pis, with the maximum external disk speed limited especially by the USB 2.0 bus—which was shared with the network adapter, limiting its bandwidth further—even USB booting didn't make things amazing.

But the Pi 4 not only separated the network adapter from the USB bus, it also has USB 3.0, which can be 10x faster than USB 2.0 (theoretically). So when the USB boot beta was announced, I wanted to put it through its paces. And after testing it a bit, I decided to use the Pi 4 as my full-time workstation for a day, to see whether it can cope and where it falls short. I'll be posting a video and blog post with more detail on that experience very soon.

Update: I now have a video that goes along with this blog post:

Getting the Pi 4 to USB boot

First, I flashed a 32GB SanDisk Extreme Pro microSD card with the latest 64-bit beta release of the Raspberry Pi OS. In the future, you'll be able to download it from the regular Pi OS download page, but for now it's available from this forum thread.

To flash the card, I still rely on good old dd on my Mac, but you can use the Raspberry Pi Imager instead.

NOTE: You could break your Pi's firmware and render it inoperable if you do something wrong here. I cannot be held responsible for anything that happens with your Pi if you try using this beta feature!

Then I followed the USB Boot beta setup instructions and did the following:

  1. Booted the Raspberry Pi with the microSD card I just flashed.
  2. Opened Terminal in Raspberry Pi OS (note: you can do these steps from another computer via SSH if you want to set up the Pi headless).
  3. Ran the following commands:

     sudo apt update
     sudo apt full-upgrade
    
  4. Edited the rpi-eeprom-update file and changed the 'CRITICAL' value to 'BETA', using nano /etc/default/rpi-eeprom-update.

  5. I ran the command to update the EEPROM:

    sudo rpi-eeprom-update -d -f /lib/firmware/raspberrypi/bootloader/beta/pieeprom-2020-06-03.bin

  6. Reboot the Pi and check the bootloader version by opening Terminal and running:

    vcgencmd bootloader_version

This should output something like:

pi@raspberrypi:~ $ vcgencmd bootloader_version
Jun  3 2020 13:53:47
version b5de8c32f4f45a12a1fdfe107254df82965f9d56 (release)
timestamp 1591188827

Hooray, you're halfway there!

Next, you need to flash the Pi OS to the external USB SSD or HDD. First things first, it's a good idea to plug your drive into the Pi while it's booted and make sure the Pi recognizes it (it should appear on your desktop, or you can also look for it with lsusb). Not all external drives and USB to SATA adapters work out of the box.

In case you're wondering, I'm using a cheap Kingston A400 240GB SSD inside an Inateck USB 3.0 SATA enclosure (make sure to get the one with UASP support, it's a lot faster). And yes, those are affiliate links.

So plug the USB drive into your main computer (where you flashed the microSD card earlier), and flash the Raspberry Pi OS to it the same way you did to the microSD card.

Before you eject the boot volume, you need to replace some files on it with the latest versions from GitHub:

  1. Go to the raspberrypi/firmware GitHub repository and download the zip or clone the project to your computer (get the default master branch).
  2. Inside the boot folder, copy over all the files that end in .elf or .dat to the boot volume of your USB drive (replacing the same-named files that already exist there).
  3. Eject the boot volume, and unplug the USB drive.

Now, it's time to see if everything worked!

Shut down the Pi if it's currently running from the microSD card. Then unplug the microSD card, and plug in the USB drive.

Make sure you plug the drive into a USB 3.0 port (the blue-colored ones), and not one of the USB 2.0 ports (the black-colored ones), or else you'll be severely limited in throughput.

Power up the Pi, and after a minute or so (it has to expand the USB drive to fill the volume and then reboot), it should boot up!

If you're like me, though, you may run into this screen after the soft reboot:

Raspberry Pi 4 failed to soft reboot with external USB SSD

That's okay; it seems related to the bug Bootloader can't boot via USB-HDD after system reboot, and the solution (for now) is to unplug the Pi to power it down completely, then plug it back in.

Now, assuming your USB drive or SATA adapter is compatible, the Pi should boot right up, and you'll find it much faster than before, when it was booting off a microSD card!

Benchmarks

To get a sense of the raw performance difference between disk access while booted from the microSD card (one of the best you can buy, according to my 2019 Raspberry Pi microSD card comparison) versus booting from the Kingston USB SSD.

The first benchmark gets a synthesis of large file write activity, plus small (4K) file random access read/write activity. Both are important, but in many ways, differences in the latter are amplified when using a general purpose computer like the Pi, because computers don't just read and write large files all day (lots of sequential access) like video cameras or drones do (that's the use case for which most microSD cards are optimized).

If you want to run this benchmark on your own; it's documented in the Pi Dramble Wiki's disk access benchmark page. Note that for the USB SSD, I modified the hdparm test to use the path /dev/sda1 instead of /dev/mmcblk0.

Pi 4 Disk Benchmarks comparing microSD to USB SSD

Before you ask: yes, I ran these benchmarks four times (discarding the first result). I ran them on a clean brand new just-flashed system. I flashed both the microSD card and USB drive with the exact same Raspberry Pi OS 64-bit beta image file. I had the Pi running in a case with a fan and the Pi did not throttle at any time.

The results really speak for themselves. For sequential operations, using a USB SSD is 3-4x faster than using a microSD card. And for random access, random reads are a bit faster, but writes are about 6x faster! This makes a difference in many activities, like launching apps, running a web browser with many tabs. And the sequential performance means you should be able to stream and/or record 4K or HD video easily while the Pi has enough bandwidth for other things too (assuming you're not re-encoding via the CPU—this kills the CPU).

To get an idea of how the disk access affects the performance of a real-world application I'm familiar with (and am able to benchmark thoroughly with highly accurate results), I also ran a set of Drupal benchmarks, using the Pi Dramble Drupal benchmarks I've been running on Pis for years. I ran them against an installation of Drupal Pi, which runs Drupal and MariaDB in Docker containers, accessed through Nginx.

Here are the results:

Drupal PHP Installation Benchmarks on Pi 4 with microSD vs USB SSD

Same as before, I ran all tests 4x, and since it was such a difference, I went ahead and rebooted and ran them again; all results were less than 0.5% apart on the same configuration, so it's pretty apparent the SSD makes a huge different in many operations—Drupal's installation and first page load result in hundreds of files being accessed and/or written to disk, so it makes sense it's a lot faster on the SSD.

I also tested another scenario: loading and reloading pages in Drupal:

Drupal Page Load benchmarks - microSD vs SSD

These benchmarks show that for some operations, the disk IO performance is not that important. In Drupal's case, the data required to process and return the response for each page load is cached (both in PHP's case, in the opcache, and in the database's cache, in its query caches), so actual reads and writes are minimized.

Summary

So where do we go from here? For the first time, I think I'm convincing myself that a Raspberry Pi 4 could be a competent general purpose computer for some people. In the past it's mostly been a fun aside for general computing, with a number of severe limitations. Since the original model B, each of the limitations has been removed, and outside of a few nice-to-haves (e.g. built-in fast NVMe storage, maybe full USB-C support, and more speed from the CPU), it's actually not that bad. And it's silent, if you use it caseless or with a decent heat-sink case like the Flirc (which I reviewed alongside other options in The best way to keep your cool running a Raspberry Pi 4).

And I'm not just speaking platitudes: today, I used the Raspberry Pi for all my daily work, to see what rough edges lay in the path towards considering it as a daily general purpose workstation.

In comparison to my Dell XPS 13 and MacBook Pro 16, the option of silence, the very low power consumption, and the comparatively low cost of entry make the Pi 4 appealing for more use cases than ever before.

Before you dive into the comments and tear apart the last few paragraphs above, please consider that I don't need 'death star'-level power available at all times. I know a 64-core Threadripper with 1 TB of memory and a RAID array of 1 TB NVMe SSDs is going to beat the pants off a Raspberry Pi 4 at full tilt. It's even more power efficient (watts per unit of computing)!

But I like to compare it to a truck: if you want to haul 80,000 lbs of materials, you're going to need a beefy semi tractor-trailer rig (a Threadripper). But would you like to park that in your driveway for day-to-day light hauling activities when you could get a small pickup (a Pi 4) for 1/10th the cost? And is it more fun (as a general rule...) to drive the pickup, or the semi?

Though stretching the analogy a bit further, the Pi is probably more like a Chevy Aveo compared to a Macbook Air or Dell XPS being the pickup truck.

Comments

Hi Jeff, thanks for the article. Great to see some speed comparisons as well, definitely worth it!

I'm going to do the same thing soon with my Pi 4, would you know if this is also possible with the 20.04 Ubuntu image? Love to have that native Wireguard module...

Regards, Dom

I can confirm that following the steps or a Pi OS usb boot works fine but then just flashing the 20.04 image instead of Pi OS does not work :( . I was also hoping to run 20.04 directly from an ssd over USB.

(I'm using an ORICO TCM2-C3 NVMe SSD enclosure and a WD blue SN550 500Gb SSD if anyone is looking for adapters that work - NVMe only though not SATA)

It's not ideal, but you can always boot Ubuntu from the SD card and then have the root filesystem run from the SSD like we did before USB boot was a thing.

At least until somebody figures out how to mod the Ubuntu 20.04 image to support USB booting.

It is already possible yes, has some videos on YouTube of how to use Ubuntu serve and clone another distribution on top of Ubuntu server

Frankly, after some decades of computer use I am finally (as in "this year") realizing how important it is to NOT have internal storage on a computer. Can't boot? Then can't get your data. Plus, never ever upgrade a primary system; always have a second system and modify that instead. When the secondary system works better than the first, then the second becomes primary and the first is the new test bed (or, kept as a backup and use a third for testing upgrades). A system like the Raspberry Pi makes maintaining reliability with less surprises amazingly affordable, and lack of internal storage actually supports that even further. Just buy separately how ever much storage is actually needed, and how ever fast or stacked is desired.

I'm hoping this will speed up the Gitlab experience on my 4GB Raspberry Pi. It's definitely usable, but page loads just feel slightly sluggish. RAM usage is pretty acceptable as well so thinking this will make more of an impact than an 8GB model.

great guide - worked perfectly with the new 8GB RPi 4 and SanDisk 256GB Extreme PRO USB 3.1 Solid State Flash Drive - lightning fast!

Handy shortcut Matt. Thanks.
Which version did you install?

I was enjoying using my verbatim usb boot until it started freezing on rainbow screen with POE hat in use.

Not sure how to troubleshoot that but
basic power start is working as long as i start my usb hub first

Will this still work if you flash the 32 bit version of Raspberry Pi OS to the SSD?

Thanks for this post and the video. I plan to give this a shot. For the USB drive enclosure, it is available either with or without UASP. Some googling suggest UASP should be better performance, but does anyone know if the UASP variant works on the Pi4?

Awesome article Jeff!

How safe is it exactly to completely unplug the PI after a shutdown command when an ssd is connected?
As far as i know drives are still powered after the shutdown command.
I didn't really care about my old 2,5 hdd's, but i rather not damage my ssd.

It should be save after the Pi does it's normal "I'm shut down now" rapid blinks of the green activity light. Once it does that, the external drive should not be seeing any activity, and that would be a safe time to unplug.

Hi Jeff,

Thanks for the wonderful guide. Got it working just fine. I have a question regarding running apt-get update.

When running it, it was expecting to update all of these packages:
ca-certificates firmware-atheros firmware-brcm80211 firmware-libertas firmware-misc-nonfree firmware-realtek libgnutls30 libraspberrypi-bin libraspberrypi-dev libraspberrypi-doc libraspberrypi0 raspberrypi-bootloader raspberrypi-kernel raspi-config rpi-eeprom rpi-eeprom-images

Should I be concerned that it wanted to update rpi-eeprom? Would this kill the RPi ability to boot from USB3?

Thanks! Tim

It should be good to go now, because the stable apt repositories have the firmware updates that you had installed from the beta channel earlier.

As part of the update the .elf files were replaced. Other sundry problems ensue. Good thing I kept an image of the original setup before.

There be dragons here.

--Tim

Hello, with the new raspberry os can you boot directly on 2 usb drive with a MDADM raid 1 ? (without the sd card)

thank you

Nice. Walkthrough worked flawlessly. Thank you. :D

I have a very small configuration with a M-SATA extention board and a 64 GB M-SATA. Booting from M-SATA works well and quick but mouse an keyboard are not recognized by the system. Can somebody help please?

Hi
Can you give us the dd command you used to copy an image onto USB?

THanks

Because you say: "So plug the USB drive into your main computer (where you flashed the microSD card earlier), and flash the Raspberry Pi OS to it the same way you did to the microSD card."
But the PI imager won't write to USB, so I presume you used dd.

I think Tim touched on this above but not sure. When I did this I used the 5-15 version of the bin which is older than what you used. Is there a need to update this? And any complications if I am already running from SSD?

Cheers
Donnie

Hey mate, great tutorial!! Congrats, for real! Quick question, I actually got the same setup as you but I have the inatek case with UASP instead. Noticed something odd: whenever I reboot my pi I have to manually press the inatek external case power button to power it up or my pi would be in boot loop searching for a bootable device (I removed the micro SD card). Any idea on how to solve this one? Thanks in advance and great job!!

It could be connected to this issue. If the drive has to be turned off/on or unplugged/replugged, then you might need to update the firmware to a newer version that has the power on/off feature built in.

Hello,

So I flashed the new stable version of the bootloader and it works just fine, but I can;t figure out how to change the boot order from sd to the usb! I tried following the instructions on the main raspiberri pi websitye, but I cant figure out how to open that bootconf.txt file to change the one line of code so it will boot from usb. Could someone direct me how to do this? Agsin I have already flashed my eeprom and the bootloader works it just only boots from sd card.

Thanks,
Nate

Have you tried pulling out the microSD card and seeing if it boots from the USB drive that way? I know depending on the version of Pi, some will always boot from microSD if a microSD with a valid /boot/bootcode.bin file exists there. Unlike the BIOS on some PCs, you can't really change the boot device order on a Pi.