Ever since the Pi 2 model B went to a 4-core processor, disk IO has often been the primary bottleneck for my Pi projects.
You can use microSD cards, which aren't horrible, but... well, nevermind, they're pretty bad as a primary disk. Or you can plug in a USB 3.0 SSD and get decent speed, but you end up with a cabling mess and lose bandwidth and latency to a USB-to-SATA or USB-to-NVMe adapter.
The Pi 4 actually has an x1 PCI Express gen 2.0 lane, but the USB 3.0 controller chip populates that bus on the model B. The Compute Module 4, however doesn't presume anything—it exposes the PCIe lane directly to any card it plugs into.
And in the case of Oratek's TOFU, it's exposed through an M.2 slot, making this board the first one I've used that can accept native NVMe storage, directly under the Pi:
Now, a single x1 lane at gen 2 speeds tops out around 400 MiB/sec in real-world usage, so many NVMe drives are still underpowered connected to the Pi, but as you'll see in a bit, a cheap KingSpec SSD was 3x faster for random IO than a similar SSD plugged in via USB 3.0.
I'd like to thank the Swiss company Oratek for sending me this TOFU board. The board is just 9 centimeters square, yet packs in most of the same features on the much larger Compute Module 4 IO Board:
I also posted a video that goes along with this blog post on my YouTube channel:
M.2 on a Pi
Let's start on the bottom, and take a look at the M.2 slot on the TOFU. Some other Compute Module 4 boards I've been keeping my eye on are designed around the more popular 'M key' slot in the 80mm form factor.
Other boards have 30mm 'A+E' key slots, that accept things like the Google Coral TPU, or WiFi 6 cards like the Intel AX200 I tested.
But the TOFU has a 'B' type M.2 2242 socket, meaning it accepts only B or B+M key devices.
And the slot on this board only has one standoff at 42 millimeters, so you can't fit shorter devices like a 30 mm 'B+M key' NVMe, like the WD SN520. Of course, if you taped it down it would work, but that's not a great solution:
But 42mm devices work great, as long as driver support is present on the Pi—and for NVMe drives, at least, Pi OS includes support out of the box.
I bought a KingSpec 128GB 2242 NVMe drive, and after installing it, it showed up using
lsblk, and I could format and mount it like any other drive.
I compared this setup to a similarly-priced SSD I reviewed in the Argon One M.2, which connects an SSD inside the same case as a Pi 4, but using a USB 3.0 to SATA adapter.
Since the NVMe drive communicates directly over the Pi's PCI express bus, it performs much better on every benchmark.
And yes, the two drives are not the exact same; but after testing a few dozen SATA SSDs and NVMe drives, both natively and through USB adapters, I can say with confidence that the USB adapters soak up at least 10-20% of the bandwidth, and it's worse for random IO, which requires better latency. Direct SATA and NVMe connection is a lot faster, plus cabling (especially for NVMe... where there is none) is a lot cleaner.
Unfortunately, you can't boot off an NVMe drive—at least not yet. The Pi's firmware needs to be updated to support this feature. It seems like it may happen, but there's no timeline for it yet.
Update: Apparently this week NVMe SSD Boot was just added as a beta feature in the Pi firmware, though it requires a bit of a process to use it: NVMe SSD Boot (BETA).
NVMe drives are only one of the accessories you can install on the TOFU, though. What it's really designed for is a 4G LTE module, like a Sierra Wireless AirPrime. The TOFU includes a built-in SIM tray, so you can get mobile service on the board.
As more industrial systems require more Internet connectivity, it's important to make robust control boards with both wireless and wired networking. And the Raspberry Pi Compute Module is a great platform to build on, since it can run a full Linux OS and is easy to remotely administer.
The TOFU has another feature that's extremely useful in industrial settings: It accepts a wide range of power inputs.
You can either use a barrel plug (like the regular Compute Module IO board) or a 3.5mm terminal block, and power the board with between 7.5 to 28V of DC power.
It also includes a 5 volt, 3.5 amp fuse to protect against overcurrent, and also supports Power over Ethernet through the standard Pi PoE HAT, since it includes the extra four pins used to extract power from the ethernet connector and deliver it to the HAT.
Outside of commercial use, the $100 dollar price tag is a little high for the 'impulse buy then stick it in a drawer' use case most of us using Pis are familiar with.
But it's a great, compact board for things like Pi clusters, remote camera installations, or an interactive display system.
It preserves most of the ports you would get on a full size IO board in half the space, including:
- 1 CSI camera connector
- 1 DSI display connector
- 1 full-size HDMI port
- 3 USB 2.0 ports (which need to be enabled to work)
- 1 Gigabit Ethernet port
- 40-pin GPIO header
- microSD slot (only usable with CM4 Lite)
- 1 USB-C port (used for flashing CM4 eMMC models—hold down the 'boot' button while plugging into a host computer)
If powering the board via PoE or USB-C, the power lines to the M.2 slot are disabled, since these power methods can't guarantee enough current for the M.2 spec—to use an M.2 slot you still need to power the board through one of the 12V options.
On the bottom, there's:
- 1 M.2 2242 slot
- 1 microSIM slot for wireless cards
A case for the TOFU
Oratek was kind to provide the TOFU carrier board, but so far they don't have a case for it. That's something they may be offering in the future, but for now, I wanted something to house the board on my desk, so I designed my first-ever 3D object: a case for the TOFU.
I built it in Fusion 360 following Shawn Victor's great PCB case design tutorial, and printed it on my Ender 3 V2 printer. I had to go through a few revisions before I got to the final form, but it is a nice little enclosure, about the same size as the Argon One M.2 I mentioned earlier.
I posted the case design to Thingiverse: TOFU Enclosure by geerlingguy
Someday I'll learn how to make a top for the enclosure with a cutout for a fan to cool the TOFU. For now, it just looks nicer and protects the bottom of the TOFU, where the SSD is exposed.
At over $100 shipped to the US, I don't expect everyone to go out and buy one of these things. But I like the design of the board, and there are some applications where it is a great value, like for remote installations, using solar power and a battery, for camera surveillance or remote monitoring or control.
You can pre-order the TOFU at tofu.oratek.com, and see all the Compute Module 4-based boards I've been tracking on my Pi PCI website.
I Have a Laptop that's running Linux and the WD Black M.2/NVMe Drive requires an addition to the Grub boot options in order to Boot Linux so This:
And that has to be added after the quiet splash part of the existing Grub boot option/environment variables settings.
So I'd be very careful with any WD M.2/NVMe as Linux will not Boot on an ASUS Tuf FX505DY-WH51 laptop(Ryzen 5 3550H/Radeon RX 560X discrete GPU)! And the real annoying part is that I always keep the Linux Mint Live USB handy to boot the laptop if the Grub Boot needs to get repaired and I have to hand type in that: nvme_core.default_ps_max_latency_us=5500 every time that I want to Boot from the Mint Live USB Image on that ASUS Laptop. So I'll be shopping soon for a Samsung M.2 NVMe SSD and out with that WD M.2 SSD to some external/internal M.2 to USB adapter, for some external drive usage more than likely!
I have a Linux PC. I run Ubuntu. I also have a RPi Zero W. I have not had to add anything to my Linux Box to boot of M.2, same should be true of the RPi if the firmware recognizes an M.2. However M.2s were not originally available for RPi's. Wjat some fail to realize is the Linux OS does not need for the firmware to recognize the hardware. Back inthe day I added drives to my Linux system that my DOS/Windows could not recognize. Dual Booting DOS/Windows left my system not seeing the drives even though I made sure to format it in FAT32. At that time the system I was using could not run a 64 bit OS, and the 32 bit WindowsOS did not understand the drive, since the BIOS did not understand it. Linux However did, as it did not use the BIOS to configure or communicate with the drive. The RPi OSes are Linux based, and with the correct tweaking Linux can speak to hardware that DOS or Windows can not. Yes it is true that out of the box, there may no be drivers for certain hardware and they may be unusable, but not if you are a hardware/software expert. You can simply write a driver as long as you know the architecture of the hardware. The problem is, many manufacturers do not issue datasheets on proprietary hardware, so some circuits have NO official spec out in the wild, only within the companies "top secret" corporate files.
As far as issues with the WD NVMe Once you make the proper changes to the proper files you should NEVER need to add lines each time you boot. Some files that are accessible to you can be changed to allow the NVMe to be used, however often there are two sets of these files. If you modify the wrong one, it will get updated or overwritten by other files, eliminating all of you work. I remember making changes to my system in the past, then finding out the advice i received off the internet was wrong. The reason it was wrong is because it was outdated. The file I modified was still in the system, and could still be changed and would work the was the advice said, however any software or system updates done from the GUI or apt/aptget would overwrite my work. I found out later the the system keeps a duplicate, I assume if the duplicate is understood by the computer, it copies it to the one the computer uses for booting. If it does not understand it it retains the original.
That being said, I think there are many different files that work that way, and why I am not being specific about what file. If you edit the "master" (for lack of a better term), then the second copy gets over written, Try to determine if the file you are changing is over written by the system under specific circumstances... if it is, then changing to another NVMe won't help anything, unless the drive for some reason is recognized while the other is not, and the system updates the config based upon this recognition. Otherwise you will have the same situation.
I wonder why they didn't fix the USB-C so it did full PD, then you wouldn't be forced to use one of the other power methods, 12V 3A would be plenty. I mean it can't be to save money since it's a $100 board without the compute module.
I had some hopes for the Tofu board but it's a little expensive to build a system out of that. Especially considering about $50 gets you a carrier board with PoE and case for the previous gen compute module, CM3+. While those don't have M.2 they still have eMMC as an option. Slightly faster and slightly more reliable than just an SD card and CM3+ modules will be in production until 2026.
I do find this part needing a little more explanation: "If powering the board via PoE or USB-C, the power lines to the M.2 slot are disabled, since these power methods can't guarantee enough current for the M.2 spec—to use an M.2 slot you still need to power the board through one of the 12V options." What does the M.2 spec say about power? That specific Kingspec SSD requires all of 2W and with a quick look I found PCIe x1 slots provide max 10W which I'd assume is the same for single lane M.2 as used here? PoE can usually provide at least 15W per port, USB PD way more. So it seems it's more about design choices here.
The Pi itself consumes power too, remember — and the USB-C port is not necessarily supporting PD; power supplies like the Pi Foundations' own only supports 5V/2A. It is a design decision, but it's more a decision to avoid any potential power issues with M.2 cards that do consume more power (especially if you're driving the CM4 hard and have USB accessories plugged in pulling down power on their own).
Oratek are now offering a case for the Tofu board also an NVME M2 adapter. Visit the website.
I was hoping to use the CM4 with NVME storage to replace my RPi4B as my Astronomy control platform. I currently use my RPi4B and connect my ZWO imaging camera via the USB3 port. But, both this TOFU board and also the MirkoPC don't have USB3 so I am still stuck. So I guess I will have to wait to see if anyone else comes up with a solution.
Most boards will provide either PCIe (NVMe) or USB 3, but not both, because the USB 3 on the RPi4 is provided by a PCIe USB 3 controller. To provide both, the board would have to have a PCIe switch on it.
I have bought two TOFU boards and they are running raspi-os from micro-sd just fine, but no luck with nvme yet.
The only difference is I am using CM4 Lite with no eMMC, just 2G and no wi-fi. As you know the full featured boards are hard to find!
At boot time I get the message "brcm-pcie fd500000.pcie: link down" and lspci shows no devices.
I have used rpi-update and have the newest kernel and modules.
The one thing I have not updated is the board's eeprom, but that should not be needed, right?
Have you tried TOFU with a Lite module?