A PCIe Coral TPU FINALLY works on Raspberry Pi 5

Coral.ai TPUs are AI accelerators used for tasks like machine vision and audio processing. Raspberry Pis are often integrated into small robotics and IoT products—or used to analyze live video feeds with Frigate.

Until today, nobody I know of has been able to get a PCI Express Coral TPU working on the Raspberry Pi. The Compute Module 4, unfortunately, had some quirks in its PCIe implementation, preventing the use of the Coral over PCIe.

Google Coral TPU running over PCIe on Raspberry Pi 5

The Raspberry Pi 5 has a much improved PCIe bus—capable of reaching Gen 3 speeds even!—and I've already tested the first PCIe NVMe HATs for Pi 5.

So can the Pi 5 handle the Coral TPU natively over PCIe?

Yes. Though currently, you need to tweak a few things to get it working.

PCIe bringup for Coral TPU on Pi 5

Buckle up, to get the TPU working, we are going to need to overcome some hurdles:

  • Coral's drivers only work on 4K page size, so we need to switch from the default Pi kernel
  • The Coral is a bit picky with PCIe timings, so (for now at least) we need to disable PCIe ASPM
  • The Pi's default device tree sets up the PCIe bus to not have enough MSI-X interrupts, so we need to change it
  • Pi OS 12 'Bookworm' ships with Python 3.11, but Coral's PyCoral library only runs on 3.9, so we need to run inside Docker (or install an alternate system-wide Python version)
  • There's no A+E key adapter/HAT for the Raspberry Pi (yet), so we need a hardware interface to plug the Coral TPU into the Pi 5's PCIe header

Pi Config Changes

Some of the changes required to get the Coral working are pretty easy—changing some configuration and rebooting.

First, to switch from the default 16k page size to 4k pages (confirm which kernel you're running with uname -a), you can add the line kernel=kernel8.img to the bottom of your /boot/firmware/config.txt file.

Then, to enable the external PCI Express connector, add the following two lines to the same file (again, at the bottom):

dtparam=pciex1
dtparam=pciex1_gen=2

Note that I tested with Gen 1, 2, and 3—the TPU was functional at all three speeds, but I had a few link errors (that did not cause any noticeable problems) at Gen 3 with my own setup.

The final configuration change is disabling ASPM for PCI Express devices—this might not be absolutely necessary, but it does clear up some noisy link error correction messages in the system logs.

Edit /boot/firmware/cmdline.txt and in the configuration line, add in pcie_aspm=off somewhere (I added it prior to rootwait.

Save that file and the config.txt changes and reboot, and you can confirm the kernel was switched by running uname -a:

pi@pi5:~ $ uname -a
Linux pi5 6.1.0-rpi4-rpi-v8 #1 SMP PREEMPT Debian 1:6.1.54-1+rpt2 (2023-10-05) aarch64 GNU/Linux

Yours will look a little different, but the key there is to see the -v8 at the end of the kernel version string.

Device Tree Change

This next bit is slightly more complicated, as most Raspberry Pi owners don't mess around with Device Trees (the little files that describe the hardware to Linux), but it's not so bad.

In fact, I have a guide for the process on this very blog: How to customize the dtb (device tree overlay) on the Raspberry Pi.

In fact, the exact steps for modifying the PCIe bus to use the correct msi-parent value is shown in that blog post—so follow it to make the correct change.

Why switch the value? See this post from jdb on the Pi forums.

Python 3.9 inside Docker

The last step—assuming you have the hardware to connect up a PCIe Coral TPU to the Pi 5's PCIe connector (more on that later)—is to install Docker and run the PyCoral library inside a Debian 10 Docker container, which is much quicker and easier than trying to get PyCoral installed on the system Python on Debian 12.

And wouldn't you know? I have a guide for how to do exactly that elsewhere on this blog: Testing the Coral TPU Accelerator (M.2 or PCIe) in Docker

Follow the Python 3.10 and 3.11 support issue on the PyCoral GitHub for any updates about getting it installed on the version of Python that ships with the latest Pi OS.

Physically Plugging in a Coral TPU to the Raspberry Pi 5

Right now, unfortunately, there are no commercially-available Pi 5 HATs or adapter boards that go from the proprietary PCIe FFC connector on the Raspberry Pi 5 to either a standard PCIe slot, or to an A+E key M.2 connector.

I am certain both of those will exist at some point (hopefully soon!), but for now, your best bet is to order a HatDrive! Top or Bottom from Pineberry Pi and then order a PCIe M.2 M-key to A+E key adapter. Or use the Coral B+M key module with the 'Bottom' version (since it's 2280-size, and won't fit in the 'Top' HAT).

For my own testing, I've been using a prototype board that Mirkotronics / Pineberry Pi sent called 'uPCity'. I am hoping they can sell a version of that board at some point, because it is extremely fun to mess around with so many different PCI Express devices on the Raspberry Pi 5!

Conclusion

Once everything is wired up, and you install the apex PCIe driver—and can see /dev/apex_0—you could run an image classification example inside the edgetpu examples repo. Give it an image of this bird:

Coral Edge TPU Example Bird image

And then it should spit out:

root@7bb44c28f378:~# python3 /usr/share/edgetpu/examples/classify_image.py --model /usr/share/edgetpu/examples/models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --label /usr/share/edgetpu/examples/models/inat_bird_labels.txt --image /usr/share/edgetpu/examples/images/bird.bmp
---------------------------
Poecile atricapillus (Black-capped Chickadee)
Score :  0.44140625
---------------------------
Poecile carolinensis (Carolina Chickadee)
Score :  0.29296875

Further reading

coral
tpu
ai
machine learning
raspberry pi
pcie

Comments

I am not 100% I understood the problem but you said "There's no A+E key adapter/HAT for the Raspberry Pi (yet), so we need a hardware interface to plug the Coral TPU into the Pi 5's PCIe header" -- there are A+E to M key adapters https://www.amazon.com/dp/B07WBXP632 wouldn't that help

He was saying there was no FPC adapter directly, and that you could use an adapter like the one you suggested with the bottom hat that takes a 2280. What’s shown in the article is a coral module inserted into a PCIe WiFi adapter card without the antennae ports connected

Does the same go for the Coral USB accelerator? I was hoping I could use the CM4 module in conjunction with the Coral USB accelerator on my drone for object detection.

I spent a bunch of time getting the USB accelerator working today, and all the dependencies for Coral have decayed. It took me a fair bit of hunting to get it working on a current installation of Raspberry Pi OS. I'll see if I can write it up somewhere.

Here's what I did, let me know if it works for you:

# Install pyenv and python 3.8.18. Latest version I could get to work
sudo apt-get install -y make build-essential libssl-dev zlib1g-dev \
libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev \
libncursesw5-dev xz-utils tk-dev libffi-dev liblzma-dev
curl https://pyenv.run | bash

echo << EOF >> ~/.bashrc
export PYENV_ROOT="$HOME/.pyenv"
[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"

eval "$(pyenv virtualenv-init -)"
EOF

# reboot here

pyenv install 3.8.18
pyenv global 3.8.18
# Install Pillow 9.5, as that's the last release with support for Antialias
python3 -m pip install Pillow==9.5.0
# Install the Coral Edge TPU Runtime
echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | sudo tee /etc/apt/sources.list.d/coral-edgetpu.list
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
sudo apt-get update
# Install the PyCoral library
python3 -m pip install --extra-index-url https://google-coral.github.io/py-repo/ pycoral~=2.0

I have gotten the M.2 version to work using a Python 3.9 Miniforge environment, without docker at all. Perhaps that would work for the USB version as well.

It's also worth noting that in my limited testing of the USB accelerator on rpi5, I'm seeing times in the 4-7 ms range (after the first, longer time).

Since a motherboard is basically a 4x pcie multiplier with hard attached devices, it will be nice to start to see a pcie multiplier that can be attached to the m.2 or pci connection (and dont need extra power or has its own psu that accepts a range of voltage, so that you dont need too many extra things), to attach 2 or 4 things to something that has only one expansion slot. That pcie v2.0 bifurcation chip used on those mining cards are cheap and between having 1 to x but slow(but still faster than sc cards, usb,etc, and with more iops for less laggy experience), sometimes having that possibility will be great, also some devices are slow so having a faster pcie will do nothing anyway. It will be like converting a budget motherboard to a pro, and it could be like that for the sbcs, having a few pcie slots, usb, wifi, lan, etc, like the motherboard for the CM, only that you already have the basic on a sbc and you only want more expansion and connections.
I believe that with something like that, sbcs could have been more popular because it have been easier to add more and faster storage and conectivity to be useful your desktop or server

I’m happy to report that I have got the HatDrive! Bottom working on the Pi 5 8gb with a M.2 M>E(A=E)-key adaptor from amazon and Coral edge TPU A+E following these guides. The image classifier worked nicely at the end.

I am hoping to get this working with frigate soon. The Coral device showed up in my first install of Frigate, but I somehow managed to make a mess of frigate container, so starting fresh.

Thanks for the guide Jeff, amazing work!

Awesome! I somehow got the video feed working first time, then changed the config and couldn't get it to work again.

Looking forward to the video / blog with any tips on setting up on the pi 5. Don't know much about Ansible, but looks interesting.

Thanks for this work and recent new comments. Really interested in simple approach to AI on new Rpi 5 with a library that has already been established. Additional Coral is great, with best approach either USB or PCIE. I don't see such a thread on Raspberry forums. Thanks again.

Hi Jeff, I enjoy following your experiments and have been inspired to try and get a Coral device working on my Rpi5. I have purchased the AI hat from pieberry and a coral dual device. I have followed the instructions for installed the pci driver etc and can see the /dev/apex_0 device on the client. The only issue is that when I run the sample above inside docker I get the following error:
RuntimeError: Error in device opening (/dev/apex_0)!
I'm assuming I've missed something but any help in debugger this issue would be appreciated, even just pointing me to the correct place to look.

Following up on my previous comment, did some more digging and followed the steps again and found the issue. I made the msi-parent change on the wrong pic parent, doh! All working now let the experiments begin

Hi! I´ve been using a Coral TPU (USB version) on a RPi IO Board, and it´s awesome!

Now, I´m really interested on your PCIe setup, so, would you mind to share it?

If I´m right, I need: A RPi5, Pineberry AI Hat, M.2 Accelerator with Dual Edge TPU (M.2 E-Key) and of course a Picamera.

am I right? Thank you in advance!

you really don't have to use pycoral, and can just use tflite runtime (install with pip install tflite-runtime --break-system-packages or just apt install python3-tflite-runtime if that works for you) It works just as good and is fully compatible with the tutorial from above. This eliminates using docker, which can be use full in some cases.

And btw: Rapsberry pi recently updated device tree, and the update does undo changes made to it. (at least, this happend to me)

Hi Jeff,
thanks to your help i got frigate running with a pcie-coral M-Key on the pimoroni nvme adapter.

I followed these instructions to the letter but cannot get it to work. It seems that Raspberry Pi 5 and M.2 Coral are not ready to be used together.

very nice tutorial thanks for the great work ;-)

Hi Jeff,

Thanks so much for your work on this! You inspired me to get started with accelerators on my Bookworm RPi5.

I bought a couple of the Coral M.2 Accelerators (A+E key) and two of the Pineberry Pi Hat AI! for Raspberry Pi 5 boards. I had no problem with the Hat and the Coral M.2 showed up right away. I first tried following Google's Debian directions (https://coral.ai/software/#debian-packages), and ran into the Python version limit (I'm on 3.11.2), then the page size difference. I was able to find GitHub user oberluz's pycoral and tflite_runtime wheels for my version of Python and TensorFlow 2.13, to wit:

https://github.com/oberluz/pycoral/releases/download/2.13.0/tflite_runt…
https://github.com/oberluz/pycoral/releases/download/2.13.0/pycoral-2.1…

After that, I had to go back and follow your directions (lol). I ran into a couple of snags though - for one, my entry for pcie110000 in the device tree file had a value of `msi-parent = <0x2c>` and not 0x2f. I also have a pcie@120000, so wasn't sure which one to change. I ended up changing pcie110000, and that seems to be working. Maybe your board has more PCIe slots?

The second snag was when trying to validate the install by running the bird recognition/parrot model. I got a segmentation fault when using the EdgeTPU model. I didn't install it because the Debian directions above said that `python3-edgetpu` and `edgetpu-examples`. Does that sound right though? Am I not able to run EdgeTPU models? It's kind of an academic question at this point - this Pi5 and its Coral are going to be part of a Kubernetes cluster, so I really just wanted to see if I could get it working and am hoping it will show up as an accelerator resource for the cluster.

It seems like that this tutorial doesn't work anymore because of recent updates to the kernel (At least for me). A Debian installation that I got it working on, dint boot after updating (apt update && apt upgrade), and when I try it on a new installation, gasket-dkms wont install.

You can build gasket-dkms from source not to hard. There might be more dependencies and your dependencies might change depending on distro. This is what I did as part of figuring this out.
sudo apt update
sudo apt install git
sudo apt install dh-dkms
sudo apt install dkms
git clone https://github.com/google/gasket-driver.git
cd gasket-driver
sudo debuild -us -uc -tc -b
cd ..
sudo dpkg -i gasket-dkms_1.0-18_all.deb

Thanks for all the info, and everyone who's shared some knowledge here.

With the new 6.6 kernel.

You'll need to build gasket-driver from source as mentioned above.

But also use a different value for the device tree update. Mentioned here: https://gist.github.com/dataslayermedia/714ec5a9601249d9ee754919dea49c7…

Instead of 0x66, use 0x67.
`sed -i '/pcie@110000 {/,/msi-parent = <0x2[fc]>;/{s/<0x2f>/<0x67>/; s/<0x2c>/<0x67>/}' $SCRATCH_DTS`

When using 0x66 my apex_0 device would get lost on reboot. It works with 0x67:

```
root@68ea3c82bf63:~# python3 /usr/share/edgetpu/examples/classify_image.py --model /usr/share/edgetpu/examples/models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --label /usr/share/edgetpu/examples/models/inat_bird_labels.txt --image /usr/share/edgetpu/examples/images/parrot.jpg
---------------------------
Ara macao (Scarlet Macaw)
Score : 0.6796875
---------------------------
Platycercus elegans (Crimson Rosella)
Score : 0.12109375
```

Thanks for everyone's efforts here.