PTP and IEEE-1588 hardware timestamping on the Raspberry Pi CM4

I've been following the issue CM4 is missing IEEE1588-2008 support through BCM54210PE since I heard about IEEE1588-2008 support on the Compute Module 4 last year.

Broadcom NIC on Raspberry Pi Compute Module 4

Apparently the little NIC included on every Compute Module 4—the BCM54210PE, which is different than the NIC on the Pi 4 model B (a BCM54213PE)—includes support for a feature called PTP, or Precision Time Protocol.

Update: I also posted a video about PTP on the CM4 on my YouTube channel: It's About Time (PTP on the Raspberry Pi CM4).

If you thought NTP was great, with its millisecond-level accuracy when synchronizing clocks over a network, you'll love PTP—it can sync down to nanoseconds, or in some cases even picoseconds!

PTP is more prevalent in HFT (High Frequency Trading) in finance, scientific research, and distributed databases that need highly accurate timings.

But with a tiny Compute Module 4 able to control hardware timestamps—plus the plethora of available high-quality GPS HATs for the Pi—maybe highly-accurate time could be more democratized!

Getting PTP and PPS

Pin 9 Raspberry Pi Compute Module 4 IO Board J2

The Compute Module 4 IO Board comes with two pins (8 and 9 on J2) labeled SYNC_IN and SYNC_OUT. Those pins are supposedly wired to the PHY with 1.8v signalling (see the CM4 datasheet)—but through a bunch of experimentation, it seems only pin 9 is wired correctly.

So on the Compute Module 4, you can get a PPS input or output through pin 9, but not both at the same time—at least when using the official CM4 IO Board.

The PPS is helpful for debug purposes, and if you need to distribute a PPS to any equipment that uses a PPS input for time sync. Otherwise, PTP is useful for synchronizing time over the Pi's Ethernet connection to any 'slave' devices on the same network.

I'll have more on this in an upcoming video, but I wanted to post instructions for how to set up a CM4 as a PTP master / time server.

Patches were only recently added to the Pi OS kernel fork, for example:

And the same patchset is already upstream in later versions of the Linux kernel (yay!). But to get these running on a Pi today, you need to run sudo rpi-update to get the latest kernel version (which is not yet installed when running sudo apt upgrade).

Once you do that and reboot, you should see a new ptp device:

$ ls /dev/ptp*
/dev/ptp0

$ cat /sys/class/ptp/ptp0/clock_name 
bcm_phy_ptp

And then you should see its hardware timestamping capabilities listed using ethtool:

$ ethtool -T eth0
Time stamping parameters for eth0:
Capabilities:
    hardware-transmit
    hardware-receive
    hardware-raw-clock
PTP Hardware Clock: 0
Hardware Transmit Timestamp Modes:
    off
    on
    onestep-sync
    onestep-p2p
Hardware Receive Filter Modes:
    none
    ptpv2-event

And for a final test, assuming you have an oscilloscope hooked to pin 9 and a ground pin, you can enable PPS output and monitor it on a scope:

$ sudo ./testptp -d /dev/ptp0 -L0,2
set pin function okay
$ sudo ./testptp -d /dev/ptp0 -p 1000000000
periodic output request okay

PPS output on Tektronix oscilloscope from Raspberry Pi Compute Module 4

What's next?

Well, remember the open source Time Card I mentioned last year? Well, using it—or something similar, with GPS and a better oscillator, like temperature-controlled oscillator (TCXO) or a chip-scale atomic clock, you could build a very competent (and relatively inexpensive) master clock / stratum-1 time server for both PTP and NTP.

This can be useful for the aforementioned database, HFT, and scientific projects... but it could also bring better timings to other applications as well. Programming changes a bit when you can guarantee you're within nanoseconds of UTC—assuming you have a good GPS chip and decent time holdover for GPS dropouts.

Raspberry Pi Compute Module 4 PTP Sync PPS on Oscilloscope

On two Compute Module 4s, using ptp4l, I'm able to sync two Compute Module 4's on a LAN to within about 10-15 nanoseconds of each other (that's a 10ns/division time scale in the above picture).

The commands I ran to set up and test PTP were:

# Run this on the 'master' Pi:
sudo ptp4l -i eth0 --masterOnly 1 -m --tx_timestamp_timeout 200

# Run this on any 'slave' Pis:
sudo ptp4l -i eth0 --slaveOnly 1 -m --tx_timestamp_timeout 200

Thanks especially to Ahmad Byagowi and his team over at Meta spearheading the Time Appliances Project, and to Lasse Johnsen at Timebeat for their help in getting this moving. It seemed like things were stuck for a while, but they were able to work with engineers at Raspberry Pi and Broadcom to get the necessary patches written and tested!

I'll cover more on PTP/IEEE1588 and PPS on the CM4 in my upcoming video on my YouTube channel, so subscribe if you want to see that. I will also continue to publish my findings with the Time Card in this GitHub issue.

Comments

Is there any trick to get "testptp"?
Followed the instructions, but testptp cant be found.

Is anyone aware of other single board computers that support IEEE-1588? While I’d love to use a CM4, I’m not willing to pay a scalper and supply doesn’t seem to get any better (as of early 2023).

Unfortunately none of the other SBCs I've used do, though many will work with the PTP support built into a PCIe card like one of Intel's NICs.

Any recommendations for a low/moderate cost 8 port switch that supports IEEE 1588 hardware time stamping? Is hardware time stamping part of AVB (Audio Video Bridging)?

I am trying to set up a PTP lab using the TimeBeat CM4 GPS module. My understanding is that a subnet of PTP servers requires hardware time stamping to get ultra precise time synchronization.

This is part of a project to use a CM4 to provide all of the network infrastructure services like PTP, DHCP, DNS and IGMP required for the new SMPTE ST 2110 media over IP suite of protocols using ansible and docker.

PS, my TimeBeat CM4 GPS module arrived yesterday. Yea!!!
PPS, SMPTE, Society of Motion Picture and Television Engineers.

Dear Jeff
Exceptional work. I would like to thank you and all the contributing people for the development of the driver set. I would like report a case which you might find interesting. While the case you outlined works for 64-bit OS, it does not for 32-bit OS. In case of 32-bit OS,
all the test results are fine including "ethtool -T eth0". However when we use ptp4l in master and slave form, both sides give errors. Master side gives "sync failed", "time-out while polling", "Faulty to listening" errors. Slave side gives HWTSTAMP error. It seems that there is a driver problem in 32-bit OS (bullseye, Debian:11, Kernel:5.15).

32-bit OS is used by many already and migration to 64-bit is not straightforward since there is no guarantee for any smooth transition or performance. It would be good if we have the same functionality in 32-bit OS if there is no loss in time accuracy.

Hi Jeff - great work on this topic! I've found SYNC_OUT to be operating at 3.3V as opposed to the 1.8V levels advertised in the CM4 datasheet. Have you found the same? I tested this simply by running the testptp program as you showed above to generate a PPS signal. I also measured a 4 microsecond pulse. Do you know if SYNC_OUT can tolerate 3.3V when configured as an input? Thanks!

When running sudo ./testptp -d /dev/ptp0 -p 1000000000 I continuously get the error that: "clock_gettime: Connection timed out" Does anyone know why this is happening or what it means? It seems to be random when it says periodic output function okay. I am not sure what is going on and any feedback would be helpful!

When running testptp.c, I query for ptp clock time (sudo ./testptp -d /dev/ptp0 -g) and the output says "clock_gettime: Connection timed out". Do I need to hook up my CM4 to a network in order to get a time reference? Currently, I do not have wifi nor any ethernet connection which allows the CM4 to be hooked up to the internet/network.

Hi,
thanks for this tutorial! I am currently experimenting with the Compute Module.
The PTP capability is great. However I wonder how I can use the SYNC_IN signal. According to the schematics of the Compute Module IO Board, the SYNC_IN pin is definitely connected to the Raspberry Pi connector.
I have a GPS Module that provides 1PPS signals, so it would make a lot of sense to connect the GPS module to SYNC_IN! But when I do that, how can I tell ptp4l to use the 1PPS signal from SYNC_IN?