[Linuxptp-users] Hardware timestamping does not work

[Jan Deinhard]

Hello,

I have problems setting up hardware timestamping with ptp4l version 1.6.
Probably it is a problem with my system but I am not sure how figure out
what is going wrong.

I am using a Debian workstation and a i.MX6UL evaluation board running on a
Linux build with Yocto. According to ethtool -T eth0 both systems support
PTP hardware timestamping.

With the workstation as master

$ ptp4l -i eth0 -m -H

and the eval board as slave

$ ptp4l -i eth0 -m -H -s

I see get the following output on the slave:

ptp4l[26422.241]: selected /dev/ptp0 as PTP clock
ptp4l[26422.248]: driver changed our HWTSTAMP options
ptp4l[26422.251]: tx_type 1 not 1
ptp4l[26422.253]: rx_filter 1 not 12
ptp4l[26422.255]: port 1: INITIALIZING to LISTENING on INITIALIZE
ptp4l[26422.257]: port 0: INITIALIZING to LISTENING on INITIALIZE
ptp4l[26423.632]: port 1: new foreign master d05099.fffe.82569e-1
ptp4l[26427.632]: selected best master clock d05099.fffe.82569e
ptp4l[26427.633]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE
ptp4l[26428.632]: master offset 9027724697 s0 freq [phone] path
delay 0
ptp4l[26429.632]: master offset 10110507943 s1 freq [phone] path delay
-339142602
ptp4l[26431.632]: master offset 1443965638 s2 freq [phone] path delay
-295834797
ptp4l[26431.634]: port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED
ptp4l[26432.632]: master offset 2187585778 s2 freq [phone] path delay
-295834797
ptp4l[26433.632]: master offset 2925385526 s2 freq [phone] path delay
-290010958
ptp4l[26434.632]: master offset 3663195859 s2 freq [phone] path delay
-284187119
ptp4l[26435.632]: master offset 4399753214 s2 freq [phone] path delay
-277121915

It is not getting better after some minutes. Master offset is alway
increasing and freq is always the same value.

With the eval board as master and the workstation as slave I see the
following output on the slave:

ptp4l[28320.840]: selected /dev/ptp0 as PTP clock
ptp4l[28320.840]: port 1: INITIALIZING to LISTENING on INITIALIZE
ptp4l[28320.841]: port 0: INITIALIZING to LISTENING on INITIALIZE
ptp4l[28322.088]: port 1: new foreign master 00049f.fffe.03fc1c-1
ptp4l[28326.088]: selected best master clock 00049f.fffe.03fc1c
ptp4l[28326.088]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE
ptp4l[28327.087]: master offset 125878074591 s0 freq -23999996 path
delay 0
ptp4l[28328.088]: master offset 125134345169 s1 freq -23999999 path
delay 0
ptp4l[28329.088]: master offset -737638119 s2 freq -23999999 path
delay 0
ptp4l[28329.088]: port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED
ptp4l[28330.088]: clockcheck: clock jumped backward or running slower than
expected!
ptp4l[28330.088]: master offset -1475235977 s0 freq -23999999 path
delay 0
ptp4l[28330.088]: port 1: SLAVE to UNCALIBRATED on SYNCHRONIZATION_FAULT
ptp4l[28331.088]: clockcheck: clock jumped backward or running slower than
expected!
ptp4l[28331.088]: master offset -2419666334 s0 freq -23999999 path delay
206861774
ptp4l[28332.088]: clockcheck: clock jumped backward or running slower than
expected!
ptp4l[28332.088]: master offset -3089364053 s0 freq -23999999 path delay
138983131
ptp4l[28333.088]: clockcheck: clock jumped backward or running slower than
expected!
ptp4l[28333.089]: master offset -3866489723 s0 freq -23999999 path delay
178522823
ptp4l[28334.088]: clockcheck: clock jumped backward or running slower than
expected!
ptp4l[28334.089]: master offset -4617886979 s0 freq -23999999 path delay
192326258

It works somewhat better when I use software ts on the workstation and
hardware ts on the eval board but "freq" remains at a very high value on
the slave:

ptp4l[28788.508]: port 1: INITIALIZING to LISTENING on INITIALIZE
ptp4l[28788.508]: port 0: INITIALIZING to LISTENING on INITIALIZE
ptp4l[28794.533]: selected best master clock d05099.fffe.82569e
ptp4l[28799.399]: port 1: new foreign master 00049f.fffe.03fc1c-1
ptp4l[28801.119]: selected best master clock d05099.fffe.82569e
ptp4l[28803.399]: selected best master clock 00049f.fffe.03fc1c
ptp4l[28803.399]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE
ptp4l[28805.399]: master offset 166527119067 s0 freq [phone] path
delay 57163
(removed 15 lines)
ptp4l[28821.402]: master offset 166527038073 s1 freq [phone] path
delay 61780
ptp4l[28823.402]: master offset 44731 s2 freq [phone] path delay
61458
ptp4l[28823.402]: port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED
ptp4l[28824.402]: master offset 14076 s2 freq [phone] path delay
66101
ptp4l[28825.403]: master offset -5574 s2 freq [phone] path delay
66101
ptp4l[28826.403]: master offset -3251 s2 freq [phone] path delay
66101
ptp4l[28827.403]: master offset -1000 s2 freq [phone] path delay
66101
ptp4l[28828.403]: master offset 5890 s2 freq [phone] path delay
60522
ptp4l[28829.403]: master offset 11802 s2 freq [phone] path delay
55710
ptp4l[28830.403]: master offset 12450 s2 freq [phone] path delay
55553
ptp4l[28831.404]: master offset 12918 s2 freq [phone] path delay
55386
ptp4l[28832.404]: master offset 32581 s2 freq [phone] path delay
55386

My current guess is a problem on my Linux system.

Here is more information on my setup:

Workstation:

$ uname
-a

Linux polaris 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u2 (2016-10-19)
x86_64 GNU/Linux

$ lspci
00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (2)
I219-V (rev 31)

$ ethtool -T
eth0

Time stamping parameters for eth0:
Capabilities:
hardware-transmit (SOF_TIMESTAMPING_TX_HARDWARE)
software-transmit (SOF_TIMESTAMPING_TX_SOFTWARE)
hardware-receive (SOF_TIMESTAMPING_RX_HARDWARE)
software-receive (SOF_TIMESTAMPING_RX_SOFTWARE)
software-system-clock (SOF_TIMESTAMPING_SOFTWARE)
hardware-raw-clock (SOF_TIMESTAMPING_RAW_HARDWARE)
PTP Hardware Clock: 0
Hardware Transmit Timestamp Modes:
off (HWTSTAMP_TX_OFF)
on (HWTSTAMP_TX_ON)
Hardware Receive Filter Modes:
none (HWTSTAMP_FILTER_NONE)
all (HWTSTAMP_FILTER_ALL)
ptpv1-l4-sync (HWTSTAMP_FILTER_PTP_V1_L4_SYNC)
ptpv1-l4-delay-req (HWTSTAMP_FILTER_PTP_V1_L4_DELAY_REQ)
ptpv2-l4-sync (HWTSTAMP_FILTER_PTP_V2_L4_SYNC)
ptpv2-l4-delay-req (HWTSTAMP_FILTER_PTP_V2_L4_DELAY_REQ)
ptpv2-l2-sync (HWTSTAMP_FILTER_PTP_V2_L2_SYNC)
ptpv2-l2-delay-req (HWTSTAMP_FILTER_PTP_V2_L2_DELAY_REQ)
ptpv2-event (HWTSTAMP_FILTER_PTP_V2_EVENT)
ptpv2-sync (HWTSTAMP_FILTER_PTP_V2_SYNC)
ptpv2-delay-req (HWTSTAMP_FILTER_PTP_V2_DELAY_REQ)



Eval board:

$ uname -a
Linux imx6ulevk 4.1.15-2.0.0+gb63f3f5 #1 SMP PREEMPT Wed Nov 2 21:42:19 CET
2016 armv7l armv7l armv7l GNU/Linux

$ ethtool -T eth0
Time stamping parameters for eth0:
Capabilities:
hardware-transmit (SOF_TIMESTAMPING_TX_HARDWARE)
software-transmit (SOF_TIMESTAMPING_TX_SOFTWARE)
hardware-receive (SOF_TIMESTAMPING_RX_HARDWARE)
software-receive (SOF_TIMESTAMPING_RX_SOFTWARE)
software-system-clock (SOF_TIMESTAMPING_SOFTWARE)
hardware-raw-clock (SOF_TIMESTAMPING_RAW_HARDWARE)
PTP Hardware Clock: 0
Hardware Transmit Timestamp Modes:
off (HWTSTAMP_TX_OFF)
on (HWTSTAMP_TX_ON)
Hardware Receive Filter Modes:
none (HWTSTAMP_FILTER_NONE)
all (HWTSTAMP_FILTER_ALL)


What can I do to find out what the problem is?

Or am I misinterpreting the output?

What does the "freq" value mean?

Best regards
Jan

[Richard Cochran]

Post by Jan Deinhard
I am using a Debian workstation and a i.MX6UL evaluation board running on a
Linux build with Yocto. According to ethtool -T eth0 both systems support
PTP hardware timestamping.

...

Post by Jan Deinhard
ptp4l[26435.632]: master offset 4399753214 s2 freq [phone] path delay
-277121915
It is not getting better after some minutes. Master offset is alway
increasing and freq is always the same value.

The frequency is locked here at maximum offset, and the device is not
converging. This is most likely a driver or HW problem on the imx6.

Post by Jan Deinhard
$ uname
-a
Linux polaris 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u2 (2016-10-19)
x86_64 GNU/Linux
$ lspci
00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (2)
I219-V (rev 31)

If this i219 is working with the igb driver, and the driver shows HW
time stamping enabled, then it should work well. So this isn't the
problem, most likely.

Post by Jan Deinhard
$ uname -a
Linux imx6ulevk 4.1.15-2.0.0+gb63f3f5 #1 SMP PREEMPT Wed Nov 2 21:42:19 CET
2016 armv7l armv7l armv7l GNU/Linux

I have never tested the ptp driver for the imx6, and I am highly
suspicious of Freescale "quality" drivers.

Post by Jan Deinhard
What can I do to find out what the problem is?

I guess that either the imx6 driver is broken by design, or that it is
mis-configured on your board. I would verify that the input clock to
the time stamping unit is the same frequency as the driver expects, or
if the driver detects the frequency, that this is correct.

You will probably have to dig into the data sheet and into the driver
source.

Here is one simple sanity check for the clock. Run

testptp -g; sleep 60; testptp -g

and verify that the difference between the two outputs is about 60
seconds.

Post by Jan Deinhard
Or am I misinterpreting the output?

Looks like a problem on the imx6. You can run a test between the i219
and any other linux PC (even if that PC only has SW time stamping) in
order to exclude a problem with the i219.

Post by Jan Deinhard
What does the "freq" value mean?

This is the adjustment in parts per billion. Normally the magnitute
of this value should never exceed 100 ppm or so.

HTH,
Richard

[Richard Cochran]

Post by Richard Cochran
testptp -g; sleep 60; testptp -g

But first reset the frequency offset, so do

testptp -f 0; testptp -g; sleep 60; testptp -g

Thanks,
Richard

[Jan Deinhard]

Hi Richard,

Post by Richard Cochran

Post by Jan Deinhard
What can I do to find out what the problem is?

I guess that either the imx6 driver is broken by design, or that it is
mis-configured on your board. I would verify that the input clock to
the time stamping unit is the same frequency as the driver expects, or
if the driver detects the frequency, that this is correct.
You will probably have to dig into the data sheet and into the driver
source.
Here is one simple sanity check for the clock. Run
testptp -g; sleep 60; testptp -g
and verify that the difference between the two outputs is about 60
seconds.

The sanity check on the eval board seems ok:

$ ./a.out -f 0; ./a.out -g; sleep 60; ./a.out -g
frequency adjustment okay
clock time: 1478391262.040584932 or Sun Nov 6 00:14:22 2016
clock time: 1478391322.082633172 or Sun Nov 6 00:15:22 2016

The difference is 60.04204824 seconds so I have to check my PC:

$ ./a.out -f 0; ./a.out -g; sleep 60; ./a.out -g
frequency adjustment okay
clock time: 1478445034.316473288 or Sun Nov 6 16:10:34 2016
clock time: 1478445049.319217444 or Sun Nov 6 16:10:49 2016

The difference is 15.002744156 seconds. Ouch!

I guess I have to check the drivers used on my PC or is there anything else
I can do here?

Thanks,
Jan

Post by Richard Cochran

Post by Jan Deinhard
I am using a Debian workstation and a i.MX6UL evaluation board running

on a

Post by Jan Deinhard
Linux build with Yocto. According to ethtool -T eth0 both systems support
PTP hardware timestamping.

...

Post by Jan Deinhard
ptp4l[26435.632]: master offset 4399753214 s2 freq [phone] path delay
-277121915
It is not getting better after some minutes. Master offset is alway
increasing and freq is always the same value.

The frequency is locked here at maximum offset, and the device is not
converging. This is most likely a driver or HW problem on the imx6.

Post by Jan Deinhard
$ uname
-a
Linux polaris 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u2 (2016-10-19)
x86_64 GNU/Linux
$ lspci
00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (2)
I219-V (rev 31)

If this i219 is working with the igb driver, and the driver shows HW
time stamping enabled, then it should work well. So this isn't the
problem, most likely.

Post by Jan Deinhard
$ uname -a
Linux imx6ulevk 4.1.15-2.0.0+gb63f3f5 #1 SMP PREEMPT Wed Nov 2 21:42:19

CET

Post by Jan Deinhard
2016 armv7l armv7l armv7l GNU/Linux

I have never tested the ptp driver for the imx6, and I am highly
suspicious of Freescale "quality" drivers.

Post by Jan Deinhard
What can I do to find out what the problem is?

I guess that either the imx6 driver is broken by design, or that it is
mis-configured on your board. I would verify that the input clock to
the time stamping unit is the same frequency as the driver expects, or
if the driver detects the frequency, that this is correct.
You will probably have to dig into the data sheet and into the driver
source.
Here is one simple sanity check for the clock. Run
testptp -g; sleep 60; testptp -g
and verify that the difference between the two outputs is about 60
seconds.

Post by Jan Deinhard
Or am I misinterpreting the output?

Looks like a problem on the imx6. You can run a test between the i219
and any other linux PC (even if that PC only has SW time stamping) in
order to exclude a problem with the i219.

Post by Jan Deinhard
What does the "freq" value mean?

This is the adjustment in parts per billion. Normally the magnitute
of this value should never exceed 100 ppm or so.
HTH,
Richard

[Richard Cochran]

Post by Jan Deinhard
The difference is 15.002744156 seconds. Ouch!
I guess I have to check the drivers used on my PC or is there anything else
I can do here?

Ok, so I haven't heard of the i219 before, and somehow the igb driver
is treating it like a i210, but the i219 is obviously different.
You'll have to take this up with Intel. They do have a mailing list,
e1000-devel.

The test result is pretty clear, and the Intel people are usually
responsive.

Thanks,
Richard

[Keller, Jacob E]

This was posted on the linuxptp mailing list possibly highlighting an issue with the igb driver and i219. I thought I would forward it to e1000-devel so that the right people can get a look at it.

Thanks,
Jake

From: Jan Deinhard [mailto:***@gmail.com]
Sent: Sunday, November 06, 2016 10:57 AM
To: linuxptp-***@lists.sourceforge.net
Subject: Re: [Linuxptp-users] Hardware timestamping does not work

Hi Richard,

Post by Jan Deinhard
What can I do to find out what the problem is?

I guess that either the imx6 driver is broken by design, or that it is
mis-configured on your board. I would verify that the input clock to
the time stamping unit is the same frequency as the driver expects, or
if the driver detects the frequency, that this is correct.

You will probably have to dig into the data sheet and into the driver
source.

Here is one simple sanity check for the clock. Run

testptp -g; sleep 60; testptp -g

and verify that the difference between the two outputs is about 60
seconds.

The sanity check on the eval board seems ok:

$ ./a.out -f 0; ./a.out -g; sleep 60; ./a.out -g
frequency adjustment okay
clock time: 1478391262.040584932<tel:[phone]> or Sun Nov 6 00:14:22 2016
clock time: 1478391322.082633172<tel:[phone]> or Sun Nov 6 00:15:22 2016
The difference is 60.04204824<tel:[phone]> seconds so I have to check my PC:

$ ./a.out -f 0; ./a.out -g; sleep 60; ./a.out -g
frequency adjustment okay
clock time: 1478445034.316473288 or Sun Nov 6 16:10:34 2016
clock time: 1478445049.319217444 or Sun Nov 6 16:10:49 2016
The difference is 15.002744156 seconds. Ouch!

I guess I have to check the drivers used on my PC or is there anything else I can do here?

Thanks,
Jan

Post by Jan Deinhard
I am using a Debian workstation and a i.MX6UL evaluation board running on a
Linux build with Yocto. According to ethtool -T eth0 both systems support
PTP hardware timestamping.

...

Post by Jan Deinhard
ptp4l[26435.632]: master offset 4399753214 s2 freq [phone] path delay
-277121915
It is not getting better after some minutes. Master offset is alway
increasing and freq is always the same value.

The frequency is locked here at maximum offset, and the device is not
converging. This is most likely a driver or HW problem on the imx6.

Post by Jan Deinhard
$ uname
-a
Linux polaris 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u2 (2016-10-19)
x86_64 GNU/Linux
$ lspci
00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (2)
I219-V (rev 31)

If this i219 is working with the igb driver, and the driver shows HW
time stamping enabled, then it should work well. So this isn't the
problem, most likely.

Post by Jan Deinhard
$ uname -a
Linux imx6ulevk 4.1.15-2.0.0+gb63f3f5 #1 SMP PREEMPT Wed Nov 2 21:42:19 CET
2016 armv7l armv7l armv7l GNU/Linux

I have never tested the ptp driver for the imx6, and I am highly
suspicious of Freescale "quality" drivers.

Post by Jan Deinhard
What can I do to find out what the problem is?

I guess that either the imx6 driver is broken by design, or that it is
mis-configured on your board. I would verify that the input clock to
the time stamping unit is the same frequency as the driver expects, or
if the driver detects the frequency, that this is correct.

You will probably have to dig into the data sheet and into the driver
source.

Here is one simple sanity check for the clock. Run

testptp -g; sleep 60; testptp -g

and verify that the difference between the two outputs is about 60
seconds.

Post by Jan Deinhard
Or am I misinterpreting the output?

Looks like a problem on the imx6. You can run a test between the i219
and any other linux PC (even if that PC only has SW time stamping) in
order to exclude a problem with the i219.

Post by Jan Deinhard
What does the "freq" value mean?

This is the adjustment in parts per billion. Normally the magnitute
of this value should never exceed 100 ppm or so.

HTH,
Richard