[Linuxptp-users] Problems with ptp4l via a Cisco Nexus 5000 Switch

[Christian]

Hello,

I'm currently trying to set up a ptp4l session between 3 servers, which are connected via a Cisco Nexus 5000 switch


Server A is supposed to be the grand master, Server B and C should be the slaves.



The Grandmaster is working fine, the Switch does accept it, but the Slaves are not working properly.

Here is the ptp log of one of the slave servers:

@s0002794:~$ ptp4l[618557.966]: selected /dev/ptp0 as PTP clock
ptp4l[618557.967]: driver changed our HWTSTAMP options
ptp4l[618557.967]: tx_type 1 not 1
ptp4l[618557.967]: rx_filter 1 not 12
ptp4l[618557.967]: port 1: INITIALIZING to LISTENING on INITIALIZE
ptp4l[618557.967]: port 0: INITIALIZING to LISTENING on INITIALIZE
ptp4l[618564.664]: driver changed our HWTSTAMP options
ptp4l[618564.664]: tx_type 1 not 1
ptp4l[618564.664]: rx_filter 1 not 12
ptp4l[618564.664]: selected best master clock a0369f.fffe.a1b68c <-- MAC Addr. of the Slave server
ptp4l[618564.840]: port 1: new foreign master 002a6a.fffe.ac97fc-16 <-- MAC Addr. of the Switch port
ptp4l[618571.137]: driver changed our HWTSTAMP options
ptp4l[618571.137]: tx_type 1 not 1
ptp4l[618571.137]: rx_filter 1 not 12
ptp4l[618571.137]: selected best master clock a0369f.fffe.a1b68c
ptp4l[618578.192]: driver changed our HWTSTAMP options
ptp4l[618578.192]: tx_type 1 not 1
ptp4l[618578.192]: rx_filter 1 not 12
ptp4l[618578.192]: selected best master clock a0369f.fffe.a1b68c
ptp4l[618580.850]: selected best master clock a0369f.fffe.a1b688 <-- MAC Addr. of the Master server
ptp4l[618580.850]: foreign master not using PTP timescale
ptp4l[618580.850]: running in a temporal vortex
ptp4l[618580.850]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE
ptp4l[618582.669]: port 1: minimum delay request interval 2^4
ptp4l[618582.851]: master offset 1299422 s0 freq +4989 path delay -11798
ptp4l[618584.852]: master offset 1321175 s1 freq +15865 path delay -9425
ptp4l[618587.572]: driver changed our HWTSTAMP options
ptp4l[618587.572]: tx_type 1 not 1
ptp4l[618587.572]: rx_filter 1 not 12
ptp4l[618587.572]: port 1: UNCALIBRATED to LISTENING on ANNOUNCE_RECEIPT_TIMEOUT_EXPIRES
ptp4l[618587.572]: selected best master clock a0369f.fffe.a1b68c
ptp4l[618594.597]: driver changed our HWTSTAMP options
ptp4l[618594.597]: tx_type 1 not 1
ptp4l[618594.597]: rx_filter 1 not 12
ptp4l[618594.597]: selected best master clock a0369f.fffe.a1b68c
ptp4l[618596.851]: selected best master clock a0369f.fffe.a1b688
ptp4l[618596.851]: foreign master not using PTP timescale
ptp4l[618596.851]: running in a temporal vortex
ptp4l[618596.851]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE
ptp4l[618598.850]: master offset 43778 s2 freq +37754 path delay 0
ptp4l[618598.850]: port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED
ptp4l[618600.852]: master offset 6710 s2 freq +25787 path delay 0

there also was this line every one in a while:

ptp4l[5828.626]: port 1: minimum delay request interval 2^^4


I am starting the Master with a config File.
ptp4l -f /etc/ptp.config

with following config file:

[global]
verbose 1
path_trace_enabled 1
time_stamping hardware
priority1 1
priority2 1
[eth4]


The Slaves are started via: ptp4l -A -m -i eth4 -s

I also wrote a config file for them, but if I use the file, they do select themself as the best clock.



***@s0002794:~$ ptp4l[619043.020]: selected best master clock a0369f.fffe.a1b68c <--MAC Addr. of the Slave
ptp4l[619050.271]: selected best master clock a0369f.fffe.a1b68c
ptp4l[619056.348]: selected best master clock a0369f.fffe.a1b68c
ptp4l[619062.726]: selected best master clock a0369f.fffe.a1b68c

with following config File:

[global]
verbose 1
path_trace_enabled 1
time_stamping hardware
slaveOnly 1
priority1 255
priority2 255
[eth4]


Phc2sys is running on both, the Master and the Slaves.
Master: phc2sys -s CLOCK_REALTIME -c eth4 -w &
Slaves: phc2sys -s eth4 -w &


Here is the configuration of the Switch:


ptp brief:

PTP port status
-----------------------
Port State
------- --------------
Eth1/17 Master <--Server C
Eth1/19 Master <--Server B
Eth1/31 Slave <--Server A

ptp clock:

PTP Device Type: Boundary clock
Clock Identity : 00:2a:6a:ff:fe:ac:97:fc
Clock Domain: 0
Number of PTP ports: 3
Priority1 : 2
Priority2 : 2
Clock Quality:
Class : 248
Accuracy : 254
Offset (log variance) : 65535
Offset From Master : 467
Mean Path Delay : 33580
Steps removed : 1
Local clock time:Thu Jun 23 13:52:14 2016

Slave port:
PTP Port Dataset: Eth1/17
Port identity: clock identity: 00:2a:6a:ff:fe:ac:97:fc
Port identity: port number: 16
PTP version: 2
Port state: Master
VLAN info: 1
Delay request interval(log mean): 4
Announce receipt time out: 2
Peer mean path delay: 0
Announce interval(log mean): 3
Sync interval(log mean): 1
Delay Mechanism: End to End

Master port:
PTP Port Dataset: Eth1/31
Port identity: clock identity: 00:2a:6a:ff:fe:ac:97:fc
Port identity: port number: 30
PTP version: 2
Port state: Slave
VLAN info: 1
Delay request interval(log mean): 4
Announce receipt time out: 2
Peer mean path delay: 0
Announce interval(log mean): 3
Sync interval(log mean): 1
Delay Mechanism: End to End


Thank you in advance.


Greeting,
Christian

[Richard Cochran]

Post by Christian
ptp4l[5828.626]: port 1: minimum delay request interval 2^^4

That is informational only, not an error.

Post by Christian
I am starting the Master with a config File.
ptp4l -f /etc/ptp.config
[global]
verbose 1
path_trace_enabled 1

...

Post by Christian
The Slaves are started via: ptp4l -A -m -i eth4 -s
[global]
verbose 1
path_trace_enabled 1

You have path_trace_enabled, but why? That is only for use in gPTP
according to IEEE 802.1AS-2011.

Post by Christian
PTP Device Type: Boundary clock

Your switch is a BC. Maybe it cannot deal with the path trace
option.

Thanks,
Richard

[Christian]

I took it out and it still has the same Problem.

Post by Richard Cochran
You have path_trace_enabled, but why? That is only for use in gPTP
according to IEEE 802.1AS-2011.

Post by Christian
PTP Device Type: Boundary clock

Your switch is a BC. Maybe it cannot deal with the path trace
option.

[Christian]

Hello,

As I have not made any progress in the matter, I have made a new log, so
you can maybe see better where the Problem is.
I have taken out the path trace enable option and upped the logging
level to 7, else all the configuration is still the same.


Here are pictures of the logs:
Server s0002796 is the master s0002792 ands0002794 are the slaves.

ptplog7start is the log it produced right after starting ptp4l.:
Loading Image...
ptplog7 is shortly thereafter: Loading Image...
ptplog7_2 is a bit later.: Loading Image...


After a while I checked the PHC clocks via phc_ctl and 94 seemed to be
synchronising, while 92 was not.


Thanks again for every little bit of help.

Greetings,
Christian

Post by Christian
Hello,
I'm currently trying to set up a ptp4l session between 3 servers, which are connected via a Cisco Nexus 5000 switch
Server A is supposed to be the grand master, Server B and C should be the slaves.
The Grandmaster is working fine, the Switch does accept it, but the Slaves are not working properly.
@s0002794:~$ ptp4l[618557.966]: selected /dev/ptp0 as PTP clock
ptp4l[618557.967]: driver changed our HWTSTAMP options
ptp4l[618557.967]: tx_type 1 not 1
ptp4l[618557.967]: rx_filter 1 not 12
ptp4l[618557.967]: port 1: INITIALIZING to LISTENING on INITIALIZE
ptp4l[618557.967]: port 0: INITIALIZING to LISTENING on INITIALIZE
ptp4l[618564.664]: driver changed our HWTSTAMP options
ptp4l[618564.664]: tx_type 1 not 1
ptp4l[618564.664]: rx_filter 1 not 12
ptp4l[618564.664]: selected best master clock a0369f.fffe.a1b68c <-- MAC Addr. of the Slave server
ptp4l[618564.840]: port 1: new foreign master 002a6a.fffe.ac97fc-16 <-- MAC Addr. of the Switch port
ptp4l[618571.137]: driver changed our HWTSTAMP options
ptp4l[618571.137]: tx_type 1 not 1
ptp4l[618571.137]: rx_filter 1 not 12
ptp4l[618571.137]: selected best master clock a0369f.fffe.a1b68c
ptp4l[618578.192]: driver changed our HWTSTAMP options
ptp4l[618578.192]: tx_type 1 not 1
ptp4l[618578.192]: rx_filter 1 not 12
ptp4l[618578.192]: selected best master clock a0369f.fffe.a1b68c
ptp4l[618580.850]: selected best master clock a0369f.fffe.a1b688 <-- MAC Addr. of the Master server
ptp4l[618580.850]: foreign master not using PTP timescale
ptp4l[618580.850]: running in a temporal vortex
ptp4l[618580.850]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE
ptp4l[618582.669]: port 1: minimum delay request interval 2^4
ptp4l[618582.851]: master offset 1299422 s0 freq +4989 path delay -11798
ptp4l[618584.852]: master offset 1321175 s1 freq +15865 path delay -9425
ptp4l[618587.572]: driver changed our HWTSTAMP options
ptp4l[618587.572]: tx_type 1 not 1
ptp4l[618587.572]: rx_filter 1 not 12
ptp4l[618587.572]: port 1: UNCALIBRATED to LISTENING on ANNOUNCE_RECEIPT_TIMEOUT_EXPIRES
ptp4l[618587.572]: selected best master clock a0369f.fffe.a1b68c
ptp4l[618594.597]: driver changed our HWTSTAMP options
ptp4l[618594.597]: tx_type 1 not 1
ptp4l[618594.597]: rx_filter 1 not 12
ptp4l[618594.597]: selected best master clock a0369f.fffe.a1b68c
ptp4l[618596.851]: selected best master clock a0369f.fffe.a1b688
ptp4l[618596.851]: foreign master not using PTP timescale
ptp4l[618596.851]: running in a temporal vortex
ptp4l[618596.851]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE
ptp4l[618598.850]: master offset 43778 s2 freq +37754 path delay 0
ptp4l[618598.850]: port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED
ptp4l[618600.852]: master offset 6710 s2 freq +25787 path delay 0
ptp4l[5828.626]: port 1: minimum delay request interval 2^^4
I am starting the Master with a config File.
ptp4l -f /etc/ptp.config
[global]
verbose 1
path_trace_enabled 1
time_stamping hardware
priority1 1
priority2 1
[eth4]
The Slaves are started via: ptp4l -A -m -i eth4 -s
I also wrote a config file for them, but if I use the file, they do select themself as the best clock.
ptp4l[619050.271]: selected best master clock a0369f.fffe.a1b68c
ptp4l[619056.348]: selected best master clock a0369f.fffe.a1b68c
ptp4l[619062.726]: selected best master clock a0369f.fffe.a1b68c
[global]
verbose 1
path_trace_enabled 1
time_stamping hardware
slaveOnly 1
priority1 255
priority2 255
[eth4]
Phc2sys is running on both, the Master and the Slaves.
Master: phc2sys -s CLOCK_REALTIME -c eth4 -w &
Slaves: phc2sys -s eth4 -w &
PTP port status
-----------------------
Port State
------- --------------
Eth1/17 Master <--Server C
Eth1/19 Master <--Server B
Eth1/31 Slave <--Server A
PTP Device Type: Boundary clock
Clock Identity : 00:2a:6a:ff:fe:ac:97:fc
Clock Domain: 0
Number of PTP ports: 3
Priority1 : 2
Priority2 : 2
Class : 248
Accuracy : 254
Offset (log variance) : 65535
Offset From Master : 467
Mean Path Delay : 33580
Steps removed : 1
Local clock time:Thu Jun 23 13:52:14 2016
PTP Port Dataset: Eth1/17
Port identity: clock identity: 00:2a:6a:ff:fe:ac:97:fc
Port identity: port number: 16
PTP version: 2
Port state: Master
VLAN info: 1
Delay request interval(log mean): 4
Announce receipt time out: 2
Peer mean path delay: 0
Announce interval(log mean): 3
Sync interval(log mean): 1
Delay Mechanism: End to End
PTP Port Dataset: Eth1/31
Port identity: clock identity: 00:2a:6a:ff:fe:ac:97:fc
Port identity: port number: 30
PTP version: 2
Port state: Slave
VLAN info: 1
Delay request interval(log mean): 4
Announce receipt time out: 2
Peer mean path delay: 0
Announce interval(log mean): 3
Sync interval(log mean): 1
Delay Mechanism: End to End
Thank you in advance.
Greeting,
Christian
------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Linuxptp-users mailing list
https://lists.sourceforge.net/lists/listinfo/linuxptp-users

[Dale Smith]

On Thu, Jun 30, 2016 at 8:14 AM, Christian <

Post by Christian
Server s0002796 is the master s0002792 ands0002794 are the slaves.
https://s32.postimg.org/i6af9ejx1/ptplog7start.png
https://s32.postimg.org/bnsuv6rat/ptplog7.png
ptplog7_2 is a bit later.: https://s32.postimg.org/jmtjfahhh/ptplog7_2.png

Howdy Christian,

I think it would be far better to submit the logs as text instead of
images. Those images are very difficult to read and impossible to edit
(the text).

-Dale

[Richard Cochran]

Post by Dale Smith
I think it would be far better to submit the logs as text instead of
images. Those images are very difficult to read and impossible to edit
(the text).

+1

In addition, can you post a short pcap file showing the PTP traffic on
the non-working port?

Thanks,
Richard

[Christian]

I have attached 3 .txt files with the logs. Again, 96 is the master
server and 94 and 92 are the slaves.
And I have attached a pcap file of the Server 94(not captured at the
same moment, but the behavior was the same).

Post by Richard Cochran

Post by Dale Smith
I think it would be far better to submit the logs as text instead of
images. Those images are very difficult to read and impossible to edit
(the text).

+1
In addition, can you post a short pcap file showing the PTP traffic on
the non-working port?
Thanks,
Richard

[Richard Cochran]

I have attached 3 .txt files with the logs. Again, 96 is the master server
and 94 and 92 are the slaves.
And I have attached a pcap file of the Server 94(not captured at the same
moment, but the behavior was the same).

Sorry about the delay. I finally took a look at your problem...

In the PCAP, the Announce messages are only sent by the Cisco every
seven seconds. The PCAP frames are truncated, so we can't see what
the Cisco advertises for the logMessageInterval (logAnnounceInterval).
I guess it is set to 3 (2^3 = 8 seconds).

Anyhow, you can fix this by configuring the Cisco and your client to
the same logAnnounceInterval value.

Thanks,
Richard

[Richard Cochran]

ptp4l[622912.203]: port 0: INITIALIZING to LISTENING on INITIALIZE
ptp4l[622912.209]: port 0: setting asCapable
ptp4l[622918.492]: port 0: announce timeout
ptp4l[622925.985]: port 0: announce timeout
ptp4l[622932.071]: port 0: announce timeout

Port 0 is the UDS port and should never see an announce message
timeout. This is a regression in v1.6. I just posted a fix for it on
the linuxptp-devel mailing list.

Thanks,
Richard