[Linuxptp-users] (no subject)

[Wolfgang Wallner]

Hello LinuxPTP community,

We (B&R) are currently evaluating the available open-source PTP implementations (LinxPTP, PTPd, OpenAVB) for the usage as IEEE 802.1AS stacks.
As part of this effort, I have spent some time to test LinuxPTP on a B&R industrial PC with an Intel i210 NIC.
LinuxPTP already supports most of 802.1AS, but I have also found some issues, which I would like to discuss with you.
I have listed the individual issues below, feedback on any of these issues would be welcome.
Especially if they are already known, if someone is working on them, or if the LinuxPTP community would be interested in contributions for these issues.

kind regards,
Wolfgang Wallner

PS: I tried to scan the archives before posting this message.
Unfortunately I always get "Error 403: Read access required" when I try to access https://sourceforge.net/p/linuxptp/mailman/linuxptp-users/
Do I need any further permissions for my sourceforge account to view the archives?


LinuxPTP 802.1AS issues:

1) phc2sys does not work if transportSpecific is set to 1

When I set transportSpecific to 1 (as required by 802.1AS) in my config file, the phc2sys tool does not work.
My test config file looks as follows:

$ cat TestConfig.cfg

[global]
transportSpecific 1

When I call ptp4l with "sudo ptp4l -f TestConfig.cfg -i enp2s0 -p /dev/ptp0 -m" (without that config file) and phc2sys with "sudo phc2sys -a -r -r -m -S 0.2 -F 0.2" it only prints "phc2sys[5236.261]: Waiting for ptp4l..." every second. If I start ptp4l without that config file or comment out that one line, phc2sys works as expected.

2) Differences in the BMC algorithm

Is it possible to specify a different best master clock algorithm than that of the default PTP profile?
I could not find any such option, please correct me if I have overlooked something.

The BMCA of the default PTP profile and 802.1AS are similar, but not the same.
Noticable differences are:

*) There is no pre-master state in 802.1AS
*) There is no foreign master qualification in 802.1AS
*) If a master stops sending sync messages but still sends announce messages, this should trigger a state change in the slave according to 802.1AS (configured via syncReceiptTimeoutTime), while it is fine in default PTP.

3) PDelay uses wrong clock

802.1AS specifies in 11.1.2 the following:

"In practice, t1 and t4 are measured relative to the LocalClock entity of the initiator time-aware system, and t2 and t3 are measured relative to the LocalClock entity of the responder time-aware system."

The LocalClock in 802.1AS is a local free running oscillator, in contrast to the recovered clock that is created via PTP.
However, I see in the t2 and t3 timestamps of my LinuxPTP instances times refering to the time that is transfered via PTP.

4) Path Trace TLV

Announce messages without a Path Trace TLV are ignored by my device if I set path_trace_enabled to 1.
I think in 802.1AS a Path Trace TLV should be attached to announce messages, but announce messages should not be dropped if a received message does not have one.

5) The PPS signal produced by the i210 seems to be unreliable

I know that the right place to discuss i210 problems would be LMKL, I just mention this here as I assume several of you might have experience with this chip.
I observe that if I enable the PPS signal on the software definable pins of the i210 it initially works fine.
But if there are state changes in the PTP stack because I unplug/replug other devices to the network, the PPS signal of the i210 can disappear.
Calling the corresponding ioctls again (and thus setting the register of the i210 again) restores the signal.
Does this match your experience?

[Richard Cochran]

Wolfgang,

In general I am not aware of any issues regarding AS compatibility.
We tried very had to follow the standard as closely as possible.
Delio (on CC) might have more to say about this. He maintains his own
AS branch on github.

https://github.com/audioscience/linuxptp

I just briefly compared v1.4 with his asi1230-master, and nothing
earth shaking stood out.

On Mon, Oct 31, 2016 at 08:58:57AM +0100, Wolfgang Wallner wrote:
> Do I need any further permissions for my sourceforge account to view the archives?

I just fiddled with the SF settings for the lists. Can you see it now?

(I have been recommending gmane instead, but that site went down.)

> LinuxPTP 802.1AS issues:
>
> 1) phc2sys does not work if transportSpecific is set to 1
>
> When I set transportSpecific to 1 (as required by 802.1AS) in my config file, the phc2sys tool does not work.
> My test config file looks as follows:
>
> $ cat TestConfig.cfg
>
> [global]
> transportSpecific 1
>
> When I call ptp4l with "sudo ptp4l -f TestConfig.cfg -i enp2s0 -p
> /dev/ptp0 -m" (without that config file) and phc2sys with "sudo
> phc2sys -a -r -r -m -S 0.2 -F 0.2" it only prints
> "phc2sys[5236.261]: Waiting for ptp4l..." every second. If I start
> ptp4l without that config file or comment out that one line, phc2sys
> works as expected.

Can you try this instead?

[global]
[enp2s0]
#
# Not in the global section!
#
transportSpecific 1

This is bug where the gloabl transportSpecific setting is being
applied to the UDS interface. I'll fix this soon.

> 2) Differences in the BMC algorithm
>
> Is it possible to specify a different best master clock algorithm than that of the default PTP profile?

No, not yet. I have some patches in the works for allowing different
BMC variants, because some profiles will require it.

However, the AS mode should work fine with the default BMC...

> The BMCA of the default PTP profile and 802.1AS are similar, but not the same.
> Noticable differences are:
>
> *) There is no pre-master state in 802.1AS

This only causes a master to wait an additional announce period before
sending the first announce and sync messages. It doesn't affect how
we inter-operate in a gPTP network.

> *) There is no foreign master qualification in 802.1AS

Again, this doesn't affect correctness WRT how we inter-operate. This
only affects the slave, causing a short delay before starting
(internal) synchronization.

> *) If a master stops sending sync messages but still sends announce messages, this should trigger a state change in the slave according to 802.1AS (configured via syncReceiptTimeoutTime), while it is fine in default PTP.

This is implemented. Did you test this?

> 3) PDelay uses wrong clock
>
> 802.1AS specifies in 11.1.2 the following:
>
> "In practice, t1 and t4 are measured relative to the LocalClock entity of the initiator time-aware system, and t2 and t3 are measured relative to the LocalClock entity of the responder time-aware system."
>
> The LocalClock in 802.1AS is a local free running oscillator, in contrast to the recovered clock that is created via PTP.
> However, I see in the t2 and t3 timestamps of my LinuxPTP instances times refering to the time that is transfered via PTP.

No, the interval T3-T2 is converted into the local time base.
See get_raw_delay().

> 4) Path Trace TLV
>
> Announce messages without a Path Trace TLV are ignored by my device if I set path_trace_enabled to 1.
> I think in 802.1AS a Path Trace TLV should be attached to announce messages, but announce messages should not be dropped if a received message does not have one.

First you complain that we don't follow AS exactly, and then you
complain that we *do* follow it exactly!

Path Trace is specified in 10.5.3, and there is no hint of it being
optional. In fact, this is required in the PICS. Any master who
sends an Announce must conform. Look at page 240.

MIMSTR-8 Does the Announce message body comply
with the requirements in 10.5.3.1 and
Table 10-7?

@Delio, I see you allow announce without the TLV:

https://github.com/audioscience/linuxptp/commit/0b6d8ca73332391af9bddc25177df254b066d669

Can you tell us why?

> 5) The PPS signal produced by the i210 seems to be unreliable
>
> I know that the right place to discuss i210 problems would be LMKL, I just mention this here as I assume several of you might have experience with this chip.
> I observe that if I enable the PPS signal on the software definable pins of the i210 it initially works fine.
> But if there are state changes in the PTP stack because I unplug/replug other devices to the network, the PPS signal of the i210 can disappear.
> Calling the corresponding ioctls again (and thus setting the register of the i210 again) restores the signal.
> Does this match your experience?

Yes. You must stop and restart again whenever the time jumps. You
should write a script to monitor ptp4l and restart when needed.

Thanks,
Richard

[Wolfgang Wallner]

Hi Delio,

> >> 2) Differences in the BMC algorithm
> >>
> >> Is it possible to specify a different best master clock algorithm than that of the default PTP profile?
> >
> > No, not yet. I have some patches in the works for allowing different
> > BMC variants, because some profiles will require it.
> >
> > However, the AS mode should work fine with the default BMC…
>
> I can confirm AS mode works with the default BMC. As Richard says below the differences result in extra delays but peers eventually “converge”.

Ok, thanks for the clarification.

> >> 4) Path Trace TLV
> > @Delio, I see you allow announce without the TLV:
> >
> > https://github.com/audioscience/linuxptp/commit/0b6d8ca73332391af9bddc25177df254b066d669
> >
> > Can you tell us why?
>
> This is tricky because in some places 802.1AS-2011 seems to imply a TLV is mandatory but when you dig down into the implementation details you find that: in 10.3.13.2.1 (f) txAnnounce() appends the TLV *only if* it doesn’t exceed the transport’s MTU and in 10.3.10.2.1 (c) and (d) qualifyAnnounce() accepts an announce message without a TLV. So a TLV is present *if* it fits in the transport’s MTU but a compliant implementation has to accept an announce message without path trace TLV.

I haven't studied the 802.1AS specification in detail yet, but your description matches my current understanding of how the path trace TLVs work.

regards, Wolfgang

[Richard Cochran]

On Mon, Oct 31, 2016 at 02:07:01PM +0100, Delio Brignoli wrote:
> > @Delio, I see you allow announce without the TLV:
> >
> > https://github.com/audioscience/linuxptp/commit/0b6d8ca73332391af9bddc25177df254b066d669
> >
> > Can you tell us why?
>
> This is tricky because in some places 802.1AS-2011 seems to imply a
> TLV is mandatory but when you dig down into the implementation
> details you find that: in 10.3.13.2.1 (f) txAnnounce() appends the
> TLV *only if* it doesn’t exceed the transport’s MTU and in
> 10.3.10.2.1 (c) and (d) qualifyAnnounce() accepts an announce
> message without a TLV. So a TLV is present *if* it fits in the
> transport’s MTU but a compliant implementation has to accept an
> announce message without path trace TLV.

Those passages permit dropping the TLV when it would cause the message
to exceed the frame size. Let's do a little math.

The Announce message takes 78 bytes, the TLV takes 4 + 8N bytes, and
an Ethernet frame can be as large as 1500 bytes. So the gPTP network
would have to have 177 hops before dropping the TLV is allowed.

Was the switch you had operating in such a large network?

If the TLV is missing, chances are that the sender is out of
compliance, don't you think?

> Additionally the path trace TLV could be one of many TLVs appended
> to the announce message and it doesn’t have to be the first. Even
> with my patch applied linuxptp expects the path trace TLV to be the
> first one to appear. I would like to fix it but we considered it a
> non-urgent issue and had to shift focus to something else so it
> never got done.

You are right, in the presence of multiple TLVs, the Path Trace TLV
could come appear at another position, and you are right that it is a
non-issue.

When we come to the place where 177 switch AVB networks are common and
many more TLVs are defined, then we'll have to fix these points.

Thanks,
Richard

[Delio Brignoli]

> On 31 Oct 2016, at 16:06, Richard Cochran <***@gmail.com> wrote:
>
> On Mon, Oct 31, 2016 at 02:07:01PM +0100, Delio Brignoli wrote:
>>> @Delio, I see you allow announce without the TLV:
>>>
>>> https://github.com/audioscience/linuxptp/commit/0b6d8ca73332391af9bddc25177df254b066d669
>>>
>>> Can you tell us why?
>>
>> This is tricky because in some places 802.1AS-2011 seems to imply a
>> TLV is mandatory but when you dig down into the implementation
>> details you find that: in 10.3.13.2.1 (f) txAnnounce() appends the
>> TLV *only if* it doesn’t exceed the transport’s MTU and in
>> 10.3.10.2.1 (c) and (d) qualifyAnnounce() accepts an announce
>> message without a TLV. So a TLV is present *if* it fits in the
>> transport’s MTU but a compliant implementation has to accept an
>> announce message without path trace TLV.
>
> Those passages permit dropping the TLV when it would cause the message
> to exceed the frame size. Let's do a little math.
[...]
> Was the switch you had operating in such a large network?

Or the MTU may be artificially restricted, who knows... but I am not saying linuxptp should be changed to conditionally drop the TLV.

> If the TLV is missing, chances are that the sender is out of
> compliance, don't you think?

Yes, but I stated as much in my previous message: the switch was not compliant. However, linuxptp isn’t compliant either on the RX side which is the reason for my patch.

Regards
—
Delio

[Richard Cochran]

On Mon, Oct 31, 2016 at 05:29:30PM +0100, Delio Brignoli wrote:
>
> Yes, but I stated as much in my previous message: the switch was not
> compliant. However, linuxptp isn’t compliant either on the RX side
> which is the reason for my patch.

The whole reason for the Path Trace is to avoid loops. At face value,
this is not a bad idea. But the exception means that really large
networks will not have the check. Brilliant. Just where the feature
make the most sense, they turn it off.

I am hoping that the next editions of 802.1AS and 1588 will remove
some of the brain farts and not add too many new ones.

One can always hope...

Thanks,
Richard

[Richard Cochran]

On Mon, Oct 31, 2016 at 07:31:35PM +0100, Richard Cochran wrote:
> On Mon, Oct 31, 2016 at 05:29:30PM +0100, Delio Brignoli wrote:
> >
> > Yes, but I stated as much in my previous message: the switch was not
> > compliant. However, linuxptp isn’t compliant either on the RX side
> > which is the reason for my patch.
>
> The whole reason for the Path Trace is to avoid loops.

Having thought about it more, I would argue that we already are
compliant. Let me explain.

The Path Trace TLV is meant to avoid loops, but according to the
standard it must be dropped if the clock list does not fit into the
MTU. This is a property of the network, not of a particular node. If
the TLV will be dropped in a switch, then there is no point for the
master to send one in the first place.

If a network cannot support the TLV, then the administrator must know
this, and he must have some other means to prevent loops, otherwise
the gPTP protocol will fail.

Our implementation doesn't have a simple AVB on/off option, but rather
each aspect can be configured individually. In a non-Path-Trace
capable network, the admin simply sets "path_trace_enabled 0" in the
configuration file.

See?

Thanks,
Richard

[Richard Cochran]

On Mon, Oct 31, 2016 at 12:17:54PM +0100, Richard Cochran wrote:
> This is bug where the gloabl transportSpecific setting is being
> applied to the UDS interface. I'll fix this soon.

I take this back. I thought about it some more and decided not to
change this. Here is why.

First of all, if you run AVB according to the standard, then you
should not run phc2sys in the first place. The standard tells you to
leave the local clock free running and calculate the phase and
frequency numerically. So strictly speaking, you would write a
program similar to phc2sys that queries TIME_STATUS_NP and uses that
information to tune the system clock.

[ Ok, so that is rather weak, since the end station can and should
tune its PHC if it wants to. Who cares what the standard says. In
fact, our gPTP.cfg doesn't even set the free_running option. But
the next reason is a better one. ]

Secondly, we already have UDS clients that expect the global
transportSpecific flag to appear in the UDS messages. The pmc program
has a command line option for this. If we change the UDS port to mask
the flag away, this will break existing management clients.

So I think the work around is a reasonable solution for phc2sys. If
anything, phc2sys should also support the transportSpecific flag, but
this can wait until we can pass a config file to it. We don't want
yet another command line option, IMHO.

Thanks,
Richard

[Wolfgang Wallner]

Hello Richard,

> > Do I need any further permissions for my sourceforge account to view the archives?
>
> I just fiddled with the SF settings for the lists. Can you see it now?
>
> (I have been recommending gmane instead, but that site went down.)

I still get the same error (Error 403, Read access required).

> > LinuxPTP 802.1AS issues:
> >
> > 1) phc2sys does not work if transportSpecific is set to 1
> >
> > When I set transportSpecific to 1 (as required by 802.1AS) in my config file, the phc2sys tool does not work.
> > My test config file looks as follows:
> >
> > $ cat TestConfig.cfg
> >
> > [global]
> > transportSpecific 1
> >
> > When I call ptp4l with "sudo ptp4l -f TestConfig.cfg -i enp2s0 -p
> > /dev/ptp0 -m" (without that config file) and phc2sys with "sudo
> > phc2sys -a -r -r -m -S 0.2 -F 0.2" it only prints
> > "phc2sys[5236.261]: Waiting for ptp4l..." every second. If I start
> > ptp4l without that config file or comment out that one line, phc2sys
> > works as expected.
>
> Can you try this instead?
>
> [global]
> [enp2s0]
> #
> # Not in the global section!
> #
> transportSpecific 1
>
> This is bug where the gloabl transportSpecific setting is being
> applied to the UDS interface. I'll fix this soon.

Thanks for the hint, with this configuration it works as expected.

> > 2) Differences in the BMC algorithm
> >
> > Is it possible to specify a different best master clock algorithm than that of the default PTP profile?
>
> No, not yet. I have some patches in the works for allowing different
> BMC variants, because some profiles will require it.
>
> However, the AS mode should work fine with the default BMC...
>

Ok, thanks for the clarification.

> > The BMCA of the default PTP profile and 802.1AS are similar, but not the same.
> > Noticable differences are:
> >
> > *) There is no pre-master state in 802.1AS
>
> This only causes a master to wait an additional announce period before
> sending the first announce and sync messages. It doesn't affect how
> we inter-operate in a gPTP network.
>
> > *) There is no foreign master qualification in 802.1AS
>
> Again, this doesn't affect correctness WRT how we inter-operate. This
> only affects the slave, causing a short delay before starting
> (internal) synchronization.
>
> > *) If a master stops sending sync messages but still sends announce messages, this should trigger a state change in the slave according to 802.1AS (configured via syncReceiptTimeoutTime), while it is fine in default PTP.
>
> This is implemented. Did you test this?

Sorry, this is my fault.
I just checked my config and syncReceiptTimeout is set to 0.
I thought I had already changed it on my test device.

> > 3) PDelay uses wrong clock
> >
> > 802.1AS specifies in 11.1.2 the following:
> >
> > "In practice, t1 and t4 are measured relative to the LocalClock entity of the initiator time-aware system, and t2 and t3 are measured relative to the LocalClock entity of the responder time-aware system."
> >
> > The LocalClock in 802.1AS is a local free running oscillator, in contrast to the recovered clock that is created via PTP.
> > However, I see in the t2 and t3 timestamps of my LinuxPTP instances times refering to the time that is transfered via PTP.
>
> No, the interval T3-T2 is converted into the local time base.
> See get_raw_delay().

I will have a lock at it.

> > 4) Path Trace TLV
> >
> > Announce messages without a Path Trace TLV are ignored by my device if I set path_trace_enabled to 1.
> > I think in 802.1AS a Path Trace TLV should be attached to announce messages, but announce messages should not be dropped if a received message does not have one.
>
> First you complain that we don't follow AS exactly, and then you
> complain that we *do* follow it exactly!

Please don't get me wrong, I'm not complaining :)
I just want to discuss what seemed non-compliant.

> Path Trace is specified in 10.5.3, and there is no hint of it being
> optional. In fact, this is required in the PICS. Any master who
> sends an Announce must conform. Look at page 240.
>
> MIMSTR-8 Does the Announce message body comply
> with the requirements in 10.5.3.1 and
> Table 10-7?
>
> @Delio, I see you allow announce without the TLV:
>
> https://github.com/audioscience/linuxptp/commit/0b6d8ca73332391af9bddc25177df254b066d669
>
> Can you tell us why?
>
> > 5) The PPS signal produced by the i210 seems to be unreliable
> >
> > I know that the right place to discuss i210 problems would be LMKL, I just mention this here as I assume several of you might have experience with this chip.
> > I observe that if I enable the PPS signal on the software definable pins of the i210 it initially works fine.
> > But if there are state changes in the PTP stack because I unplug/replug other devices to the network, the PPS signal of the i210 can disappear.
> > Calling the corresponding ioctls again (and thus setting the register of the i210 again) restores the signal.
> > Does this match your experience?
>
> Yes. You must stop and restart again whenever the time jumps. You
> should write a script to monitor ptp4l and restart when needed.

Ok, thanks for the information!

This leads to another point that I have not figured out yet.
If I would like to get notified in another program about PTP state changes, what is the preferred interface?

regards, Wolfgang

PS: I forgot to set the mail subject in my initial mailing list post in the morning.
Should we change the subject to something meaningful like "802.1AS discussion" now or would that cause trouble with the e.g. the thread view in mailing list archives?

[Richard Cochran]

This post might be inappropriate. Click to display it.

[Richard Cochran]

On Mon, Oct 31, 2016 at 03:49:39PM +0100, Wolfgang Wallner wrote:
> I still get the same error (Error 403, Read access required).

Changed settings again. Does it work for you now?

Thanks,
Richard

[Wolfgang Wallner]

-----Richard Cochran <***@gmail.com> schrieb: -----
> > I still get the same error (Error 403, Read access required).
>
> Changed settings again. Does it work for you now?

Yes, now it works fine (even without logging in to SourceForge).
Thanks for fixing this!

regards, Wolfgang

[Keller, Jacob E]

On Mon, 2016-10-31 at 08:58 +0100, Wolfgang Wallner wrote:
> 5) The PPS signal produced by the i210 seems to be unreliable
>
> I know that the right place to discuss i210 problems would be LMKL, I
> just mention this here as I assume several of you might have
> experience with this chip.
> I observe that if I enable the PPS signal on the software definable
> pins of the i210 it initially works fine.
> But if there are state changes in the PTP stack because I
> unplug/replug other devices to the network, the PPS signal of the
> i210 can disappear.
> Calling the corresponding ioctls again (and thus setting the register
> of the i210 again) restores the signal.
> Does this match your experience?

What version of i210 driver are you using? An in-kernel version? What
kernel version? There have been some fixes to the upstream driver
recently which may have resolved this issue. The i210 driver was not
resetting the PTP signals when it changed link status, and I believe
this may have been fixed. If not, providing more detail may be useful
in forwarding this to the current driver owner.

Thanks,
Jake

>
> -------------------------------------------------------------------
> -----------
> The Command Line: Reinvented for Modern Developers
> Did the resurgence of CLI tooling catch you by surprise?
> Reconnect with the command line and become more productive. 
> Learn the new .NET and ASP.NET CLI. Get your free copy!
> http://sdm.link/telerik
> _______________________________________________
> Linuxptp-users mailing list
> Linuxptp-***@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linuxptp-users