RE: pulse-per-second API status

["Terry Moore" <tmm%mcci.com@localhost>]

Hi all,

I do USB for a living. 

USB is not a very good way to do PPS synchronization if you're looking for
usec resolution -- there are huge and unpredictable delays that can be
introduced from a number of sources.

Some of the sources are inherent and some are due to system issues.

Sources of variation:
*) the USB com port may be using an interrupt pipe to report changes in the
handshaking lines. Depending on the vendor, there may be variation in the
requested frequency of interrupt polling.  On high-speed and superspeed
devices, USB polling may occur as quickly as every 125us; on full and
low-speed devices, the polling interval drops to 1ms max. However, when you
get polled within a given 125us or 1ms window will vary based on other
devices on the bus. You cannot control this, in general, unless you dedicate
a host controller and only plug one device into the bus.  Furthermore, even
on an unloaded bus, your sampling (polling) frequency will be slaved to the
clock of the USB host controller, with additional jitter introduced by the
clock of the USB device controller.

*) the USB com port may be using a bulk pipe to report changes in the
handshaking lines.  On a bus with multiple devices, the polling for any
particular endpoint will have gaps that are not predictable. Even on an
unloaded bus, there are gaps close to end of each frame or microframe during
which polls will not occur. 

*) after the USB I/O operation completes at the hardware, there is
unpredictable latency introduced by the USB subsystem. This latency often
applies even if you have a dedicated USB host controller (because you have a
limited number of CPUs, and even with concurrent processing, activity on
other USB busses will often slow down completion processing).

Audio devices get around this by disciplining the clock at a higher level.
It is easy to imagine a USB device that measures long term clock rate on the
bus (using the SOF mechanisms or simply by pushing data over an isochronous
pipe) and then feeds back clock difference information.  (The device would
schedule data to go to the host once per disciplined millisecond; this would
either cause gaps in the isoch traffic, or overruns, which would be
reported.) The driver on the host would use this to monitor clock drift and
feed the information into the kernel clock. 

But using a serial port handshaking line over an emulated com port over USB
is not likely to be terribly wonderful. Long-term accuracy (tick count)
probably no problem, but jitter.... it will depend on how that's filtered. I
suspect that the kernel's clock discipline loop is not likely to be happy if
there's jitter on this particular reference.

Best regards,
--Terry

> -----Original Message-----
> From: tech-kern-owner%NetBSD.org@localhost 
> [mailto:tech-kern-owner%NetBSD.org@localhost] On
> Behalf Of Mouse
> Sent: Friday, November 01, 2013 14:05
> To: tech-kern%NetBSD.org@localhost
> Subject: Re: pulse-per-second API status
> 
> >>> * ucom(4) needs pps support
> >> It does?  I thought USB introduced delays which are both uncertain
> >> and, in this context, large, [...]
> > The delays introduced by USB are said to be on the order of 1 ms.
> 
> I find "said to be" somewhat less than reassuring.  It's been a while
> since I read the relevant doc, but I think this depends heavily on how
> often the host interface is configured to poll the device.  The fuzzy
> memory also reports that there are different intervals configurable; I
> don't remember what they were, but I also don't know what NetBSD does
> for that, so that doesn't mean much.
> 
> Also, see below - 1ms strikes me as pretty bad for PPS.
> 
> [in another message]
> 
> > This patch to netbsd-5 adds pps support to ucom(4), [...]
> 
> > I have tested it with a GR301-W and the in-tree (-5) ntpd.  I am
> > seeing offsets of about -100ms [...]
> 
> 100ms is a lot worse than 1ms, and surely bad enough that stamping the
> implicit approval of "it's in the tree" on this is a bad idea.  Or do
> you really think your network is bad enough that 99ms of that is coming
> from network asymmetry?
> 
> PPS with a lot of wobble in it may be better than a crappy network
> connection.  But if NetBSD enables PPS on ucom, there's going to be an
> expectation that it is good enough for stratum-1 timekeeping, like PPS
> on real serial ports.  You, Greg, know better.  J-randoms who see what
> looks to them like PPS support on ucom won't.
> 
> [back to the first message]
> 
> > So it depends on whether 1 ms of fuzz is "doesn't work".
> 
> Not quite: it also depends on how close (your impression of) what "[is]
> said to be" is to reality.
> 
> > I would say that if the next best option is worse than that (nmea
> > timecode over the same USB, or asymmetic delays over the Internet),
> > then it does work, even if a real serial port would be better.
> 
> It does work in the sense that it's the best chime available in that
> dismal a situation.
> 
> But it still may not work in the sense of living up to the expectations
> people have come to have for PPS on serial ports.
> 
> My worry is not that it's not the best time available in some
> circumstances.  My worry is that putting it into the tree will lead to
> its getting used as if it were as good as PPS on anything else, leading
> both to timeservers that claim stratum 1 but give bad chime and to
> people blaming NetBSD for its crappy PPS support when the real problem
> is that they don't understand the USB issues and it _looks_ like any
> other PPS support until you test the resulting time carefully.
> 
> /~\ The ASCII                           Mouse
> \ / Ribbon Campaign
>  X  Against HTML              mouse%rodents-montreal.org@localhost
> / \ Email!         7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B