Re: [PATCH for-next V2 0/9] Add completion timestamping support

[Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]

[-- Attachment #1: Type: text/plain, Size: 3288 bytes --]

On Sat, 2015-06-06 at 03:16 -0500, Christoph Lameter wrote:
> On Wed, 3 Jun 2015, Jason Gunthorpe wrote:
> 
> > On Wed, Jun 03, 2015 at 07:55:58PM -0500, Christoph Lameter wrote:
> >
> > > I thknk the raw cycles and the rought oscillator speed are fine.
> >
> > Time keeping is designed to adjust for 100's of ppm drift between
> > clocks.
> 
> What time keeping? Ntp? pptp is supposed to be accurate to 10s of ns and
> we would need an accuracy in that range.
> 
> > A communications clock source will be spec'd to be below 200ppm in
> > accuracy. IB clocks are below 100 ppm, and PCI-E is 300ppm (approx, I
> > didn't check, order of magnitue is close)
> 
> Well that is not usable. ns are a billionth of a second which is the unit
> of measurement of these activities here. A send action can be around 600-1000ns.
> If we are off by 200ppm then that is 200 microseconds meaning 200000 ns.
> And its our experience that these clocks can be off by milliseconds in
> practice.

The ppm rating is based upon the speed of the clock, not time.  It's how
many cycles of variance you are allowed from the target speed given in
cycles / millions of cycles of the target clock frequency.  If you have
a 312.5MHz clock, and your accuracy is specified as 100ppm, then the
total clock variability is 312.5 * 100 = 31250 cycles (I suspect that
this is an absolute variance, and so the tolerance would be +-1/2 of the
total amount, but I don't know that for certain).

> > That translates into 0.0625 Hz. for a 312.5 MHz ethernet reference clock

I don't know how this number is derived, but 0.0625Hz sounds like an odd
variance.

> Ok that is around 3ns per cycle? And you think the accuracy is therefore
> in femtoseconds? I have never seen something that accurate. Wish something
> like that would exist. Maybe in some labs that provide the source of
> global timekeeping?
> 
> > Compared to 5,000,000 Hz in error from rounding.
> 
> Huh?

He's pointing out that the design as specified passes the clock
frequency to the user space library in terms of integer MHz.  The
standard Ethernet clock frequency is 312.5MHz.  That .5MHz, or
500,000Hz, must be rounded off as it is passed from the kernel to the
user space library.  And that 500,000 cycle per second error in the
stated speed of the clock is *way* larger than the +- error variance in
the clocks you are using.  If you are having problems keeping your time
numbers synchronized, then this is likely a bigger problem than the
variance of the clocks.

> > So no, I disagree that rough is fine for anything.
> 
> I am sorry but the practical issues that we are dealing with in
> timekeeping today shows just the opposite. For a true comparison of clocks
> with nanosecond accuracy you would need time corrected values and that is
> a challenge due to the variances of the clocks that we see.

Jason's point, and one that isn't addressed yet, is that this might not
be variance in the clocks and instead might be a design flaw in the API
you are using and the way the clock speeds are passed to user space.
Changing from int MHz to int KHz might solve your problem.


-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
              GPG KeyID: 0E572FDD


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]