Calibrating TSC with HPET timer

Calibrating TSC with HPET timer

[Steve Rosenbluth]

On the last community call I promised to report back the results of my
RTDM module TSC calibration vs the kernel's TSC calibration.

The original problem was that of varying TSC frequency on boot of x86
i5/i7/Atom processors. This is not runtime TSC drift, this is the TSC
starting up at a significantly different frequency at boot time and
then staying at that frequency. The TSC rate variation manifests
itself as over 1.0usec discrepancy in the period of a Xenomai periodic
task - such that a Xenomai RT periodic task doesn't run at the same
rate across cold boots.

Recently Hongzhan created an ipipe patch to re-read the kernels
ongoing calibration, which is available as of Xenomai 3 kernels
4.19.246 and above.
https://source.denx.de/Xenomai/ipipe-x86/-/commit/24c2fbb40b03aef678fe9cc062badc2e8cfd985c
The patch adds to tsc.c : __ipipe_report_clockfreq_update()

My manual calibration consisted of busy-looping in a RTDM module for 5
seconds, and comparing TSC clock ticks to HPET clock ticks. Applying
the resultant calibration factor I could get my RT-userspace periodic
task to run at a consistent rate across cold boots of the PC, with a
period deviation of only about +/-2nsec as measured on a calibrated
external frequency counter.

So the answer to the big question is that Hongzhan's patch for using
the kernel calibration achieves exactly the same thing, the RT task
period variation is equally as accurate (+/-2nsec). As a matter of
fact it appears the kernel is using the HPET for its calibration too,
because I see on the frequency counter the same constant error in the
kernel's calibration versus my RTDM calibration (HPET is always off by
about 40nsec, but with a static scaling factor the RT task period is
made perfect). I tested with Xeno 3.1.2 and the latest ipipe patch
ipipe-core-4.19.264-cip84-x86-24.patch.

An example of printed output from dmesg is as follows:
[ 7235.365194] * Xeno TSC monotonic clock delta was 4999510139 ns
[ 7235.365196] * hpet actual clock delta was 5000000139 ns
 * calib adjustment factor == 0.999902000

The way I did the calibration in an RTDM module on Xenomai 3 is in the
code below. We are using this approach now in our legacy Xenomai 2.6.5
systems and it works well. Note that you can't do floating point in an
RTDM module on Xeno 2.6 but you can pass the TSC/HPET clock ticks via
shared memory from the RTDM module to your RT periodic task where you
can calculate the calibration scaling factor.

hpet_freq_p = (int *) kallsyms_lookup_name("hpet_freq");  // HPET frequency
if (hpet_freq_p)            // if kernel has exported the needed symbol
{
   rtdm_printk("* hpet clock freq, according to kernel, is: %d\n",
*hpet_freq_p);
   hpet_period = 1.0/(double)*hpet_freq_p;        // hpet_freq has
been determined by the kernel
   calib_period_ns =  5000000000;                  // calibrate for
this many nsec
   read_hpet_p = (void *) kallsyms_lookup_name("read_hpet");    // get
access to kernel's read function
   if (read_hpet_p > 0)
   {
      rtdm_printk("* Starting TSC calibration, calib duration will be
%lld sec.\n",calib_period_ns);
      tod1 = (*read_hpet_p)(NULL);      // get (hpet) wall clock
      xeno_start_time_ns = rtdm_clock_read_monotonic();  // Xeno
monotonic clock, based on TSE
      do
      {
          tod2 = (*read_hpet_p)(NULL);      // Xeno reads wall clock
      }
      while ( (tod2*hpet_period*1000000000.0) <
((tod1*hpet_period*1000000000.0) + calib_period_ns) );
      xeno_stop_time_ns = rtdm_clock_read_monotonic();
      delta_monotonic_ns = xeno_stop_time_ns - xeno_start_time_ns;
      delta_hpet_ns = (long long)((tod2-tod1)*hpet_period*1000000000.0);
      shmem_p->calib_clock_adjust =
(double)delta_monotonic_ns/(double)delta_hpet_ns;  // will use this to
adjust period
      rtdm_printk("* Xeno TSC monotonic clock delta was %lld ns\n",
delta_monotonic_ns );
      rtdm_printk("* hpet actual clock delta was %lld ns\n", delta_hpet_ns);
   }
   else
   {
      rtdm_printk("* Problem getting read_hpet() handle!\n");
      shmem_p->calib_clock_adjust = 1.0;         // don't use this
clock for calibration
   }
}
else
   rtdm_printk("* Problem getting 'hpet_freq' handle! Will skip
calibration.\n");

-Steve Rosenbluth