[PATCH 0/4] avoid double timer interrupt with nohz and Intel TSC

[PATCH 0/4] avoid double timer interrupt with nohz and Intel TSC

[Nicolai Stange]

With a single task running on a NOHZ CPU on an Intel Haswell, I recognized
that I did not only get the one expected local_timer APIC interrupt, but
two per second at minimum.

Further tracing showed that the first one preceedes the programmed deadline
by up to ~50us and hence, it did nothing except for reprogramming the TSC
deadline clockevent device to trigger shortly thereafter again.

FYI, the trace looks like this:

  <...>-2938  [007] d.h.   420.753164: local_timer_entry: vector=239
  <...>-2938  [007] d.h.   420.753164: __hrtimer_run_queues <-hrtimer_interrupt
  <...>-2938  [007] d.h.   420.753184: local_timer_entry: vector=239
  <...>-2938  [007] d.h.   420.753184: __hrtimer_run_queues <-hrtimer_interrupt
  <...>-2938  [007] d.h.   420.753195: tick_sched_timer <-__hrtimer_run_queues
  <...>-2938  [007] d.h.   421.752170: local_timer_entry: vector=239
  <...>-2938  [007] d.h.   421.752171: __hrtimer_run_queues <-hrtimer_interrupt
  <...>-2938  [007] d.h.   421.752202: local_timer_entry: vector=239
  <...>-2938  [007] d.h.   421.752202: __hrtimer_run_queues <-hrtimer_interrupt
  <...>-2938  [007] d.h.   421.752202: tick_sched_timer <-__hrtimer_run_queues

It turns out that this too early programmed TSC deadline is caused by
inaccuracies in some frequency calculations which become significant if
the timer periods become large as it is the case for nohz with one task
(delta = 10^9ns).

The first three patches address inaccuracies entering the TSC deadline
clockevent devices' frequency.

The fourth patch is the most important one as it addresses the error
of largest relative magnitude. It is caused by the assumption in the
clockevents core that the ratio of the monotonic clock's frequency to that
of the clockevent device's is a constant. Since the monotonic clock's
frequency gets dynamically adjusted in order to compensate for NTP errors,
this is not true.

With this patchset applied, the trace looks like this:

  <...>-23609 [007] d.h.  1811.586658: local_timer_entry: vector=239
  <...>-23609 [007] d.h.  1811.586680: __hrtimer_run_queues <-hrtimer_interrupt
  <...>-23609 [007] d.h.  1811.586680: tick_sched_timer <-__hrtimer_run_queues
  <...>-23609 [007] d.h.  1812.585659: local_timer_entry: vector=239
  <...>-23609 [007] d.h.  1812.585666: __hrtimer_run_queues <-hrtimer_interrupt
  <...>-23609 [007] d.h.  1812.585666: tick_sched_timer <-__hrtimer_run_queues
  <...>-23609 [007] d.h.  1813.584661: local_timer_entry: vector=239
  <...>-23609 [007] d.h.  1813.584668: __hrtimer_run_queues <-hrtimer_interrupt
  <...>-23609 [007] d.h.  1813.584668: tick_sched_timer <-__hrtimer_run_queues

Please note that the first three TSC-patches might not be necessary to
get this result. In fact, [3/4] ("arch, x86, tsc: inform TSC deadline
clockevent device about") is somewhat counterproductive in the sense that
on my system, it usually corrects the TSC deadline device's frequency
towards lower values and thus, facilitates the too-early interrupt
behaviour initially described. However, I decided to send them along with
the fourth patch because
 - I tested the fourth patch in this setting
 - I believe that a greater accurracy of the TSC deadline device is
   worthwhile on its own

Applicable to linux-next-20160708. The individual patches don't depend on
each other.

Nicolai Stange (4):
  arch, x86, tsc deadline clockevent dev: reduce frequency roundoff
    error
  arch, x86, tsc deadline clockevent dev: reduce TSC_DIVISOR to 2
  arch, x86, tsc: inform TSC deadline clockevent device about
    recalibration
  kernel/time/clockevents: compensate for monotonic clock's dynamic
    frequency

 arch/x86/include/asm/apic.h |  1 +
 arch/x86/kernel/apic/apic.c | 29 ++++++++++++++++++++++++--
 arch/x86/kernel/tsc.c       |  4 ++++
 kernel/time/clockevents.c   |  1 +
 kernel/time/timekeeping.c   | 50 +++++++++++++++++++++++++++++++++++++++++++++
 kernel/time/timekeeping.h   |  1 +
 6 files changed, 84 insertions(+), 2 deletions(-)

-- 
2.9.0

[PATCH 1/4] arch, x86, tsc deadline clockevent dev: reduce frequency roundoff error

[Nicolai Stange]

In setup_APIC_timer(), the registered clockevent device's frequency
is calculated by first dividing tsc_khz by TSC_DIVISOR and multiplying
it with 1000 afterwards.

The multiplication with 1000 is done for converting from kHz to Hz and the
division by TSC_DIVISOR is carried out in order to make sure that the final
result fits into an u32.

However, with the order given in this calculation, the roundoff error
introduced by the division gets magnified by a factor of 1000 by the
following multiplication.

Increase the accuracy by reversing the order of the division and
multiplication. In order not to overflow during this calculation, cast
temporarily to u64.

Signed-off-by: Nicolai Stange <nicstange@gmail.com>
---
 arch/x86/kernel/apic/apic.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index 89a5bce..dce654c 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -563,7 +563,8 @@ static void setup_APIC_timer(void)
 				    CLOCK_EVT_FEAT_DUMMY);
 		levt->set_next_event = lapic_next_deadline;
 		clockevents_config_and_register(levt,
-						(tsc_khz / TSC_DIVISOR) * 1000,
+						(u32)(((u64)tsc_khz * 1000) /
+							TSC_DIVISOR),
 						0xF, ~0UL);
 	} else
 		clockevents_register_device(levt);
-- 
2.9.0

[PATCH 2/4] arch, x86, tsc deadline clockevent dev: reduce TSC_DIVISOR to 2

[Nicolai Stange]

In order to avoid overflowing an u32, the TSC deadline clockevent device's
frequency is divided by TSC_DIVISOR at registration.

The TSC_DIVISOR is currently defined as equaling 32 which allows for a
TSC frequency as high as 2^32 / 10^9ns * 32 = 137 GHz.

OTOH, larger values of TSC_DIVISOR introduce bigger roundoff errors into
the device's frequency.

A value of 2 for TSC_DIVISOR allows for a TSC frequency of
2^32 / 10^9ns * 2 = 8.5 GHz which is still way larger than anything to
expect in the next years.

Reduce the TSC deadline clockevent device's TSC_DIVISOR from 32 down to 2.

Signed-off-by: Nicolai Stange <nicstange@gmail.com>
---
 arch/x86/kernel/apic/apic.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index dce654c..1d22c72 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -311,7 +311,7 @@ int lapic_get_maxlvt(void)
 
 /* Clock divisor */
 #define APIC_DIVISOR 16
-#define TSC_DIVISOR  32
+#define TSC_DIVISOR  2
 
 /*
  * This function sets up the local APIC timer, with a timeout of
-- 
2.9.0

[PATCH 3/4] arch, x86, tsc: inform TSC deadline clockevent device about recalibration

[Nicolai Stange]

The TSC deadline clockevent devices' configuration and registration
happens before the TSC frequency calibration is refined in
tsc_refine_calibration_work().

This results in the TSC clocksource and the TSC deadline clockevent
devices being configured with slightly different frequencies: the former
gets the refined one and the latter are configured with the inaccurate
frequency detected earlier by means of the
"Fast TSC calibration using PIT".

Within the APIC code, introduce the notifier function
lapic_update_tsc_freq() which reconfigures all per-CPU TSC deadline
clockevent devices with the current tsc_khz.

Call it from the TSC code after TSC calibration refinement has happened.

Signed-off-by: Nicolai Stange <nicstange@gmail.com>
---
 arch/x86/include/asm/apic.h |  1 +
 arch/x86/kernel/apic/apic.c | 24 ++++++++++++++++++++++++
 arch/x86/kernel/tsc.c       |  4 ++++
 3 files changed, 29 insertions(+)

diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index bc27611..971f446 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -135,6 +135,7 @@ extern void init_apic_mappings(void);
 void register_lapic_address(unsigned long address);
 extern void setup_boot_APIC_clock(void);
 extern void setup_secondary_APIC_clock(void);
+extern void lapic_update_tsc_freq(void);
 extern int APIC_init_uniprocessor(void);
 
 #ifdef CONFIG_X86_64
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index 1d22c72..85238a8 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -571,6 +571,30 @@ static void setup_APIC_timer(void)
 }
 
 /*
+ * Install the updated TSC frequency from recalibration at the TSC
+ * deadline clockevent devices.
+ */
+static void __lapic_update_tsc_freq(void *info)
+{
+	struct clock_event_device *levt = this_cpu_ptr(&lapic_events);
+
+	if (!this_cpu_has(X86_FEATURE_TSC_DEADLINE_TIMER))
+		return;
+
+	clockevents_config(levt, (u32)(((u64)tsc_khz * 1000) / TSC_DIVISOR));
+}
+
+void lapic_update_tsc_freq(void)
+{
+	/*
+	 * The clockevent device's ->mult and ->shift can both be
+	 * changed. In order to avoid races, schedule the frequency
+	 * update code on each CPU.
+	 */
+	on_each_cpu(__lapic_update_tsc_freq, NULL, 0);
+}
+
+/*
  * In this functions we calibrate APIC bus clocks to the external timer.
  *
  * We want to do the calibration only once since we want to have local timer
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index 38ba6de..6ae5126 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -22,6 +22,7 @@
 #include <asm/nmi.h>
 #include <asm/x86_init.h>
 #include <asm/geode.h>
+#include <asm/apic.h>
 
 unsigned int __read_mostly cpu_khz;	/* TSC clocks / usec, not used here */
 EXPORT_SYMBOL(cpu_khz);
@@ -1193,6 +1194,9 @@ static void tsc_refine_calibration_work(struct work_struct *work)
 		(unsigned long)tsc_khz / 1000,
 		(unsigned long)tsc_khz % 1000);
 
+	/* Inform the TSC deadline clockevent devices about the recalibration */
+	lapic_update_tsc_freq();
+
 out:
 	if (boot_cpu_has(X86_FEATURE_ART))
 		art_related_clocksource = &clocksource_tsc;
-- 
2.9.0

[PATCH 4/4] kernel/time/clockevents: compensate for monotonic clock's dynamic frequency

[Nicolai Stange]

With NOHZ_FULL and one single well-isolated, CPU consumptive task, one
would expect approximately one clockevent interrupt per second. However, on
my Intel Haswell where the monotonic clock is the TSC monotonic clock and
the clockevent device is the TSC deadline device, it turns out that every
second, there are two such interrupts: the first one arrives always
approximately ~50us before the scheduled deadline as programmed by
tick_nohz_stop_sched_tick() through the hrtimer API. The
__hrtimer_run_queues() called in this interrupt detects that the queued
tick_sched_timer hasn't expired yet and simply does nothing except
reprogramming the clock event device to fire shortly after again.

These too early programmed deadlines are explained as follows:
clockevents_program_event() programs the clockevent device to fire
after
  f_event * delta_t_progr
clockevent device cycles where f_event is the clockevent device's hardware
frequency and delta_t_progr is the requested time interval. After that many
clockevent device cycles have elapsed, the device underlying the monotonic
clock, that is the monotonic raw clock has seen f_raw / f_event as many
cycles.
The ktime_get() called from __hrtimer_run_queues() interprets those
cycles to run at the frequency of the monotonic clock. Summarizing:
  delta_t_perc = 1/f_mono * f_raw/f_event * f_event * delta_t_progr
               = f_raw / f_mono * delta_t_progr
with f_mono being the monotonic clock's frequency and delta_t_perc being
the elapsed time interval as perceived by __hrtimer_run_queues().

Now, f_mono is not a constant, but is dynamically adjusted in
timekeeping_adjust() in order to compensate for the NTP error. With the
large values of delta_t_progr of 10^9ns with NOHZ_FULL, the error made
becomes significant and results in the double timer interrupts described
above.

Compensate for this error by multiplying delta_t_progr with f_mono / f_raw
in clockevents_program_event() before actually programming the clockevent
device.

Namely, introduce a helper, timekeeping_mono_interval_to_raw(), which
converts a given time interval from the monotonic clock's perception to
that of the raw monotonic clock by multiplying the value by f_mono / f_raw.
Call that helper from clockevents_program_event() in order to obtain a
suitable time interval to program the clockevent device with.

Signed-off-by: Nicolai Stange <nicstange@gmail.com>
---
 kernel/time/clockevents.c |  1 +
 kernel/time/timekeeping.c | 50 +++++++++++++++++++++++++++++++++++++++++++++++
 kernel/time/timekeeping.h |  1 +
 3 files changed, 52 insertions(+)

diff --git a/kernel/time/clockevents.c b/kernel/time/clockevents.c
index a9b76a4..4bccf04 100644
--- a/kernel/time/clockevents.c
+++ b/kernel/time/clockevents.c
@@ -329,6 +329,7 @@ int clockevents_program_event(struct clock_event_device *dev, ktime_t expires,
 		return dev->set_next_ktime(expires, dev);
 
 	delta = ktime_to_ns(ktime_sub(expires, ktime_get()));
+	delta = timekeeping_mono_interval_to_raw(delta);
 	if (delta <= 0)
 		return force ? clockevents_program_min_delta(dev) : -ETIME;
 
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index dcd5ce6..b1e7a41 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -2133,6 +2133,56 @@ out:
 }
 
 /**
+ * timekeeper_mono_interval_to_raw - Convert mono interval to raw's perception.
+ * @interval: Time interval as measured by the mono clock.
+ *
+ * Converts the given time interval as measured by the monotonic clock to
+ * what would have been measured by the raw monotonic clock in the meanwhile.
+ * The monotonic clock's frequency gets dynamically adjusted every now and then
+ * in order to compensate for the differences to NTP. OTOH, the clockevents
+ * devices are not affected by this adjustment, i.e. they keep ticking at some
+ * fixed hardware frequency which may be assumed to have a constant ratio to
+ * the fixed raw monotonic clock's frequency. This function provides a means
+ * to convert time intervals from the dynamic frequency monotonic clock to
+ * the fixed frequency hardware world.
+ *
+ * The returned value is clamped to [0, KTIME_MAX]. In particular,
+ * KTIME_MAX is returned on overflow and zero for negative values of interval.
+ */
+s64 timekeeping_mono_interval_to_raw(s64 interval)
+{
+	struct timekeeper *tk = &tk_core.timekeeper;
+	u32 raw_mult = tk->tkr_raw.mult, mono_mult = tk->tkr_mono.mult;
+	s64 raw;
+
+	/* The overflow checks below can't deal with negative intervals. */
+	if (interval <= 0)
+		return interval;
+
+	/*
+	 * Calculate
+	 *   raw = f_mono / f_raw * interval
+	 *       = (raw_mult / 2^raw_shift) / (mono_mult / 2^mono_shift)
+	 *            * interval
+	 * where f_mono and f_raw denote the frequencies of the monotonic
+	 * and raw clock respectively.
+	 *
+	 * Note that the monotonic and raw clock shifts' are equal and fixed,
+	 * that is they cancel.
+	 */
+	if (KTIME_MAX / raw_mult < interval)
+		return KTIME_MAX;
+	raw = interval * raw_mult;
+
+	if (KTIME_MAX - mono_mult / 2 < raw)
+		return KTIME_MAX;
+	raw += mono_mult / 2; /* round properly */
+	raw /= mono_mult;
+
+	return raw;
+}
+
+/**
  * getboottime64 - Return the real time of system boot.
  * @ts:		pointer to the timespec64 to be set
  *
diff --git a/kernel/time/timekeeping.h b/kernel/time/timekeeping.h
index 704f595..40a0fa9 100644
--- a/kernel/time/timekeeping.h
+++ b/kernel/time/timekeeping.h
@@ -18,6 +18,7 @@ extern void timekeeping_resume(void);
 
 extern void do_timer(unsigned long ticks);
 extern void update_wall_time(void);
+extern s64 timekeeping_mono_interval_to_raw(s64 interval);
 
 extern seqlock_t jiffies_lock;
 
-- 
2.9.0

Re: [PATCH 3/4] arch, x86, tsc: inform TSC deadline clockevent device about recalibration

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 1477 bytes --]

Hi,

[auto build test ERROR on tip/timers/core]
[also build test ERROR on v4.7-rc6 next-20160708]
[cannot apply to tip/x86/core]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Nicolai-Stange/avoid-double-timer-interrupt-with-nohz-and-Intel-TSC/20160710-202914
config: i386-tinyconfig (attached as .config)
compiler: gcc-6 (Debian 6.1.1-1) 6.1.1 20160430
reproduce:
        # save the attached .config to linux build tree
        make ARCH=i386 

All errors (new ones prefixed by >>):

   arch/x86/kernel/tsc.c: In function 'tsc_refine_calibration_work':
>> arch/x86/kernel/tsc.c:1198:2: error: implicit declaration of function 'lapic_update_tsc_freq' [-Werror=implicit-function-declaration]
     lapic_update_tsc_freq();
     ^~~~~~~~~~~~~~~~~~~~~
   cc1: some warnings being treated as errors

vim +/lapic_update_tsc_freq +1198 arch/x86/kernel/tsc.c

  1192		tsc_khz = freq;
  1193		pr_info("Refined TSC clocksource calibration: %lu.%03lu MHz\n",
  1194			(unsigned long)tsc_khz / 1000,
  1195			(unsigned long)tsc_khz % 1000);
  1196	
  1197		/* Inform the TSC deadline clockevent devices about the recalibration */
> 1198		lapic_update_tsc_freq();
  1199	
  1200	out:
  1201		if (boot_cpu_has(X86_FEATURE_ART))

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 6350 bytes --]

Re: [PATCH 4/4] kernel/time/clockevents: compensate for monotonic clock's dynamic frequency

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 1006 bytes --]

Hi,

[auto build test ERROR on tip/timers/core]
[also build test ERROR on v4.7-rc6 next-20160708]
[cannot apply to tip/x86/core]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Nicolai-Stange/avoid-double-timer-interrupt-with-nohz-and-Intel-TSC/20160710-202914
config: i386-randconfig-s1-201628 (attached as .config)
compiler: gcc-6 (Debian 6.1.1-1) 6.1.1 20160430
reproduce:
        # save the attached .config to linux build tree
        make ARCH=i386 

All errors (new ones prefixed by >>):

   kernel/built-in.o: In function `timekeeping_mono_interval_to_raw':
>> (.text+0x66436): undefined reference to `__divdi3'
   kernel/built-in.o: In function `timekeeping_mono_interval_to_raw':
   (.text+0x6649c): undefined reference to `__divdi3'

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 22648 bytes --]

Re: [PATCH 4/4] kernel/time/clockevents: compensate for monotonic clock's dynamic frequency

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 1414 bytes --]

Hi,

[auto build test ERROR on tip/timers/core]
[also build test ERROR on v4.7-rc6 next-20160708]
[cannot apply to tip/x86/core]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Nicolai-Stange/avoid-double-timer-interrupt-with-nohz-and-Intel-TSC/20160710-202914
config: sh-sh7785lcr_32bit_defconfig (attached as .config)
compiler: sh4-linux-gnu-gcc (Debian 5.3.1-8) 5.3.1 20160205
reproduce:
        wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        make.cross ARCH=sh 

All errors (new ones prefixed by >>):

   kernel/built-in.o: In function `timekeeping_mono_interval_to_raw':
>> kernel/time/timekeeping.c:2164: undefined reference to `__divdi3'

vim +2164 kernel/time/timekeeping.c

  2158		 * that is they cancel.
  2159		 */
  2160		if (KTIME_MAX / raw_mult < interval)
  2161			return KTIME_MAX;
  2162		raw = interval * raw_mult;
  2163	
> 2164		if (KTIME_MAX - mono_mult / 2 < raw)
  2165			return KTIME_MAX;
  2166		raw += mono_mult / 2; /* round properly */
  2167		raw /= mono_mult;

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 16948 bytes --]

Re: [PATCH 4/4] kernel/time/clockevents: compensate for monotonic clock's dynamic frequency

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 12533 bytes --]

Hi,

[auto build test ERROR on tip/timers/core]
[also build test ERROR on v4.7-rc6 next-20160708]
[cannot apply to tip/x86/core]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Nicolai-Stange/avoid-double-timer-interrupt-with-nohz-and-Intel-TSC/20160710-202914
config: frv-defconfig (attached as .config)
compiler: frv-linux-gcc (GCC) 4.9.0
reproduce:
        wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        make.cross ARCH=frv 

All errors (new ones prefixed by >>):

   frv-linux-ld: Warning: size of symbol `read_persistent_clock' changed from 160 in arch/frv/kernel/built-in.o to 12 in kernel/built-in.o
   frv-linux-ld: Warning: size of symbol `sched_clock' changed from 56 in arch/frv/kernel/built-in.o to 40 in kernel/built-in.o
   frv-linux-ld: Warning: size of symbol `arch_cpu_idle' changed from 64 in arch/frv/kernel/built-in.o to 20 in kernel/built-in.o
   frv-linux-ld: Warning: size of symbol `arch_show_interrupts' changed from 84 in arch/frv/kernel/built-in.o to 8 in kernel/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_fadvise64_64' changed from 8 in kernel/built-in.o to 780 in mm/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_munlockall' changed from 8 in kernel/built-in.o to 84 in mm/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_swapoff' changed from 8 in kernel/built-in.o to 1268 in mm/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_process_vm_writev' changed from 8 in kernel/built-in.o to 36 in mm/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_memfd_create' changed from 8 in kernel/built-in.o to 548 in mm/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_mlock2' changed from 8 in kernel/built-in.o to 52 in mm/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_remap_file_pages' changed from 8 in kernel/built-in.o to 708 in mm/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_fadvise64' changed from 8 in kernel/built-in.o to 28 in mm/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_munlock' changed from 8 in kernel/built-in.o to 128 in mm/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_mincore' changed from 8 in kernel/built-in.o to 596 in mm/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_msync' changed from 8 in kernel/built-in.o to 588 in mm/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_swapon' changed from 8 in kernel/built-in.o to 3720 in mm/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_madvise' changed from 8 in kernel/built-in.o to 1880 in mm/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_process_vm_readv' changed from 8 in kernel/built-in.o to 32 in mm/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_mlockall' changed from 8 in kernel/built-in.o to 356 in mm/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_mprotect' changed from 8 in kernel/built-in.o to 624 in mm/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_mlock' changed from 8 in kernel/built-in.o to 16 in mm/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_mremap' changed from 8 in kernel/built-in.o to 1108 in mm/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_execveat' changed from 8 in kernel/built-in.o to 124 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_signalfd4' changed from 8 in kernel/built-in.o to 536 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_io_setup' changed from 8 in kernel/built-in.o to 1984 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_copy_file_range' changed from 8 in kernel/built-in.o to 736 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_io_cancel' changed from 8 in kernel/built-in.o to 352 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_eventfd' changed from 8 in kernel/built-in.o to 16 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_epoll_create' changed from 8 in kernel/built-in.o to 32 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_epoll_pwait' changed from 8 in kernel/built-in.o to 400 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_epoll_ctl' changed from 8 in kernel/built-in.o to 2616 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_open_by_handle_at' changed from 8 in kernel/built-in.o to 12 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_epoll_wait' changed from 8 in kernel/built-in.o to 1072 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_inotify_add_watch' changed from 8 in kernel/built-in.o to 836 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_sysfs' changed from 8 in kernel/built-in.o to 444 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_signalfd' changed from 8 in kernel/built-in.o to 16 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_eventfd2' changed from 8 in kernel/built-in.o to 164 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_epoll_create1' changed from 8 in kernel/built-in.o to 376 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_timerfd_settime' changed from 8 in kernel/built-in.o to 1252 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_inotify_init' changed from 8 in kernel/built-in.o to 16 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_inotify_init1' changed from 8 in kernel/built-in.o to 364 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_io_submit' changed from 8 in kernel/built-in.o to 16 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_bdflush' changed from 8 in kernel/built-in.o to 132 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_timerfd_create' changed from 8 in kernel/built-in.o to 468 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_name_to_handle_at' changed from 8 in kernel/built-in.o to 648 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_flock' changed from 8 in kernel/built-in.o to 436 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_timerfd_gettime' changed from 8 in kernel/built-in.o to 512 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_inotify_rm_watch' changed from 8 in kernel/built-in.o to 208 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_io_getevents' changed from 8 in kernel/built-in.o to 744 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_io_destroy' changed from 8 in kernel/built-in.o to 236 in fs/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_msgget' changed from 8 in kernel/built-in.o to 64 in ipc/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_mq_getsetattr' changed from 8 in kernel/built-in.o to 576 in ipc/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_mq_unlink' changed from 8 in kernel/built-in.o to 308 in ipc/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_shmdt' changed from 8 in kernel/built-in.o to 456 in ipc/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_msgrcv' changed from 8 in kernel/built-in.o to 20 in ipc/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_mq_open' changed from 8 in kernel/built-in.o to 792 in ipc/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_semget' changed from 8 in kernel/built-in.o to 116 in ipc/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_mq_timedsend' changed from 8 in kernel/built-in.o to 712 in ipc/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_semctl' changed from 8 in kernel/built-in.o to 608 in ipc/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_mq_notify' changed from 8 in kernel/built-in.o to 1020 in ipc/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_semtimedop' changed from 8 in kernel/built-in.o to 2400 in ipc/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_shmctl' changed from 8 in kernel/built-in.o to 584 in ipc/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_mq_timedreceive' changed from 8 in kernel/built-in.o to 1124 in ipc/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_msgsnd' changed from 8 in kernel/built-in.o to 92 in ipc/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_semop' changed from 8 in kernel/built-in.o to 16 in ipc/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_shmat' changed from 8 in kernel/built-in.o to 52 in ipc/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_msgctl' changed from 8 in kernel/built-in.o to 192 in ipc/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_shmget' changed from 8 in kernel/built-in.o to 64 in ipc/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_ipc' changed from 8 in kernel/built-in.o to 624 in ipc/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_ioprio_set' changed from 8 in kernel/built-in.o to 636 in block/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_ioprio_get' changed from 8 in kernel/built-in.o to 616 in block/built-in.o
   frv-linux-ld: Warning: size of symbol `pcibios_enable_device' changed from 100 in arch/frv/mb93090-mb00/built-in.o to 12 in drivers/built-in.o
   frv-linux-ld: Warning: size of symbol `pcibios_setup' changed from 76 in arch/frv/mb93090-mb00/built-in.o to 4 in drivers/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_sendmmsg' changed from 8 in kernel/built-in.o to 12 in net/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_sendto' changed from 8 in kernel/built-in.o to 256 in net/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_recv' changed from 8 in kernel/built-in.o to 20 in net/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_send' changed from 8 in kernel/built-in.o to 20 in net/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_connect' changed from 8 in kernel/built-in.o to 184 in net/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_getpeername' changed from 8 in kernel/built-in.o to 188 in net/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_shutdown' changed from 8 in kernel/built-in.o to 128 in net/built-in.o
   frv-linux-ld: Warning: size of symbol `bpf_helper_changes_skb_data' changed from 8 in kernel/built-in.o to 104 in net/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_bind' changed from 8 in kernel/built-in.o to 176 in net/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_getsockname' changed from 8 in kernel/built-in.o to 188 in net/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_socket' changed from 8 in kernel/built-in.o to 228 in net/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_recvmsg' changed from 8 in kernel/built-in.o to 12 in net/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_setsockopt' changed from 8 in kernel/built-in.o to 188 in net/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_recvfrom' changed from 8 in kernel/built-in.o to 296 in net/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_listen' changed from 8 in kernel/built-in.o to 152 in net/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_accept' changed from 8 in kernel/built-in.o to 16 in net/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_recvmmsg' changed from 8 in kernel/built-in.o to 316 in net/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_socketpair' changed from 8 in kernel/built-in.o to 604 in net/built-in.o
   frv-linux-ld: Warning: size of symbol `skb_copy_bits' changed from 8 in kernel/built-in.o to 560 in net/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_sendmsg' changed from 8 in kernel/built-in.o to 12 in net/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_accept4' changed from 8 in kernel/built-in.o to 456 in net/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_socketcall' changed from 8 in kernel/built-in.o to 508 in net/built-in.o
   frv-linux-ld: Warning: size of symbol `sys_getsockopt' changed from 8 in kernel/built-in.o to 168 in net/built-in.o
   kernel/built-in.o: In function `timekeeping_mono_interval_to_raw':
>> (.text+0x47d98): undefined reference to `__divll'
   kernel/built-in.o: In function `timekeeping_mono_interval_to_raw':
   (.text+0x47e24): undefined reference to `__divll'

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 8693 bytes --]

Re: [PATCH 0/4] avoid double timer interrupt with nohz and Intel TSC

[Nicolai Stange]

Please drop in favour of v2.

Thanks,

Nicolai