[RFC 1/2] time: workaround crappy hpet

From: Shaohua Li <shli@fb.com>
To: lkml <linux-kernel@vger.kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>,
	John Stultz <john.stultz@linaro.org>, <calvinowens@fb.com>
Subject: [RFC 1/2] time: workaround crappy hpet
Date: Mon, 11 Apr 2016 17:57:56 -0700	[thread overview]
Message-ID: <09c4f19409012995595db6fd0a12f326c292af1a.1460422356.git.shli@fb.com> (raw)

Calvin found 'perf record -a --call-graph dwarf -- sleep 5' making clocksource
switching to hpet. We found similar symptom in another machine. Here is an example:

[8224517.520885] timekeeping watchdog: Marking clocksource 'tsc' as unstable, because the skew is too large:
[8224517.540032]        'hpet' wd_now: ffffffff wd_last: b39c0bd mask: ffffffff
[8224517.553092]        'tsc' cs_now: 48ceac7013714e cs_last: 48ceac25be34ac mask: ffffffffffffffff
[8224517.569849] Switched to clocksource hpet

In both machines, wd_now is 0xffffffff. The tsc time looks correct, the cpu is 2.5G
(0x48ceac7013714e - 0x48ceac25be34ac)/2500000 = 0.4988s
0.4988s matches WATCHDOG_INTERVAL. Since hpet reads to 0xffffffff in both
machines, this sounds not coincidence, hept is crappy.

This patch tries to workaround this issue. We do retry if hpet has 0xffffff value.
In the relevant machine, the hpet counter doesn't read to 0xffffffff later.
The chance hpet has 0xffffffff counter is very small, this patch should have no
impact for good hpet.

I'm open if there is better solution.

Reported-by: Calvin Owens<calvinowens@fb.com>
Signed-off-by: Shaohua Li <shli@fb.com>
---
 arch/x86/kernel/hpet.c | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/hpet.c b/arch/x86/kernel/hpet.c
index a1f0e4a..333b57c 100644
--- a/arch/x86/kernel/hpet.c
+++ b/arch/x86/kernel/hpet.c
@@ -763,7 +763,23 @@ static int hpet_cpuhp_notify(struct notifier_block *n,
  */
 static cycle_t read_hpet(struct clocksource *cs)
 {
-	return (cycle_t)hpet_readl(HPET_COUNTER);
+	unsigned int ret;
+	static bool checked;
+	ret = hpet_readl(HPET_COUNTER);
+
+	if (unlikely(ret == 0xffffffff && !checked)) {
+		int i;
+		for (i = 0; i < 20; i++) {
+			ret = hpet_readl(HPET_COUNTER);
+			if (ret != 0xffffffff)
+				break;
+		}
+		if (i == 20) {
+			WARN_ONCE(true, "HPET counter value is abnormal\n");
+			checked = true;
+		}
+	}
+	return (cycle_t)ret;
 }
 
 static struct clocksource clocksource_hpet = {
-- 
2.8.0.rc2
