From: "Paul E. McKenney" <paulmck@kernel.org>
To: Thomas Gleixner <tglx@linutronix.de>
Cc: Feng Tang <feng.tang@intel.com>,
Xing Zhengjun <zhengjun.xing@linux.intel.com>,
John Stultz <john.stultz@linaro.org>,
Stephen Boyd <sboyd@kernel.org>, Jonathan Corbet <corbet@lwn.net>,
Mark Rutland <Mark.Rutland@arm.com>,
Marc Zyngier <maz@kernel.org>, Andi Kleen <ak@linux.intel.com>,
Chris Mason <clm@fb.com>, LKML <linux-kernel@vger.kernel.org>,
lkp@lists.01.org, lkp@intel.com
Subject: Re: [LKP] Re: [clocksource] 6c52b5f3cf: stress-ng.opcode.ops_per_sec -14.4% regression
Date: Fri, 23 Apr 2021 14:14:44 -0700 [thread overview]
Message-ID: <20210423211444.GO975577@paulmck-ThinkPad-P17-Gen-1> (raw)
In-Reply-To: <87tunwvmmu.ffs@nanos.tec.linutronix.de>
On Fri, Apr 23, 2021 at 09:14:49PM +0200, Thomas Gleixner wrote:
> On Thu, Apr 22 2021 at 07:24, Paul E. McKenney wrote:
> > On Thu, Apr 22, 2021 at 03:41:26PM +0800, Feng Tang wrote:
> > So what are our options?
> >
> > 1. Clear CLOCK_SOURCE_MUST_VERIFY from tsc-early.
> >
> > 2. #1, but add tsc-early into the watchdog list and set
> > CLOCK_SOURCE_MUST_VERIFY once it is better calibrated.
> >
> > 3. Add a field to struct clocksource that, if non-zero, gives
> > the maximum drift in nanoseconds per half second (AKA
> > WATCHDOG_INTERVAL). If zero, the WATCHDOG_MAX_SKEW value
> > is used. Set this to (say) 150,000ns for tsc-early.
> >
> > 4. As noted earlier, increase WATCHDOG_MAX_SKEW to 150 microseconds,
> > which again is not a good approach given the real-world needs
> > of real-world applications.
> >
> > 5. Your ideas here.
>
> #3 or add a flag to the clocksource which says 'frequency is guesswork' and
> increase the threshold based on that.
>
> If that flag is still set max_drift is != 0 after 20 seconds yell.
I made it 60 seconds based on recent experience with large systems,
but sounds good!
And the calls to clocksource_unregister(&clocksource_tsc_early) mean
that it is not necessary to actually clear the .max_drift field, if
I understand correctly.
It looks to me that init_tsc_clocksource() is invoked at device_initcall()
time, and that it either immediately unregisters clocksource_tsc_early
(known TSC frequency), or schedules a delayed work to make
tsc_refine_calibration_work() do the calibration and the unregister.
Please see below for an untested patch.
Thanx, Paul
------------------------------------------------------------------------
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index 56289170753c..7192b8950322 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -1127,6 +1127,7 @@ static int tsc_cs_enable(struct clocksource *cs)
static struct clocksource clocksource_tsc_early = {
.name = "tsc-early",
.rating = 299,
+ .max_drift = 5 * NSEC_PER_MSEC,
.read = read_tsc,
.mask = CLOCKSOURCE_MASK(64),
.flags = CLOCK_SOURCE_IS_CONTINUOUS |
diff --git a/include/linux/clocksource.h b/include/linux/clocksource.h
index 83a3ebff7456..0c4418b381bf 100644
--- a/include/linux/clocksource.h
+++ b/include/linux/clocksource.h
@@ -93,6 +93,7 @@ struct clocksource {
u32 shift;
u64 max_idle_ns;
u32 maxadj;
+ u32 max_drift;
#ifdef CONFIG_ARCH_CLOCKSOURCE_DATA
struct arch_clocksource_data archdata;
#endif
diff --git a/kernel/time/clocksource.c b/kernel/time/clocksource.c
index a1f90e2b1039..871837c46c62 100644
--- a/kernel/time/clocksource.c
+++ b/kernel/time/clocksource.c
@@ -126,6 +126,7 @@ static void __clocksource_change_rating(struct clocksource *cs, int rating);
*/
#define WATCHDOG_INTERVAL (HZ >> 1)
#define WATCHDOG_THRESHOLD (200 * NSEC_PER_USEC)
+#define WATCHDOG_SYNC_FORGIVENESS (HZ * 60UL)
/*
* Maximum permissible delay between two readouts of the watchdog
@@ -377,6 +378,7 @@ static void clocksource_watchdog(struct timer_list *unused)
int next_cpu, reset_pending;
int64_t wd_nsec, cs_nsec;
struct clocksource *cs;
+ u32 md;
spin_lock(&watchdog_lock);
if (!watchdog_running)
@@ -423,6 +425,12 @@ static void clocksource_watchdog(struct timer_list *unused)
continue;
/* Check the deviation from the watchdog clocksource. */
+ if (!cs->max_drift) {
+ md = WATCHDOG_MAX_SKEW;
+ } else {
+ WARN_ON_ONCE(time_after(jiffies, WATCHDOG_SYNC_FORGIVENESS));
+ md = cs->max_drift;
+ }
if (abs(cs_nsec - wd_nsec) > WATCHDOG_THRESHOLD) {
pr_warn("timekeeping watchdog on CPU%d: Marking clocksource '%s' as unstable because the skew is too large:\n",
smp_processor_id(), cs->name);
next prev parent reply other threads:[~2021-04-23 21:14 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-20 6:49 [clocksource] 6c52b5f3cf: stress-ng.opcode.ops_per_sec -14.4% regression kernel test robot
2021-04-20 13:43 ` Paul E. McKenney
2021-04-20 14:05 ` Paul E. McKenney
2021-04-21 6:07 ` [LKP] " Xing, Zhengjun
2021-04-21 13:42 ` Paul E. McKenney
2021-04-22 6:58 ` Xing Zhengjun
2021-04-22 7:41 ` Feng Tang
2021-04-22 14:24 ` Paul E. McKenney
2021-04-22 16:57 ` Paul E. McKenney
2021-04-23 6:11 ` Feng Tang
2021-04-23 14:02 ` Paul E. McKenney
2021-04-24 12:29 ` Feng Tang
2021-04-24 17:53 ` Paul E. McKenney
2021-04-25 2:14 ` Feng Tang
2021-04-25 3:14 ` Feng Tang
2021-04-25 19:15 ` Paul E. McKenney
2021-04-25 19:14 ` Paul E. McKenney
2021-04-26 12:39 ` Thomas Gleixner
2021-04-26 14:05 ` Feng Tang
2021-04-26 14:33 ` Thomas Gleixner
2021-04-26 15:12 ` Feng Tang
2021-04-23 2:15 ` Xing Zhengjun
2021-04-23 4:12 ` Paul E. McKenney
2021-04-23 19:14 ` Thomas Gleixner
2021-04-23 21:14 ` Paul E. McKenney [this message]
2021-04-23 23:39 ` Paul E. McKenney
2021-04-23 19:09 ` Thomas Gleixner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210423211444.GO975577@paulmck-ThinkPad-P17-Gen-1 \
--to=paulmck@kernel.org \
--cc=Mark.Rutland@arm.com \
--cc=ak@linux.intel.com \
--cc=clm@fb.com \
--cc=corbet@lwn.net \
--cc=feng.tang@intel.com \
--cc=john.stultz@linaro.org \
--cc=linux-kernel@vger.kernel.org \
--cc=lkp@intel.com \
--cc=lkp@lists.01.org \
--cc=maz@kernel.org \
--cc=sboyd@kernel.org \
--cc=tglx@linutronix.de \
--cc=zhengjun.xing@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).