From: "tip-bot2 for Thomas Gleixner" <tip-bot2@linutronix.de>
To: linux-tip-commits@vger.kernel.org
Cc: Thomas Gleixner <tglx@linutronix.de>,
x86@kernel.org, linux-kernel@vger.kernel.org
Subject: [tip: timers/core] hrtimer: Force clock_was_set() handling for the HIGHRES=n, NOHZ=y case
Date: Tue, 10 Aug 2021 16:02:16 -0000 [thread overview]
Message-ID: <162861133625.395.538177983573699889.tip-bot2@tip-bot2> (raw)
In-Reply-To: <20210713135158.288697903@linutronix.de>
The following commit has been merged into the timers/core branch of tip:
Commit-ID: e71a4153b7c256ec103e79875398553808aeffd2
Gitweb: https://git.kernel.org/tip/e71a4153b7c256ec103e79875398553808aeffd2
Author: Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Tue, 13 Jul 2021 15:39:49 +02:00
Committer: Thomas Gleixner <tglx@linutronix.de>
CommitterDate: Tue, 10 Aug 2021 17:57:22 +02:00
hrtimer: Force clock_was_set() handling for the HIGHRES=n, NOHZ=y case
When CONFIG_HIGH_RES_TIMERS is disabled, but NOHZ is enabled then
clock_was_set() is not doing anything. With HIGHRES=n the kernel relies on
the periodic tick to update the clock offsets, but when NOHZ is enabled and
active then CPUs which are in a deep idle sleep do not have a periodic tick
which means the expiry of timers affected by clock_was_set() can be
arbitrarily delayed up to the point where the CPUs are brought out of idle
again.
Make the clock_was_set() logic unconditionaly available so that idle CPUs
are kicked out of idle to handle the update.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/r/20210713135158.288697903@linutronix.de
---
kernel/time/hrtimer.c | 87 ++++++++++++++++++++++++++++--------------
1 file changed, 59 insertions(+), 28 deletions(-)
diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 7ebf642..214fd65 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -739,23 +739,7 @@ static inline int hrtimer_is_hres_enabled(void)
return hrtimer_hres_enabled;
}
-/*
- * Retrigger next event is called after clock was set
- *
- * Called with interrupts disabled via on_each_cpu()
- */
-static void retrigger_next_event(void *arg)
-{
- struct hrtimer_cpu_base *base = this_cpu_ptr(&hrtimer_bases);
-
- if (!__hrtimer_hres_active(base))
- return;
-
- raw_spin_lock(&base->lock);
- hrtimer_update_base(base);
- hrtimer_force_reprogram(base, 0);
- raw_spin_unlock(&base->lock);
-}
+static void retrigger_next_event(void *arg);
/*
* Switch to high resolution mode
@@ -781,9 +765,50 @@ static void hrtimer_switch_to_hres(void)
static inline int hrtimer_is_hres_enabled(void) { return 0; }
static inline void hrtimer_switch_to_hres(void) { }
-static inline void retrigger_next_event(void *arg) { }
#endif /* CONFIG_HIGH_RES_TIMERS */
+/*
+ * Retrigger next event is called after clock was set with interrupts
+ * disabled through an SMP function call or directly from low level
+ * resume code.
+ *
+ * This is only invoked when:
+ * - CONFIG_HIGH_RES_TIMERS is enabled.
+ * - CONFIG_NOHZ_COMMON is enabled
+ *
+ * For the other cases this function is empty and because the call sites
+ * are optimized out it vanishes as well, i.e. no need for lots of
+ * #ifdeffery.
+ */
+static void retrigger_next_event(void *arg)
+{
+ struct hrtimer_cpu_base *base = this_cpu_ptr(&hrtimer_bases);
+
+ /*
+ * When high resolution mode or nohz is active, then the offsets of
+ * CLOCK_REALTIME/TAI/BOOTTIME have to be updated. Otherwise the
+ * next tick will take care of that.
+ *
+ * If high resolution mode is active then the next expiring timer
+ * must be reevaluated and the clock event device reprogrammed if
+ * necessary.
+ *
+ * In the NOHZ case the update of the offset and the reevaluation
+ * of the next expiring timer is enough. The return from the SMP
+ * function call will take care of the reprogramming in case the
+ * CPU was in a NOHZ idle sleep.
+ */
+ if (!__hrtimer_hres_active(base) && !tick_nohz_active)
+ return;
+
+ raw_spin_lock(&base->lock);
+ hrtimer_update_base(base);
+ if (__hrtimer_hres_active(base))
+ hrtimer_force_reprogram(base, 0);
+ else
+ hrtimer_update_next_event(base);
+ raw_spin_unlock(&base->lock);
+}
/*
* When a timer is enqueued and expires earlier than the already enqueued
@@ -842,22 +867,28 @@ static void hrtimer_reprogram(struct hrtimer *timer, bool reprogram)
}
/*
- * Clock realtime was set
- *
- * Change the offset of the realtime clock vs. the monotonic
- * clock.
+ * Clock was set. This might affect CLOCK_REALTIME, CLOCK_TAI and
+ * CLOCK_BOOTTIME (for late sleep time injection).
*
- * We might have to reprogram the high resolution timer interrupt. On
- * SMP we call the architecture specific code to retrigger _all_ high
- * resolution timer interrupts. On UP we just disable interrupts and
- * call the high resolution interrupt code.
+ * This requires to update the offsets for these clocks
+ * vs. CLOCK_MONOTONIC. When high resolution timers are enabled, then this
+ * also requires to eventually reprogram the per CPU clock event devices
+ * when the change moves an affected timer ahead of the first expiring
+ * timer on that CPU. Obviously remote per CPU clock event devices cannot
+ * be reprogrammed. The other reason why an IPI has to be sent is when the
+ * system is in !HIGH_RES and NOHZ mode. The NOHZ mode updates the offsets
+ * in the tick, which obviously might be stopped, so this has to bring out
+ * the remote CPU which might sleep in idle to get this sorted.
*/
void clock_was_set(void)
{
-#ifdef CONFIG_HIGH_RES_TIMERS
+ if (!hrtimer_hres_active() && !tick_nohz_active)
+ goto out_timerfd;
+
/* Retrigger the CPU local events everywhere */
on_each_cpu(retrigger_next_event, NULL, 1);
-#endif
+
+out_timerfd:
timerfd_clock_was_set();
}
next prev parent reply other threads:[~2021-08-10 16:03 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-13 13:39 [patch V2 00/10] hrtimer: Reprogramming and clock_was_set() overhaul Thomas Gleixner
2021-07-13 13:39 ` [patch V2 01/10] hrtimer: Avoid double reprogramming in __hrtimer_start_range_ns() Thomas Gleixner
2021-08-10 16:02 ` [tip: timers/core] " tip-bot2 for Thomas Gleixner
2021-07-13 13:39 ` [patch V2 02/10] hrtimer: Consolidate reprogramming code Thomas Gleixner
2021-08-10 16:02 ` [tip: timers/core] " tip-bot2 for Peter Zijlstra
2021-08-12 7:19 ` Mike Galbraith
2021-08-12 14:11 ` Thomas Gleixner
2021-08-12 14:32 ` Thomas Gleixner
2021-08-12 15:04 ` Mike Galbraith
2021-08-12 15:22 ` Thomas Gleixner
2021-08-12 15:31 ` Mike Galbraith
2021-08-12 16:58 ` Thomas Gleixner
2021-08-12 15:27 ` Mike Galbraith
2021-08-12 20:32 ` [PATCH] hrtimer: Unbreak hrtimer_force_reprogram() Thomas Gleixner
2021-08-12 20:40 ` [tip: timers/core] " tip-bot2 for Thomas Gleixner
2021-08-13 7:58 ` [PATCH] " Thomas Gleixner
2021-08-12 20:31 ` [PATCH] hrtimer: Use raw_cpu_ptr() in clock_was_set() Thomas Gleixner
2021-08-12 20:40 ` [tip: timers/core] " tip-bot2 for Thomas Gleixner
2021-08-13 6:42 ` [PATCH] " Marek Szyprowski
[not found] ` <CGME20210812130945eucas1p117fc1e90f31c8d9fd177932cd1a18512@eucas1p1.samsung.com>
2021-08-12 13:09 ` [patch V2 02/10] hrtimer: Consolidate reprogramming code Marek Szyprowski
2021-07-13 13:39 ` [patch V2 03/10] hrtimer: Ensure timerfd notification for HIGHRES=n Thomas Gleixner
2021-08-10 16:02 ` [tip: timers/core] " tip-bot2 for Thomas Gleixner
2021-07-13 13:39 ` [patch V2 04/10] hrtimer: Force clock_was_set() handling for the HIGHRES=n, NOHZ=y case Thomas Gleixner
2021-08-10 16:02 ` tip-bot2 for Thomas Gleixner [this message]
2021-07-13 13:39 ` [patch V2 05/10] timerfd: Provide timerfd_resume() Thomas Gleixner
2021-08-10 16:02 ` [tip: timers/core] " tip-bot2 for Thomas Gleixner
2021-07-13 13:39 ` [patch V2 06/10] timekeeping: Distangle resume and clock-was-set events Thomas Gleixner
2021-08-10 16:02 ` [tip: timers/core] " tip-bot2 for Thomas Gleixner
2021-07-13 13:39 ` [patch V2 07/10] time/timekeeping: Avoid invoking clock_was_set() twice Thomas Gleixner
2021-08-10 16:02 ` [tip: timers/core] " tip-bot2 for Thomas Gleixner
2021-07-13 13:39 ` [patch V2 08/10] hrtimer: Add bases argument to clock_was_set() Thomas Gleixner
2021-08-10 16:02 ` [tip: timers/core] " tip-bot2 for Thomas Gleixner
2021-07-13 13:39 ` [patch V2 09/10] hrtimer: Avoid unnecessary SMP function calls in clock_was_set() Thomas Gleixner
2021-08-10 16:02 ` [tip: timers/core] " tip-bot2 for Marcelo Tosatti
2021-07-13 13:39 ` [patch V2 10/10] hrtimer: Avoid more " Thomas Gleixner
2021-08-10 16:02 ` [tip: timers/core] " tip-bot2 for Thomas Gleixner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=162861133625.395.538177983573699889.tip-bot2@tip-bot2 \
--to=tip-bot2@linutronix.de \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-tip-commits@vger.kernel.org \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).