From: Mike Galbraith <umgwanakikbuti@gmail.com>
To: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: linux-rt-users <linux-rt-users@vger.kernel.org>,
LKML <linux-kernel@vger.kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
rostedt@goodmis.org, John Kacur <jkacur@redhat.com>
Subject: [rfc patch v2] rt,nohz_full: fix nohz_full for PREEMPT_RT_FULL
Date: Fri, 13 Mar 2015 05:53:25 +0100 [thread overview]
Message-ID: <1426222405.17611.7.camel@gmail.com> (raw)
In-Reply-To: <1426153093.4190.2.camel@gmail.com>
First of all, a task being ticked and trying to shut the tick down will
fail to do so due to having just awakened ksoftirqd, so let ksoftirqd
try to do that after SOFTIRQ_TIMER processing. Secondly, should the
tick be shut down, we may livelock in hrtimer-cancel() because in -rt
a callback may be running. Break the loop, and let tick_nohz_restart()
know that the timer is busy so it can bail.
Signed-off-by: Mike Galbraith <umgwanakikbuti@gmail.com>
---
kernel/sched/core.c | 8 +++++++-
kernel/softirq.c | 17 +++++++++++++++++
kernel/time/hrtimer.c | 9 +++++++++
kernel/time/tick-sched.c | 14 +++++++++++++-
4 files changed, 46 insertions(+), 2 deletions(-)
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -783,12 +783,18 @@ static inline bool got_nohz_idle_kick(vo
#ifdef CONFIG_NO_HZ_FULL
bool sched_can_stop_tick(void)
{
+ int ksoftirqd = !!(IS_ENABLED(CONFIG_PREEMPT_RT_FULL) &&
+ current == this_cpu_ksoftirqd());
+
/*
* More than one running task need preemption.
* nr_running update is assumed to be visible
* after IPI is sent from wakers.
+ *
+ * NOTE, RT: ksoftirqd tries to stop the tick for
+ * tasks as they exit irq, ergo subtracts itself.
*/
- if (this_rq()->nr_running > 1)
+ if (this_rq()->nr_running - ksoftirqd > 1)
return false;
return true;
--- a/kernel/softirq.c
+++ b/kernel/softirq.c
@@ -498,6 +498,22 @@ static void unlock_softirq(int which)
local_unlock(local_softirq_locks[which]);
}
+/*
+ * Let ksoftirqd try to shut down the tick when awakened via
+ * timer_interrupt->irq_exit()->invoke_softirq(), as the task
+ * then calling tick_nohz_irq_exit() WILL fail to do so due
+ * to that very wakeup having made rq->nr_running > 1.
+ */
+static void tick_nohz_sirq_timer_exit(int which)
+{
+ if (!IS_ENABLED(CONFIG_NO_HZ_FULL))
+ return;
+ if (!IS_ENABLED(CONFIG_PREEMPT_RT_FULL))
+ return;
+ if (which != TIMER_SOFTIRQ || current != this_cpu_ksoftirqd())
+ return;
+ tick_nohz_irq_exit();
+}
static void do_single_softirq(int which, int need_rcu_bh_qs)
{
unsigned long old_flags = current->flags;
@@ -513,6 +529,7 @@ static void do_single_softirq(int which,
current->flags &= ~PF_IN_SOFTIRQ;
vtime_account_irq_enter(current);
tsk_restore_flags(current, old_flags, PF_MEMALLOC);
+ tick_nohz_sirq_timer_exit(which);
}
/*
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -1162,6 +1162,15 @@ int hrtimer_try_to_cancel(struct hrtimer
if (!hrtimer_callback_running(timer))
ret = remove_hrtimer(timer, base);
+ else if (IS_ENABLED(CONFIG_PREEMPT_RT_FULL) &&
+ tick_nohz_full_cpu(raw_smp_processor_id())) {
+ /*
+ * Let tick_nohz_restart() know that the timer is
+ * active lest we spin forever in hrtimer_cancel().
+ */
+ if (in_irq() && timer->irqsafe)
+ ret = HRTIMER_STATE_CALLBACK;
+ }
unlock_hrtimer_base(timer, &flags);
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -222,7 +222,12 @@ void __tick_nohz_full_check(void)
static void nohz_full_kick_work_func(struct irq_work *work)
{
+ unsigned long flags;
+
+ /* ksoftirqd processes sirqs with interrupts enabled */
+ local_irq_save(flags);
__tick_nohz_full_check();
+ local_irq_restore(flags);
}
static DEFINE_PER_CPU(struct irq_work, nohz_full_kick_work) = {
@@ -879,7 +884,14 @@ ktime_t tick_nohz_get_sleep_length(void)
static void tick_nohz_restart(struct tick_sched *ts, ktime_t now)
{
- hrtimer_cancel(&ts->sched_timer);
+ int cpu = raw_smp_processor_id();
+
+ /* In an RT kernel, a callback may be running. If so, bail */
+ if (IS_ENABLED(CONFIG_PREEMPT_RT_FULL) && tick_nohz_full_cpu(cpu)) {
+ if (hrtimer_cancel(&ts->sched_timer) == HRTIMER_STATE_CALLBACK)
+ return;
+ } else
+ hrtimer_cancel(&ts->sched_timer);
hrtimer_set_expires(&ts->sched_timer, ts->last_tick);
while (1) {
next prev parent reply other threads:[~2015-03-13 4:53 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-02-16 11:18 [ANNOUNCE] 3.18.7-rt1 Sebastian Andrzej Siewior
2015-02-16 18:13 ` Sebastian Andrzej Siewior
2015-02-16 20:12 ` Richard Cochran
2015-02-16 20:21 ` Steven Rostedt
2015-02-17 8:37 ` Daniel Wagner
2015-02-19 9:36 ` Joakim Hernberg
2015-02-25 13:55 ` Sebastian Andrzej Siewior
2015-02-26 8:48 ` Sebastian Andrzej Siewior
2015-02-27 6:40 ` Daniel Wagner
2015-02-27 7:58 ` Sebastian Andrzej Siewior
2015-02-18 11:21 ` [patch-3.18.7-rt1] sched/context-tracking: fix PREEMPT_LAZY explosions Mike Galbraith
2015-02-18 11:27 ` [rfc patch-3.18.7-rt1] rt/nohz_full: Fix can_stop_full_tick() gripe within softirq Mike Galbraith
2015-02-19 4:50 ` Mike Galbraith
2015-02-19 6:30 ` [patch-3.18.7-rt1] sched/context-tracking: fix PREEMPT_LAZY explosions Mike Galbraith
2015-03-09 13:45 ` Sebastian Andrzej Siewior
2015-03-09 14:36 ` Mike Galbraith
2015-03-09 14:45 ` Sebastian Andrzej Siewior
2015-03-10 9:35 ` Mike Galbraith
2015-03-11 6:18 ` Mike Galbraith
2015-03-12 9:38 ` [rfc patch] rt,nohz_full: fix nohz_full for PREEMPT_RT_FULL Mike Galbraith
2015-03-12 15:09 ` Steven Rostedt
2015-03-13 2:13 ` Mike Galbraith
2015-03-13 3:03 ` Steven Rostedt
2015-03-13 4:53 ` Mike Galbraith [this message]
2015-03-16 20:24 ` [rfc patch v2] " Sebastian Andrzej Siewior
2015-03-17 1:53 ` Mike Galbraith
2015-03-17 4:45 ` Mike Galbraith
2015-04-10 14:15 ` Sebastian Andrzej Siewior
2015-04-10 14:24 ` Mike Galbraith
2015-04-10 14:28 ` Mike Galbraith
2015-04-11 13:15 ` Mike Galbraith
2015-04-11 13:36 ` Mike Galbraith
2015-04-11 14:22 ` Mike Galbraith
2015-04-13 9:43 ` Sebastian Andrzej Siewior
2015-04-13 9:41 ` Sebastian Andrzej Siewior
2015-02-18 14:09 ` [patch-3.18.7-rt1] snd/pcm: fix snd_pcm_stream_lock*() irqs_disabled() splats Mike Galbraith
2015-02-25 14:08 ` Sebastian Andrzej Siewior
2015-02-18 15:05 ` [patch-3.18.7-rt1]sunrpc: make svc_xprt_do_enqueue() use get_cpu_light() Mike Galbraith
2015-02-25 14:14 ` Sebastian Andrzej Siewior
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1426222405.17611.7.camel@gmail.com \
--to=umgwanakikbuti@gmail.com \
--cc=bigeasy@linutronix.de \
--cc=jkacur@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rt-users@vger.kernel.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).