From: Dmitry Osipenko <digetx@gmail.com>
To: Frederic Weisbecker <frederic@kernel.org>,
LKML <linux-kernel@vger.kernel.org>
Cc: "Rafael J . Wysocki" <rafael.j.wysocki@intel.com>,
Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@kernel.org>,
Fabio Estevam <festevam@gmail.com>,
stable@vger.kernel.org, Thomas Gleixner <tglx@linutronix.de>,
"Paul E . McKenney" <paulmck@kernel.org>,
Len Brown <lenb@kernel.org>,
Pengutronix Kernel Team <kernel@pengutronix.de>,
NXP Linux Team <linux-imx@nxp.com>,
Daniel Lezcano <daniel.lezcano@linaro.org>,
Shawn Guo <shawnguo@kernel.org>,
Sascha Hauer <s.hauer@pengutronix.de>,
"linux-tegra@vger.kernel.org" <linux-tegra@vger.kernel.org>
Subject: Re: [PATCH 2/4] cpuidle: Fix missing need_resched() check after rcu_idle_enter()
Date: Tue, 5 Jan 2021 20:25:41 +0300 [thread overview]
Message-ID: <e882be10-548a-8e90-9bc6-acea453a5241@gmail.com> (raw)
In-Reply-To: <20201222013712.15056-3-frederic@kernel.org>
22.12.2020 04:37, Frederic Weisbecker пишет:
> Entering RCU idle mode may cause a deferred wake up of an RCU NOCB_GP
> kthread (rcuog) to be serviced.
>
> Usually a wake up happening while running the idle task is spotted in
> one of the need_resched() checks carefully placed within the idle loop
> that can break to the scheduler.
>
> Unfortunately within cpuidle the call to rcu_idle_enter() is already
> beyond the last generic need_resched() check. Some drivers may perform
> their own checks like with mwait_idle_with_hints() but many others don't
> and we may halt the CPU with a resched request unhandled, leaving the
> task hanging.
>
> Fix this with performing a last minute need_resched() check after
> calling rcu_idle_enter().
>
> Reported-by: Paul E. McKenney <paulmck@kernel.org>
> Fixes: 1098582a0f6c (sched,idle,rcu: Push rcu_idle deeper into the idle path)
> Cc: stable@vger.kernel.org
> Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@kernel.org>
> Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
> ---
> drivers/cpuidle/cpuidle.c | 33 +++++++++++++++++++++++++--------
> 1 file changed, 25 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c
> index ef2ea1b12cd8..4cc1ba49ce05 100644
> --- a/drivers/cpuidle/cpuidle.c
> +++ b/drivers/cpuidle/cpuidle.c
> @@ -134,8 +134,8 @@ int cpuidle_find_deepest_state(struct cpuidle_driver *drv,
> }
>
> #ifdef CONFIG_SUSPEND
> -static void enter_s2idle_proper(struct cpuidle_driver *drv,
> - struct cpuidle_device *dev, int index)
> +static int enter_s2idle_proper(struct cpuidle_driver *drv,
> + struct cpuidle_device *dev, int index)
> {
> ktime_t time_start, time_end;
> struct cpuidle_state *target_state = &drv->states[index];
> @@ -151,7 +151,14 @@ static void enter_s2idle_proper(struct cpuidle_driver *drv,
> stop_critical_timings();
> if (!(target_state->flags & CPUIDLE_FLAG_RCU_IDLE))
> rcu_idle_enter();
> - target_state->enter_s2idle(dev, drv, index);
> + /*
> + * Last need_resched() check must come after rcu_idle_enter()
> + * which may wake up RCU internal tasks.
> + */
> + if (!need_resched())
> + target_state->enter_s2idle(dev, drv, index);
> + else
> + index = -EBUSY;
> if (WARN_ON_ONCE(!irqs_disabled()))
> local_irq_disable();
> if (!(target_state->flags & CPUIDLE_FLAG_RCU_IDLE))
> @@ -159,10 +166,13 @@ static void enter_s2idle_proper(struct cpuidle_driver *drv,
> tick_unfreeze();
> start_critical_timings();
>
> - time_end = ns_to_ktime(local_clock());
> + if (index > 0) {
index=0 is valid too
> + time_end = ns_to_ktime(local_clock());
> + dev->states_usage[index].s2idle_time += ktime_us_delta(time_end, time_start);
> + dev->states_usage[index].s2idle_usage++;
> + }
>
> - dev->states_usage[index].s2idle_time += ktime_us_delta(time_end, time_start);
> - dev->states_usage[index].s2idle_usage++;
> + return index;
> }
>
> /**
> @@ -184,7 +194,7 @@ int cpuidle_enter_s2idle(struct cpuidle_driver *drv, struct cpuidle_device *dev)
> */
> index = find_deepest_state(drv, dev, U64_MAX, 0, true);
> if (index > 0) {
> - enter_s2idle_proper(drv, dev, index);
> + index = enter_s2idle_proper(drv, dev, index);
> local_irq_enable();
> }
> return index;
> @@ -234,7 +244,14 @@ int cpuidle_enter_state(struct cpuidle_device *dev, struct cpuidle_driver *drv,
> stop_critical_timings();
> if (!(target_state->flags & CPUIDLE_FLAG_RCU_IDLE))
> rcu_idle_enter();
> - entered_state = target_state->enter(dev, drv, index);
> + /*
> + * Last need_resched() check must come after rcu_idle_enter()
> + * which may wake up RCU internal tasks.
> + */
> + if (!need_resched())
> + entered_state = target_state->enter(dev, drv, index);
> + else
> + entered_state = -EBUSY;
> if (!(target_state->flags & CPUIDLE_FLAG_RCU_IDLE))
> rcu_idle_exit();
> start_critical_timings();
>
This patch causes a hardlock on NVIDIA Tegra using today's linux-next.
Disabling coupled idling state helps. Please fix thanks in advance.
next prev parent reply other threads:[~2021-01-05 17:26 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-12-22 1:37 [PATCH 0/4] sched/idle: Fix missing need_resched() checks after rcu_idle_enter() Frederic Weisbecker
2020-12-22 1:37 ` [PATCH 1/4] sched/idle: Fix missing need_resched() check " Frederic Weisbecker
2020-12-22 4:20 ` Paul E. McKenney
2020-12-22 1:37 ` [PATCH 2/4] cpuidle: " Frederic Weisbecker
2021-01-05 17:25 ` Dmitry Osipenko [this message]
2021-01-05 18:10 ` Dmitry Osipenko
2021-01-05 19:17 ` Paul E. McKenney
2020-12-22 1:37 ` [PATCH 3/4] ARM: imx6q: " Frederic Weisbecker
2020-12-22 1:37 ` [PATCH 4/4] ACPI: processor: " Frederic Weisbecker
2020-12-22 16:19 ` [PATCH 0/4] sched/idle: Fix missing need_resched() checks " Rafael J. Wysocki
2021-01-01 16:05 ` Paul E. McKenney
2021-01-04 15:20 [PATCH 0/4] sched/idle: Fix missing need_resched() checks after rcu_idle_enter() v2 Frederic Weisbecker
2021-01-04 15:20 ` [PATCH 2/4] cpuidle: Fix missing need_resched() check after rcu_idle_enter() Frederic Weisbecker
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e882be10-548a-8e90-9bc6-acea453a5241@gmail.com \
--to=digetx@gmail.com \
--cc=daniel.lezcano@linaro.org \
--cc=festevam@gmail.com \
--cc=frederic@kernel.org \
--cc=kernel@pengutronix.de \
--cc=lenb@kernel.org \
--cc=linux-imx@nxp.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-tegra@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=paulmck@kernel.org \
--cc=peterz@infradead.org \
--cc=rafael.j.wysocki@intel.com \
--cc=s.hauer@pengutronix.de \
--cc=shawnguo@kernel.org \
--cc=stable@vger.kernel.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).