From: Frederic Weisbecker <frederic@kernel.org>
To: "Paul E. McKenney" <paulmck@linux.ibm.com>
Cc: fweisbec@gmail.com, tglx@linutronix.de, mingo@kernel.org,
linux-kernel@vger.kernel.org, rcu@vger.kernel.org,
peterz@infradead.org, paulmckrcu@gmail.com
Subject: Re: How to turn scheduler tick on for current nohz_full CPU?
Date: Tue, 30 Jul 2019 18:43:10 +0200 [thread overview]
Message-ID: <20190730164309.GA962@lenoir> (raw)
In-Reply-To: <20190729223238.GF14271@linux.ibm.com>
On Mon, Jul 29, 2019 at 03:32:38PM -0700, Paul E. McKenney wrote:
> On Wed, Jul 24, 2019 at 06:12:43PM -0700, Paul E. McKenney wrote:
>
> The patch below (which includes your patch) does help considerably.
> However, it does have some shortcomings:
>
> 1. Adds an atomic operation (albeit a cache-local one) to
> the scheduler fastpath. One approach would be to have
> a way of testing this bit and clearing it only if set.
>
> Another approach would be to instead clear it on the
> transition to nohz_full userspace or to idle.
Well, the latter would be costly as it is going to restart the tick on every
user -> kernel transitions.
>
> 2. There are a lot of other places in the kernel that are in
> need of this bit being set. I am therefore considering making
> multi_cpu_stop() or its callers set this bit on all CPUs upon
> entry and clear it upon exit. While in this state, it is
> likely necessary to disable clearing this bit. Or it would
> be necessary to make multi_cpu_stop() repeat clearing the bit
> every so often.
>
> As it stands, I have CPU hotplug removal operations taking
> more than 400 seconds.
>
> 3. It was tempting to ask for this bit to be tracked on a per-task
> basis, but from what I can see that adds at least as much
> complexity as it removes.
Yeah I forgot to answer, you can use tick_dep_set_task() for that.
>
> Thoughts?
>
> Thanx, Paul
>
> PS. Outage on @linux.ibm.com, hence the CC of my gmail address.
>
> ------------------------------------------------------------------------
>
> diff --git a/include/linux/tick.h b/include/linux/tick.h
> index 196a0a7bfc4f..0dea6fb33a11 100644
> --- a/include/linux/tick.h
> +++ b/include/linux/tick.h
> @@ -108,7 +108,8 @@ enum tick_dep_bits {
> TICK_DEP_BIT_POSIX_TIMER = 0,
> TICK_DEP_BIT_PERF_EVENTS = 1,
> TICK_DEP_BIT_SCHED = 2,
> - TICK_DEP_BIT_CLOCK_UNSTABLE = 3
> + TICK_DEP_BIT_CLOCK_UNSTABLE = 3,
> + TICK_DEP_BIT_RCU = 4
> };
>
> #define TICK_DEP_MASK_NONE 0
> @@ -116,6 +117,7 @@ enum tick_dep_bits {
> #define TICK_DEP_MASK_PERF_EVENTS (1 << TICK_DEP_BIT_PERF_EVENTS)
> #define TICK_DEP_MASK_SCHED (1 << TICK_DEP_BIT_SCHED)
> #define TICK_DEP_MASK_CLOCK_UNSTABLE (1 << TICK_DEP_BIT_CLOCK_UNSTABLE)
> +#define TICK_DEP_MASK_RCU (1 << TICK_DEP_BIT_RCU)
>
> #ifdef CONFIG_NO_HZ_COMMON
> extern bool tick_nohz_enabled;
> @@ -258,6 +260,9 @@ static inline bool tick_nohz_full_enabled(void) { return false; }
> static inline bool tick_nohz_full_cpu(int cpu) { return false; }
> static inline void tick_nohz_full_add_cpus_to(struct cpumask *mask) { }
>
> +static inline void tick_nohz_dep_set_cpu(int cpu, enum tick_dep_bits bit) { }
> +static inline void tick_nohz_dep_clear_cpu(int cpu, enum tick_dep_bits bit) { }
And I gave you the wrong APIs. Please consider_using tick_dep_set_cpu()
and tick_dep_clear_cpu() that first check if the CPU uses nohz_full.
Those should have the !CONFIG_NO_HZ_FULL stub implemented as well.
Thanks.
next prev parent reply other threads:[~2019-07-30 16:43 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-07-24 11:53 How to turn scheduler tick on for current nohz_full CPU? Paul E. McKenney
2019-07-24 13:22 ` Frederic Weisbecker
2019-07-24 13:52 ` Paul E. McKenney
2019-07-24 14:30 ` Frederic Weisbecker
2019-07-25 1:12 ` Paul E. McKenney
2019-07-29 22:32 ` Paul E. McKenney
2019-07-30 16:43 ` Frederic Weisbecker [this message]
2019-07-30 17:36 ` Paul E. McKenney
2019-08-19 2:14 ` Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190730164309.GA962@lenoir \
--to=frederic@kernel.org \
--cc=fweisbec@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=paulmck@linux.ibm.com \
--cc=paulmckrcu@gmail.com \
--cc=peterz@infradead.org \
--cc=rcu@vger.kernel.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).