From: Frederic Weisbecker <fweisbec@gmail.com>
To: Chris Metcalf <cmetcalf@ezchip.com>
Cc: Don Zickus <dzickus@redhat.com>, Ingo Molnar <mingo@kernel.org>,
Andrew Morton <akpm@linux-foundation.org>,
Andrew Jones <drjones@redhat.com>,
chai wen <chaiw.fnst@cn.fujitsu.com>,
Ulrich Obergfell <uobergfe@redhat.com>,
Fabian Frederick <fabf@skynet.be>,
Aaron Tomlin <atomlin@redhat.com>, Ben Zhang <benzh@chromium.org>,
Christoph Lameter <cl@linux.com>,
Gilad Ben-Yossef <gilad@benyossef.com>,
Steven Rostedt <rostedt@goodmis.org>,
linux-kernel@vger.kernel.org, Jonathan Corbet <corbet@lwn.net>,
linux-doc@vger.kernel.org, Thomas Gleixner <tglx@linutronix.de>,
Peter Zijlstra <peterz@infradead.org>
Subject: Re: [PATCH v10 1/3] smpboot: allow excluding cpus from the smpboot threads
Date: Fri, 1 May 2015 10:53:58 +0200 [thread overview]
Message-ID: <20150501085356.GA14149@lerouge> (raw)
In-Reply-To: <1430422766-19703-2-git-send-email-cmetcalf@ezchip.com>
On Thu, Apr 30, 2015 at 03:39:24PM -0400, Chris Metcalf wrote:
> This change allows some cores to be excluded from running the
> smp_hotplug_thread tasks. The following commit to update
> kernel/watchdog.c to use this functionality is the motivating
> example, and more information on the motivation is provided there.
>
> A new smp_hotplug_thread field is introduced, "cpumask", which
> is cpumask field managed by the smpboot subsystem that indicates whether
> or not the given smp_hotplug_thread should run on that core; the
> cpumask is checked when deciding whether to unpark the thread.
>
> To limit the cpumask to less than cpu_possible, you must call
> smpboot_update_cpumask_percpu_thread() after registering.
>
> Signed-off-by: Chris Metcalf <cmetcalf@ezchip.com>
> ---
> include/linux/smpboot.h | 5 +++++
> kernel/smpboot.c | 55 ++++++++++++++++++++++++++++++++++++++++++++++++-
> 2 files changed, 59 insertions(+), 1 deletion(-)
>
> diff --git a/include/linux/smpboot.h b/include/linux/smpboot.h
> index d600afb21926..7c42153edfac 100644
> --- a/include/linux/smpboot.h
> +++ b/include/linux/smpboot.h
> @@ -27,6 +27,8 @@ struct smpboot_thread_data;
> * @pre_unpark: Optional unpark function, called before the thread is
> * unparked (cpu online). This is not guaranteed to be
> * called on the target cpu of the thread. Careful!
> + * @cpumask: Internal state. To update which threads are unparked,
> + * call smpboot_update_cpumask_percpu_thread().
> * @selfparking: Thread is not parked by the park function.
> * @thread_comm: The base name of the thread
> */
> @@ -41,11 +43,14 @@ struct smp_hotplug_thread {
> void (*park)(unsigned int cpu);
> void (*unpark)(unsigned int cpu);
> void (*pre_unpark)(unsigned int cpu);
> + struct cpumask cpumask;
I believe it should be allocated dynamically, otherwise it gets the size of NR_CPUS
instead of nr_cpus_bits. It's not _that_ much space spared but think there should be
several struct smp_hotplug_thread registered.
> bool selfparking;
> const char *thread_comm;
> };
>
> int smpboot_register_percpu_thread(struct smp_hotplug_thread *plug_thread);
> void smpboot_unregister_percpu_thread(struct smp_hotplug_thread *plug_thread);
> +int smpboot_update_cpumask_percpu_thread(struct smp_hotplug_thread *plug_thread,
> + const struct cpumask *);
>
> #endif
> diff --git a/kernel/smpboot.c b/kernel/smpboot.c
> index c697f73d82d6..209750ab7031 100644
> --- a/kernel/smpboot.c
> +++ b/kernel/smpboot.c
> @@ -232,7 +232,8 @@ void smpboot_unpark_threads(unsigned int cpu)
>
> mutex_lock(&smpboot_threads_lock);
> list_for_each_entry(cur, &hotplug_threads, list)
> - smpboot_unpark_thread(cur, cpu);
> + if (cpumask_test_cpu(cpu, &cur->cpumask))
> + smpboot_unpark_thread(cur, cpu);
> mutex_unlock(&smpboot_threads_lock);
> }
>
> @@ -258,6 +259,15 @@ static void smpboot_destroy_threads(struct smp_hotplug_thread *ht)
> {
> unsigned int cpu;
>
> + /* Unpark any threads that were voluntarily parked. */
> + for_each_cpu_not(cpu, &ht->cpumask) {
> + if (cpu_online(cpu)) {
> + struct task_struct *tsk = *per_cpu_ptr(ht->store, cpu);
> + if (tsk)
> + kthread_unpark(tsk);
I'm still not clear why we are doing that. kthread_stop() should be able
to handle parked kthreads, otherwise it needs to be fixed.
> + }
> + }
> +
> /* We need to destroy also the parked threads of offline cpus */
> for_each_possible_cpu(cpu) {
> struct task_struct *tsk = *per_cpu_ptr(ht->store, cpu);
> @@ -281,6 +291,7 @@ int smpboot_register_percpu_thread(struct smp_hotplug_thread *plug_thread)
> unsigned int cpu;
> int ret = 0;
>
> + cpumask_copy(&plug_thread->cpumask, cpu_possible_mask);
> get_online_cpus();
> mutex_lock(&smpboot_threads_lock);
> for_each_online_cpu(cpu) {
> @@ -316,6 +327,48 @@ void smpboot_unregister_percpu_thread(struct smp_hotplug_thread *plug_thread)
> }
> EXPORT_SYMBOL_GPL(smpboot_unregister_percpu_thread);
>
> +/**
> + * smpboot_update_cpumask_percpu_thread - Adjust which per_cpu hotplug threads stay parked
> + * @plug_thread: Hotplug thread descriptor
> + * @new: Revised mask to use
> + *
> + * The cpumask field in the smp_hotplug_thread must not be updated directly
> + * by the client, but only by calling this function.
> + */
> +int smpboot_update_cpumask_percpu_thread(struct smp_hotplug_thread *plug_thread,
> + const struct cpumask *new)
> +{
> + struct cpumask *old = &plug_thread->cpumask;
> + cpumask_var_t tmp;
> + unsigned int cpu;
> +
> + if (!alloc_cpumask_var(&tmp, GFP_KERNEL))
> + return -ENOMEM;
> +
> + get_online_cpus();
> + mutex_lock(&smpboot_threads_lock);
> +
> + /* Park threads that were exclusively enabled on the old mask. */
> + cpumask_andnot(tmp, old, new);
> + for_each_cpu_and(cpu, tmp, cpu_online_mask)
> + smpboot_park_thread(plug_thread, cpu);
> +
> + /* Unpark threads that are exclusively enabled on the new mask. */
> + cpumask_andnot(tmp, new, old);
> + for_each_cpu_and(cpu, tmp, cpu_online_mask)
> + smpboot_unpark_thread(plug_thread, cpu);
> +
> + cpumask_copy(old, new);
> +
> + mutex_unlock(&smpboot_threads_lock);
> + put_online_cpus();
> +
> + free_cpumask_var(tmp);
> +
> + return 0;
> +}
> +EXPORT_SYMBOL_GPL(smpboot_update_cpumask_percpu_thread);
> +
> static DEFINE_PER_CPU(atomic_t, cpu_hotplug_state) = ATOMIC_INIT(CPU_POST_DEAD);
>
> /*
> --
> 2.1.2
>
next prev parent reply other threads:[~2015-05-01 8:54 UTC|newest]
Thread overview: 106+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-03-30 18:51 [PATCH] watchdog: nohz: don't run watchdog on nohz_full cores cmetcalf
2015-03-30 19:12 ` Don Zickus
2015-03-30 19:32 ` [PATCH v2] " Chris Metcalf
2015-03-30 20:02 ` Don Zickus
2015-04-02 15:19 ` Frederic Weisbecker
2015-03-31 2:04 ` [PATCH] " Mike Galbraith
2015-03-31 6:34 ` Mike Galbraith
2015-03-31 18:32 ` Chris Metcalf
2015-03-31 7:25 ` Ingo Molnar
2015-03-31 18:30 ` Chris Metcalf
2015-04-02 13:35 ` Don Zickus
2015-04-02 13:49 ` Chris Metcalf
2015-04-02 14:15 ` Don Zickus
2015-04-02 15:38 ` Frederic Weisbecker
2015-04-02 15:42 ` Chris Metcalf
2015-04-02 16:08 ` Don Zickus
2015-04-02 16:48 ` Frederic Weisbecker
2015-04-02 17:39 ` [PATCH v3] watchdog: add watchdog_cpumask sysctl to assist nohz cmetcalf
2015-04-02 18:06 ` Peter Zijlstra
2015-04-02 18:16 ` Chris Metcalf
2015-04-02 18:33 ` Peter Zijlstra
2015-04-02 18:49 ` Chris Metcalf
2015-04-02 18:45 ` Don Zickus
2015-04-03 16:08 ` [PATCH v4 1/2] smpboot: allow excluding cpus from the smpboot threads cmetcalf
2015-04-03 16:08 ` [PATCH v4 2/2] watchdog: add watchdog_exclude sysctl to assist nohz cmetcalf
2015-04-05 16:46 ` Ulrich Obergfell
2015-04-06 19:45 ` [PATCH v5 0/2] nohz/watchdog/smp_hotplug_thread changes cmetcalf
2015-04-06 19:45 ` [PATCH v5 1/2] smpboot: allow excluding cpus from the smpboot threads cmetcalf
2015-04-08 13:28 ` Frederic Weisbecker
2015-04-08 14:06 ` Chris Metcalf
2015-04-08 17:29 ` Frederic Weisbecker
2015-04-06 19:45 ` [PATCH v5 2/2] watchdog: add watchdog_exclude sysctl to assist nohz cmetcalf
2015-04-07 15:44 ` Don Zickus
2015-04-07 15:56 ` Sasha Levin
2015-04-07 17:49 ` Chris Metcalf
2015-04-08 14:01 ` Frederic Weisbecker
2015-04-08 19:11 ` [PATCH v6 1/2] smpboot: allow excluding cpus from the smpboot threads cmetcalf
2015-04-08 19:11 ` [PATCH v6 2/2] watchdog: add watchdog_cpumask sysctl to assist nohz cmetcalf
2015-04-08 20:37 ` [PATCH v6 1/2] smpboot: allow excluding cpus from the smpboot threads Thomas Gleixner
2015-04-09 20:29 ` [PATCH] " Chris Metcalf
2015-04-10 1:58 ` Frederic Weisbecker
2015-04-10 16:33 ` Chris Metcalf
2015-04-12 19:14 ` Frederic Weisbecker
2015-04-13 16:06 ` Chris Metcalf
2015-04-13 21:54 ` Frederic Weisbecker
2015-04-14 19:37 ` [PATCH v8 1/3] " Chris Metcalf
2015-04-14 19:37 ` [PATCH v8 2/3] watchdog: add watchdog_cpumask sysctl to assist nohz Chris Metcalf
2015-04-16 10:46 ` Ulrich Obergfell
2015-04-17 15:41 ` Chris Metcalf
2015-04-22 8:20 ` Ulrich Obergfell
2015-04-28 17:52 ` Chris Metcalf
2015-04-29 8:48 ` Ulrich Obergfell
2015-04-17 1:31 ` Chai Wen
2015-04-17 16:10 ` Chris Metcalf
2015-04-14 19:37 ` [PATCH v8 3/3] procfs: treat parked tasks as sleeping for task state Chris Metcalf
2015-04-16 15:28 ` [PATCH v8 1/3] smpboot: allow excluding cpus from the smpboot threads Frederic Weisbecker
2015-04-16 15:50 ` Chris Metcalf
2015-04-16 16:48 ` Frederic Weisbecker
2015-04-17 16:17 ` Chris Metcalf
2015-04-17 18:37 ` [PATCH v9 " Chris Metcalf
2015-04-17 18:37 ` [PATCH v9 2/3] watchdog: add watchdog_cpumask sysctl to assist nohz Chris Metcalf
2015-04-21 12:32 ` Ulrich Obergfell
2015-04-28 18:07 ` Chris Metcalf
2015-04-29 9:49 ` Ulrich Obergfell
2015-04-29 13:10 ` Don Zickus
2015-04-21 14:07 ` Ulrich Obergfell
2015-04-22 15:18 ` Don Zickus
2015-04-25 15:42 ` Ulrich Obergfell
2015-04-22 11:02 ` Ulrich Obergfell
2015-04-22 15:21 ` Don Zickus
2015-04-27 20:27 ` Chris Metcalf
2015-04-28 15:17 ` Don Zickus
2015-04-28 19:42 ` Andrew Morton
2015-04-30 19:39 ` [PATCH v10 0/3] add watchdog_cpumask to help nohz_full Chris Metcalf
2015-04-30 19:39 ` [PATCH v10 1/3] smpboot: allow excluding cpus from the smpboot threads Chris Metcalf
2015-05-01 8:53 ` Frederic Weisbecker [this message]
2015-05-01 19:57 ` Chris Metcalf
2015-05-01 21:23 ` Frederic Weisbecker
2015-05-04 22:06 ` Chris Metcalf
2015-06-03 2:34 ` Don Zickus
2015-06-04 17:25 ` Chris Metcalf
2015-05-01 20:00 ` [PATCH] smpboot: dynamically allocate the cpumask Chris Metcalf
2015-04-30 19:39 ` [PATCH v10 2/3] watchdog: add watchdog_cpumask sysctl to assist nohz Chris Metcalf
2015-04-30 20:00 ` Don Zickus
2015-04-30 20:09 ` Chris Metcalf
2015-05-01 13:46 ` Don Zickus
2015-04-30 19:39 ` [PATCH v10 3/3] procfs: treat parked tasks as sleeping for task state Chris Metcalf
2015-04-29 22:26 ` [PATCH v9 2/3] watchdog: add watchdog_cpumask sysctl to assist nohz Andrew Morton
2015-04-29 22:26 ` Andrew Morton
2015-04-17 18:37 ` [PATCH v9 3/3] procfs: treat parked tasks as sleeping for task state Chris Metcalf
2015-04-29 22:26 ` Andrew Morton
2015-04-29 22:26 ` [PATCH v9 1/3] smpboot: allow excluding cpus from the smpboot threads Andrew Morton
2015-04-30 16:07 ` Chris Metcalf
2015-04-14 15:23 ` [PATCH] " Frederic Weisbecker
2015-04-14 15:39 ` Chris Metcalf
2015-04-14 17:57 ` Thomas Gleixner
2015-04-10 20:48 ` [PATCH v7 1/3] " Chris Metcalf
2015-04-10 20:48 ` [PATCH v7 2/3] watchdog: add watchdog_cpumask sysctl to assist nohz Chris Metcalf
2015-04-10 20:48 ` [PATCH v7 3/3] procfs: treat parked tasks as sleeping for task state Chris Metcalf
2015-04-10 21:11 ` [PATCH v7 1/3] smpboot: allow excluding cpus from the smpboot threads Andrew Morton
2015-04-13 15:48 ` Chris Metcalf
2015-04-08 19:21 ` [PATCH v5 2/2] watchdog: add watchdog_exclude sysctl to assist nohz Chris Metcalf
2015-04-08 22:31 ` Frederic Weisbecker
2015-03-31 10:17 ` [PATCH] watchdog: nohz: don't run watchdog on nohz_full cores Christoph Lameter
2015-03-31 18:39 ` Chris Metcalf
2015-04-02 14:13 ` Christoph Lameter
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150501085356.GA14149@lerouge \
--to=fweisbec@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=atomlin@redhat.com \
--cc=benzh@chromium.org \
--cc=chaiw.fnst@cn.fujitsu.com \
--cc=cl@linux.com \
--cc=cmetcalf@ezchip.com \
--cc=corbet@lwn.net \
--cc=drjones@redhat.com \
--cc=dzickus@redhat.com \
--cc=fabf@skynet.be \
--cc=gilad@benyossef.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=uobergfe@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).