From: Randy Dunlap <rdunlap@infradead.org>
To: Marcelo Tosatti <mtosatti@redhat.com>, linux-kernel@vger.kernel.org
Cc: Frederic Weisbecker <fweisbec@gmail.com>,
Chris Friesen <chris.friesen@windriver.com>,
Thomas Gleixner <tglx@linutronix.de>,
Peter Zijlstra <peterz@infradead.org>,
Andrew Morton <akpm@linux-foundation.org>,
Jim Somerville <Jim.Somerville@windriver.com>,
Christoph Lameter <cl@linux.com>
Subject: Re: [patch 3/4] isolcpus: affine kernel threads to housekeeping cpus
Date: Wed, 1 Apr 2020 15:32:43 -0700 [thread overview]
Message-ID: <a5e70c3b-2927-6ee3-fe75-8b9ece0df94f@infradead.org> (raw)
In-Reply-To: <20200401121342.979811840@redhat.com>
On 4/1/20 5:10 AM, Marcelo Tosatti wrote:
> This is a kernel enhancement that configures the cpu affinity of kernel
> threads via kernel boot option nohz_full=.
>
> When this option is specified, the cpumask is immediately applied upon
> thread launch. This does not affect kernel threads that specify cpu
> and node.
>
> This allows CPU isolation (that is not allowing certain threads
> to execute on certain CPUs) without using the isolcpus=domain parameter,
> making it possible to enable load balancing on such CPUs
> during runtime (see kernel-parameters.txt).
>
> Note-1: this is based off on Wind River's patch at
> https://github.com/starlingx-staging/stx-integ/blob/master/kernel/kernel-std/centos/patches/affine-compute-kernel-threads.patch
>
> Difference being that this patch is limited to modifying
> kernel thread cpumask: Behaviour of other threads can
> be controlled via cgroups or sched_setaffinity.
>
> Note-2: Wind River's patch was based off Christoph Lameter's patch at
> https://lwn.net/Articles/565932/ with the only difference being
> the kernel parameter changed from kthread to kthread_cpus.
>
> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
>
Hi Marcelo,
> ---
> Documentation/admin-guide/kernel-parameters.txt | 8 ++++++++
Patch is missing those Docum bits.
> include/linux/sched/isolation.h | 1 +
> kernel/kthread.c | 6 ++++--
> kernel/sched/isolation.c | 6 ++++++
> 4 files changed, 19 insertions(+), 2 deletions(-)
>
> Index: linux-2.6/include/linux/sched/isolation.h
> ===================================================================
> --- linux-2.6.orig/include/linux/sched/isolation.h
> +++ linux-2.6/include/linux/sched/isolation.h
> @@ -14,6 +14,7 @@ enum hk_flags {
> HK_FLAG_DOMAIN = (1 << 5),
> HK_FLAG_WQ = (1 << 6),
> HK_FLAG_MANAGED_IRQ = (1 << 7),
> + HK_FLAG_KTHREAD = (1 << 8),
> };
>
> #ifdef CONFIG_CPU_ISOLATION
> Index: linux-2.6/kernel/kthread.c
> ===================================================================
> --- linux-2.6.orig/kernel/kthread.c
> +++ linux-2.6/kernel/kthread.c
> @@ -23,6 +23,7 @@
> #include <linux/ptrace.h>
> #include <linux/uaccess.h>
> #include <linux/numa.h>
> +#include <linux/sched/isolation.h>
> #include <trace/events/sched.h>
>
> static DEFINE_SPINLOCK(kthread_create_lock);
> @@ -347,7 +348,8 @@ struct task_struct *__kthread_create_on_
> * The kernel thread should not inherit these properties.
> */
> sched_setscheduler_nocheck(task, SCHED_NORMAL, ¶m);
> - set_cpus_allowed_ptr(task, cpu_possible_mask);
> + set_cpus_allowed_ptr(task,
> + housekeeping_cpumask(HK_FLAG_KTHREAD));
> }
> kfree(create);
> return task;
> @@ -572,7 +574,7 @@ int kthreadd(void *unused)
> /* Setup a clean context for our children to inherit. */
> set_task_comm(tsk, "kthreadd");
> ignore_signals(tsk);
> - set_cpus_allowed_ptr(tsk, cpu_possible_mask);
> + set_cpus_allowed_ptr(tsk, housekeeping_cpumask(HK_FLAG_KTHREAD));
> set_mems_allowed(node_states[N_MEMORY]);
>
> current->flags |= PF_NOFREEZE;
> Index: linux-2.6/kernel/sched/isolation.c
> ===================================================================
> --- linux-2.6.orig/kernel/sched/isolation.c
> +++ linux-2.6/kernel/sched/isolation.c
> @@ -141,7 +141,7 @@ static int __init housekeeping_nohz_full
> unsigned int flags;
>
> flags = HK_FLAG_TICK | HK_FLAG_WQ | HK_FLAG_TIMER | HK_FLAG_RCU |
> - HK_FLAG_MISC | HK_FLAG_SCHED;
> + HK_FLAG_MISC | HK_FLAG_SCHED | HK_FLAG_KTHREAD;
>
> return housekeeping_setup(str, flags);
> }
>
>
thanks.
--
~Randy
next prev parent reply other threads:[~2020-04-01 22:32 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-04-01 12:10 [patch 0/4] affine kernel threads to nohz_full= cpumask (v4) Marcelo Tosatti
2020-04-01 12:10 ` [patch 1/4] kthread: switch to cpu_possible_mask Marcelo Tosatti
2020-04-01 12:10 ` [patch 2/4] isolation: set HK_FLAG_SCHED on nohz_full CPUs Marcelo Tosatti
2020-04-01 12:10 ` [patch 3/4] isolcpus: affine kernel threads to housekeeping cpus Marcelo Tosatti
2020-04-01 22:32 ` Randy Dunlap [this message]
2020-04-03 10:37 ` [patch 3/4 v2] " Marcelo Tosatti
2020-04-01 12:10 ` [patch 4/4] isolcpus: undeprecate on documentation Marcelo Tosatti
2020-04-20 16:02 ` [patch 0/4] affine kernel threads to nohz_full= cpumask (v4) Frederic Weisbecker
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a5e70c3b-2927-6ee3-fe75-8b9ece0df94f@infradead.org \
--to=rdunlap@infradead.org \
--cc=Jim.Somerville@windriver.com \
--cc=akpm@linux-foundation.org \
--cc=chris.friesen@windriver.com \
--cc=cl@linux.com \
--cc=fweisbec@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mtosatti@redhat.com \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).