From: Peter Zijlstra <peterz@infradead.org>
To: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: LKML <linux-kernel@vger.kernel.org>,
Lai Jiangshan <laijs@linux.alibaba.com>,
Hillf Danton <hdanton@sina.com>,
Valentin Schneider <valentin.schneider@arm.com>,
Qian Cai <cai@redhat.com>,
Vincent Donnefort <vincent.donnefort@arm.com>,
Tejun Heo <tj@kernel.org>
Subject: Re: [PATCH 00/10] workqueue: break affinity initiatively
Date: Tue, 15 Dec 2020 09:49:14 +0100 [thread overview]
Message-ID: <20201215084914.GD3040@hirez.programming.kicks-ass.net> (raw)
In-Reply-To: <CAJhGHyA=8vbamdFKwPGFHtL4iObJ929DR+iasVhmODV-u5UNfw@mail.gmail.com>
On Tue, Dec 15, 2020 at 04:14:26PM +0800, Lai Jiangshan wrote:
> On Tue, Dec 15, 2020 at 3:50 PM Peter Zijlstra <peterz@infradead.org> wrote:
> >
> > On Tue, Dec 15, 2020 at 01:44:53PM +0800, Lai Jiangshan wrote:
> > > I don't know how the scheduler distinguishes all these
> > > different cases under the "new assumption".
> >
> > The special case is:
> >
> > (p->flags & PF_KTHREAD) && p->nr_cpus_allowed == 1
> >
> >
>
> So unbound per-node workers can possibly match this test. So there is code
> needed to handle for unbound workers/pools which is done by this patchset.
Curious; how could a per-node worker match this? Only if the node is a
single CPU, or otherwise too?
> Is this the code of is_per_cpu_kthread()? I think I should have also
> used this function in workqueue and don't break affinity for unbound
> workers have more than 1 cpu.
Yes, that function captures it. If you want to use it, feel free to move
it to include/linux/sched.h.
This class of threads is 'special', since it needs to violate the
regular hotplug rules, and migrate_disable() made it just this little
bit more special. It basically comes down to how we need certain per-cpu
kthreads to run on a CPU while it's brought up, before userspace is
allowed on, and similarly they need to run on the CPU after userspace is
no longer allowed on in order to bring it down.
(IOW, they must be allowed to violate the active mask)
Due to migrate_disable() we had to move the migration code from the very
last cpu-down stage, to earlier. This in turn brought the expectation
(which is normally met) that per-cpu kthreads will stop/park or
otherwise make themselves scarce when the CPU goes down. We can no
longer force migrate them.
Workqueues are the sole exception to that, they've got some really
'dodgy' hotplug behaviour.
next prev parent reply other threads:[~2020-12-15 8:50 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-12-14 15:54 [PATCH 00/10] workqueue: break affinity initiatively Lai Jiangshan
2020-12-14 15:54 ` [PATCH 01/10] workqueue: restore unbound_workers' cpumask correctly Lai Jiangshan
2020-12-14 15:54 ` [PATCH 02/10] workqueue: use cpu_possible_mask instead of cpu_active_mask to break affinity Lai Jiangshan
2020-12-14 17:25 ` Peter Zijlstra
2020-12-15 8:33 ` Lai Jiangshan
2020-12-15 8:40 ` Peter Zijlstra
2020-12-16 14:32 ` Tejun Heo
2020-12-14 15:54 ` [PATCH 03/10] workqueue: Manually break affinity on pool detachment Lai Jiangshan
2020-12-14 15:54 ` [PATCH 04/10] workqueue: don't set the worker's cpumask when kthread_bind_mask() Lai Jiangshan
2020-12-16 14:39 ` Tejun Heo
2020-12-14 15:54 ` [PATCH 05/10] workqueue: introduce wq_online_cpumask Lai Jiangshan
2020-12-14 15:54 ` [PATCH 06/10] workqueue: use wq_online_cpumask in restore_unbound_workers_cpumask() Lai Jiangshan
2020-12-14 15:54 ` [PATCH 07/10] workqueue: Manually break affinity on hotplug for unbound pool Lai Jiangshan
2020-12-16 14:50 ` Tejun Heo
2020-12-14 15:54 ` [PATCH 08/10] workqueue: reorganize workqueue_online_cpu() Lai Jiangshan
2020-12-14 15:54 ` [PATCH 09/10] workqueue: reorganize workqueue_offline_cpu() unbind_workers() Lai Jiangshan
2020-12-14 15:54 ` [PATCH 10/10] workqueue: Fix affinity of kworkers when attaching into pool Lai Jiangshan
2020-12-15 15:03 ` Valentin Schneider
2020-12-14 17:36 ` [PATCH 00/10] workqueue: break affinity initiatively Peter Zijlstra
2020-12-15 5:44 ` Lai Jiangshan
2020-12-15 7:50 ` Peter Zijlstra
2020-12-15 8:14 ` Lai Jiangshan
2020-12-15 8:49 ` Peter Zijlstra [this message]
2020-12-15 9:46 ` Lai Jiangshan
2020-12-16 14:30 ` Tejun Heo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201215084914.GD3040@hirez.programming.kicks-ass.net \
--to=peterz@infradead.org \
--cc=cai@redhat.com \
--cc=hdanton@sina.com \
--cc=jiangshanlai@gmail.com \
--cc=laijs@linux.alibaba.com \
--cc=linux-kernel@vger.kernel.org \
--cc=tj@kernel.org \
--cc=valentin.schneider@arm.com \
--cc=vincent.donnefort@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.