LKML Archive on lore.kernel.org
 help / color / Atom feed
From: "Paul E. McKenney" <paulmck@kernel.org>
To: Hillf Danton <hdanton@sina.com>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>,
	linux-kernel@vger.kernel.org,
	Valentin Schneider <valentin.schneider@arm.com>,
	Peter Zijlstra <peterz@infradead.org>, Qian Cai <cai@redhat.com>,
	Vincent Donnefort <vincent.donnefort@arm.com>,
	Lai Jiangshan <laijs@linux.alibaba.com>,
	Tejun Heo <tj@kernel.org>
Subject: Re: [PATCH -tip V2 00/10] workqueue: break affinity initiatively
Date: Sat, 26 Dec 2020 06:52:39 -0800
Message-ID: <20201226145239.GJ2657@paulmck-ThinkPad-P72> (raw)
In-Reply-To: <20201226103421.6616-1-hdanton@sina.com>

On Sat, Dec 26, 2020 at 06:34:21PM +0800, Hillf Danton wrote:
> On Wed, 23 Dec 2020 11:49:51 -0800 "Paul E. McKenney" wrote:
> >On Sat, Dec 19, 2020 at 01:09:09AM +0800, Lai Jiangshan wrote:
> >> From: Lai Jiangshan <laijs@linux.alibaba.com>
> >> 
> >> 06249738a41a ("workqueue: Manually break affinity on hotplug")
> >> said that scheduler will not force break affinity for us.
> >> 
> >> But workqueue highly depends on the old behavior. Many parts of the codes
> >> relies on it, 06249738a41a ("workqueue: Manually break affinity on hotplug")
> >> is not enough to change it, and the commit has flaws in itself too.
> >> 
> >> It doesn't handle for worker detachment.
> >> It doesn't handle for worker attachement, mainly worker creation
> >>   which is handled by Valentin Schneider's patch [1].
> >> It doesn't handle for unbound workers which might be possible
> >> per-cpu-kthread.
> >> 
> >> We need to thoroughly update the way workqueue handles affinity
> >> in cpu hot[un]plug, what is this patchset intends to do and
> >> replace the Valentin Schneider's patch [1].  The equivalent patch
> >> is patch 10.
> >> 
> >> Patch 1 fixes a flaw reported by Hillf Danton <hdanton@sina.com>.
> >> I have to include this fix because later patches depends on it.
> >> 
> >> The patchset is based on tip/master rather than workqueue tree,
> >> because the patchset is a complement for 06249738a41a ("workqueue:
> >> Manually break affinity on hotplug") which is only in tip/master by now.
> >> 
> >> And TJ acked to route the series through tip.
> >> 
> >> Changed from V1:
> >> 	Add TJ's acked-by for the whole patchset
> >> 
> >> 	Add more words to the comments and the changelog, mainly derived
> >> 	from discussion with Peter.
> >> 
> >> 	Update the comments as TJ suggested.
> >> 	
> >> 	Update a line of code as Valentin suggested.
> >> 
> >> 	Add Valentin's ack for patch 10 because "Seems alright to me." and
> >> 	add Valentin's comments to the changelog which is integral.
> >> 
> >> [1]: https://lore.kernel.org/r/ff62e3ee994efb3620177bf7b19fab16f4866845.camel@redhat.com
> >> [V1 patcheset]: https://lore.kernel.org/lkml/20201214155457.3430-1-jiangshanlai@gmail.com/
> >> 
> >> Cc: Hillf Danton <hdanton@sina.com>
> >> Cc: Valentin Schneider <valentin.schneider@arm.com>
> >> Cc: Qian Cai <cai@redhat.com>
> >> Cc: Peter Zijlstra <peterz@infradead.org>
> >> Cc: Vincent Donnefort <vincent.donnefort@arm.com>
> >> Cc: Tejun Heo <tj@kernel.org>
> >
> >And rcutorture hits this, so thank you for the fix!
> 
> Can you please specify a bit what you encountered in rcutorture
> before this patchset? You know we cant have a correct estimation
> of the fix diameter without your help.

It triggers the following in sched_cpu_dying() in kernel/sched/core.c,
exactly the same as for Lai Jiangshan:

	BUG_ON(rq->nr_running != 1 || rq_has_pinned_tasks(rq))

Which is in fact the "this" in my earlier "rcutorture hits this".  ;-)

							Thanx, Paul

> >Tested-by: Paul E. McKenney <paulmck@kernel.org>
> >
> >> Lai Jiangshan (10):
> >>   workqueue: restore unbound_workers' cpumask correctly
> >>   workqueue: use cpu_possible_mask instead of cpu_active_mask to break
> >>     affinity
> >>   workqueue: Manually break affinity on pool detachment
> >>   workqueue: don't set the worker's cpumask when kthread_bind_mask()
> >>   workqueue: introduce wq_online_cpumask
> >>   workqueue: use wq_online_cpumask in restore_unbound_workers_cpumask()
> >>   workqueue: Manually break affinity on hotplug for unbound pool
> >>   workqueue: reorganize workqueue_online_cpu()
> >>   workqueue: reorganize workqueue_offline_cpu() unbind_workers()
> >>   workqueue: Fix affinity of kworkers when attaching into pool
> >> 
> >>  kernel/workqueue.c | 214 ++++++++++++++++++++++++++++-----------------
> >>  1 file changed, 132 insertions(+), 82 deletions(-)
> >> 
> >> -- 
> >> 2.19.1.6.gb485710b

  parent reply index

Thread overview: 23+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-12-18 17:09 Lai Jiangshan
2020-12-18 17:09 ` [PATCH -tip V2 01/10] workqueue: restore unbound_workers' cpumask correctly Lai Jiangshan
2020-12-18 17:09 ` [PATCH -tip V2 02/10] workqueue: use cpu_possible_mask instead of cpu_active_mask to break affinity Lai Jiangshan
2020-12-18 17:09 ` [PATCH -tip V2 03/10] workqueue: Manually break affinity on pool detachment Lai Jiangshan
2020-12-18 17:09 ` [PATCH -tip V2 04/10] workqueue: don't set the worker's cpumask when kthread_bind_mask() Lai Jiangshan
2020-12-24  6:33   ` [workqueue] 6094661b16: WARNING:at_kernel/sched/core.c:#__set_cpus_allowed_ptr kernel test robot
2020-12-18 17:09 ` [PATCH -tip V2 05/10] workqueue: introduce wq_online_cpumask Lai Jiangshan
2020-12-18 17:09 ` [PATCH -tip V2 06/10] workqueue: use wq_online_cpumask in restore_unbound_workers_cpumask() Lai Jiangshan
2020-12-18 17:09 ` [PATCH -tip V2 07/10] workqueue: Manually break affinity on hotplug for unbound pool Lai Jiangshan
2020-12-18 17:09 ` [PATCH -tip V2 08/10] workqueue: reorganize workqueue_online_cpu() Lai Jiangshan
2020-12-18 17:09 ` [PATCH -tip V2 09/10] workqueue: reorganize workqueue_offline_cpu() unbind_workers() Lai Jiangshan
2020-12-18 17:09 ` [PATCH -tip V2 10/10] workqueue: Fix affinity of kworkers when attaching into pool Lai Jiangshan
2020-12-18 17:59   ` Valentin Schneider
2020-12-19  1:11     ` Lai Jiangshan
2020-12-22 21:39 ` [PATCH -tip V2 00/10] workqueue: break affinity initiatively Dexuan-Linux Cui
2020-12-23 11:32   ` Lai Jiangshan
2020-12-23 15:01   ` Lai Jiangshan
2020-12-23 20:27     ` Dexuan Cui
2020-12-23 20:39       ` Dexuan Cui
2020-12-23 19:49 ` Paul E. McKenney
     [not found] ` <20201226103421.6616-1-hdanton@sina.com>
2020-12-26 14:52   ` Paul E. McKenney [this message]
2020-12-27 14:08     ` Lai Jiangshan
2020-12-27 16:02       ` Paul E. McKenney

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201226145239.GJ2657@paulmck-ThinkPad-P72 \
    --to=paulmck@kernel.org \
    --cc=cai@redhat.com \
    --cc=hdanton@sina.com \
    --cc=jiangshanlai@gmail.com \
    --cc=laijs@linux.alibaba.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=peterz@infradead.org \
    --cc=tj@kernel.org \
    --cc=valentin.schneider@arm.com \
    --cc=vincent.donnefort@arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git
	git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git
	git clone --mirror https://lore.kernel.org/lkml/10 lkml/git/10.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git