All of lore.kernel.org
 help / color / mirror / Atom feed
From: Imran Khan <imran.f.khan@oracle.com>
To: Tejun Heo <tj@kernel.org>, Mukesh Ojha <quic_mojha@quicinc.com>
Cc: "Michal Koutný" <mkoutny@suse.com>,
	"Xuewen Yan" <xuewen.yan94@gmail.com>,
	lizefan.x@bytedance.com, hannes@cmpxchg.org, tglx@linutronix.de,
	steven.price@arm.com, peterz@infradead.org,
	cgroups@vger.kernel.org, linux-kernel@vger.kernel.org,
	"Zhao Gongyi" <zhaogongyi@huawei.com>,
	"Zhang Qiao" <zhangqiao22@huawei.com>
Subject: Re: [PATCH cgroup/for-6.0-fixes] cgroup: Fix threadgroup_rwsem <-> cpus_read_lock() deadlock
Date: Wed, 17 Aug 2022 06:20:08 +1000	[thread overview]
Message-ID: <fa9b19c7-65db-9e94-6627-31ae205b76ee@oracle.com> (raw)
In-Reply-To: <YvrWaml3F+x9Dk+T@slm.duckdns.org>

Hello Tejun,

On 16/8/22 9:27 am, Tejun Heo wrote:
> Bringing up a CPU may involve creating new tasks which requires read-locking
> threadgroup_rwsem, so threadgroup_rwsem nests inside cpus_read_lock().
> However, cpuset's ->attach(), which may be called with thredagroup_rwsem
> write-locked, also wants to disable CPU hotplug and acquires
> cpus_read_lock(), leading to a deadlock.
> 
> Fix it by guaranteeing that ->attach() is always called with CPU hotplug
> disabled and removing cpus_read_lock() call from cpuset_attach().
> 
> Signed-off-by: Tejun Heo <tj@kernel.org>
> ---
> Hello, sorry about the delay.
> 
> So, the previous patch + the revert isn't quite correct because we sometimes
> elide both cpus_read_lock() and threadgroup_rwsem together and
> cpuset_attach() woudl end up running without CPU hotplug enabled. Can you
> please test whether this patch fixes the problem?
> 

This fixes the issue seen in my setup. As my setup is 5.4 based I used
cgroup_attach_lock/unlock(true) in the backport version of your patch.

Feel free to add my

Reviewed-and-tested-by: Imran Khan <imran.f.khan@oracle.com>

Thanks,
-- Imran

WARNING: multiple messages have this Message-ID (diff)
From: Imran Khan <imran.f.khan-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
To: Tejun Heo <tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
	Mukesh Ojha <quic_mojha-jfJNa2p1gH1BDgjK7y7TUQ@public.gmane.org>
Cc: "Michal Koutný" <mkoutny-IBi9RG/b67k@public.gmane.org>,
	"Xuewen Yan"
	<xuewen.yan94-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
	lizefan.x-EC8Uxl6Npydl57MIdRCFDg@public.gmane.org,
	hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org,
	tglx-hfZtesqFncYOwBW4kG4KsQ@public.gmane.org,
	steven.price-5wv7dgnIgG8@public.gmane.org,
	peterz-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org,
	cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	"Zhao Gongyi"
	<zhaogongyi-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>,
	"Zhang Qiao"
	<zhangqiao22-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Subject: Re: [PATCH cgroup/for-6.0-fixes] cgroup: Fix threadgroup_rwsem <-> cpus_read_lock() deadlock
Date: Wed, 17 Aug 2022 06:20:08 +1000	[thread overview]
Message-ID: <fa9b19c7-65db-9e94-6627-31ae205b76ee@oracle.com> (raw)
In-Reply-To: <YvrWaml3F+x9Dk+T-NiLfg/pYEd1N0TnZuCh8vA@public.gmane.org>

Hello Tejun,

On 16/8/22 9:27 am, Tejun Heo wrote:
> Bringing up a CPU may involve creating new tasks which requires read-locking
> threadgroup_rwsem, so threadgroup_rwsem nests inside cpus_read_lock().
> However, cpuset's ->attach(), which may be called with thredagroup_rwsem
> write-locked, also wants to disable CPU hotplug and acquires
> cpus_read_lock(), leading to a deadlock.
> 
> Fix it by guaranteeing that ->attach() is always called with CPU hotplug
> disabled and removing cpus_read_lock() call from cpuset_attach().
> 
> Signed-off-by: Tejun Heo <tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
> ---
> Hello, sorry about the delay.
> 
> So, the previous patch + the revert isn't quite correct because we sometimes
> elide both cpus_read_lock() and threadgroup_rwsem together and
> cpuset_attach() woudl end up running without CPU hotplug enabled. Can you
> please test whether this patch fixes the problem?
> 

This fixes the issue seen in my setup. As my setup is 5.4 based I used
cgroup_attach_lock/unlock(true) in the backport version of your patch.

Feel free to add my

Reviewed-and-tested-by: Imran Khan <imran.f.khan-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>

Thanks,
-- Imran

  reply	other threads:[~2022-08-16 20:21 UTC|newest]

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <8245b710-8acb-d8e6-7045-99a5f71dad4e@oracle.com>
2022-07-20  2:38 ` Query regarding deadlock involving cgroup_threadgroup_rwsem and cpu_hotplug_lock Imran Khan
2022-07-20  3:27   ` Imran Khan
2022-07-20  3:27     ` Imran Khan
2022-07-20 11:06     ` Mukesh Ojha
2022-07-20 11:06       ` Mukesh Ojha
2022-07-20 12:01       ` Mukesh Ojha
2022-07-20 12:01         ` Mukesh Ojha
2022-07-20 18:05         ` Tejun Heo
2022-07-20 18:05           ` Tejun Heo
2022-07-27 19:33           ` Tejun Heo
2022-07-27 19:33             ` Tejun Heo
2022-08-12 10:27             ` Mukesh Ojha
2022-08-12 10:27               ` Mukesh Ojha
2022-08-15  9:05               ` Michal Koutný
2022-08-15  9:25                 ` Xuewen Yan
2022-08-15  9:25                   ` Xuewen Yan
2022-08-15  9:39                   ` Michal Koutný
2022-08-15  9:39                     ` Michal Koutný
2022-08-15 10:59                     ` Mukesh Ojha
2022-08-15 10:59                       ` Mukesh Ojha
2022-08-15 23:27                       ` [PATCH cgroup/for-6.0-fixes] cgroup: Fix threadgroup_rwsem <-> cpus_read_lock() deadlock Tejun Heo
2022-08-15 23:27                         ` Tejun Heo
2022-08-16 20:20                         ` Imran Khan [this message]
2022-08-16 20:20                           ` Imran Khan
2022-08-17  6:55                         ` Xuewen Yan
2022-08-17  6:55                           ` Xuewen Yan
2022-08-17 17:40                         ` Tejun Heo
2022-08-17 17:40                           ` Tejun Heo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=fa9b19c7-65db-9e94-6627-31ae205b76ee@oracle.com \
    --to=imran.f.khan@oracle.com \
    --cc=cgroups@vger.kernel.org \
    --cc=hannes@cmpxchg.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lizefan.x@bytedance.com \
    --cc=mkoutny@suse.com \
    --cc=peterz@infradead.org \
    --cc=quic_mojha@quicinc.com \
    --cc=steven.price@arm.com \
    --cc=tglx@linutronix.de \
    --cc=tj@kernel.org \
    --cc=xuewen.yan94@gmail.com \
    --cc=zhangqiao22@huawei.com \
    --cc=zhaogongyi@huawei.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.