All of lore.kernel.org
 help / color / mirror / Atom feed
From: Luis Chamberlain <mcgrof@kernel.org>
To: David Laight <David.Laight@aculab.com>
Cc: "tj@kernel.org" <tj@kernel.org>,
	"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"minchan@kernel.org" <minchan@kernel.org>,
	"jeyu@kernel.org" <jeyu@kernel.org>,
	"shuah@kernel.org" <shuah@kernel.org>,
	"rdunlap@infradead.org" <rdunlap@infradead.org>,
	"rafael@kernel.org" <rafael@kernel.org>,
	"masahiroy@kernel.org" <masahiroy@kernel.org>,
	"ndesaulniers@google.com" <ndesaulniers@google.com>,
	"yzaikin@google.com" <yzaikin@google.com>,
	"nathan@kernel.org" <nathan@kernel.org>,
	"ojeda@kernel.org" <ojeda@kernel.org>,
	"vitor@massaru.org" <vitor@massaru.org>,
	"elver@google.com" <elver@google.com>,
	"jarkko@kernel.org" <jarkko@kernel.org>,
	"glider@google.com" <glider@google.com>,
	"rf@opensource.cirrus.com" <rf@opensource.cirrus.com>,
	"stephen@networkplumber.org" <stephen@networkplumber.org>,
	"bvanassche@acm.org" <bvanassche@acm.org>,
	"jolsa@kernel.org" <jolsa@kernel.org>,
	"andriy.shevchenko@linux.intel.com" 
	<andriy.shevchenko@linux.intel.com>,
	"trishalfonso@google.com" <trishalfonso@google.com>,
	"andreyknvl@gmail.com" <andreyknvl@gmail.com>,
	"jikos@kernel.org" <jikos@kernel.org>,
	"mbenes@suse.com" <mbenes@suse.com>,
	"ngupta@vflare.org" <ngupta@vflare.org>,
	"sergey.senozhatsky.work@gmail.com" 
	<sergey.senozhatsky.work@gmail.com>,
	"reinette.chatre@intel.com" <reinette.chatre@intel.com>,
	"fenghua.yu@intel.com" <fenghua.yu@intel.com>,
	"bp@alien8.de" <bp@alien8.de>, "x86@kernel.org" <x86@kernel.org>,
	"hpa@zytor.com" <hpa@zytor.com>,
	"lizefan.x@bytedance.com" <lizefan.x@bytedance.com>,
	"hannes@cmpxchg.org" <hannes@cmpxchg.org>,
	"daniel.vetter@ffwll.ch" <daniel.vetter@ffwll.ch>,
	"bhelgaas@google.com" <bhelgaas@google.com>,
	"kw@linux.com" <kw@linux.com>,
	"dan.j.williams@intel.com" <dan.j.williams@intel.com>,
	"senozhatsky@chromium.org" <senozhatsky@chromium.org>,
	"hch@lst.de" <hch@lst.de>, "joe@perches.com" <joe@perches.com>,
	"hkallweit1@gmail.com" <hkallweit1@gmail.com>,
	"axboe@kernel.dk" <axboe@kernel.dk>,
	"jpoimboe@redhat.com" <jpoimboe@redhat.com>,
	"tglx@linutronix.de" <tglx@linutronix.de>,
	"keescook@chromium.org" <keescook@chromium.org>,
	"rostedt@goodmis.org" <rostedt@goodmis.org>,
	"peterz@infradead.org" <peterz@infradead.org>,
	"linux-spdx@vger.kernel.org" <linux-spdx@vger.kernel.org>,
	"linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>,
	"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
	"linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
	"linux-kselftest@vger.kernel.org"
	<linux-kselftest@vger.kernel.org>,
	"cgroups@vger.kernel.org" <cgroups@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"copyleft-next@lists.fedorahosted.org" 
	<copyleft-next@lists.fedorahosted.org>,
	Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp>
Subject: Re: [PATCH v7 09/12] sysfs: fix deadlock race with module removal
Date: Tue, 21 Sep 2021 08:48:49 -0700	[thread overview]
Message-ID: <CAB=NE6WjupsJFwsj94sC_j3gcYn2Qo0sx1=tMv=WUZ83jq_DFw@mail.gmail.com> (raw)
In-Reply-To: <c70dcb03e27e43c5b5311e184357df39@AcuMS.aculab.com>

On Tue, Sep 21, 2021 at 1:24 AM David Laight <David.Laight@aculab.com> wrote:
>
> From: Luis Chamberlain
> > Sent: 17 September 2021 20:47
> >
> > When sysfs attributes use a lock also used on module removal we can
> > race to deadlock. This happens when for instance a sysfs file on
> > a driver is used, then at the same time we have module removal call
> > trigger. The module removal call code holds a lock, and then the sysfs
> > file entry waits for the same lock. While holding the lock the module
> > removal tries to remove the sysfs entries, but these cannot be removed
> > yet as one is waiting for a lock. This won't complete as the lock is
> > already held. Likewise module removal cannot complete, and so we deadlock.
>
> Isn't the real problem the race between a sysfs file action and the
> removal of the sysfs node?

Nope, that is taken care of by kernfs.

> This isn't really related to module unload - except that may
> well remove some sysfs nodes.

Nope, the issue is a deadlock that can happen due to a shared lock on
module removal and a driver sysfs operation.

> This is the same problem as removing any other kind of driver callback.
> There are three basic solutions:
> 1) Use a global lock - not usually useful.
> 2) Have the remove call sleep until any callbacks are complete.
> 3) Have the remove just request removal and have a final
>    callback (from a different context).

Kernfs already does a sort of combination of 1) and 2) but 1) is using
atomic reference counts.

> If the remove can sleep (as in 2) then there is a requirement
> on the driver code to not hold any locks across the 'remove'
> that can be acquired during the callbacks.

And this is the part that kernfs has no control over since the removal
and sysfs operation are implementation specific.

> Now, for sysfs, you probably only want to sleep the remove code
> while a read/write is in progress - not just because the node
> is open.
> That probably requires marking an open node 'invalid' and
> deferring delete to close.

This is already done by kernfs.

> None of this requires a reference count on the module.

You are missing the point to the other aspect of the try_module_get(),
it lets you also check if module exit has been entered. By using
try_module_get() you let the module exit trump proceeding with an
operation, therefore also preventing any potential use of a shared
lock on module exit and the driver specific sysfs operation.

  Luis

WARNING: multiple messages have this Message-ID (diff)
From: Luis Chamberlain <mcgrof-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
To: David Laight <David.Laight-JxhZ9S5GRejQT0dZR+AlfA@public.gmane.org>
Cc: "tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org"
	<tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
	"gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org"
	<gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org>,
	"akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org"
	<akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>,
	"minchan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org"
	<minchan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
	"jeyu-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org"
	<jeyu-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
	"shuah-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org"
	<shuah-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
	"rdunlap-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org"
	<rdunlap-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>,
	"rafael-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org"
	<rafael-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
	"masahiroy-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org"
	<masahiroy-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
	"ndesaulniers-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org"
	<ndesaulniers-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
	"yzaikin-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org"
	<yzaikin-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
	"nathan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org"
	<nathan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
	"ojeda-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org"
	<ojeda-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
	"vitor-HdruG0TBZYlAfugRpC6u6w@public.gmane.org"
	<vitor-HdruG0TBZYlAfugRpC6u6w@public.gmane.org>,
	"elver-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org"
	<elver-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
	"jarkko-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org"
	<jarkko-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
	"glider-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org"
	<glider-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
	rf-yzvPICuk2AB0ssr/7FiavQ@public.gmane.org
Subject: Re: [PATCH v7 09/12] sysfs: fix deadlock race with module removal
Date: Tue, 21 Sep 2021 08:48:49 -0700	[thread overview]
Message-ID: <CAB=NE6WjupsJFwsj94sC_j3gcYn2Qo0sx1=tMv=WUZ83jq_DFw@mail.gmail.com> (raw)
In-Reply-To: <c70dcb03e27e43c5b5311e184357df39-1XygrNkDbNvwg4NCKwmqgw@public.gmane.org>

On Tue, Sep 21, 2021 at 1:24 AM David Laight <David.Laight-JxhZ9S5GRejQT0dZR+AlfA@public.gmane.org> wrote:
>
> From: Luis Chamberlain
> > Sent: 17 September 2021 20:47
> >
> > When sysfs attributes use a lock also used on module removal we can
> > race to deadlock. This happens when for instance a sysfs file on
> > a driver is used, then at the same time we have module removal call
> > trigger. The module removal call code holds a lock, and then the sysfs
> > file entry waits for the same lock. While holding the lock the module
> > removal tries to remove the sysfs entries, but these cannot be removed
> > yet as one is waiting for a lock. This won't complete as the lock is
> > already held. Likewise module removal cannot complete, and so we deadlock.
>
> Isn't the real problem the race between a sysfs file action and the
> removal of the sysfs node?

Nope, that is taken care of by kernfs.

> This isn't really related to module unload - except that may
> well remove some sysfs nodes.

Nope, the issue is a deadlock that can happen due to a shared lock on
module removal and a driver sysfs operation.

> This is the same problem as removing any other kind of driver callback.
> There are three basic solutions:
> 1) Use a global lock - not usually useful.
> 2) Have the remove call sleep until any callbacks are complete.
> 3) Have the remove just request removal and have a final
>    callback (from a different context).

Kernfs already does a sort of combination of 1) and 2) but 1) is using
atomic reference counts.

> If the remove can sleep (as in 2) then there is a requirement
> on the driver code to not hold any locks across the 'remove'
> that can be acquired during the callbacks.

And this is the part that kernfs has no control over since the removal
and sysfs operation are implementation specific.

> Now, for sysfs, you probably only want to sleep the remove code
> while a read/write is in progress - not just because the node
> is open.
> That probably requires marking an open node 'invalid' and
> deferring delete to close.

This is already done by kernfs.

> None of this requires a reference count on the module.

You are missing the point to the other aspect of the try_module_get(),
it lets you also check if module exit has been entered. By using
try_module_get() you let the module exit trump proceeding with an
operation, therefore also preventing any potential use of a shared
lock on module exit and the driver specific sysfs operation.

  Luis

       reply	other threads:[~2021-09-21 15:49 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <20210917194709.3562413-1-mcgrof@kernel.org>
     [not found] ` <20210917194709.3562413-10-mcgrof@kernel.org>
     [not found]   ` <c70dcb03e27e43c5b5311e184357df39@AcuMS.aculab.com>
2021-09-21 15:48     ` Luis Chamberlain [this message]
2021-09-21 15:48       ` [PATCH v7 09/12] sysfs: fix deadlock race with module removal Luis Chamberlain
     [not found] <20210918050430.3671227-1-mcgrof@kernel.org>
2021-09-18  5:04 ` Luis Chamberlain
2021-09-18  5:04   ` Luis Chamberlain
2021-09-20 17:53   ` Tejun Heo
2021-09-20 17:53     ` Tejun Heo
2021-09-20 19:15     ` Luis Chamberlain
2021-09-20 19:15       ` Luis Chamberlain
2021-09-20 19:22       ` Tejun Heo
2021-09-20 19:22         ` Tejun Heo
2021-09-20 19:38         ` Luis Chamberlain
2021-09-20 19:38           ` Luis Chamberlain
2021-09-20 20:52   ` Dan Williams
2021-09-20 20:52     ` Dan Williams
2021-09-20 21:17     ` Luis Chamberlain
2021-09-20 21:17       ` Luis Chamberlain
2021-09-20 21:55       ` Dan Williams
2021-09-20 21:55         ` Dan Williams
2021-09-21  0:03         ` Luis Chamberlain
2021-09-21  0:03           ` Luis Chamberlain
2021-09-20 21:36   ` Bart Van Assche
2021-09-20 21:36     ` Bart Van Assche
2021-09-20 21:43     ` Luis Chamberlain
2021-09-20 21:43       ` Luis Chamberlain

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAB=NE6WjupsJFwsj94sC_j3gcYn2Qo0sx1=tMv=WUZ83jq_DFw@mail.gmail.com' \
    --to=mcgrof@kernel.org \
    --cc=David.Laight@aculab.com \
    --cc=akpm@linux-foundation.org \
    --cc=andreyknvl@gmail.com \
    --cc=andriy.shevchenko@linux.intel.com \
    --cc=axboe@kernel.dk \
    --cc=bhelgaas@google.com \
    --cc=bp@alien8.de \
    --cc=bvanassche@acm.org \
    --cc=cgroups@vger.kernel.org \
    --cc=copyleft-next@lists.fedorahosted.org \
    --cc=dan.j.williams@intel.com \
    --cc=daniel.vetter@ffwll.ch \
    --cc=elver@google.com \
    --cc=fenghua.yu@intel.com \
    --cc=glider@google.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=hannes@cmpxchg.org \
    --cc=hch@lst.de \
    --cc=hkallweit1@gmail.com \
    --cc=hpa@zytor.com \
    --cc=jarkko@kernel.org \
    --cc=jeyu@kernel.org \
    --cc=jikos@kernel.org \
    --cc=joe@perches.com \
    --cc=jolsa@kernel.org \
    --cc=jpoimboe@redhat.com \
    --cc=keescook@chromium.org \
    --cc=kw@linux.com \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=linux-spdx@vger.kernel.org \
    --cc=lizefan.x@bytedance.com \
    --cc=masahiroy@kernel.org \
    --cc=mbenes@suse.com \
    --cc=minchan@kernel.org \
    --cc=nathan@kernel.org \
    --cc=ndesaulniers@google.com \
    --cc=ngupta@vflare.org \
    --cc=ojeda@kernel.org \
    --cc=penguin-kernel@i-love.sakura.ne.jp \
    --cc=peterz@infradead.org \
    --cc=rafael@kernel.org \
    --cc=rdunlap@infradead.org \
    --cc=reinette.chatre@intel.com \
    --cc=rf@opensource.cirrus.com \
    --cc=rostedt@goodmis.org \
    --cc=senozhatsky@chromium.org \
    --cc=sergey.senozhatsky.work@gmail.com \
    --cc=shuah@kernel.org \
    --cc=stephen@networkplumber.org \
    --cc=tglx@linutronix.de \
    --cc=tj@kernel.org \
    --cc=trishalfonso@google.com \
    --cc=vitor@massaru.org \
    --cc=x86@kernel.org \
    --cc=yzaikin@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.