From: SeongJae Park <sj38.park@gmail.com>
To: "Jürgen Groß" <jgross@suse.com>
Cc: axboe@kernel.dk, linux-block@vger.kernel.org,
SeongJae Park <sjpark@amazon.com>,
konrad.wilk@oracle.com, pdurrant@amazon.com,
SeongJae Park <sjpark@amazon.de>,
linux-kernel@vger.kernel.org, sj38.park@gmail.com,
xen-devel@lists.xenproject.org, roger.pau@citrix.com
Subject: Re: [Xen-devel] [PATCH v11 2/6] xenbus/backend: Protect xenbus callback with lock
Date: Tue, 17 Dec 2019 18:27:38 +0100 [thread overview]
Message-ID: <20191217172738.20787-1-sj38.park@gmail.com> (raw)
In-Reply-To: <f9a601af-4413-ed1d-f7f4-89343118a2f1@suse.com>
On Tue, 17 Dec 2019 18:10:19 +0100 "Jürgen Groß" <jgross@suse.com> wrote:
> On 17.12.19 17:24, SeongJae Park wrote:
> > On Tue, 17 Dec 2019 17:13:42 +0100 "Jürgen Groß" <jgross@suse.com> wrote:
> >
> >> On 17.12.19 17:07, SeongJae Park wrote:
> >>> From: SeongJae Park <sjpark@amazon.de>
> >>>
> >>> 'reclaim_memory' callback can race with a driver code as this callback
> >>> will be called from any memory pressure detected context. To deal with
> >>> the case, this commit adds a spinlock in the 'xenbus_device'. Whenever
> >>> 'reclaim_memory' callback is called, the lock of the device which passed
> >>> to the callback as its argument is locked. Thus, drivers registering
> >>> their 'reclaim_memory' callback should protect the data that might race
> >>> with the callback with the lock by themselves.
> >>>
> >>> Signed-off-by: SeongJae Park <sjpark@amazon.de>
> >>> ---
> >>> drivers/xen/xenbus/xenbus_probe.c | 1 +
> >>> drivers/xen/xenbus/xenbus_probe_backend.c | 10 ++++++++--
> >>> include/xen/xenbus.h | 2 ++
> >>> 3 files changed, 11 insertions(+), 2 deletions(-)
> >>>
> >>> diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c
> >>> index 5b471889d723..b86393f172e6 100644
> >>> --- a/drivers/xen/xenbus/xenbus_probe.c
> >>> +++ b/drivers/xen/xenbus/xenbus_probe.c
> >>> @@ -472,6 +472,7 @@ int xenbus_probe_node(struct xen_bus_type *bus,
> >>> goto fail;
> >>>
> >>> dev_set_name(&xendev->dev, "%s", devname);
> >>> + spin_lock_init(&xendev->reclaim_lock);
> >>>
> >>> /* Register with generic device framework. */
> >>> err = device_register(&xendev->dev);
> >>> diff --git a/drivers/xen/xenbus/xenbus_probe_backend.c b/drivers/xen/xenbus/xenbus_probe_backend.c
> >>> index 7e78ebef7c54..516aa64b9967 100644
> >>> --- a/drivers/xen/xenbus/xenbus_probe_backend.c
> >>> +++ b/drivers/xen/xenbus/xenbus_probe_backend.c
> >>> @@ -251,12 +251,18 @@ static int backend_probe_and_watch(struct notifier_block *notifier,
> >>> static int backend_reclaim_memory(struct device *dev, void *data)
> >>> {
> >>> const struct xenbus_driver *drv;
> >>> + struct xenbus_device *xdev;
> >>> + unsigned long flags;
> >>>
> >>> if (!dev->driver)
> >>> return 0;
> >>> drv = to_xenbus_driver(dev->driver);
> >>> - if (drv && drv->reclaim_memory)
> >>> - drv->reclaim_memory(to_xenbus_device(dev));
> >>> + if (drv && drv->reclaim_memory) {
> >>> + xdev = to_xenbus_device(dev);
> >>> + spin_trylock_irqsave(&xdev->reclaim_lock, flags);
> >>
> >> You need spin_lock_irqsave() here. Or maybe spin_lock() would be fine,
> >> too? I can't see a reason why you'd want to disable irqs here.
> >
> > I needed to diable irq here as this is called from the memory shrinker context.
>
> Okay.
>
> >
> > Also, used 'trylock' because the 'probe()' and 'remove()' code of the driver
> > might include memory allocation. And the xen-blkback actually does. If the
> > allocation shows a memory pressure during the allocation, it will trigger this
> > shrinker callback again and then deadlock.
>
> In that case you need to either return when you didn't get the lock or
Yes, it should. Cannot believe how I posted this code. Seems I made some
terrible mistake while formatting patches. Anyway, will return if fail to
acquire the lock, in the next version.
Thanks,
SeongJae Park
>
> - when obtaining the lock during probe() and remove() set a variable
> containing the current cpu number
> - and reset that to e.g NR_CPUS before releasing the lock again
> - in the shrinker callback do trylock, and if you didn't get the lock
> test whether the cpu-variable above is set to your current cpu and
> continue only if yes; if not, redo the the trylock
>
>
> Juergen
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
next prev parent reply other threads:[~2019-12-17 17:28 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-12-17 16:07 [Xen-devel] [PATCH v11 0/6] xenbus/backend: Add a memory pressure handler callback SeongJae Park
2019-12-17 16:07 ` [Xen-devel] [PATCH v11 1/6] xenbus/backend: Add " SeongJae Park
2019-12-17 16:07 ` [Xen-devel] [PATCH v11 2/6] xenbus/backend: Protect xenbus callback with lock SeongJae Park
2019-12-17 16:13 ` Jürgen Groß
2019-12-17 16:24 ` SeongJae Park
2019-12-17 17:10 ` Jürgen Groß
2019-12-17 17:27 ` SeongJae Park [this message]
2019-12-17 16:07 ` [Xen-devel] [PATCH v11 3/6] xen/blkback: Squeeze page pools if a memory pressure is detected SeongJae Park
2019-12-17 16:07 ` [Xen-devel] [PATCH v11 4/6] xen/blkback: Protect 'reclaim_memory()' with 'reclaim_lock' SeongJae Park
2019-12-17 16:15 ` Jürgen Groß
2019-12-17 16:31 ` [Xen-devel] [PATCH 1/3] xen/blkback: Squeeze page pools if a memory pressure is detected SeongJae Park
2019-12-17 16:10 ` [Xen-devel] [PATCH v11 5/6] xen/blkback: Remove unnecessary static variable name prefixes SeongJae Park
2019-12-17 16:10 ` [Xen-devel] [PATCH v11 6/6] xen/blkback: Consistently insert one empty line between functions SeongJae Park
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191217172738.20787-1-sj38.park@gmail.com \
--to=sj38.park@gmail.com \
--cc=axboe@kernel.dk \
--cc=jgross@suse.com \
--cc=konrad.wilk@oracle.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=pdurrant@amazon.com \
--cc=roger.pau@citrix.com \
--cc=sjpark@amazon.com \
--cc=sjpark@amazon.de \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).