From: Parav Pandit <parav@mellanox.com>
To: Alex Williamson <alex.williamson@redhat.com>,
Kirti Wankhede <kwankhede@nvidia.com>
Cc: "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: RE: [PATCH 7/8] vfio/mdev: Fix aborting mdev child device removal if one fails
Date: Mon, 25 Mar 2019 21:36:42 +0000 [thread overview]
Message-ID: <VI1PR0501MB2271F32EF956C838A19D460CD15E0@VI1PR0501MB2271.eurprd05.prod.outlook.com> (raw)
In-Reply-To: <20190325144935.0696668a@x1.home>
> -----Original Message-----
> From: Alex Williamson <alex.williamson@redhat.com>
> Sent: Monday, March 25, 2019 3:50 PM
> To: Kirti Wankhede <kwankhede@nvidia.com>
> Cc: Parav Pandit <parav@mellanox.com>; kvm@vger.kernel.org; linux-
> kernel@vger.kernel.org
> Subject: Re: [PATCH 7/8] vfio/mdev: Fix aborting mdev child device removal if
> one fails
>
> On Tue, 26 Mar 2019 01:05:34 +0530
> Kirti Wankhede <kwankhede@nvidia.com> wrote:
>
> > On 3/23/2019 4:50 AM, Parav Pandit wrote:
> > > device_for_each_child() stops executing callback function for
> > > remaining child devices, if callback hits an error.
> > > Each child mdev device is independent of each other.
> > > While unregistering parent device, mdev core must remove all child
> > > mdev devices.
> > > Therefore, mdev_device_remove_cb() always returns success so that
> > > device_for_each_child doesn't abort if one child removal hits error.
> > >
> >
> > When unregistering parent device, force_remove is set to true amd
> > mdev_device_remove_ops() always returns success.
>
> Can we know that? mdev_device_remove() doesn't guarantee to return
> zero.
>
> > > While at it, improve remove and unregister functions for below
> simplicity.
> > >
> > > There isn't need to pass forced flag pointer during mdev parent
> > > removal which invokes mdev_device_remove().
> >
> > There is a need to pass the flag, pasting here the comment above
> > mdev_device_remove_ops() which explains why the flag is needed:
> >
> > /*
> > * mdev_device_remove_ops gets called from sysfs's 'remove' and when
> > parent
> > * device is being unregistered from mdev device framework.
> > * - 'force_remove' is set to 'false' when called from sysfs's 'remove'
> > which
> > * indicates that if the mdev device is active, used by VMM or userspace
> > * application, vendor driver could return error then don't remove the
> > device.
> > * - 'force_remove' is set to 'true' when called from
> > mdev_unregister_device()
> > * which indicate that parent device is being removed from mdev device
> > * framework so remove mdev device forcefully.
> > */
>
> I don't see that this changes the force behavior, it's simply noting that in
> order to continue the device_for_each_child() iterator, we need to return
> zero, regardless of what mdev_device_remove() returns, and the parent
> remove path is the only caller of mdev_device_remove_cb(), so we can
> assume force = true when calling mdev_device_remove(). Aside from maybe
> a WARN_ON if mdev_device_remove() returns non-zero, that much looks
> reasonable to me.
>
> > So simplify the flow.
> > >
> > > mdev_device_remove() is called from two paths.
> > > 1. mdev_unregister_driver()
> > > mdev_device_remove_cb()
> > > mdev_device_remove()
> > > 2. remove_store()
> > > mdev_device_remove()
> > >
> > > When device is removed by user using remote_store(), device under
> > > removal is mdev device.
> > > When device is removed during parent device removal using generic
> > > child iterator, mdev check is already done using dev_is_mdev().
> > >
> > > Hence, remove the unnecessary loop in mdev_device_remove().
>
> I don't think knowing the device type is the only reason for this loop though.
> Both paths you mention above can race with each other, so we need to
> serialize them and pick a winner. The mdev_list_lock allows us to do that.
> Additionally...
>
> > >
> > > Fixes: 7b96953bc640 ("vfio: Mediated device Core driver")
> > > Signed-off-by: Parav Pandit <parav@mellanox.com>
> > > ---
> > > drivers/vfio/mdev/mdev_core.c | 24 +++++-------------------
> > > 1 file changed, 5 insertions(+), 19 deletions(-)
> > >
> > > diff --git a/drivers/vfio/mdev/mdev_core.c
> > > b/drivers/vfio/mdev/mdev_core.c index ab05464..944a058 100644
> > > --- a/drivers/vfio/mdev/mdev_core.c
> > > +++ b/drivers/vfio/mdev/mdev_core.c
> > > @@ -150,10 +150,10 @@ static int mdev_device_remove_ops(struct
> > > mdev_device *mdev, bool force_remove)
> > >
> > > static int mdev_device_remove_cb(struct device *dev, void *data) {
> > > - if (!dev_is_mdev(dev))
> > > - return 0;
> > > + if (dev_is_mdev(dev))
> > > + mdev_device_remove(dev, true);
> > >
> > > - return mdev_device_remove(dev, data ? *(bool *)data : true);
> > > + return 0;
> > > }
> > >
> > > /*
> > > @@ -241,7 +241,6 @@ int mdev_register_device(struct device *dev,
> > > const struct mdev_parent_ops *ops) void
> > > mdev_unregister_device(struct device *dev) {
> > > struct mdev_parent *parent;
> > > - bool force_remove = true;
> > >
> > > mutex_lock(&parent_list_lock);
> > > parent = __find_parent_device(dev); @@ -255,8 +254,7 @@ void
> > > mdev_unregister_device(struct device *dev)
> > > list_del(&parent->next);
> > > class_compat_remove_link(mdev_bus_compat_class, dev, NULL);
> > >
> > > - device_for_each_child(dev, (void *)&force_remove,
> > > - mdev_device_remove_cb);
> > > + device_for_each_child(dev, NULL, mdev_device_remove_cb);
> > >
> > > parent_remove_sysfs_files(parent);
> > >
> > > @@ -346,24 +344,12 @@ int mdev_device_create(struct kobject *kobj,
> > > struct device *dev, uuid_le uuid)
> > >
> > > int mdev_device_remove(struct device *dev, bool force_remove) {
> > > - struct mdev_device *mdev, *tmp;
> > > + struct mdev_device *mdev;
> > > struct mdev_parent *parent;
> > > struct mdev_type *type;
> > > int ret;
> > >
> > > mdev = to_mdev_device(dev);
> > > -
> > > - mutex_lock(&mdev_list_lock);
>
> Acquiring the lock is removed, but...
>
Crap. Missed the lower part.
> > > - list_for_each_entry(tmp, &mdev_list, next) {
> > > - if (tmp == mdev)
> > > - break;
> > > - }
> > > -
> > > - if (tmp != mdev) {
> > > - mutex_unlock(&mdev_list_lock);
> > > - return -ENODEV;
> > > - }
> > > -
> > > if (!mdev->active) {
> > > mutex_unlock(&mdev_list_lock);
> > > return -EAGAIN;
> > >
>
> We still release it in this path and the code below here. If we don't find the
> device on the list under lock, then we're working with a stale device and
> playing with the 'active' flag of that device outside of any sort of mutual
> exclusion is racy. Thanks,
Subsequent patch makes the order sane.
I think I should merge this change with patch-8 in the series.
next prev parent reply other threads:[~2019-03-25 21:36 UTC|newest]
Thread overview: 49+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-03-22 23:20 [PATCH 0/8] vfio/mdev: Improve vfio/mdev core module Parav Pandit
2019-03-22 23:20 ` [PATCH 1/8] vfio/mdev: Fix to not do put_device on device_register failure Parav Pandit
2019-03-25 11:48 ` Maxim Levitsky
2019-03-25 18:17 ` Kirti Wankhede
2019-03-25 19:21 ` Alex Williamson
2019-03-25 21:11 ` Parav Pandit
2019-03-22 23:20 ` [PATCH 2/8] vfio/mdev: Avoid release parent reference during error path Parav Pandit
2019-03-25 11:49 ` Maxim Levitsky
2019-03-25 18:27 ` Kirti Wankhede
2019-03-22 23:20 ` [PATCH 3/8] vfio/mdev: Removed unused kref Parav Pandit
2019-03-25 11:50 ` Maxim Levitsky
2019-03-25 18:41 ` Kirti Wankhede
2019-03-22 23:20 ` [PATCH 4/8] vfio/mdev: Drop redundant extern for exported symbols Parav Pandit
2019-03-25 11:56 ` Maxim Levitsky
2019-03-25 19:07 ` Kirti Wankhede
2019-03-25 19:49 ` Alex Williamson
2019-03-25 21:27 ` Parav Pandit
2019-03-22 23:20 ` [PATCH 5/8] vfio/mdev: Avoid masking error code to EBUSY Parav Pandit
2019-03-25 11:57 ` Maxim Levitsky
2019-03-25 19:18 ` Kirti Wankhede
2019-03-25 21:29 ` Parav Pandit
2019-03-22 23:20 ` [PATCH 6/8] vfio/mdev: Follow correct remove sequence Parav Pandit
2019-03-25 11:58 ` Maxim Levitsky
2019-03-25 20:20 ` Alex Williamson
2019-03-25 21:31 ` Parav Pandit
2019-03-22 23:20 ` [PATCH 7/8] vfio/mdev: Fix aborting mdev child device removal if one fails Parav Pandit
2019-03-25 11:58 ` Maxim Levitsky
2019-03-25 19:35 ` Kirti Wankhede
2019-03-25 20:49 ` Alex Williamson
2019-03-25 21:36 ` Parav Pandit [this message]
2019-03-25 21:52 ` Alex Williamson
2019-03-25 22:07 ` Parav Pandit
2019-03-22 23:20 ` [PATCH 8/8] vfio/mdev: Improve the create/remove sequence Parav Pandit
2019-03-25 13:24 ` Maxim Levitsky
2019-03-25 21:42 ` Parav Pandit
2019-03-25 23:18 ` Alex Williamson
2019-03-25 23:34 ` Parav Pandit
2019-03-26 0:05 ` Alex Williamson
2019-03-26 1:43 ` Parav Pandit
2019-03-26 2:16 ` Alex Williamson
2019-03-26 3:19 ` Parav Pandit
2019-03-26 5:53 ` Parav Pandit
2019-03-26 15:21 ` Alex Williamson
2019-03-26 7:06 ` Kirti Wankhede
2019-03-26 15:26 ` Alex Williamson
2019-03-27 3:19 ` Parav Pandit
2019-03-26 15:30 ` Parav Pandit
2019-03-28 17:20 ` Kirti Wankhede
2019-03-29 14:49 ` Alex Williamson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=VI1PR0501MB2271F32EF956C838A19D460CD15E0@VI1PR0501MB2271.eurprd05.prod.outlook.com \
--to=parav@mellanox.com \
--cc=alex.williamson@redhat.com \
--cc=kvm@vger.kernel.org \
--cc=kwankhede@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).