From: Tiwei Bie <tiwei.bie@intel.com>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: mst@redhat.com, jasowang@redhat.com,
linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
virtualization@lists.linux-foundation.org,
netdev@vger.kernel.org, shahafs@mellanox.com,
rob.miller@broadcom.com, haotian.wang@sifive.com,
eperezma@redhat.com, lulu@redhat.com, parav@mellanox.com,
rdunlap@infradead.org, hch@infradead.org, jiri@mellanox.com,
hanand@xilinx.com, mhabets@solarflare.com,
maxime.coquelin@redhat.com, lingshan.zhu@intel.com,
dan.daly@intel.com, cunming.liang@intel.com,
zhihong.wang@intel.com
Subject: Re: [PATCH] vhost: introduce vDPA based backend
Date: Wed, 19 Feb 2020 10:52:38 +0800 [thread overview]
Message-ID: <20200219025217.GA971968@___> (raw)
In-Reply-To: <20200218135359.GA9608@ziepe.ca>
On Tue, Feb 18, 2020 at 09:53:59AM -0400, Jason Gunthorpe wrote:
> On Fri, Jan 31, 2020 at 11:36:51AM +0800, Tiwei Bie wrote:
>
> > +static int vhost_vdpa_alloc_minor(struct vhost_vdpa *v)
> > +{
> > + return idr_alloc(&vhost_vdpa.idr, v, 0, MINORMASK + 1,
> > + GFP_KERNEL);
> > +}
>
> Please don't use idr in new code, use xarray directly
>
> > +static int vhost_vdpa_probe(struct device *dev)
> > +{
> > + struct vdpa_device *vdpa = dev_to_vdpa(dev);
> > + const struct vdpa_config_ops *ops = vdpa->config;
> > + struct vhost_vdpa *v;
> > + struct device *d;
> > + int minor, nvqs;
> > + int r;
> > +
> > + /* Currently, we only accept the network devices. */
> > + if (ops->get_device_id(vdpa) != VIRTIO_ID_NET) {
> > + r = -ENOTSUPP;
> > + goto err;
> > + }
> > +
> > + v = kzalloc(sizeof(*v), GFP_KERNEL | __GFP_RETRY_MAYFAIL);
> > + if (!v) {
> > + r = -ENOMEM;
> > + goto err;
> > + }
> > +
> > + nvqs = VHOST_VDPA_VQ_MAX;
> > +
> > + v->vqs = kmalloc_array(nvqs, sizeof(struct vhost_virtqueue),
> > + GFP_KERNEL);
> > + if (!v->vqs) {
> > + r = -ENOMEM;
> > + goto err_alloc_vqs;
> > + }
> > +
> > + mutex_init(&v->mutex);
> > + atomic_set(&v->opened, 0);
> > +
> > + v->vdpa = vdpa;
> > + v->nvqs = nvqs;
> > + v->virtio_id = ops->get_device_id(vdpa);
> > +
> > + mutex_lock(&vhost_vdpa.mutex);
> > +
> > + minor = vhost_vdpa_alloc_minor(v);
> > + if (minor < 0) {
> > + r = minor;
> > + goto err_alloc_minor;
> > + }
> > +
> > + d = device_create(vhost_vdpa.class, NULL,
> > + MKDEV(MAJOR(vhost_vdpa.devt), minor),
> > + v, "%d", vdpa->index);
> > + if (IS_ERR(d)) {
> > + r = PTR_ERR(d);
> > + goto err_device_create;
> > + }
> > +
>
> I can't understand what this messing around with major/minor numbers
> does. Without allocating a cdev via cdev_add/etc there is only a
> single char dev in existence here. This and the stuff in
> vhost_vdpa_open() looks non-functional.
I followed the code in VFIO. Please see more details below.
>
> > +static void vhost_vdpa_remove(struct device *dev)
> > +{
> > + DEFINE_WAIT_FUNC(wait, woken_wake_function);
> > + struct vhost_vdpa *v = dev_get_drvdata(dev);
> > + int opened;
> > +
> > + add_wait_queue(&vhost_vdpa.release_q, &wait);
> > +
> > + do {
> > + opened = atomic_cmpxchg(&v->opened, 0, 1);
> > + if (!opened)
> > + break;
> > + wait_woken(&wait, TASK_UNINTERRUPTIBLE, HZ * 10);
> > + } while (1);
> > +
> > + remove_wait_queue(&vhost_vdpa.release_q, &wait);
>
> *barf* use the normal refcount pattern please
>
> read side:
>
> refcount_inc_not_zero(uses)
> //stuff
> if (refcount_dec_and_test(uses))
> complete(completer)
>
> destroy side:
> if (refcount_dec_and_test(uses))
> complete(completer)
> wait_for_completion(completer)
> // refcount now permanently == 0
>
> Use a completion in driver code
>
> > + mutex_lock(&vhost_vdpa.mutex);
> > + device_destroy(vhost_vdpa.class,
> > + MKDEV(MAJOR(vhost_vdpa.devt), v->minor));
> > + vhost_vdpa_free_minor(v->minor);
> > + mutex_unlock(&vhost_vdpa.mutex);
> > + kfree(v->vqs);
> > + kfree(v);
>
> This use after-fress vs vhost_vdpa_open prior to it setting the open
> bit. Maybe use xarray, rcu and kfree_rcu ..
>
> > +static int __init vhost_vdpa_init(void)
> > +{
> > + int r;
> > +
> > + idr_init(&vhost_vdpa.idr);
> > + mutex_init(&vhost_vdpa.mutex);
> > + init_waitqueue_head(&vhost_vdpa.release_q);
> > +
> > + /* /dev/vhost-vdpa/$vdpa_device_index */
> > + vhost_vdpa.class = class_create(THIS_MODULE, "vhost-vdpa");
> > + if (IS_ERR(vhost_vdpa.class)) {
> > + r = PTR_ERR(vhost_vdpa.class);
> > + goto err_class;
> > + }
> > +
> > + vhost_vdpa.class->devnode = vhost_vdpa_devnode;
> > +
> > + r = alloc_chrdev_region(&vhost_vdpa.devt, 0, MINORMASK + 1,
> > + "vhost-vdpa");
> > + if (r)
> > + goto err_alloc_chrdev;
> > +
> > + cdev_init(&vhost_vdpa.cdev, &vhost_vdpa_fops);
> > + r = cdev_add(&vhost_vdpa.cdev, vhost_vdpa.devt, MINORMASK + 1);
> > + if (r)
> > + goto err_cdev_add;
>
> It is very strange, is the intention to create a single global char
> dev?
No. It's to create a per-vdpa char dev named
vhost-vdpa/$vdpa_device_index in dev.
I followed the code in VFIO which creates char dev
vfio/$GROUP dynamically, e.g.:
https://github.com/torvalds/linux/blob/b1da3acc781c/drivers/vfio/vfio.c#L2164-L2180
https://github.com/torvalds/linux/blob/b1da3acc781c/drivers/vfio/vfio.c#L373-L387
https://github.com/torvalds/linux/blob/b1da3acc781c/drivers/vfio/vfio.c#L1553
Is it something unwanted?
Thanks for the review.
Regards,
Tiwei
>
> If so, why is there this:
>
> +static int vhost_vdpa_open(struct inode *inode, struct file *filep)
> +{
> + struct vhost_vdpa *v;
> + struct vhost_dev *dev;
> + struct vhost_virtqueue **vqs;
> + int nvqs, i, r, opened;
> +
> + v = vhost_vdpa_get_from_minor(iminor(inode));
>
> ?
>
> If the idea is to create a per-vdpa char dev then this stuff belongs
> in vhost_vdpa_probe(), the cdev should be part of the vhost_vdpa, and
> the above should be container_of not an idr lookup.
>
> Jason
next prev parent reply other threads:[~2020-02-19 2:53 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-31 3:36 [PATCH] vhost: introduce vDPA based backend Tiwei Bie
2020-01-31 3:56 ` Randy Dunlap
2020-01-31 5:12 ` Randy Dunlap
2020-01-31 5:54 ` Tiwei Bie
2020-01-31 5:52 ` Tiwei Bie
2020-02-04 3:30 ` Jason Wang
2020-02-04 6:01 ` Michael S. Tsirkin
2020-02-04 6:46 ` Jason Wang
2020-02-05 2:05 ` Tiwei Bie
2020-02-05 3:12 ` Jason Wang
2020-02-05 5:31 ` Michael S. Tsirkin
2020-02-05 5:50 ` Jason Wang
2020-02-05 6:30 ` Michael S. Tsirkin
2020-02-05 6:49 ` Jason Wang
2020-02-05 7:16 ` Michael S. Tsirkin
2020-02-05 7:42 ` Jason Wang
2020-02-05 9:22 ` Michael S. Tsirkin
2020-02-05 2:02 ` Tiwei Bie
2020-02-05 3:11 ` Jason Wang
2020-02-05 7:15 ` Shahaf Shuler
2020-02-05 7:50 ` Jason Wang
2020-02-05 9:23 ` Michael S. Tsirkin
2020-02-06 3:07 ` Jason Wang
2020-02-05 9:30 ` Shahaf Shuler
2020-02-05 10:33 ` Michael S. Tsirkin
2020-02-06 3:09 ` Jason Wang
2020-02-06 3:04 ` Jason Wang
2020-02-05 12:56 ` Jason Gunthorpe
2020-02-05 13:14 ` Michael S. Tsirkin
2020-02-06 3:11 ` Jason Wang
2020-02-06 3:21 ` Zhu Lingshan
2020-02-18 13:53 ` Jason Gunthorpe
2020-02-19 2:52 ` Tiwei Bie [this message]
2020-02-19 13:11 ` Jason Gunthorpe
2020-02-20 2:42 ` Tiwei Bie
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200219025217.GA971968@___ \
--to=tiwei.bie@intel.com \
--cc=cunming.liang@intel.com \
--cc=dan.daly@intel.com \
--cc=eperezma@redhat.com \
--cc=hanand@xilinx.com \
--cc=haotian.wang@sifive.com \
--cc=hch@infradead.org \
--cc=jasowang@redhat.com \
--cc=jgg@ziepe.ca \
--cc=jiri@mellanox.com \
--cc=kvm@vger.kernel.org \
--cc=lingshan.zhu@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lulu@redhat.com \
--cc=maxime.coquelin@redhat.com \
--cc=mhabets@solarflare.com \
--cc=mst@redhat.com \
--cc=netdev@vger.kernel.org \
--cc=parav@mellanox.com \
--cc=rdunlap@infradead.org \
--cc=rob.miller@broadcom.com \
--cc=shahafs@mellanox.com \
--cc=virtualization@lists.linux-foundation.org \
--cc=zhihong.wang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).