KVM Archive on lore.kernel.org
 help / color / Atom feed
From: Jason Gunthorpe <jgg@mellanox.com>
To: Jason Wang <jasowang@redhat.com>
Cc: "mst@redhat.com" <mst@redhat.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"virtualization@lists.linux-foundation.org" 
	<virtualization@lists.linux-foundation.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	"tiwei.bie@intel.com" <tiwei.bie@intel.com>,
	"maxime.coquelin@redhat.com" <maxime.coquelin@redhat.com>,
	"cunming.liang@intel.com" <cunming.liang@intel.com>,
	"zhihong.wang@intel.com" <zhihong.wang@intel.com>,
	"rob.miller@broadcom.com" <rob.miller@broadcom.com>,
	"xiao.w.wang@intel.com" <xiao.w.wang@intel.com>,
	"haotian.wang@sifive.com" <haotian.wang@sifive.com>,
	"lingshan.zhu@intel.com" <lingshan.zhu@intel.com>,
	"eperezma@redhat.com" <eperezma@redhat.com>,
	"lulu@redhat.com" <lulu@redhat.com>,
	Parav Pandit <parav@mellanox.com>,
	"kevin.tian@intel.com" <kevin.tian@intel.com>,
	"stefanha@redhat.com" <stefanha@redhat.com>,
	"rdunlap@infradead.org" <rdunlap@infradead.org>,
	"hch@infradead.org" <hch@infradead.org>,
	"aadam@redhat.com" <aadam@redhat.com>,
	"jakub.kicinski@netronome.com" <jakub.kicinski@netronome.com>,
	Jiri Pirko <jiri@mellanox.com>,
	Shahaf Shuler <shahafs@mellanox.com>,
	"hanand@xilinx.com" <hanand@xilinx.com>,
	"mhabets@solarflare.com" <mhabets@solarflare.com>
Subject: Re: [PATCH 4/5] virtio: introduce a vDPA based transport
Date: Fri, 17 Jan 2020 14:00:18 +0000
Message-ID: <20200117140013.GV20978@mellanox.com> (raw)
In-Reply-To: <8e8aa4b7-4948-5719-9618-e28daffba1a5@redhat.com>

On Fri, Jan 17, 2020 at 05:32:35PM +0800, Jason Wang wrote:
> > 
> > > +	const struct vdpa_config_ops *ops = vdpa->config;
> > > +	struct virtio_vdpa_device *vd_dev;
> > > +	int rc;
> > > +
> > > +	vd_dev = devm_kzalloc(dev, sizeof(*vd_dev), GFP_KERNEL);
> > > +	if (!vd_dev)
> > > +		return -ENOMEM;
> > This is not right, the struct device lifetime is controled by a kref,
> > not via devm. If you want to use a devm unwind then the unwind is
> > put_device, not devm_kfree.
> 
> I'm not sure I get the point here. The lifetime is bound to underlying vDPA
> device and devres allow to be freed before the vpda device is released. But
> I agree using devres of underlying vdpa device looks wired.

Once device_initialize is called the only way to free a struct device
is via put_device, while here you have a devm trigger that will
unconditionally do kfree on a struct device without respecting the
reference count.

reference counted memory must never be allocated with devm.

> > > +	vd_dev->vdev.dev.release = virtio_vdpa_release_dev;
> > > +	vd_dev->vdev.config = &virtio_vdpa_config_ops;
> > > +	vd_dev->vdpa = vdpa;
> > > +	INIT_LIST_HEAD(&vd_dev->virtqueues);
> > > +	spin_lock_init(&vd_dev->lock);
> > > +
> > > +	vd_dev->vdev.id.device = ops->get_device_id(vdpa);
> > > +	if (vd_dev->vdev.id.device == 0)
> > > +		return -ENODEV;
> > > +
> > > +	vd_dev->vdev.id.vendor = ops->get_vendor_id(vdpa);
> > > +	rc = register_virtio_device(&vd_dev->vdev);
> > > +	if (rc)
> > > +		put_device(dev);
> > And a ugly unwind like this is why you want to have device_initialize()
> > exposed to the driver,
> 
> In this context, which "driver" did you mean here? (Note, virtio-vdpa is the
> driver for vDPA bus here).

'driver' is the thing using the 'core' library calls to implement a
device, so here the 'vd_dev' is the driver and
'register_virtio_device' is the core

> > 
> > Where is the various THIS_MODULE's I expect to see in a scheme like
> > this?
> > 
> > All function pointers must be protected by a held module reference
> > count, ie the above probe/remove and all the pointers in ops.
> 
> Will double check, since I don't see this in other virtio transport drivers
> (PCI or MMIO).

pci_register_driver is a macro that provides a THIS_MODULE, and the
pci core code sets driver.owner, then the rest of the stuff related to
driver ops is supposed to work against that to protect the driver ops.

For the device module refcounting you either need to ensure that
'unregister' is a strong fence and guanentees that no device ops are
called past unregister (noting that this is impossible for release),
or you need to hold the module lock until release.

It is common to see non-core subsystems get this stuff wrong.

Jason

  reply index

Thread overview: 76+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-01-16 12:42 [PATCH 0/5] vDPA support Jason Wang
2020-01-16 12:42 ` [PATCH 1/5] vhost: factor out IOTLB Jason Wang
2020-01-17  4:14   ` Randy Dunlap
2020-01-17  9:34     ` Jason Wang
2020-01-18  0:01   ` kbuild test robot
2020-01-18  0:40   ` kbuild test robot
2020-01-16 12:42 ` [PATCH 2/5] vringh: IOTLB support Jason Wang
2020-01-17 21:54   ` kbuild test robot
2020-01-17 22:33   ` kbuild test robot
2020-01-16 12:42 ` [PATCH 3/5] vDPA: introduce vDPA bus Jason Wang
2020-01-16 15:22   ` Jason Gunthorpe
2020-01-17  3:03     ` Jason Wang
2020-01-17 13:54       ` Jason Gunthorpe
2020-01-20  7:50         ` Jason Wang
2020-01-20 12:17         ` Michael S. Tsirkin
2020-01-20 17:50           ` Jason Gunthorpe
2020-01-20 21:56             ` Michael S. Tsirkin
2020-01-21 14:12               ` Jason Gunthorpe
2020-01-21 14:15                 ` Michael S. Tsirkin
2020-01-21 14:16                   ` Jason Gunthorpe
2020-01-21  8:40       ` Tian, Kevin
2020-01-21  9:41         ` Jason Wang
2020-01-17  4:16   ` Randy Dunlap
2020-01-17  9:34     ` Jason Wang
2020-01-17 12:13   ` Michael S. Tsirkin
2020-01-17 13:52     ` Jason Wang
     [not found]       ` <CAJPjb1+fG9L3=iKbV4Vn13VwaeDZZdcfBPvarogF_Nzhk+FnKg@mail.gmail.com>
2020-01-19  9:07         ` Shahaf Shuler
2020-01-19  9:59           ` Michael S. Tsirkin
2020-01-20  8:44             ` Jason Wang
2020-01-20 12:09               ` Michael S. Tsirkin
2020-01-21  3:32                 ` Jason Wang
2020-01-20  8:43           ` Jason Wang
2020-01-20 17:49             ` Jason Gunthorpe
2020-01-20 20:51               ` Shahaf Shuler
2020-01-20 21:25                 ` Michael S. Tsirkin
2020-01-20 21:47                   ` Shahaf Shuler
2020-01-20 21:59                     ` Michael S. Tsirkin
2020-01-21  6:01                       ` Shahaf Shuler
2020-01-21  7:57                         ` Jason Wang
2020-01-21 14:07                   ` Jason Gunthorpe
2020-01-21 14:16                     ` Michael S. Tsirkin
2020-01-20 21:48               ` Michael S. Tsirkin
2020-01-21  4:00               ` Jason Wang
2020-01-21  5:47                 ` Michael S. Tsirkin
2020-01-21  8:00                   ` Jason Wang
2020-01-21  8:15                     ` Michael S. Tsirkin
2020-01-21  8:35                       ` Jason Wang
2020-01-21 11:09                         ` Shahaf Shuler
2020-01-22  6:36                           ` Jason Wang
2020-01-21 14:05                       ` Jason Gunthorpe
2020-01-21 14:17                         ` Michael S. Tsirkin
2020-01-22  6:18                           ` Jason Wang
2020-01-20  8:19         ` Jason Wang
2020-01-16 12:42 ` [PATCH 4/5] virtio: introduce a vDPA based transport Jason Wang
2020-01-16 15:38   ` Jason Gunthorpe
2020-01-17  9:32     ` Jason Wang
2020-01-17 14:00       ` Jason Gunthorpe [this message]
2020-01-20  7:52         ` Jason Wang
2020-01-17  4:10   ` Randy Dunlap
2020-01-16 12:42 ` [PATCH 5/5] vdpasim: vDPA device simulator Jason Wang
2020-01-16 15:47   ` Jason Gunthorpe
2020-01-17  9:32     ` Jason Wang
2020-01-17 14:10       ` Jason Gunthorpe
2020-01-20  8:01         ` Jason Wang
2020-02-04  4:19         ` Jason Wang
2020-01-17  4:12   ` Randy Dunlap
2020-01-17  9:35     ` Jason Wang
2020-01-18 18:18   ` kbuild test robot
2020-01-28  3:32   ` Dan Carpenter
2020-02-04  4:07     ` Jason Wang
2020-02-04  8:21   ` Zhu Lingshan
2020-02-04  8:28     ` Jason Wang
2020-02-04 12:52       ` Jason Gunthorpe
2020-02-05  3:14         ` Jason Wang
2020-01-21  8:44 ` [PATCH 0/5] vDPA support Tian, Kevin
2020-01-21  9:39   ` Jason Wang

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200117140013.GV20978@mellanox.com \
    --to=jgg@mellanox.com \
    --cc=aadam@redhat.com \
    --cc=cunming.liang@intel.com \
    --cc=eperezma@redhat.com \
    --cc=hanand@xilinx.com \
    --cc=haotian.wang@sifive.com \
    --cc=hch@infradead.org \
    --cc=jakub.kicinski@netronome.com \
    --cc=jasowang@redhat.com \
    --cc=jiri@mellanox.com \
    --cc=kevin.tian@intel.com \
    --cc=kvm@vger.kernel.org \
    --cc=lingshan.zhu@intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lulu@redhat.com \
    --cc=maxime.coquelin@redhat.com \
    --cc=mhabets@solarflare.com \
    --cc=mst@redhat.com \
    --cc=netdev@vger.kernel.org \
    --cc=parav@mellanox.com \
    --cc=rdunlap@infradead.org \
    --cc=rob.miller@broadcom.com \
    --cc=shahafs@mellanox.com \
    --cc=stefanha@redhat.com \
    --cc=tiwei.bie@intel.com \
    --cc=virtualization@lists.linux-foundation.org \
    --cc=xiao.w.wang@intel.com \
    --cc=zhihong.wang@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

KVM Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/kvm/0 kvm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 kvm kvm/ https://lore.kernel.org/kvm \
		kvm@vger.kernel.org
	public-inbox-index kvm

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.kvm


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git