Re: [net-next v2 1/1] virtual-bus: Implementation of Virtual Bus

From: Jason Wang <jasowang@redhat.com>
To: Martin Habets <mhabets@solarflare.com>,
	Parav Pandit <parav@mellanox.com>,
	Jeff Kirsher <jeffrey.t.kirsher@intel.com>,
	"davem@davemloft.net" <davem@davemloft.net>,
	"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>
Cc: Dave Ertman <david.m.ertman@intel.com>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	"linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
	"nhorman@redhat.com" <nhorman@redhat.com>,
	"sassmann@redhat.com" <sassmann@redhat.com>,
	"jgg@ziepe.ca" <jgg@ziepe.ca>,
	Kiran Patil <kiran.patil@intel.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Alex Williamson <alex.williamson@redhat.com>,
	"Bie, Tiwei" <tiwei.bie@intel.com>
Subject: Re: [net-next v2 1/1] virtual-bus: Implementation of Virtual Bus
Date: Fri, 22 Nov 2019 17:13:56 +0800	[thread overview]
Message-ID: <22dd6ae3-03f4-1432-2935-8df5e9a449de@redhat.com> (raw)
In-Reply-To: <30b968cf-0e11-a2c6-5b9f-5518df11dfb7@solarflare.com>

On 2019/11/21 下午11:10, Martin Habets wrote:
> On 19/11/2019 04:08, Jason Wang wrote:
>> On 2019/11/16 上午7:25, Parav Pandit wrote:
>>> Hi Jeff,
>>>
>>>> From: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
>>>> Sent: Friday, November 15, 2019 4:34 PM
>>>>
>>>> From: Dave Ertman <david.m.ertman@intel.com>
>>>>
>>>> This is the initial implementation of the Virtual Bus, virtbus_device and
>>>> virtbus_driver.  The virtual bus is a software based bus intended to support
>>>> lightweight devices and drivers and provide matching between them and
>>>> probing of the registered drivers.
>>>>
>>>> The primary purpose of the virual bus is to provide matching services and to
>>>> pass the data pointer contained in the virtbus_device to the virtbus_driver
>>>> during its probe call.  This will allow two separate kernel objects to match up
>>>> and start communication.
>>>>
>>> It is fundamental to know that rdma device created by virtbus_driver will be anchored to which bus for an non abusive use.
>>> virtbus or parent pci bus?
>>> I asked this question in v1 version of this patch.
>>>
>>> Also since it says - 'to support lightweight devices', documenting that information is critical to avoid ambiguity.
>>>
>>> Since for a while I am working on the subbus/subdev_bus/xbus/mdev [1] whatever we want to call it, it overlaps with your comment about 'to support lightweight devices'.
>>> Hence let's make things crystal clear weather the purpose is 'only matching service' or also 'lightweight devices'.
>>> If this is only matching service, lets please remove lightweight devices part..
>>
>> Yes, if it's matching + lightweight device, its function is almost a duplication of mdev. And I'm working on extending mdev[1] to be a generic module to support any types of virtual devices a while. The advantage of mdev is:
>>
>> 1) ready for the userspace driver (VFIO based)
>> 2) have a sysfs/GUID based management interface
> In my view this virtual-bus is more generic and more flexible than mdev.

Even after the series [1] here?

> What for you are the advantages of mdev to me are some of it's disadvantages.
>
> The way I see it we can provide rdma support in the driver using virtual-bus.

Yes, but since it does matching only, you can do everything you want. 
But it looks to me Greg does not want a bus to be an API multiplexer. So 
if a dedicated bus is desired, it won't be much of code to have a bus on 
your own.

> At the moment we would need separate mdev support in the driver for vdpa, but I hope at some point mdev
> would become a layer on top of virtual-bus.
> Besides these users we also support internal tools for our hardware factory provisioning, and for testing/debugging.
> I could easily imagine such tools using a virtual-bus device. With mdev those interfaces would be more convoluted.

Can you give me an example?

>
>> So for 1, it's not clear that how userspace driver would be supported here, or it's completely not being accounted in this series? For 2, it looks to me that this series leave it to the implementation, this means management to learn several vendor specific interfaces which seems a burden.
>>
>> Note, technically Virtual Bus could be implemented on top of [1] with the full lifecycle API.
> Seems easier to me to do that the other way around: mdev could be implemented on top of virtual-bus.

Probably, without the part of parent_ops, they are almost equal.

Thanks

>
> Best regards,
> Martin
>
>> [1] https://lkml.org/lkml/2019/11/18/261
>>
>>
>>> You additionally need modpost support for id table integration to modifo, modprobe and other tools.
>>> A small patch similar to this one [2] is needed.
>>> Please include in the series.
>>>
>>> [..]
>>
>> And probably a uevent method. But rethinking of this, matching through a single virtual bus seems not good. What if driver want to do some specific matching? E.g for virtio, we may want a vhost-net driver that only match networking device. With a single bus, it probably means you need another bus on top and provide the virtio specific matching there. This looks not straightforward as allowing multiple type of buses.
>>
>> Thanks
>>