From: Parav Pandit <parav@mellanox.com>
To: Jeff Kirsher <jeffrey.t.kirsher@intel.com>,
"davem@davemloft.net" <davem@davemloft.net>,
"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>
Cc: Dave Ertman <david.m.ertman@intel.com>,
"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
"linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
"nhorman@redhat.com" <nhorman@redhat.com>,
"sassmann@redhat.com" <sassmann@redhat.com>,
"jgg@ziepe.ca" <jgg@ziepe.ca>,
"galpress@amazon.com" <galpress@amazon.com>,
"selvin.xavier@broadcom.com" <selvin.xavier@broadcom.com>,
"sriharsha.basavapatna@broadcom.com"
<sriharsha.basavapatna@broadcom.com>,
"benve@cisco.com" <benve@cisco.com>,
"bharat@chelsio.com" <bharat@chelsio.com>,
"xavier.huwei@huawei.com" <xavier.huwei@huawei.com>,
Yishai Hadas <yishaih@mellanox.com>,
Leon Romanovsky <leonro@mellanox.com>,
"mkalderon@marvell.com" <mkalderon@marvell.com>,
"aditr@vmware.com" <aditr@vmware.com>,
Kiran Patil <kiran.patil@intel.com>,
Andrew Bowers <andrewx.bowers@intel.com>
Subject: Re: [RFC PATCH v4 01/25] virtual-bus: Implementation of Virtual Bus
Date: Fri, 14 Feb 2020 21:22:08 +0000 [thread overview]
Message-ID: <a8d8bc52-f67f-6690-8f34-6f3b1ee8254c@mellanox.com> (raw)
In-Reply-To: <20200212191424.1715577-2-jeffrey.t.kirsher@intel.com>
On 2/12/2020 1:14 PM, Jeff Kirsher wrote:
> From: Dave Ertman <david.m.ertman@intel.com>
>
> This is the initial implementation of the Virtual Bus,
> virtbus_device and virtbus_driver. The virtual bus is
> a software based bus intended to support registering
> virtbus_devices and virtbus_drivers and provide matching
> between them and probing of the registered drivers.
>
> The bus will support probe/remove shutdown and
> suspend/resume callbacks.
>
> Kconfig and Makefile alterations are included
>
> Signed-off-by: Dave Ertman <david.m.ertman@intel.com>
> Signed-off-by: Kiran Patil <kiran.patil@intel.com>
> Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
> Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
> ---
> Documentation/driver-api/virtual_bus.rst | 59 +++++
> drivers/bus/Kconfig | 11 +
> drivers/bus/Makefile | 1 +
> drivers/bus/virtual_bus.c | 267 +++++++++++++++++++++++
> include/linux/mod_devicetable.h | 8 +
> include/linux/virtual_bus.h | 57 +++++
> scripts/mod/devicetable-offsets.c | 3 +
> scripts/mod/file2alias.c | 8 +
> 8 files changed, 414 insertions(+)
> create mode 100644 Documentation/driver-api/virtual_bus.rst
> create mode 100644 drivers/bus/virtual_bus.c
> create mode 100644 include/linux/virtual_bus.h
>
> diff --git a/Documentation/driver-api/virtual_bus.rst b/Documentation/driver-api/virtual_bus.rst
> new file mode 100644
> index 000000000000..5f35c19171d7
> --- /dev/null
> +++ b/Documentation/driver-api/virtual_bus.rst
> @@ -0,0 +1,59 @@
> +===============================
> +Virtual Bus Devices and Drivers
> +===============================
> +
> +See <linux/virtual_bus.h> for the models for virtbus_device and virtbus_driver.
> +This bus is meant to be a lightweight software based bus to attach generic
> +devices and drivers to so that a chunk of data can be passed between them.
> +
> +One use case example is an rdma driver needing to connect with several
> +different types of PCI LAN devices to be able to request resources from
> +them (queue sets). Each LAN driver that supports rdma will register a
> +virtbus_device on the virtual bus for each physical function. The rdma
> +driver will register as a virtbus_driver on the virtual bus to be
> +matched up with multiple virtbus_devices and receive a pointer to a
> +struct containing the callbacks that the PCI LAN drivers support for
> +registering with them.
> +
> +Sections in this document:
> + Virtbus devices
> + Virtbus drivers
> + Device Enumeration
> + Device naming and driver binding
> + Virtual Bus API entry points
> +
> +Virtbus devices
> +~~~~~~~~~~~~~~~
> +Virtbus_devices support the minimal device functionality. Devices will
> +accept a name, and then, when added to the virtual bus, an automatically
> +generated index is concatenated onto it for the virtbus_device->name.
> +
> +Virtbus drivers
> +~~~~~~~~~~~~~~~
> +Virtbus drivers register with the virtual bus to be matched with virtbus
> +devices. They expect to be registered with a probe and remove callback,
> +and also support shutdown, suspend, and resume callbacks. They otherwise
> +follow the standard driver behavior of having discovery and enumeration
> +handled in the bus infrastructure.
> +
> +Virtbus drivers register themselves with the API entry point virtbus_drv_reg
> +and unregister with virtbus_drv_unreg.
> +
If you are mentioning API names, please keep the API documentation match
with API name in header file.
> +Device Enumeration
> +~~~~~~~~~~~~~~~~~~
> +Enumeration is handled automatically by the bus infrastructure via the
> +ida_simple methods.
> +
> +Device naming and driver binding
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +The virtbus_device.dev.name is the canonical name for the device. It is
> +built from two other parts:
> +
> + - virtbus_device.name (also used for matching).
> + - virtbus_device.id (generated automatically from ida_simple calls)
> +
> +Virtbus device IDs are always in "<name>.<instance>" format. Instances are
> +automatically selected through an ida_simple_get so are positive integers.
> +Names are taken from the device name field.
Name is taken from the device name field.
Driver IDs are simple <name>.
> +Need to extract the name from the Virtual Device compare to name of the
> +driver.
> diff --git a/drivers/bus/Kconfig b/drivers/bus/Kconfig
> index 6095b6df8a81..2e8b89c1761a 100644
> --- a/drivers/bus/Kconfig
> +++ b/drivers/bus/Kconfig
> @@ -202,4 +202,15 @@ config DA8XX_MSTPRI
>
> source "drivers/bus/fsl-mc/Kconfig"
>
> +config VIRTUAL_BUS
> + tristate "Software based Virtual Bus"
> + help
> + Provides a software bus for virtbus_devices to be added to it
> + and virtbus_drivers to be registered on it. Will create a match> + between the driver and device, then call the driver's probe with
> + the virtbus_device's struct.
below text reads better for "Will create a match.."
It matches driver and device based on id and calls driver's probe routine.
> + One example is the irdma driver needing to connect with various
> + PCI LAN drivers to request resources (queues) to be able to perform
> + its function.
> +
> endmenu
> diff --git a/drivers/bus/Makefile b/drivers/bus/Makefile
> index 1320bcf9fa9d..6721c77dc71b 100644
> --- a/drivers/bus/Makefile
> +++ b/drivers/bus/Makefile
> @@ -34,3 +34,4 @@ obj-$(CONFIG_UNIPHIER_SYSTEM_BUS) += uniphier-system-bus.o
> obj-$(CONFIG_VEXPRESS_CONFIG) += vexpress-config.o
>
> obj-$(CONFIG_DA8XX_MSTPRI) += da8xx-mstpri.o
> +obj-$(CONFIG_VIRTUAL_BUS) += virtual_bus.o
> diff --git a/drivers/bus/virtual_bus.c b/drivers/bus/virtual_bus.c
> new file mode 100644
> index 000000000000..85d2dbfa3376
> --- /dev/null
> +++ b/drivers/bus/virtual_bus.c
> @@ -0,0 +1,267 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * virtual_bus.c - lightweight software based bus for virtual devices
> + *
> + * Copyright (c) 2019-20 Intel Corporation
> + *
> + * Please see Documentation/driver-api/virtual_bus.rst for
> + * more information
> + */
> +
> +#include <linux/string.h>
> +#include <linux/virtual_bus.h>
> +#include <linux/of_irq.h>
> +#include <linux/module.h>
> +#include <linux/init.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/pm_domain.h>
> +#include <linux/acpi.h>
> +#include <linux/device.h>
> +
> +MODULE_LICENSE("GPL v2");
> +MODULE_DESCRIPTION("Lightweight Virtual Bus");
Last time Greg had comment about "Lightweight".
In the bus/Kconfig, it is named as "software based virtual bus".
Just say "Virtual bus".
> +MODULE_AUTHOR("David Ertman <david.m.ertman@intel.com>");
> +MODULE_AUTHOR("Kiran Patil <kiran.patil@intel.com>");
> +
> +static DEFINE_IDA(virtbus_dev_ida);
> +
> +static const
> +struct virtbus_dev_id *virtbus_match_id(const struct virtbus_dev_id *id,
> + struct virtbus_device *vdev)
> +{
> + while (id->name[0]) {
> + if (!strcmp(vdev->name, id->name)) {
> + vdev->matched_element = id;
I am yet to review the actual usage of matched_element field in
subsequent patches, but assigning this inside the match routine doesn't
look correct.
> + return id;
> + }
> + id++;
> + }
> + return NULL;
> +}
> +
> +static int virtbus_match(struct device *dev, struct device_driver *drv)
> +{
> + struct virtbus_driver *vdrv = to_virtbus_drv(drv);
> + struct virtbus_device *vdev = to_virtbus_dev(dev);
> +
> + return virtbus_match_id(vdrv->id_table, vdev) != NULL;
> +}
> +
> +static int virtbus_probe(struct device *dev)
> +{
> + return dev->driver->probe(dev);
> +}
> +
> +static int virtbus_remove(struct device *dev)
> +{
> + return dev->driver->remove(dev);
> +}
> +
> +static void virtbus_shutdown(struct device *dev)
> +{
> + dev->driver->shutdown(dev);
> +}
> +
> +static int virtbus_suspend(struct device *dev, pm_message_t state)
> +{
> + if (dev->driver->suspend)
> + return dev->driver->suspend(dev, state);
> +
> + return 0;
> +}
> +
> +static int virtbus_resume(struct device *dev)
> +{
> + if (dev->driver->resume)
> + return dev->driver->resume(dev);
> +
> + return 0;
> +}
> +
> +struct bus_type virtual_bus_type = {
static struct bus_type.
> + .name = "virtbus",
> + .match = virtbus_match,
> + .probe = virtbus_probe,
> + .remove = virtbus_remove,
> + .shutdown = virtbus_shutdown,
> + .suspend = virtbus_suspend,
> + .resume = virtbus_resume,
> +};
> +
> +/**
> + * virtbus_dev_release - Destroy a virtbus device
> + * @vdev: virtual device to release
> + */
> +static void virtbus_dev_release(struct device *_dev)
> +{
> + struct virtbus_device *vdev = to_virtbus_dev(_dev);
> +
> + ida_simple_remove(&virtbus_dev_ida, vdev->id);
Device is not yet released. It is getting released below. Please move
ida_simple_remove() after completing the release call. Save vdev->id first.
This makes mirror sequence of dev_register() once you move ida_get
before device_initialize() as Greg suggested.
> + vdev->release(vdev);
> +}
> +
> +/**
> + * virtbus_dev_register - add a virtual bus device
> + * @vdev: virtual bus device to add
> + */
> +int virtbus_dev_register(struct virtbus_device *vdev)
> +{
> + int ret;
> +
> + if (!vdev->release) {
> + dev_err(&vdev->dev, "virtbus_device .release callback NULL\n");
> + return -EINVAL;
> + }
> +
> + device_initialize(&vdev->dev);
> +
> + vdev->dev.bus = &virtual_bus_type;
> + vdev->dev.release = virtbus_dev_release;
> + /* All device IDs are automatically allocated */
> + ret = ida_simple_get(&virtbus_dev_ida, 0, 0, GFP_KERNEL);
> + if (ret < 0) {
> + dev_err(&vdev->dev, "get IDA idx for virtbus device failed!\n");
> + put_device(&vdev->dev);
> + return ret;
> + }
> +
> + vdev->id = ret;
> + dev_set_name(&vdev->dev, "%s.%d", vdev->name, vdev->id);
> +
> + dev_dbg(&vdev->dev, "Registering virtbus device '%s'\n",
> + dev_name(&vdev->dev));
> +
> + ret = device_add(&vdev->dev);
> + if (ret)
> + goto device_add_err;
> +
> + return 0;
> +
> +device_add_err:
> + dev_err(&vdev->dev, "Add device to virtbus failed!\n");
Print error code along with the error line is more useful.
> + put_device(&vdev->dev);
> + ida_simple_remove(&virtbus_dev_ida, vdev->id);
> +
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(virtbus_dev_register);
> +
> +/**
> + * virtbus_dev_unregister - remove a virtual bus device
> + * vdev: virtual bus device we are removing
> + */
> +void virtbus_dev_unregister(struct virtbus_device *vdev)
> +{
> + device_del(&vdev->dev);
> + put_device(&vdev->dev);
> +}
> +EXPORT_SYMBOL_GPL(virtbus_dev_unregister);
> +
> +static int virtbus_drv_probe(struct device *_dev)
> +{
> + struct virtbus_driver *vdrv = to_virtbus_drv(_dev->driver);
> + struct virtbus_device *vdev = to_virtbus_dev(_dev);
> + int ret;
> +
> + ret = dev_pm_domain_attach(_dev, true);
> + if (ret) {
> + dev_warn(_dev, "Failed to attatch to PM Domain : %d\n", ret);
> + return ret;
> + }
> +
> + ret = vdrv->probe(vdev);
> + if (ret) {
> + dev_err(&vdev->dev, "Probe returned error\n");
> + dev_pm_domain_detach(_dev, true);
> + }
> +
> + return ret;
> +}
> +
> +static int virtbus_drv_remove(struct device *_dev)
> +{
> + struct virtbus_driver *vdrv = to_virtbus_drv(_dev->driver);
> + struct virtbus_device *vdev = to_virtbus_dev(_dev);
> + int ret = 0;
> +
> + ret = vdrv->remove(vdev);
> + dev_pm_domain_detach(_dev, true);
> +
> + return ret;
> +}
> +
> +static void virtbus_drv_shutdown(struct device *_dev)
> +{
> + struct virtbus_driver *vdrv = to_virtbus_drv(_dev->driver);
> + struct virtbus_device *vdev = to_virtbus_dev(_dev);
> +
> + vdrv->shutdown(vdev);
> +}
> +
> +static int virtbus_drv_suspend(struct device *_dev, pm_message_t state)
> +{
> + struct virtbus_driver *vdrv = to_virtbus_drv(_dev->driver);
> + struct virtbus_device *vdev = to_virtbus_dev(_dev);
> +
> + if (vdrv->suspend)
> + return vdrv->suspend(vdev, state);
> +
> + return 0;
> +}
> +
> +static int virtbus_drv_resume(struct device *_dev)
> +{
> + struct virtbus_driver *vdrv = to_virtbus_drv(_dev->driver);
> + struct virtbus_device *vdev = to_virtbus_dev(_dev);
> +
> + if (vdrv->resume)
> + return vdrv->resume(vdev);
> +
> + return 0;
> +}
> +
> +/**
> + * __virtbus_drv_register - register a driver for virtual bus devices
> + * @vdrv: virtbus_driver structure
> + * @owner: owning module/driver
> + */
> +int __virtbus_drv_register(struct virtbus_driver *vdrv, struct module *owner)
> +{
Lets keep name consistent with other subsystems such as,
pci_register_driver(), spi_register_driver() etc as
virtbus_register_driver() and unregister().
> + if (!vdrv->probe || !vdrv->remove || !vdrv->shutdown || !vdrv->id_table)
> + return -EINVAL;
> +
> + vdrv->driver.owner = owner;
> + vdrv->driver.bus = &virtual_bus_type;
> + vdrv->driver.probe = virtbus_drv_probe;
> + vdrv->driver.remove = virtbus_drv_remove;
> + vdrv->driver.shutdown = virtbus_drv_shutdown;
> + vdrv->driver.suspend = virtbus_drv_suspend;
> + vdrv->driver.resume = virtbus_drv_resume;
> +
> + return driver_register(&vdrv->driver);
> +}
> +EXPORT_SYMBOL_GPL(__virtbus_drv_register);
> +
> +/**
> + * virtbus_drv_unregister - unregister a driver for virtual bus devices
> + * @drv: virtbus_driver structure
> + */
> +void virtbus_drv_unregister(struct virtbus_driver *vdrv)
> +{
> + driver_unregister(&vdrv->driver);
> +}
> +EXPORT_SYMBOL_GPL(virtbus_drv_unregister);
next prev parent reply other threads:[~2020-02-14 21:22 UTC|newest]
Thread overview: 53+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-02-12 19:13 [RFC PATCH v4 00/25] Intel Wired LAN/RDMA Driver Updates 2020-02-11 Jeff Kirsher
2020-02-12 19:14 ` [RFC PATCH v4 01/25] virtual-bus: Implementation of Virtual Bus Jeff Kirsher
2020-02-14 17:02 ` Greg KH
2020-02-14 20:34 ` Jason Gunthorpe
2020-02-14 20:43 ` Greg KH
2020-02-15 0:01 ` Jason Gunthorpe
2020-02-15 0:53 ` Greg KH
2020-02-14 20:45 ` Greg KH
2020-02-20 18:55 ` Ertman, David M
2020-02-20 19:27 ` Jason Gunthorpe
2020-02-14 21:22 ` Parav Pandit [this message]
2020-02-15 0:08 ` Jason Gunthorpe
2020-02-12 19:14 ` [RFC PATCH v4 02/25] ice: Create and register virtual bus for RDMA Jeff Kirsher
2020-02-14 20:39 ` Jason Gunthorpe
2020-02-20 18:48 ` Ertman, David M
2020-02-20 20:58 ` Jason Gunthorpe
2020-02-12 19:14 ` [RFC PATCH v4 03/25] ice: Complete RDMA peer registration Jeff Kirsher
2020-02-12 19:14 ` [RFC PATCH v4 04/25] ice: Support resource allocation requests Jeff Kirsher
2020-02-12 19:14 ` [RFC PATCH v4 05/25] ice: Enable event notifications Jeff Kirsher
2020-02-12 19:14 ` [RFC PATCH v4 06/25] ice: Allow reset operations Jeff Kirsher
2020-02-12 19:14 ` [RFC PATCH v4 07/25] ice: Pass through communications to VF Jeff Kirsher
2020-02-12 19:14 ` [RFC PATCH v4 08/25] i40e: Move client header location Jeff Kirsher
2020-02-12 19:14 ` [RFC PATCH v4 09/25] i40e: Register a virtbus device to provide RDMA Jeff Kirsher
2020-02-12 19:14 ` [RFC PATCH v4 10/25] RDMA/irdma: Add driver framework definitions Jeff Kirsher
2020-02-14 22:13 ` Parav Pandit
2020-02-18 20:42 ` Saleem, Shiraz
2020-02-20 22:24 ` Parav Pandit
2020-02-20 23:06 ` Jason Gunthorpe
2020-02-21 17:01 ` Saleem, Shiraz
2020-02-21 17:23 ` Parav Pandit
2020-02-21 18:04 ` Jason Gunthorpe
2020-03-19 11:49 ` Martin Habets
2020-02-12 19:14 ` [RFC PATCH v4 11/25] RDMA/irdma: Implement device initialization definitions Jeff Kirsher
2020-02-12 19:14 ` [RFC PATCH v4 12/25] RDMA/irdma: Implement HW Admin Queue OPs Jeff Kirsher
2020-02-12 19:14 ` [RFC PATCH v4 13/25] RDMA/irdma: Add HMC backing store setup functions Jeff Kirsher
2020-02-12 19:14 ` [RFC PATCH v4 14/25] RDMA/irdma: Add privileged UDA queue implementation Jeff Kirsher
2020-02-12 19:14 ` [RFC PATCH v4 15/25] RDMA/irdma: Add QoS definitions Jeff Kirsher
2020-02-12 19:14 ` [RFC PATCH v4 16/25] RDMA/irdma: Add connection manager Jeff Kirsher
2020-02-12 19:14 ` [RFC PATCH v4 17/25] RDMA/irdma: Add PBLE resource manager Jeff Kirsher
2020-02-12 19:14 ` [RFC PATCH v4 18/25] RDMA/irdma: Implement device supported verb APIs Jeff Kirsher
2020-02-14 14:54 ` Jason Gunthorpe
2020-02-14 15:49 ` Andrew Boyer
2020-02-14 16:45 ` Jason Gunthorpe
2020-02-18 20:43 ` Saleem, Shiraz
2020-02-12 19:14 ` [RFC PATCH v4 19/25] RDMA/irdma: Add RoCEv2 UD OP support Jeff Kirsher
2020-02-12 19:14 ` [RFC PATCH v4 20/25] RDMA/irdma: Add user/kernel shared libraries Jeff Kirsher
2020-02-12 19:14 ` [RFC PATCH v4 21/25] RDMA/irdma: Add miscellaneous utility definitions Jeff Kirsher
2020-02-12 19:14 ` [RFC PATCH v4 22/25] RDMA/irdma: Add dynamic tracing for CM Jeff Kirsher
2020-02-14 14:53 ` Jason Gunthorpe
2020-02-18 20:43 ` Saleem, Shiraz
2020-02-12 19:14 ` [RFC PATCH v4 23/25] RDMA/irdma: Add ABI definitions Jeff Kirsher
2020-02-12 19:14 ` [RFC PATCH v4 24/25] RDMA: Add irdma Kconfig/Makefile and remove i40iw Jeff Kirsher
2020-02-12 19:14 ` [RFC PATCH v4 25/25] RDMA/irdma: Update MAINTAINERS file Jeff Kirsher
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a8d8bc52-f67f-6690-8f34-6f3b1ee8254c@mellanox.com \
--to=parav@mellanox.com \
--cc=aditr@vmware.com \
--cc=andrewx.bowers@intel.com \
--cc=benve@cisco.com \
--cc=bharat@chelsio.com \
--cc=davem@davemloft.net \
--cc=david.m.ertman@intel.com \
--cc=galpress@amazon.com \
--cc=gregkh@linuxfoundation.org \
--cc=jeffrey.t.kirsher@intel.com \
--cc=jgg@ziepe.ca \
--cc=kiran.patil@intel.com \
--cc=leonro@mellanox.com \
--cc=linux-rdma@vger.kernel.org \
--cc=mkalderon@marvell.com \
--cc=netdev@vger.kernel.org \
--cc=nhorman@redhat.com \
--cc=sassmann@redhat.com \
--cc=selvin.xavier@broadcom.com \
--cc=sriharsha.basavapatna@broadcom.com \
--cc=xavier.huwei@huawei.com \
--cc=yishaih@mellanox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).