[PATCH v5 00/12] bus: attach / detach API

[PATCH v5 00/12] bus: attach / detach API

[Gaetan Rivet]

Following the work from Jan:

This patchset introduces the attach / detach API to rte_bus.
The rte_device structure is used as the generic device representation.

This API is implemented for the virtual bus.
The functions rte_eal_dev_attach and rte_eal_dev_detach are updated to
use this new interface.

-- v2

0. API rework
-------------

I would like to propose an evolution on the API developed by Jan.

The attach / detach rte_bus API is necessary to support the attach/detach
rte_dev API. Those are two different levels for one similar functionality.

Attach / detach does not allow true hotplugging, because the attach
function expects the devices operated upon to already exist within the
buses / sub-layers. This means that this API expects devices meta-datas
(bus-internal device representation and associated device information
read from the system) to be present upon attach. This part of the work
is done during scanning.

While it is best to avoid changing the public rte_dev API as it already
exists, nothing prevents this new rte_bus API from superseeding it.
It has been said during the previous release cycle that device hotplug
was a feature that interested users. True hotplug is not allowed by the
current attach / detach API. Worse, this API hinders the effort to bring
this new functionality by squatting its semantic field.

Thus, I propose to rename rte_bus attach / detach; plug / unplug. As it
is a superset of the attach / detach functionality, it can be used to
implement rte_dev attach / detach. Now is the right time to pivot to
this new feature.

This should help maintainers understanding the aim of this API and the
differences with the APIs higher-up, clarify the field and allow a new
functionality to be proposed.

The vdev bus is inherently supporting the new API, however it has been
made explicit. My implementation in the PCI bus in further patchset also
follows the rte_bus hotplug API instead of only attach / detach.

One remaining problem with the vdev bus is the rte_dev attach
implementation, which needs the rte_devargs rework to be properly fixed.

1. Additional evolutions in the patchset
----------------------------------------

The RTE_VERIFY on the find_device is too stringent I think and forces
all buses to implement a public device iterator. While it could be
argued that it would push for quicker support for the functionality, I
think it's possible that some buses are not interested at all in it and
should simply be ignored.

The bus devices iterator has been fixed.

The internal rte_device handle was not properly setup within the
net_ring PMD.

-- v3

The new API is now

typedef struct rte_device * (*rte_bus_plug_t)(struct rte_devargs *da);
typedef int (*rte_bus_unplug_t)(struct rte_device *dev);

So, plugging a device takes an rte_devargs as input and returns an rte_device.
While implementing related subsystems, I found that I usually needed
this rte_device handle upon a successful device plugging. This seems the
sensible and useful thing to do.
As such, on error NULL is returned and rte_errno is set by the bus.

Unplugging a device however now returns to the first version, which used
an rte_device. The explicit contract here is that if one has an
rte_device that has been obtained by calling bus->plug(), then this
handle can be used for bus->unplug().

Additionally, bus and device comparators now returns 0 on match,
following strcmp-like behavior.

-- v4

* rte_bus_find now takes a *start* parameter, that can be null.
  The bus search starts from this element if set.

* A few doc fixes.

* The rte_device field was fixed within the rte_ring PMD in a previous patch.
  This fix has been integrated by other means, it is not necessary anymore.

-- v5

* The commit
ethdev: use embedded rte_device to detach driver
  has been removed from this series to be sent separately.

* The PCI support for device access and hotplug is merged in this series
  instead of being proposed as a separate patchset.

* A few nitpicks to the code itself have been fixed.

* Some documentation has been reworked.

Gaetan Rivet (5):
  vdev: implement hotplug functionality
  vdev: expose bus name
  vdev: use standard bus registration function
  pci: implement find_device bus operation
  pci: implement hotplug bus operation

Jan Blunck (7):
  bus: add bus iterator to find a bus
  bus: add device iterator method
  bus: add helper to find which bus holds a device
  bus: add bus iterator to find a device
  bus: introduce hotplug functionality
  vdev: implement find_device bus operation
  eal: make virtual driver probe and remove take rte_vdev_device

 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |   3 +
 lib/librte_eal/common/eal_common_bus.c          |  71 ++++++++++++++
 lib/librte_eal/common/eal_common_dev.c          |  93 +++++++++++++-----
 lib/librte_eal/common/eal_common_pci.c          |  57 +++++++++++
 lib/librte_eal/common/eal_common_vdev.c         |  65 ++++++++++---
 lib/librte_eal/common/include/rte_bus.h         | 124 ++++++++++++++++++++++++
 lib/librte_eal/common/include/rte_dev.h         |  21 ++++
 lib/librte_eal/common/include/rte_vdev.h        |   2 +
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |   3 +
 9 files changed, 401 insertions(+), 38 deletions(-)

-- 
2.1.4

[PATCH v5 01/12] bus: add bus iterator to find a bus

[Gaetan Rivet]

From: Jan Blunck <jblunck@infradead.org>

This helper allows to iterate over all registered buses and find one
matching data used as parameter.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  1 +
 lib/librte_eal/common/eal_common_bus.c          | 20 ++++++++++++
 lib/librte_eal/common/include/rte_bus.h         | 43 +++++++++++++++++++++++++
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |  1 +
 4 files changed, 65 insertions(+)

diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
index 2e48a73..ed09ab2 100644
--- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
@@ -162,6 +162,7 @@ DPDK_17.02 {
 DPDK_17.05 {
 	global:
 
+	rte_bus_find;
 	rte_cpu_is_supported;
 	rte_log_dump;
 	rte_log_register;
diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
index 8f9baf8..4619eb2 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -145,3 +145,23 @@ rte_bus_dump(FILE *f)
 		}
 	}
 }
+
+struct rte_bus *
+rte_bus_find(rte_bus_cmp_t cmp,
+	     const void *data,
+	     const struct rte_bus *start)
+{
+	struct rte_bus *bus = NULL;
+	int started = start == NULL;
+
+	TAILQ_FOREACH(bus, &rte_bus_list, next) {
+		if (!started) {
+			if (bus == start)
+				started = 1;
+			continue;
+		}
+		if (cmp(bus, data) == 0)
+			break;
+	}
+	return bus;
+}
diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
index 5f47b82..ecf839b 100644
--- a/lib/librte_eal/common/include/rte_bus.h
+++ b/lib/librte_eal/common/include/rte_bus.h
@@ -141,6 +141,49 @@ int rte_bus_probe(void);
 void rte_bus_dump(FILE *f);
 
 /**
+ * Bus comparison function.
+ *
+ * @param bus
+ *	Bus under test.
+ *
+ * @param data
+ *	Data to compare against.
+ *
+ * @return
+ *	0 if the bus matches the data.
+ *	!0 if the bus does not match.
+ *	<0 if ordering is possible and the bus is lower than the data.
+ *	>0 if ordering is possible and the bus is greater than the data.
+ */
+typedef int (*rte_bus_cmp_t)(const struct rte_bus *bus, const void *data);
+
+/**
+ * Bus iterator to find a particular bus.
+ *
+ * This function compares each registered bus to find one that matches
+ * the data passed as parameter.
+ *
+ * If the comparison function returns zero this function will stop iterating
+ * over any more buses. To continue a search the bus of a previous search can
+ * be passed via the start parameter.
+ *
+ * @param cmp
+ *	Comparison function.
+ *
+ * @param data
+ *	 Data to pass to comparison function.
+ *
+ * @param start
+ *	Starting point for the iteration.
+ *
+ * @return
+ *	 A pointer to a rte_bus structure or NULL in case no bus matches
+ */
+struct rte_bus *rte_bus_find(rte_bus_cmp_t cmp,
+			     const void *data,
+			     const struct rte_bus *start);
+
+/**
  * Helper for Bus registration.
  * The constructor has higher priority than PMD constructors.
  */
diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
index 670bab3..6efa517 100644
--- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
@@ -166,6 +166,7 @@ DPDK_17.02 {
 DPDK_17.05 {
 	global:
 
+	rte_bus_find;
 	rte_cpu_is_supported;
 	rte_intr_free_epoll_fd;
 	rte_log_dump;
-- 
2.1.4

[PATCH v5 02/12] bus: add device iterator method

[Gaetan Rivet]

From: Jan Blunck <jblunck@infradead.org>

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/include/rte_bus.h | 19 +++++++++++++++++++
 lib/librte_eal/common/include/rte_dev.h | 21 +++++++++++++++++++++
 2 files changed, 40 insertions(+)

diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
index ecf839b..5efb76e 100644
--- a/lib/librte_eal/common/include/rte_bus.h
+++ b/lib/librte_eal/common/include/rte_bus.h
@@ -82,6 +82,24 @@ typedef int (*rte_bus_scan_t)(void);
 typedef int (*rte_bus_probe_t)(void);
 
 /**
+ * Device iterator to find a device on a bus.
+ *
+ * This function returns an rte_device if one of those held by the bus
+ * matches the data passed as parameter.
+ *
+ * @param cmp
+ *	Comparison function.
+ *
+ * @param data
+ *	Data to compare each device against.
+ *
+ * @return
+ *	The first device matching the data, NULL if none exists.
+ */
+typedef struct rte_device * (*rte_bus_find_device_t)(rte_dev_cmp_t cmp,
+						     const void *data);
+
+/**
  * A structure describing a generic bus.
  */
 struct rte_bus {
@@ -89,6 +107,7 @@ struct rte_bus {
 	const char *name;            /**< Name of the bus */
 	rte_bus_scan_t scan;         /**< Scan for devices attached to bus */
 	rte_bus_probe_t probe;       /**< Probe devices on bus */
+	rte_bus_find_device_t find_device; /**< Find a device on bus */
 };
 
 /**
diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
index de20c06..04d9c28 100644
--- a/lib/librte_eal/common/include/rte_dev.h
+++ b/lib/librte_eal/common/include/rte_dev.h
@@ -191,6 +191,27 @@ int rte_eal_dev_attach(const char *name, const char *devargs);
  */
 int rte_eal_dev_detach(const char *name);
 
+/**
+ * Device comparison function.
+ *
+ * This type of function is used to compare an rte_device with arbitrary
+ * data.
+ *
+ * @param dev
+ *   Device handle.
+ *
+ * @param data
+ *   Data to compare against. The type of this parameter is determined by
+ *   the kind of comparison performed by the function.
+ *
+ * @return
+ *   0 if the device matches the data.
+ *   !0 if the device does not match.
+ *   <0 if ordering is possible and the device is lower than the data.
+ *   >0 if ordering is possible and the device is greater than the data.
+ */
+typedef int (*rte_dev_cmp_t)(const struct rte_device *dev, const void *data);
+
 #define RTE_PMD_EXPORT_NAME_ARRAY(n, idx) n##idx[]
 
 #define RTE_PMD_EXPORT_NAME(name, idx) \
-- 
2.1.4

[PATCH v5 03/12] bus: add helper to find which bus holds a device

[Gaetan Rivet]

From: Jan Blunck <jblunck@infradead.org>

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  1 +
 lib/librte_eal/common/eal_common_bus.c          | 25 +++++++++++++++++++++++++
 lib/librte_eal/common/include/rte_bus.h         |  5 +++++
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |  1 +
 4 files changed, 32 insertions(+)

diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
index ed09ab2..f1a0765 100644
--- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
@@ -163,6 +163,7 @@ DPDK_17.05 {
 	global:
 
 	rte_bus_find;
+	rte_bus_find_by_device;
 	rte_cpu_is_supported;
 	rte_log_dump;
 	rte_log_register;
diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
index 4619eb2..d208214 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -165,3 +165,28 @@ rte_bus_find(rte_bus_cmp_t cmp,
 	}
 	return bus;
 }
+
+static int
+cmp_rte_device(const struct rte_device *dev1, const void *_dev2)
+{
+	const struct rte_device *dev2 = _dev2;
+
+	return dev1 != dev2;
+}
+
+static int
+bus_find_device(const struct rte_bus *bus, const void *_dev)
+{
+	struct rte_device *dev;
+
+	if (bus->find_device == NULL)
+		return -1;
+	dev = bus->find_device(cmp_rte_device, _dev);
+	return dev == NULL;
+}
+
+struct rte_bus *
+rte_bus_find_by_device(const struct rte_device *dev)
+{
+	return rte_bus_find(bus_find_device, (const void *)dev, NULL);
+}
diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
index 5efb76e..5441af9 100644
--- a/lib/librte_eal/common/include/rte_bus.h
+++ b/lib/librte_eal/common/include/rte_bus.h
@@ -203,6 +203,11 @@ struct rte_bus *rte_bus_find(rte_bus_cmp_t cmp,
 			     const struct rte_bus *start);
 
 /**
+ * Find the registered bus for a particular device.
+ */
+struct rte_bus *rte_bus_find_by_device(const struct rte_device *dev);
+
+/**
  * Helper for Bus registration.
  * The constructor has higher priority than PMD constructors.
  */
diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
index 6efa517..6f77222 100644
--- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
@@ -167,6 +167,7 @@ DPDK_17.05 {
 	global:
 
 	rte_bus_find;
+	rte_bus_find_by_device;
 	rte_cpu_is_supported;
 	rte_intr_free_epoll_fd;
 	rte_log_dump;
-- 
2.1.4

[PATCH v5 04/12] bus: add bus iterator to find a device

[Gaetan Rivet]

From: Jan Blunck <jblunck@infradead.org>

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  1 +
 lib/librte_eal/common/eal_common_bus.c          | 24 +++++++++++++++++++++++
 lib/librte_eal/common/include/rte_bus.h         | 26 +++++++++++++++++++++++++
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |  1 +
 4 files changed, 52 insertions(+)

diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
index f1a0765..21640d6 100644
--- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
@@ -164,6 +164,7 @@ DPDK_17.05 {
 
 	rte_bus_find;
 	rte_bus_find_by_device;
+	rte_bus_find_device;
 	rte_cpu_is_supported;
 	rte_log_dump;
 	rte_log_register;
diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
index d208214..63fd9f1 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -190,3 +190,27 @@ rte_bus_find_by_device(const struct rte_device *dev)
 {
 	return rte_bus_find(bus_find_device, (const void *)dev, NULL);
 }
+
+struct rte_device *
+rte_bus_find_device(rte_dev_cmp_t cmp, const void *data,
+		    const struct rte_device *start)
+{
+	struct rte_bus *bus;
+	struct rte_device *dev = NULL;
+	int started = start == NULL;
+
+	TAILQ_FOREACH(bus, &rte_bus_list, next) {
+		if (!bus->find_device)
+			continue;
+		if (!started) {
+			dev = bus->find_device(cmp_rte_device, start);
+			if (dev)
+				started = 1;
+			continue;
+		}
+		dev = bus->find_device(cmp, data);
+		if (dev)
+			break;
+	}
+	return dev;
+}
diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
index 5441af9..3e83227 100644
--- a/lib/librte_eal/common/include/rte_bus.h
+++ b/lib/librte_eal/common/include/rte_bus.h
@@ -203,6 +203,32 @@ struct rte_bus *rte_bus_find(rte_bus_cmp_t cmp,
 			     const struct rte_bus *start);
 
 /**
+ * Bus iterator to find a particular device.
+ *
+ * This function searches each registered bus to find a device that matches
+ * the data passed as parameter.
+ *
+ * If the comparison function returns zero this function will stop iterating
+ * over any more buses and devices. To continue a search the device of
+ * a previous search can be passed via the start parameter.
+ *
+ * @param cmp
+ *	Comparison function.
+ *
+ * @param data
+ *	Data to pass to comparison function.
+ *
+ * @param start
+ *	Starting point for the iteration.
+ *
+ * @return
+ *	A pointer to an rte_bus structure or NULL in case no device matches.
+ */
+struct rte_device *rte_bus_find_device(rte_dev_cmp_t cmp,
+				       const void *data,
+				       const struct rte_device *start);
+
+/**
  * Find the registered bus for a particular device.
  */
 struct rte_bus *rte_bus_find_by_device(const struct rte_device *dev);
diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
index 6f77222..e0a056d 100644
--- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
@@ -168,6 +168,7 @@ DPDK_17.05 {
 
 	rte_bus_find;
 	rte_bus_find_by_device;
+	rte_bus_find_device;
 	rte_cpu_is_supported;
 	rte_intr_free_epoll_fd;
 	rte_log_dump;
-- 
2.1.4

[PATCH v5 05/12] bus: introduce hotplug functionality

[Gaetan Rivet]

From: Jan Blunck <jblunck@infradead.org>

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/eal_common_bus.c  |  2 ++
 lib/librte_eal/common/include/rte_bus.h | 31 +++++++++++++++++++++++++++++++
 2 files changed, 33 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
index 63fd9f1..0035da0 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -50,6 +50,8 @@ rte_bus_register(struct rte_bus *bus)
 	/* A bus should mandatorily have the scan implemented */
 	RTE_VERIFY(bus->scan);
 	RTE_VERIFY(bus->probe);
+	/* Buses supporting hotplug also require unplug. */
+	RTE_VERIFY(!bus->plug || bus->unplug);
 
 	TAILQ_INSERT_TAIL(&rte_bus_list, bus, next);
 	RTE_LOG(DEBUG, EAL, "Registered [%s] bus.\n", bus->name);
diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
index 3e83227..187c37e 100644
--- a/lib/librte_eal/common/include/rte_bus.h
+++ b/lib/librte_eal/common/include/rte_bus.h
@@ -100,6 +100,35 @@ typedef struct rte_device * (*rte_bus_find_device_t)(rte_dev_cmp_t cmp,
 						     const void *data);
 
 /**
+ * Implementation specific probe function which is responsible for linking
+ * devices on that bus with applicable drivers.
+ * The plugged device might already have been used previously by the bus,
+ * in which case some buses might prefer to detect and re-use the relevant
+ * information pertaining to this device.
+ *
+ * @param da
+ *	Device declaration.
+ *
+ * @return
+ *	The pointer to a valid rte_device usable by the bus on success.
+ *	NULL on error. rte_errno is then set.
+ */
+typedef struct rte_device * (*rte_bus_plug_t)(struct rte_devargs *da);
+
+/**
+ * Implementation specific remove function which is responsible for unlinking
+ * devices on that bus from assigned driver.
+ *
+ * @param dev
+ *	Device pointer that was returned by a previous device plug call.
+ *
+ * @return
+ *	0 on success.
+ *	!0 on error. rte_errno is then set.
+ */
+typedef int (*rte_bus_unplug_t)(struct rte_device *dev);
+
+/**
  * A structure describing a generic bus.
  */
 struct rte_bus {
@@ -108,6 +137,8 @@ struct rte_bus {
 	rte_bus_scan_t scan;         /**< Scan for devices attached to bus */
 	rte_bus_probe_t probe;       /**< Probe devices on bus */
 	rte_bus_find_device_t find_device; /**< Find a device on bus */
+	rte_bus_plug_t plug;         /**< Probe single device for drivers */
+	rte_bus_unplug_t unplug;     /**< Remove single device from driver */
 };
 
 /**
-- 
2.1.4

[PATCH v5 06/12] vdev: implement find_device bus operation

[Gaetan Rivet]

From: Jan Blunck <jblunck@infradead.org>

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/eal_common_vdev.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_vdev.c b/lib/librte_eal/common/eal_common_vdev.c
index 0037a64..52528ef 100644
--- a/lib/librte_eal/common/eal_common_vdev.c
+++ b/lib/librte_eal/common/eal_common_vdev.c
@@ -338,9 +338,22 @@ vdev_probe(void)
 	return 0;
 }
 
+static struct rte_device *
+vdev_find_device(rte_dev_cmp_t cmp, const void *data)
+{
+	struct rte_vdev_device *dev;
+
+	TAILQ_FOREACH(dev, &vdev_device_list, next) {
+		if (cmp(&dev->device, data) == 0)
+			return &dev->device;
+	}
+	return NULL;
+}
+
 static struct rte_bus rte_vdev_bus = {
 	.scan = vdev_scan,
 	.probe = vdev_probe,
+	.find_device = vdev_find_device,
 };
 
 RTE_INIT(rte_vdev_bus_register);
-- 
2.1.4

[PATCH v5 07/12] vdev: implement hotplug functionality

[Gaetan Rivet]

Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/eal_common_vdev.c | 36 +++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_vdev.c b/lib/librte_eal/common/eal_common_vdev.c
index 52528ef..22e4640 100644
--- a/lib/librte_eal/common/eal_common_vdev.c
+++ b/lib/librte_eal/common/eal_common_vdev.c
@@ -38,11 +38,13 @@
 #include <sys/queue.h>
 
 #include <rte_eal.h>
+#include <rte_dev.h>
 #include <rte_bus.h>
 #include <rte_vdev.h>
 #include <rte_common.h>
 #include <rte_devargs.h>
 #include <rte_memory.h>
+#include <rte_errno.h>
 
 /** Double linked list of virtual device drivers. */
 TAILQ_HEAD(vdev_device_list, rte_vdev_device);
@@ -350,10 +352,44 @@ vdev_find_device(rte_dev_cmp_t cmp, const void *data)
 	return NULL;
 }
 
+static struct rte_device *
+vdev_plug(struct rte_devargs *da)
+{
+	struct rte_vdev_device *dev;
+	int ret;
+
+	ret = rte_vdev_init(da->virt.drv_name, da->args);
+	if (ret) {
+		rte_errno = -ret;
+		return NULL;
+	}
+	dev = find_vdev(da->virt.drv_name);
+	return &dev->device;
+}
+
+static int
+vdev_unplug(struct rte_device *dev)
+{
+	struct rte_devargs *da;
+	int ret;
+
+	da = dev->devargs;
+	if (da == NULL) {
+		rte_errno = EINVAL;
+		return -1;
+	}
+	ret = rte_vdev_uninit(da->virt.drv_name);
+	if (ret)
+		rte_errno = -ret;
+	return ret;
+}
+
 static struct rte_bus rte_vdev_bus = {
 	.scan = vdev_scan,
 	.probe = vdev_probe,
 	.find_device = vdev_find_device,
+	.plug = vdev_plug,
+	.unplug = vdev_unplug,
 };
 
 RTE_INIT(rte_vdev_bus_register);
-- 
2.1.4

[PATCH v5 08/12] vdev: expose bus name

[Gaetan Rivet]

Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/include/rte_vdev.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lib/librte_eal/common/include/rte_vdev.h b/lib/librte_eal/common/include/rte_vdev.h
index e6b678e..2d02c68 100644
--- a/lib/librte_eal/common/include/rte_vdev.h
+++ b/lib/librte_eal/common/include/rte_vdev.h
@@ -41,6 +41,8 @@ extern "C" {
 #include <rte_dev.h>
 #include <rte_devargs.h>
 
+#define VIRTUAL_BUS_NAME "virtual"
+
 struct rte_vdev_device {
 	TAILQ_ENTRY(rte_vdev_device) next;      /**< Next attached vdev */
 	struct rte_device device;               /**< Inherit core device */
-- 
2.1.4

[PATCH v5 09/12] vdev: use standard bus registration function

[Gaetan Rivet]

Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/eal_common_vdev.c | 18 +-----------------
 1 file changed, 1 insertion(+), 17 deletions(-)

diff --git a/lib/librte_eal/common/eal_common_vdev.c b/lib/librte_eal/common/eal_common_vdev.c
index 22e4640..a654709 100644
--- a/lib/librte_eal/common/eal_common_vdev.c
+++ b/lib/librte_eal/common/eal_common_vdev.c
@@ -54,14 +54,10 @@ static struct vdev_device_list vdev_device_list =
 struct vdev_driver_list vdev_driver_list =
 	TAILQ_HEAD_INITIALIZER(vdev_driver_list);
 
-static void rte_vdev_bus_register(void);
-
 /* register a driver */
 void
 rte_vdev_register(struct rte_vdev_driver *driver)
 {
-	rte_vdev_bus_register();
-
 	TAILQ_INSERT_TAIL(&vdev_driver_list, driver, next);
 }
 
@@ -392,16 +388,4 @@ static struct rte_bus rte_vdev_bus = {
 	.unplug = vdev_unplug,
 };
 
-RTE_INIT(rte_vdev_bus_register);
-
-static void rte_vdev_bus_register(void)
-{
-	static int registered;
-
-	if (registered)
-		return;
-
-	registered = 1;
-	rte_vdev_bus.name = RTE_STR(virtual);
-	rte_bus_register(&rte_vdev_bus);
-}
+RTE_REGISTER_BUS(VIRTUAL_BUS_NAME, rte_vdev_bus);
-- 
2.1.4

[PATCH v5 10/12] pci: implement find_device bus operation

[Gaetan Rivet]

Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/eal_common_pci.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c
index 78b097e..00d48d9 100644
--- a/lib/librte_eal/common/eal_common_pci.c
+++ b/lib/librte_eal/common/eal_common_pci.c
@@ -488,10 +488,23 @@ rte_pci_remove_device(struct rte_pci_device *pci_dev)
 	TAILQ_REMOVE(&rte_pci_bus.device_list, pci_dev, next);
 }
 
+static struct rte_device *
+pci_find_device(rte_dev_cmp_t cmp, const void *data)
+{
+	struct rte_pci_device *dev;
+
+	FOREACH_DEVICE_ON_PCIBUS(dev) {
+		if (cmp(&dev->device, data) == 0)
+			return &dev->device;
+	}
+	return NULL;
+}
+
 struct rte_pci_bus rte_pci_bus = {
 	.bus = {
 		.scan = rte_pci_scan,
 		.probe = rte_pci_probe,
+		.find_device = pci_find_device,
 	},
 	.device_list = TAILQ_HEAD_INITIALIZER(rte_pci_bus.device_list),
 	.driver_list = TAILQ_HEAD_INITIALIZER(rte_pci_bus.driver_list),
-- 
2.1.4

[PATCH v5 11/12] pci: implement hotplug bus operation

[Gaetan Rivet]

Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/eal_common_pci.c | 44 ++++++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c
index 00d48d9..286357d 100644
--- a/lib/librte_eal/common/eal_common_pci.c
+++ b/lib/librte_eal/common/eal_common_pci.c
@@ -47,6 +47,7 @@
 #include <rte_pci.h>
 #include <rte_per_lcore.h>
 #include <rte_memory.h>
+#include <rte_memcpy.h>
 #include <rte_memzone.h>
 #include <rte_eal.h>
 #include <rte_string_fns.h>
@@ -500,11 +501,54 @@ pci_find_device(rte_dev_cmp_t cmp, const void *data)
 	return NULL;
 }
 
+static struct rte_device *
+pci_plug(struct rte_devargs *da)
+{
+	struct rte_pci_device *pdev;
+	struct rte_pci_addr *addr;
+
+	addr = &da->pci.addr;
+	/*
+	 * Update eventual pci device in global list.
+	 * Insert it if none was found.
+	 */
+	if (pci_update_device(addr) < 0) {
+		rte_errno = EIO;
+		return NULL;
+	}
+	/* Find the current device holding this address in the bus. */
+	FOREACH_DEVICE_ON_PCIBUS(pdev) {
+		if (rte_eal_compare_pci_addr(&pdev->addr, addr) == 0) {
+			if (rte_pci_probe_one(addr)) {
+				rte_errno = ENODEV;
+				return NULL;
+			}
+			break;
+		}
+	}
+	return pdev ? &pdev->device : NULL;
+}
+
+static int
+pci_unplug(struct rte_device *dev)
+{
+	struct rte_pci_device *pdev;
+
+	pdev = RTE_DEV_TO_PCI(dev);
+	if (rte_pci_detach(&pdev->addr)) {
+		rte_errno = ENODEV;
+		return -1;
+	}
+	return 0;
+}
+
 struct rte_pci_bus rte_pci_bus = {
 	.bus = {
 		.scan = rte_pci_scan,
 		.probe = rte_pci_probe,
 		.find_device = pci_find_device,
+		.plug = pci_plug,
+		.unplug = pci_unplug,
 	},
 	.device_list = TAILQ_HEAD_INITIALIZER(rte_pci_bus.device_list),
 	.driver_list = TAILQ_HEAD_INITIALIZER(rte_pci_bus.driver_list),
-- 
2.1.4

[PATCH v5 12/12] eal: make virtual driver probe and remove take rte_vdev_device

[Gaetan Rivet]

From: Jan Blunck <jblunck@infradead.org>

This is a preparation to embed the generic rte_device into the rte_eth_dev
also for virtual devices.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/eal_common_dev.c | 93 ++++++++++++++++++++++++++--------
 1 file changed, 71 insertions(+), 22 deletions(-)

diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c
index a400ddd..d83ae41 100644
--- a/lib/librte_eal/common/eal_common_dev.c
+++ b/lib/librte_eal/common/eal_common_dev.c
@@ -37,6 +37,7 @@
 #include <inttypes.h>
 #include <sys/queue.h>
 
+#include <rte_bus.h>
 #include <rte_dev.h>
 #include <rte_devargs.h>
 #include <rte_debug.h>
@@ -45,50 +46,98 @@
 
 #include "eal_private.h"
 
+static int cmp_detached_dev_name(const struct rte_device *dev,
+	const void *_name)
+{
+	const char *name = _name;
+
+	/* skip attached devices */
+	if (dev->driver)
+		return 0;
+
+	return strcmp(dev->name, name);
+}
+
 int rte_eal_dev_attach(const char *name, const char *devargs)
 {
-	struct rte_pci_addr addr;
+	struct rte_device *dev;
+	int ret;
 
 	if (name == NULL || devargs == NULL) {
 		RTE_LOG(ERR, EAL, "Invalid device or arguments provided\n");
 		return -EINVAL;
 	}
 
-	if (eal_parse_pci_DomBDF(name, &addr) == 0) {
-		if (rte_pci_probe_one(&addr) < 0)
-			goto err;
+	dev = rte_bus_find_device(cmp_detached_dev_name, name, NULL);
+	if (dev) {
+		struct rte_bus *bus;
+
+		bus = rte_bus_find_by_device(dev);
+		if (!bus) {
+			RTE_LOG(ERR, EAL, "Cannot find bus for device (%s)\n",
+				name);
+			return -EINVAL;
+		}
 
-	} else {
-		if (rte_vdev_init(name, devargs))
-			goto err;
+		if (!bus->plug) {
+			RTE_LOG(ERR, EAL, "Bus function not supported\n");
+			return -ENOTSUP;
+		}
+
+		ret = (bus->plug(dev->devargs) == NULL);
+		goto out;
 	}
 
-	return 0;
+	/*
+	 * If we haven't found a bus device the user meant to "hotplug" a
+	 * virtual device instead.
+	 */
+	ret = rte_vdev_init(name, devargs);
+out:
+	if (ret)
+		RTE_LOG(ERR, EAL, "Driver cannot attach the device (%s)\n",
+			name);
+	return ret;
+}
+
+static int cmp_dev_name(const struct rte_device *dev, const void *_name)
+{
+	const char *name = _name;
 
-err:
-	RTE_LOG(ERR, EAL, "Driver cannot attach the device (%s)\n", name);
-	return -EINVAL;
+	return strcmp(dev->name, name);
 }
 
 int rte_eal_dev_detach(const char *name)
 {
-	struct rte_pci_addr addr;
+	struct rte_device *dev;
+	struct rte_bus *bus;
+	int ret;
 
 	if (name == NULL) {
 		RTE_LOG(ERR, EAL, "Invalid device provided.\n");
 		return -EINVAL;
 	}
 
-	if (eal_parse_pci_DomBDF(name, &addr) == 0) {
-		if (rte_pci_detach(&addr) < 0)
-			goto err;
-	} else {
-		if (rte_vdev_uninit(name))
-			goto err;
+	dev = rte_bus_find_device(cmp_dev_name, name, NULL);
+	if (!dev) {
+		RTE_LOG(ERR, EAL, "Cannot find device (%s)\n", name);
+		return -EINVAL;
+	}
+
+	bus = rte_bus_find_by_device(dev);
+	if (!bus) {
+		RTE_LOG(ERR, EAL, "Cannot find bus for device (%s)\n", name);
+		return -EINVAL;
+	}
+
+	if (!bus->unplug) {
+		RTE_LOG(ERR, EAL, "Bus function not supported\n");
+		return -ENOTSUP;
 	}
-	return 0;
 
-err:
-	RTE_LOG(ERR, EAL, "Driver cannot detach the device (%s)\n", name);
-	return -EINVAL;
+	ret = bus->unplug(dev);
+	if (ret)
+		RTE_LOG(ERR, EAL, "Driver cannot detach the device (%s)\n",
+			name);
+	return ret;
 }
-- 
2.1.4

Re: [PATCH v5 01/12] bus: add bus iterator to find a bus

[Bruce Richardson]

On Mon, Jun 26, 2017 at 02:21:59AM +0200, Gaetan Rivet wrote:
> From: Jan Blunck <jblunck@infradead.org>
> 
> This helper allows to iterate over all registered buses and find one
> matching data used as parameter.
> 
> Signed-off-by: Jan Blunck <jblunck@infradead.org>
> Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> ---
>  lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  1 +
>  lib/librte_eal/common/eal_common_bus.c          | 20 ++++++++++++
>  lib/librte_eal/common/include/rte_bus.h         | 43 +++++++++++++++++++++++++
>  lib/librte_eal/linuxapp/eal/rte_eal_version.map |  1 +
>  4 files changed, 65 insertions(+)
> 

Two minor suggestions below. Otherwise:

Acked-by: Bruce Richardson <bruce.richardson@intel.com>

> diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
> index 2e48a73..ed09ab2 100644
> --- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
> +++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
> @@ -162,6 +162,7 @@ DPDK_17.02 {
>  DPDK_17.05 {
>  	global:
>  
> +	rte_bus_find;
>  	rte_cpu_is_supported;
>  	rte_log_dump;
>  	rte_log_register;
> diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
> index 8f9baf8..4619eb2 100644
> --- a/lib/librte_eal/common/eal_common_bus.c
> +++ b/lib/librte_eal/common/eal_common_bus.c
> @@ -145,3 +145,23 @@ rte_bus_dump(FILE *f)
>  		}
>  	}
>  }
> +
> +struct rte_bus *
> +rte_bus_find(rte_bus_cmp_t cmp,
> +	     const void *data,
> +	     const struct rte_bus *start)
> +{
> +	struct rte_bus *bus = NULL;
> +	int started = start == NULL;

Please put brackets around the "start == NULL" to improve readability.
My first reading of this I assumed it was doing multiple assignment of
both start and started to NULL. To make it extra clear, prefix the
comparison with "!!".

> +
> +	TAILQ_FOREACH(bus, &rte_bus_list, next) {
> +		if (!started) {
> +			if (bus == start)
> +				started = 1;
> +			continue;
> +		}
> +		if (cmp(bus, data) == 0)
> +			break;
> +	}
> +	return bus;
> +}

I also think the name "started" is confusing, and might be better as the
slighly longer "start_found".

Re: [PATCH v5 02/12] bus: add device iterator method

[Bruce Richardson]

On Mon, Jun 26, 2017 at 02:22:00AM +0200, Gaetan Rivet wrote:
> From: Jan Blunck <jblunck@infradead.org>
> 
> Signed-off-by: Jan Blunck <jblunck@infradead.org>
> Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> ---
>  lib/librte_eal/common/include/rte_bus.h | 19 +++++++++++++++++++
>  lib/librte_eal/common/include/rte_dev.h | 21 +++++++++++++++++++++
>  2 files changed, 40 insertions(+)
> 
> diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
> index ecf839b..5efb76e 100644
> --- a/lib/librte_eal/common/include/rte_bus.h
> +++ b/lib/librte_eal/common/include/rte_bus.h
> @@ -82,6 +82,24 @@ typedef int (*rte_bus_scan_t)(void);
>  typedef int (*rte_bus_probe_t)(void);
>  
>  /**
> + * Device iterator to find a device on a bus.
> + *
> + * This function returns an rte_device if one of those held by the bus
> + * matches the data passed as parameter.
> + *
> + * @param cmp
> + *	Comparison function.
> + *
> + * @param data
> + *	Data to compare each device against.
> + *
> + * @return
> + *	The first device matching the data, NULL if none exists.
> + */
> +typedef struct rte_device * (*rte_bus_find_device_t)(rte_dev_cmp_t cmp,
> +						     const void *data);
> +
> +/**

The bus find function takes a third, start, parameter. Is it worthwhile
including such a parameter here, for consistency sake if nothing else?

Otherwise:

Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Re: [PATCH v5 03/12] bus: add helper to find which bus holds a device

[Bruce Richardson]

On Mon, Jun 26, 2017 at 02:22:01AM +0200, Gaetan Rivet wrote:
> From: Jan Blunck <jblunck@infradead.org>
> 
> Signed-off-by: Jan Blunck <jblunck@infradead.org>
> Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> ---
>  lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  1 +
>  lib/librte_eal/common/eal_common_bus.c          | 25 +++++++++++++++++++++++++
>  lib/librte_eal/common/include/rte_bus.h         |  5 +++++
>  lib/librte_eal/linuxapp/eal/rte_eal_version.map |  1 +
>  4 files changed, 32 insertions(+)
> 

Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Would it be useful to add a unit test for this function? That would help
with any future refactoring, and also indirectly test the other
functions added in the previous 2 patches.

Re: [PATCH v5 01/12] bus: add bus iterator to find a bus

[Gaëtan Rivet]

Hi Bruce,

Thanks for reading.

On Mon, Jun 26, 2017 at 04:30:55PM +0100, Bruce Richardson wrote:
> On Mon, Jun 26, 2017 at 02:21:59AM +0200, Gaetan Rivet wrote:
> > From: Jan Blunck <jblunck@infradead.org>
> > 
> > This helper allows to iterate over all registered buses and find one
> > matching data used as parameter.
> > 
> > Signed-off-by: Jan Blunck <jblunck@infradead.org>
> > Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> > ---
> >  lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  1 +
> >  lib/librte_eal/common/eal_common_bus.c          | 20 ++++++++++++
> >  lib/librte_eal/common/include/rte_bus.h         | 43 +++++++++++++++++++++++++
> >  lib/librte_eal/linuxapp/eal/rte_eal_version.map |  1 +
> >  4 files changed, 65 insertions(+)
> > 
> 
> Two minor suggestions below. Otherwise:
> 
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> 
> > diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
> > index 2e48a73..ed09ab2 100644
> > --- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
> > +++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
> > @@ -162,6 +162,7 @@ DPDK_17.02 {
> >  DPDK_17.05 {
> >  	global:
> >  
> > +	rte_bus_find;
> >  	rte_cpu_is_supported;
> >  	rte_log_dump;
> >  	rte_log_register;
> > diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
> > index 8f9baf8..4619eb2 100644
> > --- a/lib/librte_eal/common/eal_common_bus.c
> > +++ b/lib/librte_eal/common/eal_common_bus.c
> > @@ -145,3 +145,23 @@ rte_bus_dump(FILE *f)
> >  		}
> >  	}
> >  }
> > +
> > +struct rte_bus *
> > +rte_bus_find(rte_bus_cmp_t cmp,
> > +	     const void *data,
> > +	     const struct rte_bus *start)
> > +{
> > +	struct rte_bus *bus = NULL;
> > +	int started = start == NULL;
> 
> Please put brackets around the "start == NULL" to improve readability.
> My first reading of this I assumed it was doing multiple assignment of
> both start and started to NULL. To make it extra clear, prefix the
> comparison with "!!".
> 
> > +
> > +	TAILQ_FOREACH(bus, &rte_bus_list, next) {
> > +		if (!started) {
> > +			if (bus == start)
> > +				started = 1;
> > +			continue;
> > +		}
> > +		if (cmp(bus, data) == 0)
> > +			break;
> > +	}
> > +	return bus;
> > +}
> 
> I also think the name "started" is confusing, and might be better as the
> slighly longer "start_found".
> 

Agreed on both account, it will be fixed in the next version.

-- 
Gaëtan Rivet
6WIND

Re: [PATCH v5 02/12] bus: add device iterator method

[Gaëtan Rivet]

On Mon, Jun 26, 2017 at 05:20:37PM +0100, Bruce Richardson wrote:
> On Mon, Jun 26, 2017 at 02:22:00AM +0200, Gaetan Rivet wrote:
> > From: Jan Blunck <jblunck@infradead.org>
> > 
> > Signed-off-by: Jan Blunck <jblunck@infradead.org>
> > Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> > ---
> >  lib/librte_eal/common/include/rte_bus.h | 19 +++++++++++++++++++
> >  lib/librte_eal/common/include/rte_dev.h | 21 +++++++++++++++++++++
> >  2 files changed, 40 insertions(+)
> > 
> > diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
> > index ecf839b..5efb76e 100644
> > --- a/lib/librte_eal/common/include/rte_bus.h
> > +++ b/lib/librte_eal/common/include/rte_bus.h
> > @@ -82,6 +82,24 @@ typedef int (*rte_bus_scan_t)(void);
> >  typedef int (*rte_bus_probe_t)(void);
> >  
> >  /**
> > + * Device iterator to find a device on a bus.
> > + *
> > + * This function returns an rte_device if one of those held by the bus
> > + * matches the data passed as parameter.
> > + *
> > + * @param cmp
> > + *	Comparison function.
> > + *
> > + * @param data
> > + *	Data to compare each device against.
> > + *
> > + * @return
> > + *	The first device matching the data, NULL if none exists.
> > + */
> > +typedef struct rte_device * (*rte_bus_find_device_t)(rte_dev_cmp_t cmp,
> > +						     const void *data);
> > +
> > +/**
> 
> The bus find function takes a third, start, parameter. Is it worthwhile
> including such a parameter here, for consistency sake if nothing else?
> 

I was wondering this actually.

Doing so would deport the added complexity, to be repeated in
each bus implementation. It would in turn simplify my own calls,
ever so slightly. That means a few small-ish additional spots of
complexity instead of a single one.

I'm also thinking that maybe a few buses could avoid having linked lists
for their containers (I know it's frowned upon to roll your own, but
some might have good reasons for doing so), allowing random access, bypassing
cycling through each.

Well, I will do as you suggest, if anyone objects, please shout before I send the
next version.

> Otherwise:
> 
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>

-- 
Gaëtan Rivet
6WIND

Re: [PATCH v5 03/12] bus: add helper to find which bus holds a device

[Gaëtan Rivet]

On Mon, Jun 26, 2017 at 05:31:45PM +0100, Bruce Richardson wrote:
> On Mon, Jun 26, 2017 at 02:22:01AM +0200, Gaetan Rivet wrote:
> > From: Jan Blunck <jblunck@infradead.org>
> > 
> > Signed-off-by: Jan Blunck <jblunck@infradead.org>
> > Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> > ---
> >  lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  1 +
> >  lib/librte_eal/common/eal_common_bus.c          | 25 +++++++++++++++++++++++++
> >  lib/librte_eal/common/include/rte_bus.h         |  5 +++++
> >  lib/librte_eal/linuxapp/eal/rte_eal_version.map |  1 +
> >  4 files changed, 32 insertions(+)
> > 
> 
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> 
> Would it be useful to add a unit test for this function? That would help
> with any future refactoring, and also indirectly test the other
> functions added in the previous 2 patches.

Yes, I think it would be better to have unit-tests for those functions.
They can be a little tricky to get right, working with the layers of
callbacks.

I may be short on time for the next version though. I'll see.

-- 
Gaëtan Rivet
6WIND

Re: [PATCH v5 04/12] bus: add bus iterator to find a device

[Bruce Richardson]

On Mon, Jun 26, 2017 at 02:22:02AM +0200, Gaetan Rivet wrote:
> From: Jan Blunck <jblunck@infradead.org>
> 
> Signed-off-by: Jan Blunck <jblunck@infradead.org>
> Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> ---
>  lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  1 +
>  lib/librte_eal/common/eal_common_bus.c          | 24 +++++++++++++++++++++++
>  lib/librte_eal/common/include/rte_bus.h         | 26 +++++++++++++++++++++++++
>  lib/librte_eal/linuxapp/eal/rte_eal_version.map |  1 +
>  4 files changed, 52 insertions(+)
> 

Same comment for previous patch on naming of "started" variable.

Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Re: [PATCH v5 05/12] bus: introduce hotplug functionality

[Bruce Richardson]

On Mon, Jun 26, 2017 at 02:22:03AM +0200, Gaetan Rivet wrote:
> From: Jan Blunck <jblunck@infradead.org>
> 
> Signed-off-by: Jan Blunck <jblunck@infradead.org>
> Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> ---
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Re: [PATCH v5 06/12] vdev: implement find_device bus operation

[Bruce Richardson]

On Mon, Jun 26, 2017 at 02:22:04AM +0200, Gaetan Rivet wrote:
> From: Jan Blunck <jblunck@infradead.org>
> 
> Signed-off-by: Jan Blunck <jblunck@infradead.org>
> Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> ---
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Re: [PATCH v5 07/12] vdev: implement hotplug functionality

[Bruce Richardson]

On Mon, Jun 26, 2017 at 02:22:05AM +0200, Gaetan Rivet wrote:
> Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> ---
>  lib/librte_eal/common/eal_common_vdev.c | 36 +++++++++++++++++++++++++++++++++
>  1 file changed, 36 insertions(+)
> 
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Re: [PATCH v5 08/12] vdev: expose bus name

[Bruce Richardson]

On Mon, Jun 26, 2017 at 02:22:06AM +0200, Gaetan Rivet wrote:
> Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> ---

The reason for this change is not explained, so either it explain it, or
better, IMHO, just merge with the next patch, since it's a one-line
change.

If kept as separate patch:

Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Re: [PATCH v5 09/12] vdev: use standard bus registration function

[Bruce Richardson]

On Mon, Jun 26, 2017 at 02:22:07AM +0200, Gaetan Rivet wrote:
> Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> ---
>  lib/librte_eal/common/eal_common_vdev.c | 18 +-----------------
>  1 file changed, 1 insertion(+), 17 deletions(-)
> 
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Re: [PATCH v5 10/12] pci: implement find_device bus operation

[Bruce Richardson]

On Mon, Jun 26, 2017 at 02:22:08AM +0200, Gaetan Rivet wrote:
> Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> ---
>  lib/librte_eal/common/eal_common_pci.c | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
> 
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Re: [PATCH v5 11/12] pci: implement hotplug bus operation

[Bruce Richardson]

On Mon, Jun 26, 2017 at 02:22:09AM +0200, Gaetan Rivet wrote:
> Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> ---
>  lib/librte_eal/common/eal_common_pci.c | 44 ++++++++++++++++++++++++++++++++++
>  1 file changed, 44 insertions(+)
> 
> diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c
> index 00d48d9..286357d 100644
> --- a/lib/librte_eal/common/eal_common_pci.c
> +++ b/lib/librte_eal/common/eal_common_pci.c
> @@ -47,6 +47,7 @@
>  #include <rte_pci.h>
>  #include <rte_per_lcore.h>
>  #include <rte_memory.h>
> +#include <rte_memcpy.h>
>  #include <rte_memzone.h>
>  #include <rte_eal.h>
>  #include <rte_string_fns.h>
> @@ -500,11 +501,54 @@ pci_find_device(rte_dev_cmp_t cmp, const void *data)
>  	return NULL;
>  }
>  
> +static struct rte_device *
> +pci_plug(struct rte_devargs *da)
> +{
> +	struct rte_pci_device *pdev;
> +	struct rte_pci_addr *addr;
> +
> +	addr = &da->pci.addr;
> +	/*
> +	 * Update eventual pci device in global list.
> +	 * Insert it if none was found.
> +	 */
> +	if (pci_update_device(addr) < 0) {
> +		rte_errno = EIO;
> +		return NULL;
> +	}
> +	/* Find the current device holding this address in the bus. */
> +	FOREACH_DEVICE_ON_PCIBUS(pdev) {
> +		if (rte_eal_compare_pci_addr(&pdev->addr, addr) == 0) {
> +			if (rte_pci_probe_one(addr)) {

Please put the != 0, or == -1 in the condition, to make it clear it's an
error leg.

> +				rte_errno = ENODEV;
> +				return NULL;
> +			}
> +			break;
> +		}
> +	}
> +	return pdev ? &pdev->device : NULL;
> +}

Please put in explicit != NULL, as per coding standards here.

> +
> +static int
> +pci_unplug(struct rte_device *dev)
> +{
> +	struct rte_pci_device *pdev;
> +
> +	pdev = RTE_DEV_TO_PCI(dev);
> +	if (rte_pci_detach(&pdev->addr)) {

As above, please check for == or != some value.

> +		rte_errno = ENODEV;
> +		return -1;
> +	}
> +	return 0;
> +}
> +
>  struct rte_pci_bus rte_pci_bus = {
>  	.bus = {
>  		.scan = rte_pci_scan,
>  		.probe = rte_pci_probe,
>  		.find_device = pci_find_device,
> +		.plug = pci_plug,
> +		.unplug = pci_unplug,
>  	},
>  	.device_list = TAILQ_HEAD_INITIALIZER(rte_pci_bus.device_list),
>  	.driver_list = TAILQ_HEAD_INITIALIZER(rte_pci_bus.driver_list),
> -- 
> 2.1.4
> 
With above fixes,

Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Re: [PATCH v5 04/12] bus: add bus iterator to find a device

[Bruce Richardson]

On Mon, Jun 26, 2017 at 02:22:02AM +0200, Gaetan Rivet wrote:
> From: Jan Blunck <jblunck@infradead.org>
> 
> Signed-off-by: Jan Blunck <jblunck@infradead.org>
> Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> ---
>  lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  1 +
>  lib/librte_eal/common/eal_common_bus.c          | 24 +++++++++++++++++++++++
>  lib/librte_eal/common/include/rte_bus.h         | 26 +++++++++++++++++++++++++
>  lib/librte_eal/linuxapp/eal/rte_eal_version.map |  1 +
>  4 files changed, 52 insertions(+)
> 
> diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
> index f1a0765..21640d6 100644
> --- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
> +++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
> @@ -164,6 +164,7 @@ DPDK_17.05 {
>  
>  	rte_bus_find;
>  	rte_bus_find_by_device;
> +	rte_bus_find_device;
>  	rte_cpu_is_supported;
>  	rte_log_dump;
>  	rte_log_register;
> diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
> index d208214..63fd9f1 100644
> --- a/lib/librte_eal/common/eal_common_bus.c
> +++ b/lib/librte_eal/common/eal_common_bus.c
> @@ -190,3 +190,27 @@ rte_bus_find_by_device(const struct rte_device *dev)
>  {
>  	return rte_bus_find(bus_find_device, (const void *)dev, NULL);
>  }
> +
> +struct rte_device *
> +rte_bus_find_device(rte_dev_cmp_t cmp, const void *data,
> +		    const struct rte_device *start)

One additional suggestion: might it be worthwhile also returning the bus
for the device here, in an optional 4th parameter. This is, after all,
a bus API. :-)

/Bruce

Re: [PATCH v5 12/12] eal: make virtual driver probe and remove take rte_vdev_device

[Bruce Richardson]

On Mon, Jun 26, 2017 at 02:22:10AM +0200, Gaetan Rivet wrote:
> From: Jan Blunck <jblunck@infradead.org>
> 
> This is a preparation to embed the generic rte_device into the rte_eth_dev
> also for virtual devices.
> 
> Signed-off-by: Jan Blunck <jblunck@infradead.org>
> Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> ---
>  lib/librte_eal/common/eal_common_dev.c | 93 ++++++++++++++++++++++++++--------
>  1 file changed, 71 insertions(+), 22 deletions(-)
> 
> diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c
> index a400ddd..d83ae41 100644
> --- a/lib/librte_eal/common/eal_common_dev.c
> +++ b/lib/librte_eal/common/eal_common_dev.c
> @@ -37,6 +37,7 @@
>  #include <inttypes.h>
>  #include <sys/queue.h>
>  
> +#include <rte_bus.h>
>  #include <rte_dev.h>
>  #include <rte_devargs.h>
>  #include <rte_debug.h>
> @@ -45,50 +46,98 @@
>  
>  #include "eal_private.h"
>  
> +static int cmp_detached_dev_name(const struct rte_device *dev,
> +	const void *_name)
> +{
> +	const char *name = _name;
> +
> +	/* skip attached devices */
> +	if (dev->driver)
> +		return 0;
> +

Does returning 0 from this function not mean that all already-attached
devices with match? Is that really what we want, as it doesn't seem to
match the logic in the function below. Please explain if I'm wrong here.

> +	return strcmp(dev->name, name);
> +}
> +
>  int rte_eal_dev_attach(const char *name, const char *devargs)
>  {
> -	struct rte_pci_addr addr;
> +	struct rte_device *dev;
> +	int ret;
>  
>  	if (name == NULL || devargs == NULL) {
>  		RTE_LOG(ERR, EAL, "Invalid device or arguments provided\n");
>  		return -EINVAL;
>  	}
>  
> -	if (eal_parse_pci_DomBDF(name, &addr) == 0) {
> -		if (rte_pci_probe_one(&addr) < 0)
> -			goto err;
> +	dev = rte_bus_find_device(cmp_detached_dev_name, name, NULL);
> +	if (dev) {
> +		struct rte_bus *bus;
> +
> +		bus = rte_bus_find_by_device(dev);
> +		if (!bus) {
> +			RTE_LOG(ERR, EAL, "Cannot find bus for device (%s)\n",
> +				name);
> +			return -EINVAL;
> +		}
>  
> -	} else {
> -		if (rte_vdev_init(name, devargs))
> -			goto err;
> +		if (!bus->plug) {
> +			RTE_LOG(ERR, EAL, "Bus function not supported\n");
> +			return -ENOTSUP;
> +		}
> +
> +		ret = (bus->plug(dev->devargs) == NULL);
> +		goto out;
>  	}
>  
> -	return 0;
> +	/*
> +	 * If we haven't found a bus device the user meant to "hotplug" a
> +	 * virtual device instead.
> +	 */
> +	ret = rte_vdev_init(name, devargs);
> +out:
> +	if (ret)
> +		RTE_LOG(ERR, EAL, "Driver cannot attach the device (%s)\n",
> +			name);
> +	return ret;
> +}
> +
<snip>

Regards,
/Bruce

Re: [PATCH v5 12/12] eal: make virtual driver probe and remove take rte_vdev_device

[Bruce Richardson]

I'm also not sure about the commit title of this patch. It doesn't match
my understanding of what the patch does.

/Bruce

On Mon, Jun 26, 2017 at 02:22:10AM +0200, Gaetan Rivet wrote:
> From: Jan Blunck <jblunck@infradead.org>
> 
> This is a preparation to embed the generic rte_device into the rte_eth_dev
> also for virtual devices.
> 
> Signed-off-by: Jan Blunck <jblunck@infradead.org>
> Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> ---

Re: [PATCH v5 00/12] bus: attach / detach API

[Bruce Richardson]

On Mon, Jun 26, 2017 at 02:21:58AM +0200, Gaetan Rivet wrote:
> Following the work from Jan:
> 
> This patchset introduces the attach / detach API to rte_bus.
> The rte_device structure is used as the generic device representation.
> 
> This API is implemented for the virtual bus.
> The functions rte_eal_dev_attach and rte_eal_dev_detach are updated to
> use this new interface.
> 
Other than my opens on patch 12, this set all looks good to me. The
other comments I made are mostly just suggestions.

Hopefully next version can be merged.

Regards,
/Bruce

Re: [PATCH v5 12/12] eal: make virtual driver probe and remove take rte_vdev_device

[Gaëtan Rivet]

On Tue, Jun 27, 2017 at 02:58:55PM +0100, Bruce Richardson wrote:
> On Mon, Jun 26, 2017 at 02:22:10AM +0200, Gaetan Rivet wrote:
> > From: Jan Blunck <jblunck@infradead.org>
> > 
> > This is a preparation to embed the generic rte_device into the rte_eth_dev
> > also for virtual devices.
> > 
> > Signed-off-by: Jan Blunck <jblunck@infradead.org>
> > Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> > ---
> >  lib/librte_eal/common/eal_common_dev.c | 93 ++++++++++++++++++++++++++--------
> >  1 file changed, 71 insertions(+), 22 deletions(-)
> > 
> > diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c
> > index a400ddd..d83ae41 100644
> > --- a/lib/librte_eal/common/eal_common_dev.c
> > +++ b/lib/librte_eal/common/eal_common_dev.c
> > @@ -37,6 +37,7 @@
> >  #include <inttypes.h>
> >  #include <sys/queue.h>
> >  
> > +#include <rte_bus.h>
> >  #include <rte_dev.h>
> >  #include <rte_devargs.h>
> >  #include <rte_debug.h>
> > @@ -45,50 +46,98 @@
> >  
> >  #include "eal_private.h"
> >  
> > +static int cmp_detached_dev_name(const struct rte_device *dev,
> > +	const void *_name)
> > +{
> > +	const char *name = _name;
> > +
> > +	/* skip attached devices */
> > +	if (dev->driver)
> > +		return 0;
> > +
> 
> Does returning 0 from this function not mean that all already-attached
> devices with match? Is that really what we want, as it doesn't seem to
> match the logic in the function below. Please explain if I'm wrong here.
> 

Yes, this was a mistake, good catch.

> > +	return strcmp(dev->name, name);
> > +}
> > +
> >  int rte_eal_dev_attach(const char *name, const char *devargs)
> >  {
> > -	struct rte_pci_addr addr;
> > +	struct rte_device *dev;
> > +	int ret;
> >  
> >  	if (name == NULL || devargs == NULL) {
> >  		RTE_LOG(ERR, EAL, "Invalid device or arguments provided\n");
> >  		return -EINVAL;
> >  	}
> >  
> > -	if (eal_parse_pci_DomBDF(name, &addr) == 0) {
> > -		if (rte_pci_probe_one(&addr) < 0)
> > -			goto err;
> > +	dev = rte_bus_find_device(cmp_detached_dev_name, name, NULL);
> > +	if (dev) {
> > +		struct rte_bus *bus;
> > +
> > +		bus = rte_bus_find_by_device(dev);
> > +		if (!bus) {
> > +			RTE_LOG(ERR, EAL, "Cannot find bus for device (%s)\n",
> > +				name);
> > +			return -EINVAL;
> > +		}
> >  
> > -	} else {
> > -		if (rte_vdev_init(name, devargs))
> > -			goto err;
> > +		if (!bus->plug) {
> > +			RTE_LOG(ERR, EAL, "Bus function not supported\n");
> > +			return -ENOTSUP;
> > +		}
> > +
> > +		ret = (bus->plug(dev->devargs) == NULL);
> > +		goto out;
> >  	}
> >  
> > -	return 0;
> > +	/*
> > +	 * If we haven't found a bus device the user meant to "hotplug" a
> > +	 * virtual device instead.
> > +	 */
> > +	ret = rte_vdev_init(name, devargs);
> > +out:
> > +	if (ret)
> > +		RTE_LOG(ERR, EAL, "Driver cannot attach the device (%s)\n",
> > +			name);
> > +	return ret;
> > +}
> > +
> <snip>
> 
> Regards,
> /Bruce

On Tue, Jun 27, 2017 at 03:00:58PM +0100, Bruce Richardson wrote:
> I'm also not sure about the commit title of this patch. It doesn't
> match
> my understanding of what the patch does.
>
> /Bruce

I have a few issues with this patch myself. However, to properly fix
those, I need the rte_devargs evolutions coming afterward.

As an intermediate step, this is good enough to support the current
rte_dev attach / detach API and follow the new hotplug API. It makes
sense to have it as part of this series, so that the latter is self-contained
and internally consistent.

But it will evolve.

I will rewrite the commit log.

-- 
Gaëtan Rivet
6WIND

Re: [PATCH v5 00/12] bus: attach / detach API

[Gaëtan Rivet]

On Tue, Jun 27, 2017 at 03:08:17PM +0100, Bruce Richardson wrote:
> On Mon, Jun 26, 2017 at 02:21:58AM +0200, Gaetan Rivet wrote:
> > Following the work from Jan:
> > 
> > This patchset introduces the attach / detach API to rte_bus.
> > The rte_device structure is used as the generic device representation.
> > 
> > This API is implemented for the virtual bus.
> > The functions rte_eal_dev_attach and rte_eal_dev_detach are updated to
> > use this new interface.
> > 
> Other than my opens on patch 12, this set all looks good to me. The
> other comments I made are mostly just suggestions.
> 
> Hopefully next version can be merged.

Thanks for the review, your remarks were useful.
I will send the new version shortly.

> 
> Regards,
> /Bruce

-- 
Gaëtan Rivet
6WIND

Re: [PATCH v5 04/12] bus: add bus iterator to find a device

[Gaëtan Rivet]

On Tue, Jun 27, 2017 at 02:54:34PM +0100, Bruce Richardson wrote:
> On Mon, Jun 26, 2017 at 02:22:02AM +0200, Gaetan Rivet wrote:
> > From: Jan Blunck <jblunck@infradead.org>
> > 
> > Signed-off-by: Jan Blunck <jblunck@infradead.org>
> > Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> > ---
> >  lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  1 +
> >  lib/librte_eal/common/eal_common_bus.c          | 24 +++++++++++++++++++++++
> >  lib/librte_eal/common/include/rte_bus.h         | 26 +++++++++++++++++++++++++
> >  lib/librte_eal/linuxapp/eal/rte_eal_version.map |  1 +
> >  4 files changed, 52 insertions(+)
> > 
> > diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
> > index f1a0765..21640d6 100644
> > --- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
> > +++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
> > @@ -164,6 +164,7 @@ DPDK_17.05 {
> >  
> >  	rte_bus_find;
> >  	rte_bus_find_by_device;
> > +	rte_bus_find_device;
> >  	rte_cpu_is_supported;
> >  	rte_log_dump;
> >  	rte_log_register;
> > diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
> > index d208214..63fd9f1 100644
> > --- a/lib/librte_eal/common/eal_common_bus.c
> > +++ b/lib/librte_eal/common/eal_common_bus.c
> > @@ -190,3 +190,27 @@ rte_bus_find_by_device(const struct rte_device *dev)
> >  {
> >  	return rte_bus_find(bus_find_device, (const void *)dev, NULL);
> >  }
> > +
> > +struct rte_device *
> > +rte_bus_find_device(rte_dev_cmp_t cmp, const void *data,
> > +		    const struct rte_device *start)
> 
> One additional suggestion: might it be worthwhile also returning the bus
> for the device here, in an optional 4th parameter. This is, after all,
> a bus API. :-)
> 
> /Bruce

I think having the bus is a good info.
However,

1. This makes this API slightly less clean I think.
   A subjective matter, but I don't like adding cruft without good
   justification.

2. The bus will be made accessible by my future patches. The series
   afterward are introducing a generic device type representation, which
   is composed of the bus. This type info is integral to the device and
   thus the bus can be read from within the resulting rte_device here.

So, I won't add this parameter right now. If we have issues with the
rte_devargs series, it might be interesting to come back on this and
edit this API.

-- 
Gaëtan Rivet
6WIND

Re: [PATCH v5 12/12] eal: make virtual driver probe and remove take rte_vdev_device

[Bruce Richardson]

On Tue, Jun 27, 2017 at 04:47:20PM +0200, Gaëtan Rivet wrote:
> On Tue, Jun 27, 2017 at 02:58:55PM +0100, Bruce Richardson wrote:
> > On Mon, Jun 26, 2017 at 02:22:10AM +0200, Gaetan Rivet wrote:
> > > From: Jan Blunck <jblunck@infradead.org>
> > > 
> > > This is a preparation to embed the generic rte_device into the rte_eth_dev
> > > also for virtual devices.
> > > 
> > > Signed-off-by: Jan Blunck <jblunck@infradead.org>
> > > Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> > > ---
> > >  lib/librte_eal/common/eal_common_dev.c | 93 ++++++++++++++++++++++++++--------
> > >  1 file changed, 71 insertions(+), 22 deletions(-)
> > > 
> > > diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c
> > > index a400ddd..d83ae41 100644
> > > --- a/lib/librte_eal/common/eal_common_dev.c
> > > +++ b/lib/librte_eal/common/eal_common_dev.c
> > > @@ -37,6 +37,7 @@
> > >  #include <inttypes.h>
> > >  #include <sys/queue.h>
> > >  
> > > +#include <rte_bus.h>
> > >  #include <rte_dev.h>
> > >  #include <rte_devargs.h>
> > >  #include <rte_debug.h>
> > > @@ -45,50 +46,98 @@
> > >  
> > >  #include "eal_private.h"
> > >  
> > > +static int cmp_detached_dev_name(const struct rte_device *dev,
> > > +	const void *_name)
> > > +{
> > > +	const char *name = _name;
> > > +
> > > +	/* skip attached devices */
> > > +	if (dev->driver)
> > > +		return 0;
> > > +
> > 
> > Does returning 0 from this function not mean that all already-attached
> > devices with match? Is that really what we want, as it doesn't seem to
> > match the logic in the function below. Please explain if I'm wrong here.
> > 
> 
> Yes, this was a mistake, good catch.
> 
> > > +	return strcmp(dev->name, name);
> > > +}
> > > +
> > >  int rte_eal_dev_attach(const char *name, const char *devargs)
> > >  {
> > > -	struct rte_pci_addr addr;
> > > +	struct rte_device *dev;
> > > +	int ret;
> > >  
> > >  	if (name == NULL || devargs == NULL) {
> > >  		RTE_LOG(ERR, EAL, "Invalid device or arguments provided\n");
> > >  		return -EINVAL;
> > >  	}
> > >  
> > > -	if (eal_parse_pci_DomBDF(name, &addr) == 0) {
> > > -		if (rte_pci_probe_one(&addr) < 0)
> > > -			goto err;
> > > +	dev = rte_bus_find_device(cmp_detached_dev_name, name, NULL);
> > > +	if (dev) {
> > > +		struct rte_bus *bus;
> > > +
> > > +		bus = rte_bus_find_by_device(dev);
> > > +		if (!bus) {
> > > +			RTE_LOG(ERR, EAL, "Cannot find bus for device (%s)\n",
> > > +				name);
> > > +			return -EINVAL;
> > > +		}
> > >  
> > > -	} else {
> > > -		if (rte_vdev_init(name, devargs))
> > > -			goto err;
> > > +		if (!bus->plug) {
> > > +			RTE_LOG(ERR, EAL, "Bus function not supported\n");
> > > +			return -ENOTSUP;
> > > +		}
> > > +
> > > +		ret = (bus->plug(dev->devargs) == NULL);
> > > +		goto out;
> > >  	}
> > >  
> > > -	return 0;
> > > +	/*
> > > +	 * If we haven't found a bus device the user meant to "hotplug" a
> > > +	 * virtual device instead.
> > > +	 */
> > > +	ret = rte_vdev_init(name, devargs);
> > > +out:
> > > +	if (ret)
> > > +		RTE_LOG(ERR, EAL, "Driver cannot attach the device (%s)\n",
> > > +			name);
> > > +	return ret;
> > > +}
> > > +
> > <snip>
> > 
> > Regards,
> > /Bruce
> 
> On Tue, Jun 27, 2017 at 03:00:58PM +0100, Bruce Richardson wrote:
> > I'm also not sure about the commit title of this patch. It doesn't
> > match
> > my understanding of what the patch does.
> >
> > /Bruce
> 
> I have a few issues with this patch myself. However, to properly fix
> those, I need the rte_devargs evolutions coming afterward.
> 
> As an intermediate step, this is good enough to support the current
> rte_dev attach / detach API and follow the new hotplug API. It makes
> sense to have it as part of this series, so that the latter is self-contained
> and internally consistent.
> 
> But it will evolve.
> 
> I will rewrite the commit log.
> 
> -- 
Ok, with bug fix and clearer commit log:

Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Re: [PATCH v5 04/12] bus: add bus iterator to find a device

[Bruce Richardson]

On Tue, Jun 27, 2017 at 05:05:14PM +0200, Gaëtan Rivet wrote:
> On Tue, Jun 27, 2017 at 02:54:34PM +0100, Bruce Richardson wrote:
> > On Mon, Jun 26, 2017 at 02:22:02AM +0200, Gaetan Rivet wrote:
> > > From: Jan Blunck <jblunck@infradead.org>
> > > 
> > > Signed-off-by: Jan Blunck <jblunck@infradead.org>
> > > Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> > > ---
> > >  lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  1 +
> > >  lib/librte_eal/common/eal_common_bus.c          | 24 +++++++++++++++++++++++
> > >  lib/librte_eal/common/include/rte_bus.h         | 26 +++++++++++++++++++++++++
> > >  lib/librte_eal/linuxapp/eal/rte_eal_version.map |  1 +
> > >  4 files changed, 52 insertions(+)
> > > 
> > > diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
> > > index f1a0765..21640d6 100644
> > > --- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
> > > +++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
> > > @@ -164,6 +164,7 @@ DPDK_17.05 {
> > >  
> > >  	rte_bus_find;
> > >  	rte_bus_find_by_device;
> > > +	rte_bus_find_device;
> > >  	rte_cpu_is_supported;
> > >  	rte_log_dump;
> > >  	rte_log_register;
> > > diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
> > > index d208214..63fd9f1 100644
> > > --- a/lib/librte_eal/common/eal_common_bus.c
> > > +++ b/lib/librte_eal/common/eal_common_bus.c
> > > @@ -190,3 +190,27 @@ rte_bus_find_by_device(const struct rte_device *dev)
> > >  {
> > >  	return rte_bus_find(bus_find_device, (const void *)dev, NULL);
> > >  }
> > > +
> > > +struct rte_device *
> > > +rte_bus_find_device(rte_dev_cmp_t cmp, const void *data,
> > > +		    const struct rte_device *start)
> > 
> > One additional suggestion: might it be worthwhile also returning the bus
> > for the device here, in an optional 4th parameter. This is, after all,
> > a bus API. :-)
> > 
> > /Bruce
> 
> I think having the bus is a good info.
> However,
> 
> 1. This makes this API slightly less clean I think.
>    A subjective matter, but I don't like adding cruft without good
>    justification.
> 
> 2. The bus will be made accessible by my future patches. The series
>    afterward are introducing a generic device type representation, which
>    is composed of the bus. This type info is integral to the device and
>    thus the bus can be read from within the resulting rte_device here.
> 

So the device will contain a pointer to it's parent bus? If so, then ok,
let's leave out the extra parameter.

/Bruce

> So, I won't add this parameter right now. If we have issues with the
> rte_devargs series, it might be interesting to come back on this and
> edit this API.
> 
> -- 
> Gaëtan Rivet
> 6WIND

[PATCH v6 00/11] bus: attach / detach API

[Gaetan Rivet]

Following the work from Jan:

This patchset introduces the attach / detach API to rte_bus.
The rte_device structure is used as the generic device representation.

This API is implemented for the virtual bus.
The functions rte_eal_dev_attach and rte_eal_dev_detach are updated to
use this new interface.

-- v2

0. API rework
-------------

I would like to propose an evolution on the API developed by Jan.

The attach / detach rte_bus API is necessary to support the attach/detach
rte_dev API. Those are two different levels for one similar functionality.

Attach / detach does not allow true hotplugging, because the attach
function expects the devices operated upon to already exist within the
buses / sub-layers. This means that this API expects devices meta-datas
(bus-internal device representation and associated device information
read from the system) to be present upon attach. This part of the work
is done during scanning.

While it is best to avoid changing the public rte_dev API as it already
exists, nothing prevents this new rte_bus API from superseeding it.
It has been said during the previous release cycle that device hotplug
was a feature that interested users. True hotplug is not allowed by the
current attach / detach API. Worse, this API hinders the effort to bring
this new functionality by squatting its semantic field.

Thus, I propose to rename rte_bus attach / detach; plug / unplug. As it
is a superset of the attach / detach functionality, it can be used to
implement rte_dev attach / detach. Now is the right time to pivot to
this new feature.

This should help maintainers understanding the aim of this API and the
differences with the APIs higher-up, clarify the field and allow a new
functionality to be proposed.

The vdev bus is inherently supporting the new API, however it has been
made explicit. My implementation in the PCI bus in further patchset also
follows the rte_bus hotplug API instead of only attach / detach.

One remaining problem with the vdev bus is the rte_dev attach
implementation, which needs the rte_devargs rework to be properly fixed.

1. Additional evolutions in the patchset
----------------------------------------

The RTE_VERIFY on the find_device is too stringent I think and forces
all buses to implement a public device iterator. While it could be
argued that it would push for quicker support for the functionality, I
think it's possible that some buses are not interested at all in it and
should simply be ignored.

The bus devices iterator has been fixed.

The internal rte_device handle was not properly setup within the
net_ring PMD.

-- v3

The new API is now

typedef struct rte_device * (*rte_bus_plug_t)(struct rte_devargs *da);
typedef int (*rte_bus_unplug_t)(struct rte_device *dev);

So, plugging a device takes an rte_devargs as input and returns an rte_device.
While implementing related subsystems, I found that I usually needed
this rte_device handle upon a successful device plugging. This seems the
sensible and useful thing to do.
As such, on error NULL is returned and rte_errno is set by the bus.

Unplugging a device however now returns to the first version, which used
an rte_device. The explicit contract here is that if one has an
rte_device that has been obtained by calling bus->plug(), then this
handle can be used for bus->unplug().

Additionally, bus and device comparators now returns 0 on match,
following strcmp-like behavior.

-- v4

* rte_bus_find now takes a *start* parameter, that can be null.
  The bus search starts from this element if set.

* A few doc fixes.

* The rte_device field was fixed within the rte_ring PMD in a previous patch.
  This fix has been integrated by other means, it is not necessary anymore.

-- v5

* The commit
ethdev: use embedded rte_device to detach driver
  has been removed from this series to be sent separately.

* The PCI support for device access and hotplug is merged in this series
  instead of being proposed as a separate patchset.

* A few nitpicks to the code itself have been fixed.

* Some documentation has been reworked.

-- v6

* Fixed a bug in rte_eal_dev_attach with a wrong comparison callback used
  to find detached devices.

* Fixed coding style errors in PCI hotplug implementation.

* rte_bus find_device() method now takes a *start* parameter.

Gaetan Rivet (4):
  vdev: implement hotplug bus operation
  vdev: use standard bus registration function
  pci: implement find_device bus operation
  pci: implement hotplug bus operation

Jan Blunck (7):
  bus: add bus iterator to find a bus
  bus: add device iterator method
  bus: add helper to find which bus holds a device
  bus: add bus iterator to find a device
  bus: introduce hotplug functionality
  vdev: implement find_device bus operation
  dev: use new hotplug API in attach / detach

 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |   3 +
 lib/librte_eal/common/eal_common_bus.c          |  64 ++++++++++++
 lib/librte_eal/common/eal_common_dev.c          |  92 ++++++++++++----
 lib/librte_eal/common/eal_common_pci.c          |  65 ++++++++++++
 lib/librte_eal/common/eal_common_vdev.c         |  73 ++++++++++---
 lib/librte_eal/common/include/rte_bus.h         | 133 ++++++++++++++++++++++++
 lib/librte_eal/common/include/rte_dev.h         |  21 ++++
 lib/librte_eal/common/include/rte_vdev.h        |   2 +
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |   3 +
 9 files changed, 418 insertions(+), 38 deletions(-)

-- 
2.1.4

[PATCH v6 01/11] bus: add bus iterator to find a bus

[Gaetan Rivet]

From: Jan Blunck <jblunck@infradead.org>

This helper allows to iterate over all registered buses and find one
matching data used as parameter.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  1 +
 lib/librte_eal/common/eal_common_bus.c          | 20 ++++++++++++
 lib/librte_eal/common/include/rte_bus.h         | 43 +++++++++++++++++++++++++
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |  1 +
 4 files changed, 65 insertions(+)

diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
index 2e48a73..ed09ab2 100644
--- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
@@ -162,6 +162,7 @@ DPDK_17.02 {
 DPDK_17.05 {
 	global:
 
+	rte_bus_find;
 	rte_cpu_is_supported;
 	rte_log_dump;
 	rte_log_register;
diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
index 8f9baf8..86a54d1 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -145,3 +145,23 @@ rte_bus_dump(FILE *f)
 		}
 	}
 }
+
+struct rte_bus *
+rte_bus_find(rte_bus_cmp_t cmp,
+	     const void *data,
+	     const struct rte_bus *start)
+{
+	struct rte_bus *bus = NULL;
+	int start_found = !!(start == NULL);
+
+	TAILQ_FOREACH(bus, &rte_bus_list, next) {
+		if (!start_found) {
+			if (bus == start)
+				start_found = 1;
+			continue;
+		}
+		if (cmp(bus, data) == 0)
+			break;
+	}
+	return bus;
+}
diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
index 5f47b82..ecf839b 100644
--- a/lib/librte_eal/common/include/rte_bus.h
+++ b/lib/librte_eal/common/include/rte_bus.h
@@ -141,6 +141,49 @@ int rte_bus_probe(void);
 void rte_bus_dump(FILE *f);
 
 /**
+ * Bus comparison function.
+ *
+ * @param bus
+ *	Bus under test.
+ *
+ * @param data
+ *	Data to compare against.
+ *
+ * @return
+ *	0 if the bus matches the data.
+ *	!0 if the bus does not match.
+ *	<0 if ordering is possible and the bus is lower than the data.
+ *	>0 if ordering is possible and the bus is greater than the data.
+ */
+typedef int (*rte_bus_cmp_t)(const struct rte_bus *bus, const void *data);
+
+/**
+ * Bus iterator to find a particular bus.
+ *
+ * This function compares each registered bus to find one that matches
+ * the data passed as parameter.
+ *
+ * If the comparison function returns zero this function will stop iterating
+ * over any more buses. To continue a search the bus of a previous search can
+ * be passed via the start parameter.
+ *
+ * @param cmp
+ *	Comparison function.
+ *
+ * @param data
+ *	 Data to pass to comparison function.
+ *
+ * @param start
+ *	Starting point for the iteration.
+ *
+ * @return
+ *	 A pointer to a rte_bus structure or NULL in case no bus matches
+ */
+struct rte_bus *rte_bus_find(rte_bus_cmp_t cmp,
+			     const void *data,
+			     const struct rte_bus *start);
+
+/**
  * Helper for Bus registration.
  * The constructor has higher priority than PMD constructors.
  */
diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
index 670bab3..6efa517 100644
--- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
@@ -166,6 +166,7 @@ DPDK_17.02 {
 DPDK_17.05 {
 	global:
 
+	rte_bus_find;
 	rte_cpu_is_supported;
 	rte_intr_free_epoll_fd;
 	rte_log_dump;
-- 
2.1.4

[PATCH v6 02/11] bus: add device iterator method

[Gaetan Rivet]

From: Jan Blunck <jblunck@infradead.org>

This new method allows buses to expose their devices in a controlled
manner. A comparison function is provided by the user to discriminate
between devices, using arbitrary data as identifier.

It is possible to start an iteration from a specific point, in order to
continue a search.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/include/rte_bus.h | 28 ++++++++++++++++++++++++++++
 lib/librte_eal/common/include/rte_dev.h | 21 +++++++++++++++++++++
 2 files changed, 49 insertions(+)

diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
index ecf839b..51f41d9 100644
--- a/lib/librte_eal/common/include/rte_bus.h
+++ b/lib/librte_eal/common/include/rte_bus.h
@@ -82,6 +82,33 @@ typedef int (*rte_bus_scan_t)(void);
 typedef int (*rte_bus_probe_t)(void);
 
 /**
+ * Device iterator to find a device on a bus.
+ *
+ * This function returns an rte_device if one of those held by the bus
+ * matches the data passed as parameter.
+ *
+ * If the comparison function returns zero this function should stop iterating
+ * over any more devices. To continue a search the device of a previous search
+ * can be passed via the start parameter.
+ *
+ * @param cmp
+ *	Comparison function.
+ *
+ * @param data
+ *	Data to compare each device against.
+ *
+ * @param start
+ *	starting point for the iteration
+ *
+ * @return
+ *	The first device matching the data, NULL if none exists.
+ */
+typedef struct rte_device *
+(*rte_bus_find_device_t)(rte_dev_cmp_t cmp,
+			 const void *data,
+			 const struct rte_device *start);
+
+/**
  * A structure describing a generic bus.
  */
 struct rte_bus {
@@ -89,6 +116,7 @@ struct rte_bus {
 	const char *name;            /**< Name of the bus */
 	rte_bus_scan_t scan;         /**< Scan for devices attached to bus */
 	rte_bus_probe_t probe;       /**< Probe devices on bus */
+	rte_bus_find_device_t find_device; /**< Find a device on the bus */
 };
 
 /**
diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
index de20c06..04d9c28 100644
--- a/lib/librte_eal/common/include/rte_dev.h
+++ b/lib/librte_eal/common/include/rte_dev.h
@@ -191,6 +191,27 @@ int rte_eal_dev_attach(const char *name, const char *devargs);
  */
 int rte_eal_dev_detach(const char *name);
 
+/**
+ * Device comparison function.
+ *
+ * This type of function is used to compare an rte_device with arbitrary
+ * data.
+ *
+ * @param dev
+ *   Device handle.
+ *
+ * @param data
+ *   Data to compare against. The type of this parameter is determined by
+ *   the kind of comparison performed by the function.
+ *
+ * @return
+ *   0 if the device matches the data.
+ *   !0 if the device does not match.
+ *   <0 if ordering is possible and the device is lower than the data.
+ *   >0 if ordering is possible and the device is greater than the data.
+ */
+typedef int (*rte_dev_cmp_t)(const struct rte_device *dev, const void *data);
+
 #define RTE_PMD_EXPORT_NAME_ARRAY(n, idx) n##idx[]
 
 #define RTE_PMD_EXPORT_NAME(name, idx) \
-- 
2.1.4

[PATCH v6 03/11] bus: add helper to find which bus holds a device

[Gaetan Rivet]

From: Jan Blunck <jblunck@infradead.org>

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  1 +
 lib/librte_eal/common/eal_common_bus.c          | 25 +++++++++++++++++++++++++
 lib/librte_eal/common/include/rte_bus.h         |  5 +++++
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |  1 +
 4 files changed, 32 insertions(+)

diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
index ed09ab2..f1a0765 100644
--- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
@@ -163,6 +163,7 @@ DPDK_17.05 {
 	global:
 
 	rte_bus_find;
+	rte_bus_find_by_device;
 	rte_cpu_is_supported;
 	rte_log_dump;
 	rte_log_register;
diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
index 86a54d1..e12dfec 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -165,3 +165,28 @@ rte_bus_find(rte_bus_cmp_t cmp,
 	}
 	return bus;
 }
+
+static int
+cmp_rte_device(const struct rte_device *dev1, const void *_dev2)
+{
+	const struct rte_device *dev2 = _dev2;
+
+	return dev1 != dev2;
+}
+
+static int
+bus_find_device(const struct rte_bus *bus, const void *_dev)
+{
+	struct rte_device *dev;
+
+	if (bus->find_device == NULL)
+		return -1;
+	dev = bus->find_device(cmp_rte_device, _dev, NULL);
+	return dev == NULL;
+}
+
+struct rte_bus *
+rte_bus_find_by_device(const struct rte_device *dev)
+{
+	return rte_bus_find(bus_find_device, (const void *)dev, NULL);
+}
diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
index 51f41d9..ee7b8f0 100644
--- a/lib/librte_eal/common/include/rte_bus.h
+++ b/lib/librte_eal/common/include/rte_bus.h
@@ -212,6 +212,11 @@ struct rte_bus *rte_bus_find(rte_bus_cmp_t cmp,
 			     const struct rte_bus *start);
 
 /**
+ * Find the registered bus for a particular device.
+ */
+struct rte_bus *rte_bus_find_by_device(const struct rte_device *dev);
+
+/**
  * Helper for Bus registration.
  * The constructor has higher priority than PMD constructors.
  */
diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
index 6efa517..6f77222 100644
--- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
@@ -167,6 +167,7 @@ DPDK_17.05 {
 	global:
 
 	rte_bus_find;
+	rte_bus_find_by_device;
 	rte_cpu_is_supported;
 	rte_intr_free_epoll_fd;
 	rte_log_dump;
-- 
2.1.4

[PATCH v6 04/11] bus: add bus iterator to find a device

[Gaetan Rivet]

From: Jan Blunck <jblunck@infradead.org>

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  1 +
 lib/librte_eal/common/eal_common_bus.c          | 17 ++++++++++++++++
 lib/librte_eal/common/include/rte_bus.h         | 26 +++++++++++++++++++++++++
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |  1 +
 4 files changed, 45 insertions(+)

diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
index f1a0765..21640d6 100644
--- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
@@ -164,6 +164,7 @@ DPDK_17.05 {
 
 	rte_bus_find;
 	rte_bus_find_by_device;
+	rte_bus_find_device;
 	rte_cpu_is_supported;
 	rte_log_dump;
 	rte_log_register;
diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
index e12dfec..dde6c83 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -190,3 +190,20 @@ rte_bus_find_by_device(const struct rte_device *dev)
 {
 	return rte_bus_find(bus_find_device, (const void *)dev, NULL);
 }
+
+struct rte_device *
+rte_bus_find_device(rte_dev_cmp_t cmp, const void *data,
+		    const struct rte_device *start)
+{
+	struct rte_bus *bus;
+	struct rte_device *dev = NULL;
+
+	TAILQ_FOREACH(bus, &rte_bus_list, next) {
+		if (bus->find_device == NULL)
+			continue;
+		dev = bus->find_device(cmp, data, start);
+		if (dev)
+			break;
+	}
+	return dev;
+}
diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
index ee7b8f0..9aa047d 100644
--- a/lib/librte_eal/common/include/rte_bus.h
+++ b/lib/librte_eal/common/include/rte_bus.h
@@ -212,6 +212,32 @@ struct rte_bus *rte_bus_find(rte_bus_cmp_t cmp,
 			     const struct rte_bus *start);
 
 /**
+ * Bus iterator to find a particular device.
+ *
+ * This function searches each registered bus to find a device that matches
+ * the data passed as parameter.
+ *
+ * If the comparison function returns zero this function will stop iterating
+ * over any more buses and devices. To continue a search the device of
+ * a previous search can be passed via the start parameter.
+ *
+ * @param cmp
+ *	Comparison function.
+ *
+ * @param data
+ *	Data to pass to comparison function.
+ *
+ * @param start
+ *	Starting point for the iteration.
+ *
+ * @return
+ *	A pointer to an rte_bus structure or NULL in case no device matches.
+ */
+struct rte_device *rte_bus_find_device(rte_dev_cmp_t cmp,
+				       const void *data,
+				       const struct rte_device *start);
+
+/**
  * Find the registered bus for a particular device.
  */
 struct rte_bus *rte_bus_find_by_device(const struct rte_device *dev);
diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
index 6f77222..e0a056d 100644
--- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
@@ -168,6 +168,7 @@ DPDK_17.05 {
 
 	rte_bus_find;
 	rte_bus_find_by_device;
+	rte_bus_find_device;
 	rte_cpu_is_supported;
 	rte_intr_free_epoll_fd;
 	rte_log_dump;
-- 
2.1.4

[PATCH v6 05/11] bus: introduce hotplug functionality

[Gaetan Rivet]

From: Jan Blunck <jblunck@infradead.org>

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/eal_common_bus.c  |  2 ++
 lib/librte_eal/common/include/rte_bus.h | 31 +++++++++++++++++++++++++++++++
 2 files changed, 33 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
index dde6c83..83d9c07 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -50,6 +50,8 @@ rte_bus_register(struct rte_bus *bus)
 	/* A bus should mandatorily have the scan implemented */
 	RTE_VERIFY(bus->scan);
 	RTE_VERIFY(bus->probe);
+	/* Buses supporting hotplug also require unplug. */
+	RTE_VERIFY(!bus->plug || bus->unplug);
 
 	TAILQ_INSERT_TAIL(&rte_bus_list, bus, next);
 	RTE_LOG(DEBUG, EAL, "Registered [%s] bus.\n", bus->name);
diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
index 9aa047d..8c2d19c 100644
--- a/lib/librte_eal/common/include/rte_bus.h
+++ b/lib/librte_eal/common/include/rte_bus.h
@@ -109,6 +109,35 @@ typedef struct rte_device *
 			 const struct rte_device *start);
 
 /**
+ * Implementation specific probe function which is responsible for linking
+ * devices on that bus with applicable drivers.
+ * The plugged device might already have been used previously by the bus,
+ * in which case some buses might prefer to detect and re-use the relevant
+ * information pertaining to this device.
+ *
+ * @param da
+ *	Device declaration.
+ *
+ * @return
+ *	The pointer to a valid rte_device usable by the bus on success.
+ *	NULL on error. rte_errno is then set.
+ */
+typedef struct rte_device * (*rte_bus_plug_t)(struct rte_devargs *da);
+
+/**
+ * Implementation specific remove function which is responsible for unlinking
+ * devices on that bus from assigned driver.
+ *
+ * @param dev
+ *	Device pointer that was returned by a previous device plug call.
+ *
+ * @return
+ *	0 on success.
+ *	!0 on error. rte_errno is then set.
+ */
+typedef int (*rte_bus_unplug_t)(struct rte_device *dev);
+
+/**
  * A structure describing a generic bus.
  */
 struct rte_bus {
@@ -117,6 +146,8 @@ struct rte_bus {
 	rte_bus_scan_t scan;         /**< Scan for devices attached to bus */
 	rte_bus_probe_t probe;       /**< Probe devices on bus */
 	rte_bus_find_device_t find_device; /**< Find a device on the bus */
+	rte_bus_plug_t plug;         /**< Probe single device for drivers */
+	rte_bus_unplug_t unplug;     /**< Remove single device from driver */
 };
 
 /**
-- 
2.1.4

[PATCH v6 06/11] vdev: implement find_device bus operation

[Gaetan Rivet]

From: Jan Blunck <jblunck@infradead.org>

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/eal_common_vdev.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_vdev.c b/lib/librte_eal/common/eal_common_vdev.c
index 0037a64..625a8e2 100644
--- a/lib/librte_eal/common/eal_common_vdev.c
+++ b/lib/librte_eal/common/eal_common_vdev.c
@@ -338,9 +338,30 @@ vdev_probe(void)
 	return 0;
 }
 
+static struct rte_device *
+vdev_find_device(rte_dev_cmp_t cmp,
+		 const void *data,
+		 const struct rte_device *start)
+{
+	struct rte_vdev_device *dev;
+	int start_found = !!(start == NULL);
+
+	TAILQ_FOREACH(dev, &vdev_device_list, next) {
+		if (start_found == 0) {
+			if (&dev->device == start)
+				start_found = 1;
+			continue;
+		}
+		if (cmp(&dev->device, data) == 0)
+			return &dev->device;
+	}
+	return NULL;
+}
+
 static struct rte_bus rte_vdev_bus = {
 	.scan = vdev_scan,
 	.probe = vdev_probe,
+	.find_device = vdev_find_device,
 };
 
 RTE_INIT(rte_vdev_bus_register);
-- 
2.1.4

[PATCH v6 07/11] vdev: implement hotplug bus operation

[Gaetan Rivet]

Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/eal_common_vdev.c | 36 +++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_vdev.c b/lib/librte_eal/common/eal_common_vdev.c
index 625a8e2..f5c557f 100644
--- a/lib/librte_eal/common/eal_common_vdev.c
+++ b/lib/librte_eal/common/eal_common_vdev.c
@@ -38,11 +38,13 @@
 #include <sys/queue.h>
 
 #include <rte_eal.h>
+#include <rte_dev.h>
 #include <rte_bus.h>
 #include <rte_vdev.h>
 #include <rte_common.h>
 #include <rte_devargs.h>
 #include <rte_memory.h>
+#include <rte_errno.h>
 
 /** Double linked list of virtual device drivers. */
 TAILQ_HEAD(vdev_device_list, rte_vdev_device);
@@ -358,10 +360,44 @@ vdev_find_device(rte_dev_cmp_t cmp,
 	return NULL;
 }
 
+static struct rte_device *
+vdev_plug(struct rte_devargs *da)
+{
+	struct rte_vdev_device *dev;
+	int ret;
+
+	ret = rte_vdev_init(da->virt.drv_name, da->args);
+	if (ret) {
+		rte_errno = -ret;
+		return NULL;
+	}
+	dev = find_vdev(da->virt.drv_name);
+	return &dev->device;
+}
+
+static int
+vdev_unplug(struct rte_device *dev)
+{
+	struct rte_devargs *da;
+	int ret;
+
+	da = dev->devargs;
+	if (da == NULL) {
+		rte_errno = EINVAL;
+		return -1;
+	}
+	ret = rte_vdev_uninit(da->virt.drv_name);
+	if (ret)
+		rte_errno = -ret;
+	return ret;
+}
+
 static struct rte_bus rte_vdev_bus = {
 	.scan = vdev_scan,
 	.probe = vdev_probe,
 	.find_device = vdev_find_device,
+	.plug = vdev_plug,
+	.unplug = vdev_unplug,
 };
 
 RTE_INIT(rte_vdev_bus_register);
-- 
2.1.4

[PATCH v6 08/11] vdev: use standard bus registration function

[Gaetan Rivet]

Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/eal_common_vdev.c  | 18 +-----------------
 lib/librte_eal/common/include/rte_vdev.h |  2 ++
 2 files changed, 3 insertions(+), 17 deletions(-)

diff --git a/lib/librte_eal/common/eal_common_vdev.c b/lib/librte_eal/common/eal_common_vdev.c
index f5c557f..93a0ebe 100644
--- a/lib/librte_eal/common/eal_common_vdev.c
+++ b/lib/librte_eal/common/eal_common_vdev.c
@@ -54,14 +54,10 @@ static struct vdev_device_list vdev_device_list =
 struct vdev_driver_list vdev_driver_list =
 	TAILQ_HEAD_INITIALIZER(vdev_driver_list);
 
-static void rte_vdev_bus_register(void);
-
 /* register a driver */
 void
 rte_vdev_register(struct rte_vdev_driver *driver)
 {
-	rte_vdev_bus_register();
-
 	TAILQ_INSERT_TAIL(&vdev_driver_list, driver, next);
 }
 
@@ -400,16 +396,4 @@ static struct rte_bus rte_vdev_bus = {
 	.unplug = vdev_unplug,
 };
 
-RTE_INIT(rte_vdev_bus_register);
-
-static void rte_vdev_bus_register(void)
-{
-	static int registered;
-
-	if (registered)
-		return;
-
-	registered = 1;
-	rte_vdev_bus.name = RTE_STR(virtual);
-	rte_bus_register(&rte_vdev_bus);
-}
+RTE_REGISTER_BUS(VIRTUAL_BUS_NAME, rte_vdev_bus);
diff --git a/lib/librte_eal/common/include/rte_vdev.h b/lib/librte_eal/common/include/rte_vdev.h
index e6b678e..2d02c68 100644
--- a/lib/librte_eal/common/include/rte_vdev.h
+++ b/lib/librte_eal/common/include/rte_vdev.h
@@ -41,6 +41,8 @@ extern "C" {
 #include <rte_dev.h>
 #include <rte_devargs.h>
 
+#define VIRTUAL_BUS_NAME "virtual"
+
 struct rte_vdev_device {
 	TAILQ_ENTRY(rte_vdev_device) next;      /**< Next attached vdev */
 	struct rte_device device;               /**< Inherit core device */
-- 
2.1.4

[PATCH v6 09/11] pci: implement find_device bus operation

[Gaetan Rivet]

Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/eal_common_pci.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c
index 78b097e..ab55041 100644
--- a/lib/librte_eal/common/eal_common_pci.c
+++ b/lib/librte_eal/common/eal_common_pci.c
@@ -488,10 +488,31 @@ rte_pci_remove_device(struct rte_pci_device *pci_dev)
 	TAILQ_REMOVE(&rte_pci_bus.device_list, pci_dev, next);
 }
 
+static struct rte_device *
+pci_find_device(rte_dev_cmp_t cmp,
+		const void *data,
+		const struct rte_device *start)
+{
+	struct rte_pci_device *dev;
+	int start_found = !!(start == NULL);
+
+	FOREACH_DEVICE_ON_PCIBUS(dev) {
+		if (!start_found) {
+			if (&dev->device == start)
+				start_found = 1;
+			continue;
+		}
+		if (cmp(&dev->device, data) == 0)
+			return &dev->device;
+	}
+	return NULL;
+}
+
 struct rte_pci_bus rte_pci_bus = {
 	.bus = {
 		.scan = rte_pci_scan,
 		.probe = rte_pci_probe,
+		.find_device = pci_find_device,
 	},
 	.device_list = TAILQ_HEAD_INITIALIZER(rte_pci_bus.device_list),
 	.driver_list = TAILQ_HEAD_INITIALIZER(rte_pci_bus.driver_list),
-- 
2.1.4

[PATCH v6 10/11] pci: implement hotplug bus operation

[Gaetan Rivet]

Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/eal_common_pci.c | 44 ++++++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c
index ab55041..e30d2d1 100644
--- a/lib/librte_eal/common/eal_common_pci.c
+++ b/lib/librte_eal/common/eal_common_pci.c
@@ -47,6 +47,7 @@
 #include <rte_pci.h>
 #include <rte_per_lcore.h>
 #include <rte_memory.h>
+#include <rte_memcpy.h>
 #include <rte_memzone.h>
 #include <rte_eal.h>
 #include <rte_string_fns.h>
@@ -508,11 +509,54 @@ pci_find_device(rte_dev_cmp_t cmp,
 	return NULL;
 }
 
+static struct rte_device *
+pci_plug(struct rte_devargs *da)
+{
+	struct rte_pci_device *pdev;
+	struct rte_pci_addr *addr;
+
+	addr = &da->pci.addr;
+	/*
+	 * Update eventual pci device in global list.
+	 * Insert it if none was found.
+	 */
+	if (pci_update_device(addr) < 0) {
+		rte_errno = EIO;
+		return NULL;
+	}
+	/* Find the current device holding this address in the bus. */
+	FOREACH_DEVICE_ON_PCIBUS(pdev) {
+		if (rte_eal_compare_pci_addr(&pdev->addr, addr) == 0) {
+			if (rte_pci_probe_one(addr) != 0) {
+				rte_errno = ENODEV;
+				return NULL;
+			}
+			break;
+		}
+	}
+	return (pdev != NULL) ? &pdev->device : NULL;
+}
+
+static int
+pci_unplug(struct rte_device *dev)
+{
+	struct rte_pci_device *pdev;
+
+	pdev = RTE_DEV_TO_PCI(dev);
+	if (rte_pci_detach(&pdev->addr) != 0) {
+		rte_errno = ENODEV;
+		return -1;
+	}
+	return 0;
+}
+
 struct rte_pci_bus rte_pci_bus = {
 	.bus = {
 		.scan = rte_pci_scan,
 		.probe = rte_pci_probe,
 		.find_device = pci_find_device,
+		.plug = pci_plug,
+		.unplug = pci_unplug,
 	},
 	.device_list = TAILQ_HEAD_INITIALIZER(rte_pci_bus.device_list),
 	.driver_list = TAILQ_HEAD_INITIALIZER(rte_pci_bus.driver_list),
-- 
2.1.4

[PATCH v6 11/11] dev: use new hotplug API in attach / detach

[Gaetan Rivet]

From: Jan Blunck <jblunck@infradead.org>

This new generic API allows attach and detach to be compatible with all
buses supporting hotplug.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/eal_common_dev.c | 92 ++++++++++++++++++++++++++--------
 1 file changed, 70 insertions(+), 22 deletions(-)

diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c
index a400ddd..19508e6 100644
--- a/lib/librte_eal/common/eal_common_dev.c
+++ b/lib/librte_eal/common/eal_common_dev.c
@@ -37,6 +37,7 @@
 #include <inttypes.h>
 #include <sys/queue.h>
 
+#include <rte_bus.h>
 #include <rte_dev.h>
 #include <rte_devargs.h>
 #include <rte_debug.h>
@@ -45,50 +46,97 @@
 
 #include "eal_private.h"
 
+static int cmp_detached_dev_name(const struct rte_device *dev,
+	const void *_name)
+{
+	const char *name = _name;
+
+	/* skip attached devices */
+	if (dev->driver)
+		return 1;
+	return strcmp(dev->name, name);
+}
+
 int rte_eal_dev_attach(const char *name, const char *devargs)
 {
-	struct rte_pci_addr addr;
+	struct rte_device *dev;
+	int ret;
 
 	if (name == NULL || devargs == NULL) {
 		RTE_LOG(ERR, EAL, "Invalid device or arguments provided\n");
 		return -EINVAL;
 	}
 
-	if (eal_parse_pci_DomBDF(name, &addr) == 0) {
-		if (rte_pci_probe_one(&addr) < 0)
-			goto err;
+	dev = rte_bus_find_device(cmp_detached_dev_name, name, NULL);
+	if (dev) {
+		struct rte_bus *bus;
 
-	} else {
-		if (rte_vdev_init(name, devargs))
-			goto err;
+		bus = rte_bus_find_by_device(dev);
+		if (!bus) {
+			RTE_LOG(ERR, EAL, "Cannot find bus for device (%s)\n",
+				name);
+			return -EINVAL;
+		}
+
+		if (!bus->plug) {
+			RTE_LOG(ERR, EAL, "Bus function not supported\n");
+			return -ENOTSUP;
+		}
+
+		ret = (bus->plug(dev->devargs) == NULL);
+		goto out;
 	}
 
-	return 0;
+	/*
+	 * If we haven't found a bus device the user meant to "hotplug" a
+	 * virtual device instead.
+	 */
+	ret = rte_vdev_init(name, devargs);
+out:
+	if (ret)
+		RTE_LOG(ERR, EAL, "Driver cannot attach the device (%s)\n",
+			name);
+	return ret;
+}
 
-err:
-	RTE_LOG(ERR, EAL, "Driver cannot attach the device (%s)\n", name);
-	return -EINVAL;
+static int cmp_dev_name(const struct rte_device *dev, const void *_name)
+{
+	const char *name = _name;
+
+	return strcmp(dev->name, name);
 }
 
 int rte_eal_dev_detach(const char *name)
 {
-	struct rte_pci_addr addr;
+	struct rte_device *dev;
+	struct rte_bus *bus;
+	int ret;
 
 	if (name == NULL) {
 		RTE_LOG(ERR, EAL, "Invalid device provided.\n");
 		return -EINVAL;
 	}
 
-	if (eal_parse_pci_DomBDF(name, &addr) == 0) {
-		if (rte_pci_detach(&addr) < 0)
-			goto err;
-	} else {
-		if (rte_vdev_uninit(name))
-			goto err;
+	dev = rte_bus_find_device(cmp_dev_name, name, NULL);
+	if (!dev) {
+		RTE_LOG(ERR, EAL, "Cannot find device (%s)\n", name);
+		return -EINVAL;
+	}
+
+	bus = rte_bus_find_by_device(dev);
+	if (!bus) {
+		RTE_LOG(ERR, EAL, "Cannot find bus for device (%s)\n", name);
+		return -EINVAL;
+	}
+
+	if (!bus->unplug) {
+		RTE_LOG(ERR, EAL, "Bus function not supported\n");
+		return -ENOTSUP;
 	}
-	return 0;
 
-err:
-	RTE_LOG(ERR, EAL, "Driver cannot detach the device (%s)\n", name);
-	return -EINVAL;
+	ret = bus->unplug(dev);
+	if (ret)
+		RTE_LOG(ERR, EAL, "Driver cannot detach the device (%s)\n",
+			name);
+	return ret;
 }
-- 
2.1.4

Re: [PATCH v6 05/11] bus: introduce hotplug functionality

[Jan Blunck]

On Tue, Jun 27, 2017 at 6:11 PM, Gaetan Rivet <gaetan.rivet@6wind.com> wrote:
> From: Jan Blunck <jblunck@infradead.org>
>
> Signed-off-by: Jan Blunck <jblunck@infradead.org>
> Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> ---
>  lib/librte_eal/common/eal_common_bus.c  |  2 ++
>  lib/librte_eal/common/include/rte_bus.h | 31 +++++++++++++++++++++++++++++++
>  2 files changed, 33 insertions(+)
>
> diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
> index dde6c83..83d9c07 100644
> --- a/lib/librte_eal/common/eal_common_bus.c
> +++ b/lib/librte_eal/common/eal_common_bus.c
> @@ -50,6 +50,8 @@ rte_bus_register(struct rte_bus *bus)
>         /* A bus should mandatorily have the scan implemented */
>         RTE_VERIFY(bus->scan);
>         RTE_VERIFY(bus->probe);
> +       /* Buses supporting hotplug also require unplug. */
> +       RTE_VERIFY(!bus->plug || bus->unplug);
>
>         TAILQ_INSERT_TAIL(&rte_bus_list, bus, next);
>         RTE_LOG(DEBUG, EAL, "Registered [%s] bus.\n", bus->name);
> diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
> index 9aa047d..8c2d19c 100644
> --- a/lib/librte_eal/common/include/rte_bus.h
> +++ b/lib/librte_eal/common/include/rte_bus.h
> @@ -109,6 +109,35 @@ typedef struct rte_device *
>                          const struct rte_device *start);
>
>  /**
> + * Implementation specific probe function which is responsible for linking
> + * devices on that bus with applicable drivers.
> + * The plugged device might already have been used previously by the bus,
> + * in which case some buses might prefer to detect and re-use the relevant
> + * information pertaining to this device.
> + *
> + * @param da
> + *     Device declaration.
> + *
> + * @return
> + *     The pointer to a valid rte_device usable by the bus on success.
> + *     NULL on error. rte_errno is then set.
> + */
> +typedef struct rte_device * (*rte_bus_plug_t)(struct rte_devargs *da);

Shouldn't this be orthogonal to unplug() and take a rte_device. You
should only be able to plug devices that have been found by scan
before.

Jan

> +
> +/**
> + * Implementation specific remove function which is responsible for unlinking
> + * devices on that bus from assigned driver.
> + *
> + * @param dev
> + *     Device pointer that was returned by a previous device plug call.
> + *
> + * @return
> + *     0 on success.
> + *     !0 on error. rte_errno is then set.
> + */
> +typedef int (*rte_bus_unplug_t)(struct rte_device *dev);
> +
> +/**
>   * A structure describing a generic bus.
>   */
>  struct rte_bus {
> @@ -117,6 +146,8 @@ struct rte_bus {
>         rte_bus_scan_t scan;         /**< Scan for devices attached to bus */
>         rte_bus_probe_t probe;       /**< Probe devices on bus */
>         rte_bus_find_device_t find_device; /**< Find a device on the bus */
> +       rte_bus_plug_t plug;         /**< Probe single device for drivers */
> +       rte_bus_unplug_t unplug;     /**< Remove single device from driver */
>  };
>
>  /**
> --
> 2.1.4
>

Re: [PATCH v6 09/11] pci: implement find_device bus operation

[Stephen Hemminger]

On Tue, 27 Jun 2017 18:11:16 +0200
Gaetan Rivet <gaetan.rivet@6wind.com> wrote:

> +	int start_found = !!(start == NULL);
> +
> +	FOREACH_DEVICE_ON_PCIBUS(dev) {
> +		if (!start_found) {
> +			if (&dev->device == start)
> +				start_found = 1;
> +			continue;
> +		}
> +		if (cmp(&dev->device, data) == 0)
> +			return &dev->device;
> +	}
> +	return NULL;
> +}
> +

Why is start_found not a boolean?

Do you really need start_found at all? Why not just reuse existing
pointer?

	FOREACH_DEVICE_ON_PCIBUS(dev) {
		if (start) {
			if (&dev->device == start)
				start = NULL
			continue;
		}
		if (cmp(&dev->device, data) == 0)
			return &dev->device;
	}
	return NULL;
}

Re: [PATCH v6 09/11] pci: implement find_device bus operation

[Gaëtan Rivet]

On Tue, Jun 27, 2017 at 04:35:14PM -0700, Stephen Hemminger wrote:
> On Tue, 27 Jun 2017 18:11:16 +0200
> Gaetan Rivet <gaetan.rivet@6wind.com> wrote:
> 
> > +	int start_found = !!(start == NULL);
> > +
> > +	FOREACH_DEVICE_ON_PCIBUS(dev) {
> > +		if (!start_found) {
> > +			if (&dev->device == start)
> > +				start_found = 1;
> > +			continue;
> > +		}
> > +		if (cmp(&dev->device, data) == 0)
> > +			return &dev->device;
> > +	}
> > +	return NULL;
> > +}
> > +
> 
> Why is start_found not a boolean?
> 

Ah, yes, I wrote this a few times over in rte_bus and rte_vdev, and
mostly used the same scheme in the PCI implementation, without checking
for the use of stdbool in the vincinity otherwise.

I would not be opposed to using a bool if I was rewriting it, but I
don't feel this change warrants a new version.

> Do you really need start_found at all? Why not just reuse existing
> pointer?
> 

You are right, it could be reduced. But I find it a little too "clever"
in a sense, and I prefer usually to avoid rewriting input parameters. If this
function had to be refactored later, the writer would need to be careful
about start having changed. The advantage of using one variable less
does not outweight this drawback I think.

> 	FOREACH_DEVICE_ON_PCIBUS(dev) {
> 		if (start) {
> 			if (&dev->device == start)
> 				start = NULL
> 			continue;
> 		}
> 		if (cmp(&dev->device, data) == 0)
> 			return &dev->device;
> 	}
> 	return NULL;
> }

-- 
Gaëtan Rivet
6WIND

Re: [PATCH v6 09/11] pci: implement find_device bus operation

[Richardson, Bruce]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Gaëtan Rivet
> Sent: Wednesday, June 28, 2017 10:18 AM
> To: Stephen Hemminger <stephen@networkplumber.org>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v6 09/11] pci: implement find_device bus
> operation
> 
> On Tue, Jun 27, 2017 at 04:35:14PM -0700, Stephen Hemminger wrote:
> > On Tue, 27 Jun 2017 18:11:16 +0200
> > Gaetan Rivet <gaetan.rivet@6wind.com> wrote:
> >
> > > +	int start_found = !!(start == NULL);
> > > +
> > > +	FOREACH_DEVICE_ON_PCIBUS(dev) {
> > > +		if (!start_found) {
> > > +			if (&dev->device == start)
> > > +				start_found = 1;
> > > +			continue;
> > > +		}
> > > +		if (cmp(&dev->device, data) == 0)
> > > +			return &dev->device;
> > > +	}
> > > +	return NULL;
> > > +}
> > > +
> >
> > Why is start_found not a boolean?
> >
> 
> Ah, yes, I wrote this a few times over in rte_bus and rte_vdev, and mostly
> used the same scheme in the PCI implementation, without checking for the
> use of stdbool in the vincinity otherwise.
> 
> I would not be opposed to using a bool if I was rewriting it, but I don't
> feel this change warrants a new version.
> 
> > Do you really need start_found at all? Why not just reuse existing
> > pointer?
> >
> 
> You are right, it could be reduced. But I find it a little too "clever"
> in a sense, and I prefer usually to avoid rewriting input parameters. If
> this function had to be refactored later, the writer would need to be
> careful about start having changed. The advantage of using one variable
> less does not outweight this drawback I think.
> 
+1 for having an extra variable for clarity.

Re: [PATCH v6 05/11] bus: introduce hotplug functionality

[Thomas Monjalon]

27/06/2017 21:03, Jan Blunck:
> On Tue, Jun 27, 2017 at 6:11 PM, Gaetan Rivet <gaetan.rivet@6wind.com> wrote:
> > --- a/lib/librte_eal/common/include/rte_bus.h
> > +++ b/lib/librte_eal/common/include/rte_bus.h
> >  /**
> > + * Implementation specific probe function which is responsible for linking
> > + * devices on that bus with applicable drivers.
> > + * The plugged device might already have been used previously by the bus,
> > + * in which case some buses might prefer to detect and re-use the relevant
> > + * information pertaining to this device.
> > + *
> > + * @param da
> > + *     Device declaration.
> > + *
> > + * @return
> > + *     The pointer to a valid rte_device usable by the bus on success.
> > + *     NULL on error. rte_errno is then set.
> > + */
> > +typedef struct rte_device * (*rte_bus_plug_t)(struct rte_devargs *da);
> 
> Shouldn't this be orthogonal to unplug() and take a rte_device. You
> should only be able to plug devices that have been found by scan
> before.

Plugging a device that has been scanned before is a special case.
In a true hotplug scenario, we could use this plug function passing
a devargs.
I don't see any issue passing rte_devargs to plug and rte_device to unplug.

Re: [PATCH v6 05/11] bus: introduce hotplug functionality

[Jan Blunck]

On Wed, Jun 28, 2017 at 1:44 PM, Thomas Monjalon <thomas@monjalon.net> wrote:
> 27/06/2017 21:03, Jan Blunck:
>> On Tue, Jun 27, 2017 at 6:11 PM, Gaetan Rivet <gaetan.rivet@6wind.com> wrote:
>> > --- a/lib/librte_eal/common/include/rte_bus.h
>> > +++ b/lib/librte_eal/common/include/rte_bus.h
>> >  /**
>> > + * Implementation specific probe function which is responsible for linking
>> > + * devices on that bus with applicable drivers.
>> > + * The plugged device might already have been used previously by the bus,
>> > + * in which case some buses might prefer to detect and re-use the relevant
>> > + * information pertaining to this device.
>> > + *
>> > + * @param da
>> > + *     Device declaration.
>> > + *
>> > + * @return
>> > + *     The pointer to a valid rte_device usable by the bus on success.
>> > + *     NULL on error. rte_errno is then set.
>> > + */
>> > +typedef struct rte_device * (*rte_bus_plug_t)(struct rte_devargs *da);
>>
>> Shouldn't this be orthogonal to unplug() and take a rte_device. You
>> should only be able to plug devices that have been found by scan
>> before.
>
> Plugging a device that has been scanned before is a special case.
> In a true hotplug scenario, we could use this plug function passing
> a devargs.
> I don't see any issue passing rte_devargs to plug and rte_device to unplug.

What do you mean by "true hotplug"?

The problem with this is that passing just rte_devargs to plug()
requires the bus to parse and enrich the rte_devargs with bus
specifics. From there it gets folded into the to-be-created bus
specific rte_XXX_device. This makes it unnecessarily complicated and
even worse it adds a second code path how devices come alive.

When we get notified about a hotplug event we already know which bus
this event belongs to:

1. scan the bus for incoming devices
2. plug single device with devargs and probe for drivers

Makes sense?

Re: [PATCH v6 05/11] bus: introduce hotplug functionality

[Thomas Monjalon]

28/06/2017 13:58, Jan Blunck:
> On Wed, Jun 28, 2017 at 1:44 PM, Thomas Monjalon <thomas@monjalon.net> wrote:
> > 27/06/2017 21:03, Jan Blunck:
> >> On Tue, Jun 27, 2017 at 6:11 PM, Gaetan Rivet <gaetan.rivet@6wind.com> wrote:
> >> > --- a/lib/librte_eal/common/include/rte_bus.h
> >> > +++ b/lib/librte_eal/common/include/rte_bus.h
> >> >  /**
> >> > + * Implementation specific probe function which is responsible for linking
> >> > + * devices on that bus with applicable drivers.
> >> > + * The plugged device might already have been used previously by the bus,
> >> > + * in which case some buses might prefer to detect and re-use the relevant
> >> > + * information pertaining to this device.
> >> > + *
> >> > + * @param da
> >> > + *     Device declaration.
> >> > + *
> >> > + * @return
> >> > + *     The pointer to a valid rte_device usable by the bus on success.
> >> > + *     NULL on error. rte_errno is then set.
> >> > + */
> >> > +typedef struct rte_device * (*rte_bus_plug_t)(struct rte_devargs *da);
> >>
> >> Shouldn't this be orthogonal to unplug() and take a rte_device. You
> >> should only be able to plug devices that have been found by scan
> >> before.
> >
> > Plugging a device that has been scanned before is a special case.
> > In a true hotplug scenario, we could use this plug function passing
> > a devargs.
> > I don't see any issue passing rte_devargs to plug and rte_device to unplug.
> 
> What do you mean by "true hotplug"?

I mean a kernel notification of a new device.

> The problem with this is that passing just rte_devargs to plug()
> requires the bus to parse and enrich the rte_devargs with bus
> specifics. From there it gets folded into the to-be-created bus
> specific rte_XXX_device. This makes it unnecessarily complicated and
> even worse it adds a second code path how devices come alive.

Just after the notification, the rte_device does not exist yet.
So the plug function could share the same code as the scan function
to get the metadata and create the device instance.

> When we get notified about a hotplug event we already know which bus
> this event belongs to:
> 
> 1. scan the bus for incoming devices

No need to scan every devices here.

> 2. plug single device with devargs and probe for drivers
> 
> Makes sense?

I want to make sure there is no misunderstanding first :)

Re: [PATCH v6 05/11] bus: introduce hotplug functionality

[Jan Blunck]

On Wed, Jun 28, 2017 at 2:11 PM, Thomas Monjalon <thomas@monjalon.net> wrote:
> 28/06/2017 13:58, Jan Blunck:
>> On Wed, Jun 28, 2017 at 1:44 PM, Thomas Monjalon <thomas@monjalon.net> wrote:
>> > 27/06/2017 21:03, Jan Blunck:
>> >> On Tue, Jun 27, 2017 at 6:11 PM, Gaetan Rivet <gaetan.rivet@6wind.com> wrote:
>> >> > --- a/lib/librte_eal/common/include/rte_bus.h
>> >> > +++ b/lib/librte_eal/common/include/rte_bus.h
>> >> >  /**
>> >> > + * Implementation specific probe function which is responsible for linking
>> >> > + * devices on that bus with applicable drivers.
>> >> > + * The plugged device might already have been used previously by the bus,
>> >> > + * in which case some buses might prefer to detect and re-use the relevant
>> >> > + * information pertaining to this device.
>> >> > + *
>> >> > + * @param da
>> >> > + *     Device declaration.
>> >> > + *
>> >> > + * @return
>> >> > + *     The pointer to a valid rte_device usable by the bus on success.
>> >> > + *     NULL on error. rte_errno is then set.
>> >> > + */
>> >> > +typedef struct rte_device * (*rte_bus_plug_t)(struct rte_devargs *da);
>> >>
>> >> Shouldn't this be orthogonal to unplug() and take a rte_device. You
>> >> should only be able to plug devices that have been found by scan
>> >> before.
>> >
>> > Plugging a device that has been scanned before is a special case.
>> > In a true hotplug scenario, we could use this plug function passing
>> > a devargs.
>> > I don't see any issue passing rte_devargs to plug and rte_device to unplug.
>>
>> What do you mean by "true hotplug"?
>
> I mean a kernel notification of a new device.
>

Does a "false hotplug" exist, too? :)

>> The problem with this is that passing just rte_devargs to plug()
>> requires the bus to parse and enrich the rte_devargs with bus
>> specifics. From there it gets folded into the to-be-created bus
>> specific rte_XXX_device. This makes it unnecessarily complicated and
>> even worse it adds a second code path how devices come alive.
>
> Just after the notification, the rte_device does not exist yet.
> So the plug function could share the same code as the scan function
> to get the metadata and create the device instance.
>

Exactly this is what I want to avoid. The plug() function would become
a "scan-one and probe". From my point of view plug() and unplug()
should be orthogonal. The plug() and unplug() should only be
responsible for adding drivers with optional arguments. The EAL should
allow the drivers to get unplugged/re-plugged at run-time. I want to
be able to change arguments ... or even drivers :)

>> When we get notified about a hotplug event we already know which bus
>> this event belongs to:
>>
>> 1. scan the bus for incoming devices
>
> No need to scan every devices here.
>

This is a readdir followed by open+read+close for any new device. This
code belongs here anyway. Its lightweight if nothing changed. The scan
itself should be idempotent anyway.

>> 2. plug single device with devargs and probe for drivers
>>
>> Makes sense?
>
> I want to make sure there is no misunderstanding first :)

Which makes sense. That is probably my fault due to being too
distracted with other things and not communicating well enough while
Gaetan consumed my code.

Re: [PATCH v6 05/11] bus: introduce hotplug functionality

[Thomas Monjalon]

28/06/2017 15:09, Jan Blunck:
> On Wed, Jun 28, 2017 at 2:11 PM, Thomas Monjalon <thomas@monjalon.net> wrote:
> > 28/06/2017 13:58, Jan Blunck:
> >> On Wed, Jun 28, 2017 at 1:44 PM, Thomas Monjalon <thomas@monjalon.net> wrote:
> >> > 27/06/2017 21:03, Jan Blunck:
> >> >> On Tue, Jun 27, 2017 at 6:11 PM, Gaetan Rivet <gaetan.rivet@6wind.com> wrote:
> >> >> > --- a/lib/librte_eal/common/include/rte_bus.h
> >> >> > +++ b/lib/librte_eal/common/include/rte_bus.h
> >> >> >  /**
> >> >> > + * Implementation specific probe function which is responsible for linking
> >> >> > + * devices on that bus with applicable drivers.
> >> >> > + * The plugged device might already have been used previously by the bus,
> >> >> > + * in which case some buses might prefer to detect and re-use the relevant
> >> >> > + * information pertaining to this device.
> >> >> > + *
> >> >> > + * @param da
> >> >> > + *     Device declaration.
> >> >> > + *
> >> >> > + * @return
> >> >> > + *     The pointer to a valid rte_device usable by the bus on success.
> >> >> > + *     NULL on error. rte_errno is then set.
> >> >> > + */
> >> >> > +typedef struct rte_device * (*rte_bus_plug_t)(struct rte_devargs *da);
> >> >>
> >> >> Shouldn't this be orthogonal to unplug() and take a rte_device. You
> >> >> should only be able to plug devices that have been found by scan
> >> >> before.
> >> >
> >> > Plugging a device that has been scanned before is a special case.
> >> > In a true hotplug scenario, we could use this plug function passing
> >> > a devargs.
> >> > I don't see any issue passing rte_devargs to plug and rte_device to unplug.
> >>
> >> What do you mean by "true hotplug"?
> >
> > I mean a kernel notification of a new device.
> 
> Does a "false hotplug" exist, too? :)

The false hotplug was the original attach function which was just
adding a new ethdev interface.

> >> The problem with this is that passing just rte_devargs to plug()
> >> requires the bus to parse and enrich the rte_devargs with bus
> >> specifics. From there it gets folded into the to-be-created bus
> >> specific rte_XXX_device. This makes it unnecessarily complicated and
> >> even worse it adds a second code path how devices come alive.
> >
> > Just after the notification, the rte_device does not exist yet.
> > So the plug function could share the same code as the scan function
> > to get the metadata and create the device instance.
> 
> Exactly this is what I want to avoid. 

Why do you want to avoid that?
I think you mean it is not what you had in mind.

> The plug() function would become a "scan-one and probe".

Yes

> From my point of view plug() and unplug() should be orthogonal.
> The plug() and unplug() should only be responsible for adding drivers
> with optional arguments. The EAL should allow the drivers to get
> unplugged/re-plugged at run-time. I want to be able to change arguments
> ... or even drivers :)

It is a totally different thing.
We are talking about hotplug of a device,
and you are talking about changing drivers dynamically.

So I still don't understand what is the issue with the plug/unplug
functions proposed here.

> >> When we get notified about a hotplug event we already know which bus
> >> this event belongs to:
> >>
> >> 1. scan the bus for incoming devices
> >
> > No need to scan every devices here.
> 
> This is a readdir followed by open+read+close for any new device. This
> code belongs here anyway. Its lightweight if nothing changed. The scan
> itself should be idempotent anyway.
> 
> >> 2. plug single device with devargs and probe for drivers
> >>
> >> Makes sense?
> >
> > I want to make sure there is no misunderstanding first :)
> 
> Which makes sense. That is probably my fault due to being too
> distracted with other things and not communicating well enough while
> Gaetan consumed my code.

Your ideas are probably interesting, and I want to understand them.
In the meantime, we need to progress on 17.08-rc1 which must be done
in following days. Please let's separate the ideas which are not
yet implemented from what we are already able to deliver.

Re: [PATCH v6 05/11] bus: introduce hotplug functionality

[Jan Blunck]

On Wed, Jun 28, 2017 at 3:30 PM, Thomas Monjalon <thomas@monjalon.net> wrote:
> 28/06/2017 15:09, Jan Blunck:
>> On Wed, Jun 28, 2017 at 2:11 PM, Thomas Monjalon <thomas@monjalon.net> wrote:
>> > 28/06/2017 13:58, Jan Blunck:
>> >> On Wed, Jun 28, 2017 at 1:44 PM, Thomas Monjalon <thomas@monjalon.net> wrote:
>> >> > 27/06/2017 21:03, Jan Blunck:
>> >> >> On Tue, Jun 27, 2017 at 6:11 PM, Gaetan Rivet <gaetan.rivet@6wind.com> wrote:
>> >> >> > --- a/lib/librte_eal/common/include/rte_bus.h
>> >> >> > +++ b/lib/librte_eal/common/include/rte_bus.h
>> >> >> >  /**
>> >> >> > + * Implementation specific probe function which is responsible for linking
>> >> >> > + * devices on that bus with applicable drivers.
>> >> >> > + * The plugged device might already have been used previously by the bus,
>> >> >> > + * in which case some buses might prefer to detect and re-use the relevant
>> >> >> > + * information pertaining to this device.
>> >> >> > + *
>> >> >> > + * @param da
>> >> >> > + *     Device declaration.
>> >> >> > + *
>> >> >> > + * @return
>> >> >> > + *     The pointer to a valid rte_device usable by the bus on success.
>> >> >> > + *     NULL on error. rte_errno is then set.
>> >> >> > + */
>> >> >> > +typedef struct rte_device * (*rte_bus_plug_t)(struct rte_devargs *da);
>> >> >>
>> >> >> Shouldn't this be orthogonal to unplug() and take a rte_device. You
>> >> >> should only be able to plug devices that have been found by scan
>> >> >> before.
>> >> >
>> >> > Plugging a device that has been scanned before is a special case.
>> >> > In a true hotplug scenario, we could use this plug function passing
>> >> > a devargs.
>> >> > I don't see any issue passing rte_devargs to plug and rte_device to unplug.
>> >>
>> >> What do you mean by "true hotplug"?
>> >
>> > I mean a kernel notification of a new device.
>>
>> Does a "false hotplug" exist, too? :)
>
> The false hotplug was the original attach function which was just
> adding a new ethdev interface.
>
>> >> The problem with this is that passing just rte_devargs to plug()
>> >> requires the bus to parse and enrich the rte_devargs with bus
>> >> specifics. From there it gets folded into the to-be-created bus
>> >> specific rte_XXX_device. This makes it unnecessarily complicated and
>> >> even worse it adds a second code path how devices come alive.
>> >
>> > Just after the notification, the rte_device does not exist yet.
>> > So the plug function could share the same code as the scan function
>> > to get the metadata and create the device instance.
>>
>> Exactly this is what I want to avoid.
>
> Why do you want to avoid that?
> I think you mean it is not what you had in mind.
>
>> The plug() function would become a "scan-one and probe".
>
> Yes
>
>> From my point of view plug() and unplug() should be orthogonal.
>> The plug() and unplug() should only be responsible for adding drivers
>> with optional arguments. The EAL should allow the drivers to get
>> unplugged/re-plugged at run-time. I want to be able to change arguments
>> ... or even drivers :)
>
> It is a totally different thing.
> We are talking about hotplug of a device,
> and you are talking about changing drivers dynamically.
>
> So I still don't understand what is the issue with the plug/unplug
> functions proposed here.
>

I don't agree with the notion that plug() means "scan-one and probe".

>> >> When we get notified about a hotplug event we already know which bus
>> >> this event belongs to:
>> >>
>> >> 1. scan the bus for incoming devices
>> >
>> > No need to scan every devices here.
>>
>> This is a readdir followed by open+read+close for any new device. This
>> code belongs here anyway. Its lightweight if nothing changed. The scan
>> itself should be idempotent anyway.
>>
>> >> 2. plug single device with devargs and probe for drivers
>> >>
>> >> Makes sense?
>> >
>> > I want to make sure there is no misunderstanding first :)
>>
>> Which makes sense. That is probably my fault due to being too
>> distracted with other things and not communicating well enough while
>> Gaetan consumed my code.
>
> Your ideas are probably interesting, and I want to understand them.
> In the meantime, we need to progress on 17.08-rc1 which must be done
> in following days. Please let's separate the ideas which are not
> yet implemented from what we are already able to deliver.

Sorry, if you got the impression that this is taken from thin air but
this is how my original patch series looked like and I'm just
providing the rational behind it.

Re: [PATCH v6 05/11] bus: introduce hotplug functionality

[Bruce Richardson]

On Wed, Jun 28, 2017 at 05:11:46PM +0200, Jan Blunck wrote:
> On Wed, Jun 28, 2017 at 3:30 PM, Thomas Monjalon <thomas@monjalon.net> wrote:
> > 28/06/2017 15:09, Jan Blunck:
> >> On Wed, Jun 28, 2017 at 2:11 PM, Thomas Monjalon <thomas@monjalon.net> wrote:
> >> > 28/06/2017 13:58, Jan Blunck:
> >> >> On Wed, Jun 28, 2017 at 1:44 PM, Thomas Monjalon <thomas@monjalon.net> wrote:
> >> >> > 27/06/2017 21:03, Jan Blunck:
> >> >> >> On Tue, Jun 27, 2017 at 6:11 PM, Gaetan Rivet <gaetan.rivet@6wind.com> wrote:
> >> >> >> > --- a/lib/librte_eal/common/include/rte_bus.h
> >> >> >> > +++ b/lib/librte_eal/common/include/rte_bus.h
> >> >> >> >  /**
> >> >> >> > + * Implementation specific probe function which is responsible for linking
> >> >> >> > + * devices on that bus with applicable drivers.
> >> >> >> > + * The plugged device might already have been used previously by the bus,
> >> >> >> > + * in which case some buses might prefer to detect and re-use the relevant
> >> >> >> > + * information pertaining to this device.
> >> >> >> > + *
> >> >> >> > + * @param da
> >> >> >> > + *     Device declaration.
> >> >> >> > + *
> >> >> >> > + * @return
> >> >> >> > + *     The pointer to a valid rte_device usable by the bus on success.
> >> >> >> > + *     NULL on error. rte_errno is then set.
> >> >> >> > + */
> >> >> >> > +typedef struct rte_device * (*rte_bus_plug_t)(struct rte_devargs *da);
> >> >> >>
> >> >> >> Shouldn't this be orthogonal to unplug() and take a rte_device. You
> >> >> >> should only be able to plug devices that have been found by scan
> >> >> >> before.
> >> >> >
> >> >> > Plugging a device that has been scanned before is a special case.
> >> >> > In a true hotplug scenario, we could use this plug function passing
> >> >> > a devargs.
> >> >> > I don't see any issue passing rte_devargs to plug and rte_device to unplug.
> >> >>
> >> >> What do you mean by "true hotplug"?
> >> >
> >> > I mean a kernel notification of a new device.
> >>
> >> Does a "false hotplug" exist, too? :)
> >
> > The false hotplug was the original attach function which was just
> > adding a new ethdev interface.
> >
> >> >> The problem with this is that passing just rte_devargs to plug()
> >> >> requires the bus to parse and enrich the rte_devargs with bus
> >> >> specifics. From there it gets folded into the to-be-created bus
> >> >> specific rte_XXX_device. This makes it unnecessarily complicated and
> >> >> even worse it adds a second code path how devices come alive.
> >> >
> >> > Just after the notification, the rte_device does not exist yet.
> >> > So the plug function could share the same code as the scan function
> >> > to get the metadata and create the device instance.
> >>
> >> Exactly this is what I want to avoid.
> >
> > Why do you want to avoid that?
> > I think you mean it is not what you had in mind.
> >
> >> The plug() function would become a "scan-one and probe".
> >
> > Yes
> >
> >> From my point of view plug() and unplug() should be orthogonal.
> >> The plug() and unplug() should only be responsible for adding drivers
> >> with optional arguments. The EAL should allow the drivers to get
> >> unplugged/re-plugged at run-time. I want to be able to change arguments
> >> ... or even drivers :)
> >
> > It is a totally different thing.
> > We are talking about hotplug of a device,
> > and you are talking about changing drivers dynamically.
> >
> > So I still don't understand what is the issue with the plug/unplug
> > functions proposed here.
> >
> 
> I don't agree with the notion that plug() means "scan-one and probe".

What do you see as plug doing then? What do you see as the use-case and
then logic flow for hotplug?

Re: [PATCH v6 05/11] bus: introduce hotplug functionality

[Gaëtan Rivet]

On Wed, Jun 28, 2017 at 04:33:20PM +0100, Bruce Richardson wrote:
> On Wed, Jun 28, 2017 at 05:11:46PM +0200, Jan Blunck wrote:
> > On Wed, Jun 28, 2017 at 3:30 PM, Thomas Monjalon <thomas@monjalon.net> wrote:
> > > 28/06/2017 15:09, Jan Blunck:
> > >> On Wed, Jun 28, 2017 at 2:11 PM, Thomas Monjalon <thomas@monjalon.net> wrote:
> > >> > 28/06/2017 13:58, Jan Blunck:
> > >> >> On Wed, Jun 28, 2017 at 1:44 PM, Thomas Monjalon <thomas@monjalon.net> wrote:
> > >> >> > 27/06/2017 21:03, Jan Blunck:
> > >> >> >> On Tue, Jun 27, 2017 at 6:11 PM, Gaetan Rivet <gaetan.rivet@6wind.com> wrote:
> > >> >> >> > --- a/lib/librte_eal/common/include/rte_bus.h
> > >> >> >> > +++ b/lib/librte_eal/common/include/rte_bus.h
> > >> >> >> >  /**
> > >> >> >> > + * Implementation specific probe function which is responsible for linking
> > >> >> >> > + * devices on that bus with applicable drivers.
> > >> >> >> > + * The plugged device might already have been used previously by the bus,
> > >> >> >> > + * in which case some buses might prefer to detect and re-use the relevant
> > >> >> >> > + * information pertaining to this device.
> > >> >> >> > + *
> > >> >> >> > + * @param da
> > >> >> >> > + *     Device declaration.
> > >> >> >> > + *
> > >> >> >> > + * @return
> > >> >> >> > + *     The pointer to a valid rte_device usable by the bus on success.
> > >> >> >> > + *     NULL on error. rte_errno is then set.
> > >> >> >> > + */
> > >> >> >> > +typedef struct rte_device * (*rte_bus_plug_t)(struct rte_devargs *da);
> > >> >> >>
> > >> >> >> Shouldn't this be orthogonal to unplug() and take a rte_device. You
> > >> >> >> should only be able to plug devices that have been found by scan
> > >> >> >> before.
> > >> >> >
> > >> >> > Plugging a device that has been scanned before is a special case.
> > >> >> > In a true hotplug scenario, we could use this plug function passing
> > >> >> > a devargs.
> > >> >> > I don't see any issue passing rte_devargs to plug and rte_device to unplug.
> > >> >>
> > >> >> What do you mean by "true hotplug"?
> > >> >
> > >> > I mean a kernel notification of a new device.
> > >>
> > >> Does a "false hotplug" exist, too? :)
> > >
> > > The false hotplug was the original attach function which was just
> > > adding a new ethdev interface.
> > >
> > >> >> The problem with this is that passing just rte_devargs to plug()
> > >> >> requires the bus to parse and enrich the rte_devargs with bus
> > >> >> specifics. From there it gets folded into the to-be-created bus
> > >> >> specific rte_XXX_device. This makes it unnecessarily complicated and
> > >> >> even worse it adds a second code path how devices come alive.
> > >> >
> > >> > Just after the notification, the rte_device does not exist yet.
> > >> > So the plug function could share the same code as the scan function
> > >> > to get the metadata and create the device instance.
> > >>
> > >> Exactly this is what I want to avoid.
> > >
> > > Why do you want to avoid that?
> > > I think you mean it is not what you had in mind.
> > >
> > >> The plug() function would become a "scan-one and probe".
> > >
> > > Yes
> > >
> > >> From my point of view plug() and unplug() should be orthogonal.
> > >> The plug() and unplug() should only be responsible for adding drivers
> > >> with optional arguments. The EAL should allow the drivers to get
> > >> unplugged/re-plugged at run-time. I want to be able to change arguments
> > >> ... or even drivers :)
> > >
> > > It is a totally different thing.
> > > We are talking about hotplug of a device,
> > > and you are talking about changing drivers dynamically.
> > >
> > > So I still don't understand what is the issue with the plug/unplug
> > > functions proposed here.
> > >
> > 
> > I don't agree with the notion that plug() means "scan-one and probe".
> 
> What do you see as plug doing then? What do you see as the use-case and
> then logic flow for hotplug?
> 

Hi all,

We are all for having "true" hotplug support in DPDK.
By true hotplug, it means that at some point, a new device exists on the
system, while the DPDK bus on which it should be probed does not yet
have its metadata. Something needs to be done to retrieve these metadata,
one way or another.

What I see as a solution to this:

- An interrupt framework integrated to rte_bus, allowing drivers to
  describe interrupt sources (Kernel UEVENT, custom FDs, ...), to their
  buses.

- Applications should be able to pilot these interrupts for rte_bus
  (either in describing expected devices, or allowing actions to be
  taken upon device removal).

- Buses could take the responsibility of plugging in and out their own
  devices, once properly configured.

In this context, it is possible to imagine triggering a bus-rescan upon
device ADD, iff we explicitly define scan() as being idempotent (a thing
that is not part of its API yet and that we cannot expect from buses at
this point).

Then, we can limit bus->plug() to a probe, once we are sure that
metadatas for the bus are (almost) always in sync with the system.

Where we are:

- Intel is proposing a UEVENT API[1]. It might be interesting to help them
  make it respect a generic framework.

- A first plug / unplug implementation is being proposed. Plug() is
  currently effectively defined as (scan-one + probe).

What can be done to go forward:

- Define the API for rte_bus interrupt sources, configuration and
  triggers to allow the development of the needed subsystems.

- Each device event sources should offer an event parser to transform
  them in device descriptions. Depending on the specifics of the source,
  different levels of info are available.

- Redefine the requirements for bus->scan() to allow hotplugging.

- Reduce the scope of plug() from (scan-one + probe) to (probe), as
  everything is now in place.

- Further hotplugging developments are then possible: add INTR_ADD
  support, with flexible device definition for example... A thing that
  is not yet possible with the current architecture but seemed to
  interest a few people.

If we can agree that this is a way forward, do you think Jan that having
the current plug() API hinders this plan? We can reduce its scope later,
without changing current implementations as, as you said, a proper
scan should be idempotent. The future API as envisionned is already
respected, but right now the hotplug support in buses is a little more
involved. Applications that will start using plug() right now would not
have to be rewritten, as the requirements for plugging would still be
fullfilled.

We can support already hotplugging in DPDK. We can refine this support
later to make it more generic and easier to implement for PMD
maintainers. But I do not see this as a reason to block this support
from being integrated right now.

[1]: http://dpdk.org/ml/archives/dev/2017-June/069057.html

-- 
Gaëtan Rivet
6WIND

[PATCH 00/15] bus attach/detach API and hotplug rework

[Jan Blunck]

Following the work from Gaetan, who based his work on my original series.

-- v7

* Revert removal of RTE_VERIFY on find_device operation since a bus that isn't
  able to even iterate its devices is pointless

* Revert plug() changes that have been introduced in v3:
  int (*rte_bus_plug_t)(struct rte_device *dev, const char *devargs);

* Revert pointless reordering of start argument for bus iterator functions

* Add fslmc bus find_device operation

* Add support to find a bus by its name

* Add EAL hotplug API:
  int rte_eal_hotplug_add(const char *busname, const char *devname,
			  const char *devargs);
  int rte_eal_hotplug_remove(const char *busname, const char *devname);

* Keep rte_eal_dev_attach() backwards compatible and support hotplug of PCI
  and virtual devices only

* Add back patch "ethdev: use embedded rte_device to detach driver" which is
  critical for correctly detaching the correct low-level device


Gaetan Rivet (1):
  pci: implement hotplug bus operation

Jan Blunck (14):
  bus: add bus iterator to find a bus
  bus: add find_device bus operation
  vdev: implement find_device bus operation
  pci: implement find_device bus operation
  bus/fslmc: implement find_device bus operation
  bus: add helper to find which bus holds a device
  bus: add bus iterator to find a device
  bus: require buses to implement find_device operation
  bus: add rte_bus_find_by_name
  bus: introduce device plug/unplug functionality
  vdev: implement unplug bus operation
  eal: add hotplug add/remove functions
  dev: use new hotplug API in attach / detach
  ethdev: Use embedded rte_device to detach driver

 drivers/bus/fslmc/fslmc_bus.c                   |  23 ++++
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |   6 +
 lib/librte_eal/common/eal_common_bus.c          |  74 ++++++++++++
 lib/librte_eal/common/eal_common_dev.c          | 149 ++++++++++++++++++++----
 lib/librte_eal/common/eal_common_pci.c          |  55 +++++++++
 lib/librte_eal/common/eal_common_vdev.c         |  35 ++++++
 lib/librte_eal/common/include/rte_bus.h         | 137 ++++++++++++++++++++++
 lib/librte_eal/common/include/rte_dev.h         |  57 +++++++++
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |   5 +
 lib/librte_ether/rte_ethdev.c                   |   3 +-
 10 files changed, 521 insertions(+), 23 deletions(-)

--
2.9.4

[PATCH v7 01/15] bus: add bus iterator to find a bus

[Jan Blunck]

This helper allows to iterate over all registered buses and find one
matching data used as parameter.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  1 +
 lib/librte_eal/common/eal_common_bus.c          | 19 +++++++++++
 lib/librte_eal/common/include/rte_bus.h         | 42 +++++++++++++++++++++++++
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |  1 +
 4 files changed, 63 insertions(+)

diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
index 2e48a73..ed09ab2 100644
--- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
@@ -162,6 +162,7 @@ DPDK_17.02 {
 DPDK_17.05 {
 	global:
 
+	rte_bus_find;
 	rte_cpu_is_supported;
 	rte_log_dump;
 	rte_log_register;
diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
index 8f9baf8..3094daa 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -145,3 +145,22 @@ rte_bus_dump(FILE *f)
 		}
 	}
 }
+
+struct rte_bus *
+rte_bus_find(const struct rte_bus *start, rte_bus_cmp_t cmp,
+	     const void *data)
+{
+	struct rte_bus *bus = NULL;
+	bool start_found = !start;
+
+	TAILQ_FOREACH(bus, &rte_bus_list, next) {
+		if (!start_found) {
+			if (bus == start)
+				start_found = 1;
+			continue;
+		}
+		if (cmp(bus, data) == 0)
+			break;
+	}
+	return bus;
+}
diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
index 5f47b82..c52b65b 100644
--- a/lib/librte_eal/common/include/rte_bus.h
+++ b/lib/librte_eal/common/include/rte_bus.h
@@ -141,6 +141,48 @@ int rte_bus_probe(void);
 void rte_bus_dump(FILE *f);
 
 /**
+ * Bus comparison function.
+ *
+ * @param bus
+ *	Bus under test.
+ *
+ * @param data
+ *	Data to compare against.
+ *
+ * @return
+ *	0 if the bus matches the data.
+ *	!0 if the bus does not match.
+ *	<0 if ordering is possible and the bus is lower than the data.
+ *	>0 if ordering is possible and the bus is greater than the data.
+ */
+typedef int (*rte_bus_cmp_t)(const struct rte_bus *bus, const void *data);
+
+/**
+ * Bus iterator to find a particular bus.
+ *
+ * This function compares each registered bus to find one that matches
+ * the data passed as parameter.
+ *
+ * If the comparison function returns zero this function will stop iterating
+ * over any more buses. To continue a search the bus of a previous search can
+ * be passed via the start parameter.
+ *
+ * @param start
+ *	Starting point for the iteration.
+ *
+ * @param cmp
+ *	Comparison function.
+ *
+ * @param data
+ *	 Data to pass to comparison function.
+ *
+ * @return
+ *	 A pointer to a rte_bus structure or NULL in case no bus matches
+ */
+struct rte_bus *rte_bus_find(const struct rte_bus *start, rte_bus_cmp_t cmp,
+			     const void *data);
+
+/**
  * Helper for Bus registration.
  * The constructor has higher priority than PMD constructors.
  */
diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
index 670bab3..6efa517 100644
--- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
@@ -166,6 +166,7 @@ DPDK_17.02 {
 DPDK_17.05 {
 	global:
 
+	rte_bus_find;
 	rte_cpu_is_supported;
 	rte_intr_free_epoll_fd;
 	rte_log_dump;
-- 
2.9.4

[PATCH v7 02/15] bus: add find_device bus operation

[Jan Blunck]

This new method allows buses to expose their devices in a controlled
manner. A comparison function is provided by the user to discriminate
between devices, using arbitrary data as identifier.

It is possible to start an iteration from a specific point, in order to
continue a search.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/include/rte_bus.h | 27 +++++++++++++++++++++++++++
 lib/librte_eal/common/include/rte_dev.h | 21 +++++++++++++++++++++
 2 files changed, 48 insertions(+)

diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
index c52b65b..052ac8d 100644
--- a/lib/librte_eal/common/include/rte_bus.h
+++ b/lib/librte_eal/common/include/rte_bus.h
@@ -82,6 +82,32 @@ typedef int (*rte_bus_scan_t)(void);
 typedef int (*rte_bus_probe_t)(void);
 
 /**
+ * Device iterator to find a device on a bus.
+ *
+ * This function returns an rte_device if one of those held by the bus
+ * matches the data passed as parameter.
+ *
+ * If the comparison function returns zero this function should stop iterating
+ * over any more devices. To continue a search the device of a previous search
+ * can be passed via the start parameter.
+ *
+ * @param cmp
+ *	Comparison function.
+ *
+ * @param data
+ *	Data to compare each device against.
+ *
+ * @param start
+ *	starting point for the iteration
+ *
+ * @return
+ *	The first device matching the data, NULL if none exists.
+ */
+typedef struct rte_device *
+(*rte_bus_find_device_t)(const struct rte_device *start, rte_dev_cmp_t cmp,
+			 const void *data);
+
+/**
  * A structure describing a generic bus.
  */
 struct rte_bus {
@@ -89,6 +115,7 @@ struct rte_bus {
 	const char *name;            /**< Name of the bus */
 	rte_bus_scan_t scan;         /**< Scan for devices attached to bus */
 	rte_bus_probe_t probe;       /**< Probe devices on bus */
+	rte_bus_find_device_t find_device; /**< Find a device on the bus */
 };
 
 /**
diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
index de20c06..04d9c28 100644
--- a/lib/librte_eal/common/include/rte_dev.h
+++ b/lib/librte_eal/common/include/rte_dev.h
@@ -191,6 +191,27 @@ int rte_eal_dev_attach(const char *name, const char *devargs);
  */
 int rte_eal_dev_detach(const char *name);
 
+/**
+ * Device comparison function.
+ *
+ * This type of function is used to compare an rte_device with arbitrary
+ * data.
+ *
+ * @param dev
+ *   Device handle.
+ *
+ * @param data
+ *   Data to compare against. The type of this parameter is determined by
+ *   the kind of comparison performed by the function.
+ *
+ * @return
+ *   0 if the device matches the data.
+ *   !0 if the device does not match.
+ *   <0 if ordering is possible and the device is lower than the data.
+ *   >0 if ordering is possible and the device is greater than the data.
+ */
+typedef int (*rte_dev_cmp_t)(const struct rte_device *dev, const void *data);
+
 #define RTE_PMD_EXPORT_NAME_ARRAY(n, idx) n##idx[]
 
 #define RTE_PMD_EXPORT_NAME(name, idx) \
-- 
2.9.4

[PATCH v7 03/15] vdev: implement find_device bus operation

[Jan Blunck]

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/eal_common_vdev.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_vdev.c b/lib/librte_eal/common/eal_common_vdev.c
index b4db2be..9bb7427 100644
--- a/lib/librte_eal/common/eal_common_vdev.c
+++ b/lib/librte_eal/common/eal_common_vdev.c
@@ -35,6 +35,7 @@
 #include <stdio.h>
 #include <stdlib.h>
 #include <stdint.h>
+#include <stdbool.h>
 #include <sys/queue.h>
 
 #include <rte_eal.h>
@@ -334,9 +335,29 @@ vdev_probe(void)
 	return 0;
 }
 
+static struct rte_device *
+vdev_find_device(const struct rte_device *start, rte_dev_cmp_t cmp,
+		 const void *data)
+{
+	struct rte_vdev_device *dev;
+	bool start_found = !start;
+
+	TAILQ_FOREACH(dev, &vdev_device_list, next) {
+		if (start_found == 0) {
+			if (&dev->device == start)
+				start_found = 1;
+			continue;
+		}
+		if (cmp(&dev->device, data) == 0)
+			return &dev->device;
+	}
+	return NULL;
+}
+
 static struct rte_bus rte_vdev_bus = {
 	.scan = vdev_scan,
 	.probe = vdev_probe,
+	.find_device = vdev_find_device,
 };
 
 RTE_REGISTER_BUS(VIRTUAL_BUS_NAME, rte_vdev_bus);
-- 
2.9.4

[PATCH v7 04/15] pci: implement find_device bus operation

[Jan Blunck]

Signed-off-by: Jan Blunck <jblunck@infradead.org>
---
 lib/librte_eal/common/eal_common_pci.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c
index 479c62d..e449758 100644
--- a/lib/librte_eal/common/eal_common_pci.c
+++ b/lib/librte_eal/common/eal_common_pci.c
@@ -485,10 +485,31 @@ rte_pci_remove_device(struct rte_pci_device *pci_dev)
 	TAILQ_REMOVE(&rte_pci_bus.device_list, pci_dev, next);
 }
 
+static struct rte_device *
+pci_find_device(const struct rte_device *start, rte_dev_cmp_t cmp,
+		const void *data)
+{
+	struct rte_pci_device *dev;
+	bool start_found = !start;
+
+	FOREACH_DEVICE_ON_PCIBUS(dev) {
+		if (!start_found) {
+			if (&dev->device == start)
+				start_found = 1;
+			continue;
+		}
+		if (cmp(&dev->device, data) == 0)
+			return &dev->device;
+	}
+
+	return NULL;
+}
+
 struct rte_pci_bus rte_pci_bus = {
 	.bus = {
 		.scan = rte_pci_scan,
 		.probe = rte_pci_probe,
+		.find_device = pci_find_device,
 	},
 	.device_list = TAILQ_HEAD_INITIALIZER(rte_pci_bus.device_list),
 	.driver_list = TAILQ_HEAD_INITIALIZER(rte_pci_bus.driver_list),
-- 
2.9.4

[PATCH v7 05/15] bus/fslmc: implement find_device bus operation

[Jan Blunck]

Signed-off-by: Jan Blunck <jblunck@infradead.org>
---
 drivers/bus/fslmc/fslmc_bus.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/drivers/bus/fslmc/fslmc_bus.c b/drivers/bus/fslmc/fslmc_bus.c
index b24642d..1e3bbee 100644
--- a/drivers/bus/fslmc/fslmc_bus.c
+++ b/drivers/bus/fslmc/fslmc_bus.c
@@ -32,6 +32,7 @@
 
 #include <string.h>
 #include <dirent.h>
+#include <stdbool.h>
 
 #include <rte_log.h>
 #include <rte_bus.h>
@@ -105,6 +106,27 @@ rte_fslmc_probe(void)
 	return ret;
 }
 
+static struct rte_device *
+rte_fslmc_find_device(const struct rte_device *start, rte_dev_cmp_t cmp,
+		      const void *data)
+{
+	struct rte_dpaa2_device *dev;
+	bool start_found = !start;
+
+	TAILQ_FOREACH(dev, &rte_fslmc_bus.device_list, next) {
+		if (!start_found) {
+			if (&dev->device == start)
+				start_found = 1;
+			continue;
+		}
+
+		if (cmp(&dev->device, data) == 0)
+			return &dev->device;
+	}
+
+	return NULL;
+}
+
 /*register a fslmc bus based dpaa2 driver */
 void
 rte_fslmc_driver_register(struct rte_dpaa2_driver *driver)
@@ -133,6 +155,7 @@ struct rte_fslmc_bus rte_fslmc_bus = {
 	.bus = {
 		.scan = rte_fslmc_scan,
 		.probe = rte_fslmc_probe,
+		.find_device = rte_fslmc_find_device,
 	},
 	.device_list = TAILQ_HEAD_INITIALIZER(rte_fslmc_bus.device_list),
 	.driver_list = TAILQ_HEAD_INITIALIZER(rte_fslmc_bus.driver_list),
-- 
2.9.4

[PATCH v7 06/15] bus: add helper to find which bus holds a device

[Jan Blunck]

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  1 +
 lib/librte_eal/common/eal_common_bus.c          | 23 +++++++++++++++++++++++
 lib/librte_eal/common/include/rte_bus.h         |  5 +++++
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |  1 +
 4 files changed, 30 insertions(+)

diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
index ed09ab2..f1a0765 100644
--- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
@@ -163,6 +163,7 @@ DPDK_17.05 {
 	global:
 
 	rte_bus_find;
+	rte_bus_find_by_device;
 	rte_cpu_is_supported;
 	rte_log_dump;
 	rte_log_register;
diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
index 3094daa..276cce6 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -164,3 +164,26 @@ rte_bus_find(const struct rte_bus *start, rte_bus_cmp_t cmp,
 	}
 	return bus;
 }
+
+static int
+cmp_rte_device(const struct rte_device *dev1, const void *_dev2)
+{
+	const struct rte_device *dev2 = _dev2;
+
+	return dev1 != dev2;
+}
+
+static int
+bus_find_device(const struct rte_bus *bus, const void *_dev)
+{
+	struct rte_device *dev;
+
+	dev = bus->find_device(NULL, cmp_rte_device, _dev);
+	return !dev;
+}
+
+struct rte_bus *
+rte_bus_find_by_device(const struct rte_device *dev)
+{
+	return rte_bus_find(NULL, bus_find_device, (const void *)dev);
+}
diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
index 052ac8d..f8b3215 100644
--- a/lib/librte_eal/common/include/rte_bus.h
+++ b/lib/librte_eal/common/include/rte_bus.h
@@ -210,6 +210,11 @@ struct rte_bus *rte_bus_find(const struct rte_bus *start, rte_bus_cmp_t cmp,
 			     const void *data);
 
 /**
+ * Find the registered bus for a particular device.
+ */
+struct rte_bus *rte_bus_find_by_device(const struct rte_device *dev);
+
+/**
  * Helper for Bus registration.
  * The constructor has higher priority than PMD constructors.
  */
diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
index 6efa517..6f77222 100644
--- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
@@ -167,6 +167,7 @@ DPDK_17.05 {
 	global:
 
 	rte_bus_find;
+	rte_bus_find_by_device;
 	rte_cpu_is_supported;
 	rte_intr_free_epoll_fd;
 	rte_log_dump;
-- 
2.9.4

[PATCH v7 07/15] bus: add bus iterator to find a device

[Jan Blunck]

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  1 +
 lib/librte_eal/common/eal_common_bus.c          | 15 ++++++++++++++
 lib/librte_eal/common/include/rte_bus.h         | 26 +++++++++++++++++++++++++
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |  1 +
 4 files changed, 43 insertions(+)

diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
index f1a0765..21640d6 100644
--- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
@@ -164,6 +164,7 @@ DPDK_17.05 {
 
 	rte_bus_find;
 	rte_bus_find_by_device;
+	rte_bus_find_device;
 	rte_cpu_is_supported;
 	rte_log_dump;
 	rte_log_register;
diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
index 276cce6..61aa947 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -187,3 +187,18 @@ rte_bus_find_by_device(const struct rte_device *dev)
 {
 	return rte_bus_find(NULL, bus_find_device, (const void *)dev);
 }
+
+struct rte_device *
+rte_bus_find_device(const struct rte_device *start, rte_dev_cmp_t cmp,
+		    const void *data)
+{
+	struct rte_bus *bus;
+	struct rte_device *dev = NULL;
+
+	TAILQ_FOREACH(bus, &rte_bus_list, next) {
+		dev = bus->find_device(start, cmp, data);
+		if (dev)
+			break;
+	}
+	return dev;
+}
diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
index f8b3215..fea0f39 100644
--- a/lib/librte_eal/common/include/rte_bus.h
+++ b/lib/librte_eal/common/include/rte_bus.h
@@ -210,6 +210,32 @@ struct rte_bus *rte_bus_find(const struct rte_bus *start, rte_bus_cmp_t cmp,
 			     const void *data);
 
 /**
+ * Bus iterator to find a particular device.
+ *
+ * This function searches each registered bus to find a device that matches
+ * the data passed as parameter.
+ *
+ * If the comparison function returns zero this function will stop iterating
+ * over any more buses and devices. To continue a search the device of
+ * a previous search can be passed via the start parameter.
+ *
+ * @param start
+ *	Starting point for the iteration.
+ *
+ * @param cmp
+ *	Comparison function.
+ *
+ * @param data
+ *	Data to pass to comparison function.
+ *
+ * @return
+ *	A pointer to a rte_bus structure or NULL in case no device matches.
+ */
+struct rte_device *rte_bus_find_device(const struct rte_device *start,
+				       rte_dev_cmp_t cmp,
+				       const void *data);
+
+/**
  * Find the registered bus for a particular device.
  */
 struct rte_bus *rte_bus_find_by_device(const struct rte_device *dev);
diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
index 6f77222..e0a056d 100644
--- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
@@ -168,6 +168,7 @@ DPDK_17.05 {
 
 	rte_bus_find;
 	rte_bus_find_by_device;
+	rte_bus_find_device;
 	rte_cpu_is_supported;
 	rte_intr_free_epoll_fd;
 	rte_log_dump;
-- 
2.9.4

[PATCH v7 08/15] bus: require buses to implement find_device operation

[Jan Blunck]

Signed-off-by: Jan Blunck <jblunck@infradead.org>
---
 lib/librte_eal/common/eal_common_bus.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
index 61aa947..d0e652e 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -50,6 +50,7 @@ rte_bus_register(struct rte_bus *bus)
 	/* A bus should mandatorily have the scan implemented */
 	RTE_VERIFY(bus->scan);
 	RTE_VERIFY(bus->probe);
+	RTE_VERIFY(bus->find_device);
 
 	TAILQ_INSERT_TAIL(&rte_bus_list, bus, next);
 	RTE_LOG(DEBUG, EAL, "Registered [%s] bus.\n", bus->name);
-- 
2.9.4

[PATCH v7 09/15] bus: add rte_bus_find_by_name

[Jan Blunck]

Signed-off-by: Jan Blunck <jblunck@infradead.org>
---
 lib/librte_eal/common/eal_common_bus.c  | 14 ++++++++++++++
 lib/librte_eal/common/include/rte_bus.h |  5 +++++
 2 files changed, 19 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
index d0e652e..418804a 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -203,3 +203,17 @@ rte_bus_find_device(const struct rte_device *start, rte_dev_cmp_t cmp,
 	}
 	return dev;
 }
+
+static int
+cmp_bus_name(const struct rte_bus *bus, const void *_name)
+{
+	const char *name = _name;
+
+	return strcmp(bus->name, name);
+}
+
+struct rte_bus *
+rte_bus_find_by_name(const char *busname)
+{
+	return rte_bus_find(NULL, cmp_bus_name, (const void *)busname);
+}
diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
index fea0f39..08f8d46 100644
--- a/lib/librte_eal/common/include/rte_bus.h
+++ b/lib/librte_eal/common/include/rte_bus.h
@@ -241,6 +241,11 @@ struct rte_device *rte_bus_find_device(const struct rte_device *start,
 struct rte_bus *rte_bus_find_by_device(const struct rte_device *dev);
 
 /**
+ * Find the registered bus for a given name.
+ */
+struct rte_bus *rte_bus_find_by_name(const char *busname);
+
+/**
  * Helper for Bus registration.
  * The constructor has higher priority than PMD constructors.
  */
-- 
2.9.4

[PATCH v7 10/15] bus: introduce device plug/unplug functionality

[Jan Blunck]

This allows the buses to plug and probe specific devices. This is meant to
be a building block for hotplug support.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
---
 lib/librte_eal/common/eal_common_bus.c  |  2 ++
 lib/librte_eal/common/include/rte_bus.h | 32 ++++++++++++++++++++++++++++++++
 2 files changed, 34 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
index 418804a..22ff0f6 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -51,6 +51,8 @@ rte_bus_register(struct rte_bus *bus)
 	RTE_VERIFY(bus->scan);
 	RTE_VERIFY(bus->probe);
 	RTE_VERIFY(bus->find_device);
+	/* Buses supporting driver plug also require unplug. */
+	RTE_VERIFY(!bus->plug || bus->unplug);
 
 	TAILQ_INSERT_TAIL(&rte_bus_list, bus, next);
 	RTE_LOG(DEBUG, EAL, "Registered [%s] bus.\n", bus->name);
diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
index 08f8d46..4b6c73b 100644
--- a/lib/librte_eal/common/include/rte_bus.h
+++ b/lib/librte_eal/common/include/rte_bus.h
@@ -108,6 +108,36 @@ typedef struct rte_device *
 			 const void *data);
 
 /**
+ * Implementation specific probe function which is responsible for linking
+ * devices on that bus with applicable drivers.
+ *
+ * @param dev
+ *	Device pointer that was returned by a previous call to find_device.
+ *
+ * @param devargs
+ *	Device declaration.
+ *
+ * @return
+ *	The pointer to a valid rte_device usable by the bus on success.
+ *	NULL on error. rte_errno is then set.
+ */
+typedef int (*rte_bus_plug_t)(struct rte_device *dev,
+			      const char *devargs);
+
+/**
+ * Implementation specific remove function which is responsible for unlinking
+ * devices on that bus from assigned driver.
+ *
+ * @param dev
+ *	Device pointer that was returned by a previous call to find_device.
+ *
+ * @return
+ *	0 on success.
+ *	!0 on error. rte_errno is then set.
+ */
+typedef int (*rte_bus_unplug_t)(struct rte_device *dev);
+
+/**
  * A structure describing a generic bus.
  */
 struct rte_bus {
@@ -116,6 +146,8 @@ struct rte_bus {
 	rte_bus_scan_t scan;         /**< Scan for devices attached to bus */
 	rte_bus_probe_t probe;       /**< Probe devices on bus */
 	rte_bus_find_device_t find_device; /**< Find a device on the bus */
+	rte_bus_plug_t plug;         /**< Probe single device for drivers */
+	rte_bus_unplug_t unplug;     /**< Remove single device from driver */
 };
 
 /**
-- 
2.9.4

[PATCH v7 11/15] vdev: implement unplug bus operation

[Jan Blunck]

Signed-off-by: Jan Blunck <jblunck@infradead.org>
---
 lib/librte_eal/common/eal_common_vdev.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_vdev.c b/lib/librte_eal/common/eal_common_vdev.c
index 9bb7427..baf3c5b 100644
--- a/lib/librte_eal/common/eal_common_vdev.c
+++ b/lib/librte_eal/common/eal_common_vdev.c
@@ -39,11 +39,13 @@
 #include <sys/queue.h>
 
 #include <rte_eal.h>
+#include <rte_dev.h>
 #include <rte_bus.h>
 #include <rte_vdev.h>
 #include <rte_common.h>
 #include <rte_devargs.h>
 #include <rte_memory.h>
+#include <rte_errno.h>
 
 /** Double linked list of virtual device drivers. */
 TAILQ_HEAD(vdev_device_list, rte_vdev_device);
@@ -354,10 +356,22 @@ vdev_find_device(const struct rte_device *start, rte_dev_cmp_t cmp,
 	return NULL;
 }
 
+static int
+vdev_unplug(struct rte_device *dev)
+{
+	/*
+	 * The virtual bus doesn't support 'unattached' devices so this is
+	 * actually equal to hotplugging removal of it.
+	 */
+	return rte_vdev_uninit(dev->name);
+}
+
 static struct rte_bus rte_vdev_bus = {
 	.scan = vdev_scan,
 	.probe = vdev_probe,
 	.find_device = vdev_find_device,
+	/* .plug = NULL, see comment on vdev_unplug */
+	.unplug = vdev_unplug,
 };
 
 RTE_REGISTER_BUS(VIRTUAL_BUS_NAME, rte_vdev_bus);
-- 
2.9.4

[PATCH v7 12/15] pci: implement hotplug bus operation

[Jan Blunck]

From: Gaetan Rivet <gaetan.rivet@6wind.com>

Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/eal_common_pci.c | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c
index e449758..5ee100e 100644
--- a/lib/librte_eal/common/eal_common_pci.c
+++ b/lib/librte_eal/common/eal_common_pci.c
@@ -47,6 +47,7 @@
 #include <rte_pci.h>
 #include <rte_per_lcore.h>
 #include <rte_memory.h>
+#include <rte_memcpy.h>
 #include <rte_memzone.h>
 #include <rte_eal.h>
 #include <rte_string_fns.h>
@@ -505,11 +506,44 @@ pci_find_device(const struct rte_device *start, rte_dev_cmp_t cmp,
 	return NULL;
 }
 
+static int
+pci_plug(struct rte_device *dev, const char *devargs __rte_unused)
+{
+	struct rte_pci_device *pdev;
+	struct rte_pci_addr *addr;
+
+	addr = &RTE_DEV_TO_PCI(dev)->addr;
+
+	/* Find the current device holding this address in the bus. */
+	FOREACH_DEVICE_ON_PCIBUS(pdev) {
+		if (rte_eal_compare_pci_addr(&pdev->addr, addr) == 0)
+			return rte_pci_probe_one(addr);
+	}
+
+	rte_errno = ENODEV;
+	return -1;
+}
+
+static int
+pci_unplug(struct rte_device *dev)
+{
+	struct rte_pci_device *pdev;
+
+	pdev = RTE_DEV_TO_PCI(dev);
+	if (rte_pci_detach(&pdev->addr) != 0) {
+		rte_errno = ENODEV;
+		return -1;
+	}
+	return 0;
+}
+
 struct rte_pci_bus rte_pci_bus = {
 	.bus = {
 		.scan = rte_pci_scan,
 		.probe = rte_pci_probe,
 		.find_device = pci_find_device,
+		.plug = pci_plug,
+		.unplug = pci_unplug,
 	},
 	.device_list = TAILQ_HEAD_INITIALIZER(rte_pci_bus.device_list),
 	.driver_list = TAILQ_HEAD_INITIALIZER(rte_pci_bus.driver_list),
-- 
2.9.4

[PATCH v7 13/15] eal: add hotplug add/remove functions

[Jan Blunck]

Signed-off-by: Jan Blunck <jblunck@infradead.org>
---
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  2 +
 lib/librte_eal/common/eal_common_dev.c          | 68 +++++++++++++++++++++++++
 lib/librte_eal/common/include/rte_dev.h         | 28 ++++++++++
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |  2 +
 4 files changed, 100 insertions(+)

diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
index 21640d6..b7d26b2 100644
--- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
@@ -166,6 +166,8 @@ DPDK_17.05 {
 	rte_bus_find_by_device;
 	rte_bus_find_device;
 	rte_cpu_is_supported;
+	rte_eal_hotplug_add;
+	rte_eal_hotplug_remove;
 	rte_log_dump;
 	rte_log_register;
 	rte_log_get_global_level;
diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c
index a400ddd..477b4cf 100644
--- a/lib/librte_eal/common/eal_common_dev.c
+++ b/lib/librte_eal/common/eal_common_dev.c
@@ -92,3 +92,71 @@ int rte_eal_dev_detach(const char *name)
 	RTE_LOG(ERR, EAL, "Driver cannot detach the device (%s)\n", name);
 	return -EINVAL;
 }
+
+int rte_eal_hotplug_add(const char *busname, const char *devname,
+			const char *devargs)
+{
+	struct rte_bus *bus;
+	struct rte_device *dev;
+	int ret;
+
+	bus = rte_bus_find_by_name(busname);
+	if (!bus) {
+		RTE_LOG(ERR, EAL, "Cannot find bus (%s)\n", busname);
+		return -ENOENT;
+	}
+
+	if (!bus->plug) {
+		RTE_LOG(ERR, EAL, "Function plug not supported by bus (%s)\n",
+			bus->name);
+		return -ENOTSUP;
+	}
+
+	ret = bus->scan();
+	if (ret)
+		return ret;
+
+	dev = bus->find_device(NULL, cmp_detached_dev_name, devname);
+	if (!dev) {
+		RTE_LOG(ERR, EAL, "Cannot find unplugged device (%s)\n",
+			devname);
+		return -EINVAL;
+	}
+
+	ret = bus->plug(dev, devargs);
+	if (ret)
+		RTE_LOG(ERR, EAL, "Driver cannot attach the device (%s)\n",
+			dev->name);
+	return ret;
+}
+
+int rte_eal_hotplug_remove(const char *busname, const char *devname)
+{
+	struct rte_bus *bus;
+	struct rte_device *dev;
+	int ret;
+
+	bus = rte_bus_find_by_name(busname);
+	if (!bus) {
+		RTE_LOG(ERR, EAL, "Cannot find bus (%s)\n", busname);
+		return -ENOENT;
+	}
+
+	if (!bus->unplug) {
+		RTE_LOG(ERR, EAL, "Function unplug not supported by bus (%s)\n",
+			bus->name);
+		return -ENOTSUP;
+	}
+
+	dev = bus->find_device(NULL, cmp_dev_name, devname);
+	if (!dev) {
+		RTE_LOG(ERR, EAL, "Cannot find plugged device (%s)\n", devname);
+		return -EINVAL;
+	}
+
+	ret = bus->unplug(dev);
+	if (ret)
+		RTE_LOG(ERR, EAL, "Driver cannot detach the device (%s)\n",
+			dev->name);
+	return ret;
+}
diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
index 04d9c28..95440eb 100644
--- a/lib/librte_eal/common/include/rte_dev.h
+++ b/lib/librte_eal/common/include/rte_dev.h
@@ -192,6 +192,34 @@ int rte_eal_dev_attach(const char *name, const char *devargs);
 int rte_eal_dev_detach(const char *name);
 
 /**
+ * Hotplug add a given device to a specific bus.
+ *
+ * @param busname
+ *   The bus name the device is added to.
+ * @param devname
+ *   The device name. Based on this device name, eal will identify a driver
+ *   capable of handling it and pass it to the driver probing function.
+ * @param devargs
+ *   Device arguments to be passed to the driver.
+ * @return
+ *   0 on success, negative on error.
+ */
+int rte_eal_hotplug_add(const char *busname, const char *devname,
+			const char *devargs);
+
+/**
+ * Hotplug remove a given device from a specific bus.
+ *
+ * @param busname
+ *   The bus name the device is removed from.
+ * @param devname
+ *   The device name being removed.
+ * @return
+ *   0 on success, negative on error.
+ */
+int rte_eal_hotplug_remove(const char *busname, const char *devname);
+
+/**
  * Device comparison function.
  *
  * This type of function is used to compare an rte_device with arbitrary
diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
index e0a056d..a94cb7a 100644
--- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
@@ -170,6 +170,8 @@ DPDK_17.05 {
 	rte_bus_find_by_device;
 	rte_bus_find_device;
 	rte_cpu_is_supported;
+	rte_eal_hotplug_add;
+	rte_eal_hotplug_remove;
 	rte_intr_free_epoll_fd;
 	rte_log_dump;
 	rte_log_get_global_level;
-- 
2.9.4

[PATCH v7 14/15] dev: use new hotplug API in attach / detach

[Jan Blunck]

Using the new generic API allows attach and detach to be backwards
compatible while decoupling from the concrete bus implementations.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
---
 lib/librte_eal/common/eal_common_dev.c | 76 ++++++++++++++++++++++++----------
 1 file changed, 53 insertions(+), 23 deletions(-)

diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c
index 477b4cf..68c6b45 100644
--- a/lib/librte_eal/common/eal_common_dev.c
+++ b/lib/librte_eal/common/eal_common_dev.c
@@ -37,6 +37,7 @@
 #include <inttypes.h>
 #include <sys/queue.h>
 
+#include <rte_bus.h>
 #include <rte_dev.h>
 #include <rte_devargs.h>
 #include <rte_debug.h>
@@ -45,52 +46,81 @@
 
 #include "eal_private.h"
 
+static int cmp_detached_dev_name(const struct rte_device *dev,
+	const void *_name)
+{
+	const char *name = _name;
+
+	/* skip attached devices */
+	if (dev->driver)
+		return 1;
+	return strcmp(dev->name, name);
+}
+
 int rte_eal_dev_attach(const char *name, const char *devargs)
 {
-	struct rte_pci_addr addr;
+	int ret;
 
 	if (name == NULL || devargs == NULL) {
 		RTE_LOG(ERR, EAL, "Invalid device or arguments provided\n");
 		return -EINVAL;
 	}
 
-	if (eal_parse_pci_DomBDF(name, &addr) == 0) {
-		if (rte_pci_probe_one(&addr) < 0)
-			goto err;
+	ret = rte_eal_hotplug_add("PCI", name, devargs);
+	if (ret && ret != -EINVAL)
+		return ret;
 
-	} else {
-		if (rte_vdev_init(name, devargs))
-			goto err;
-	}
+	/*
+	 * If we haven't found a bus device the user meant to "hotplug" a
+	 * virtual device instead.
+	 */
+	ret = rte_vdev_init(name, devargs);
+	if (ret)
+		RTE_LOG(ERR, EAL, "Driver cannot attach the device (%s)\n",
+			name);
+	return ret;
+}
 
-	return 0;
+static int cmp_dev_name(const struct rte_device *dev, const void *_name)
+{
+	const char *name = _name;
 
-err:
-	RTE_LOG(ERR, EAL, "Driver cannot attach the device (%s)\n", name);
-	return -EINVAL;
+	return strcmp(dev->name, name);
 }
 
 int rte_eal_dev_detach(const char *name)
 {
-	struct rte_pci_addr addr;
+	struct rte_device *dev;
+	struct rte_bus *bus;
+	int ret;
 
 	if (name == NULL) {
 		RTE_LOG(ERR, EAL, "Invalid device provided.\n");
 		return -EINVAL;
 	}
 
-	if (eal_parse_pci_DomBDF(name, &addr) == 0) {
-		if (rte_pci_detach(&addr) < 0)
-			goto err;
-	} else {
-		if (rte_vdev_uninit(name))
-			goto err;
+	dev = rte_bus_find_device(NULL, cmp_dev_name, name);
+	if (!dev) {
+		RTE_LOG(ERR, EAL, "Cannot find device (%s)\n", name);
+		return -EINVAL;
+	}
+
+	bus = rte_bus_find_by_device(dev);
+	if (!bus) {
+		RTE_LOG(ERR, EAL, "Cannot find bus for device (%s)\n", name);
+		return -EINVAL;
+	}
+
+	if (!bus->unplug) {
+		RTE_LOG(ERR, EAL, "Bus function not supported\n");
+		return -ENOTSUP;
 	}
-	return 0;
 
-err:
-	RTE_LOG(ERR, EAL, "Driver cannot detach the device (%s)\n", name);
-	return -EINVAL;
+	ret = bus->unplug(dev);
+	if (ret)
+		RTE_LOG(ERR, EAL, "Driver cannot detach the device (%s)\n",
+			name);
+	return ret;
 }
 
 int rte_eal_hotplug_add(const char *busname, const char *devname,
-- 
2.9.4

[PATCH v7 15/15] ethdev: Use embedded rte_device to detach driver

[Jan Blunck]

Signed-off-by: Jan Blunck <jblunck@infradead.org>
---
 lib/librte_eal/bsdapp/eal/rte_eal_version.map |  1 +
 lib/librte_eal/common/eal_common_dev.c        | 43 ++++++++++++++++-----------
 lib/librte_eal/common/include/rte_dev.h       |  8 +++++
 lib/librte_ether/rte_ethdev.c                 |  3 +-
 4 files changed, 36 insertions(+), 19 deletions(-)

diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
index b7d26b2..df7abc5 100644
--- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
@@ -162,6 +162,7 @@ DPDK_17.02 {
 DPDK_17.05 {
 	global:
 
+	rte_eal_device_detach;
 	rte_bus_find;
 	rte_bus_find_by_device;
 	rte_bus_find_device;
diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c
index 68c6b45..3f3bfdc 100644
--- a/lib/librte_eal/common/eal_common_dev.c
+++ b/lib/librte_eal/common/eal_common_dev.c
@@ -81,6 +81,30 @@ int rte_eal_dev_attach(const char *name, const char *devargs)
 	return ret;
 }
 
+int rte_eal_device_detach(struct rte_device *dev)
+{
+	struct rte_bus *bus;
+	int ret;
+
+	bus = rte_bus_find_by_device(dev);
+	if (!bus) {
+		RTE_LOG(ERR, EAL, "Cannot find bus for device (%s)\n",
+			dev->name);
+		return -EINVAL;
+	}
+
+	if (!bus->unplug) {
+		RTE_LOG(ERR, EAL, "Bus function not supported\n");
+		return -ENOTSUP;
+	}
+
+	ret = bus->unplug(dev);
+	if (ret)
+		RTE_LOG(ERR, EAL, "Driver cannot detach the device (%s)\n",
+			dev->name);
+	return ret;
+}
+
 static int cmp_dev_name(const struct rte_device *dev, const void *_name)
 {
 	const char *name = _name;
@@ -91,8 +115,6 @@ static int cmp_dev_name(const struct rte_device *dev, const void *_name)
 int rte_eal_dev_detach(const char *name)
 {
 	struct rte_device *dev;
-	struct rte_bus *bus;
-	int ret;
 
 	if (name == NULL) {
 		RTE_LOG(ERR, EAL, "Invalid device provided.\n");
@@ -105,22 +127,7 @@ int rte_eal_dev_detach(const char *name)
 		return -EINVAL;
 	}
 
-	bus = rte_bus_find_by_device(dev);
-	if (!bus) {
-		RTE_LOG(ERR, EAL, "Cannot find bus for device (%s)\n", name);
-		return -EINVAL;
-	}
-
-	if (!bus->unplug) {
-		RTE_LOG(ERR, EAL, "Bus function not supported\n");
-		return -ENOTSUP;
-	}
-
-	ret = bus->unplug(dev);
-	if (ret)
-		RTE_LOG(ERR, EAL, "Driver cannot detach the device (%s)\n",
-			name);
-	return ret;
+	return rte_eal_device_detach(dev);
 }
 
 int rte_eal_hotplug_add(const char *busname, const char *devname,
diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
index 95440eb..a3e6d7a 100644
--- a/lib/librte_eal/common/include/rte_dev.h
+++ b/lib/librte_eal/common/include/rte_dev.h
@@ -192,6 +192,14 @@ int rte_eal_dev_attach(const char *name, const char *devargs);
 int rte_eal_dev_detach(const char *name);
 
 /**
+ * Unplug the device from the device driver.
+ *
+ * @param dev
+ *   A pointer to a rte_device structure.
+ */
+int rte_eal_device_detach(struct rte_device *dev);
+
+/**
  * Hotplug add a given device to a specific bus.
  *
  * @param busname
diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index 81a45c0..89addc6 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -439,7 +439,8 @@ rte_eth_dev_detach(uint8_t port_id, char *name)
 
 	snprintf(name, sizeof(rte_eth_devices[port_id].data->name),
 		 "%s", rte_eth_devices[port_id].data->name);
-	ret = rte_eal_dev_detach(name);
+
+	ret = rte_eal_device_detach(rte_eth_devices[0].device);
 	if (ret < 0)
 		goto err;
 
-- 
2.9.4

Re: [PATCH v6 05/11] bus: introduce hotplug functionality

[Jan Blunck]

On Thu, Jun 29, 2017 at 2:59 PM, Gaëtan Rivet <gaetan.rivet@6wind.com> wrote:
>
> Hi all,
>
> We are all for having "true" hotplug support in DPDK.
> By true hotplug, it means that at some point, a new device exists on the
> system, while the DPDK bus on which it should be probed does not yet
> have its metadata. Something needs to be done to retrieve these metadata,
> one way or another.
>
> What I see as a solution to this:
>
> - An interrupt framework integrated to rte_bus, allowing drivers to
>   describe interrupt sources (Kernel UEVENT, custom FDs, ...), to their
>   buses.
>
> - Applications should be able to pilot these interrupts for rte_bus
>   (either in describing expected devices, or allowing actions to be
>   taken upon device removal).
>
> - Buses could take the responsibility of plugging in and out their own
>   devices, once properly configured.
>

This is highly application dependent and it is up to the application
developer to decide when such events are getting processed. There is a
major difference between the data path functionality that support
interrupts and the processing of hotplug events. So from my
perspective it needs to be left as an exercise to the programmer to
add the polling of the /sysfs files into the event loop. We might
offer an example of how to do this though.

> In this context, it is possible to imagine triggering a bus-rescan upon
> device ADD, iff we explicitly define scan() as being idempotent (a thing
> that is not part of its API yet and that we cannot expect from buses at
> this point).

Hmm, so from what I can tell the PCI bus offers an idempotent scan()
and if I haven't added any bugs this is true for the virtual bus too.

> Then, we can limit bus->plug() to a probe, once we are sure that
> metadatas for the bus are (almost) always in sync with the system.
>
> Where we are:
>
> - Intel is proposing a UEVENT API[1]. It might be interesting to help them
>   make it respect a generic framework.

Just because you can do it and someone invested time doesn't mean its
a good idea. What problem is this solving? You now have a DPDK native
library that reads uevents ... Where is the benefit for the
application developer?

Although we replicate some components of an operating system in
userspace it doesn't mean that we are building one. If we don't push
back on things that don't belong here we will have a big pile of
average code soon instead of focusing on the technical problems that
need to get addressed.

> - A first plug / unplug implementation is being proposed. Plug() is
>   currently effectively defined as (scan-one + probe).
>
> What can be done to go forward:
>
> - Define the API for rte_bus interrupt sources, configuration and
>   triggers to allow the development of the needed subsystems.
>
> - Each device event sources should offer an event parser to transform
>   them in device descriptions. Depending on the specifics of the source,
>   different levels of info are available.
>
> - Redefine the requirements for bus->scan() to allow hotplugging.
>
> - Reduce the scope of plug() from (scan-one + probe) to (probe), as
>   everything is now in place.
>

Also see the series that I send out today. From my point of view we
are here already.

> - Further hotplugging developments are then possible: add INTR_ADD
>   support, with flexible device definition for example... A thing that
>   is not yet possible with the current architecture but seemed to
>   interest a few people.
>
> If we can agree that this is a way forward, do you think Jan that having
> the current plug() API hinders this plan? We can reduce its scope later,
> without changing current implementations as, as you said, a proper
> scan should be idempotent. The future API as envisionned is already
> respected, but right now the hotplug support in buses is a little more
> involved. Applications that will start using plug() right now would not
> have to be rewritten, as the requirements for plugging would still be
> fullfilled.
>
> We can support already hotplugging in DPDK. We can refine this support
> later to make it more generic and easier to implement for PMD
> maintainers. But I do not see this as a reason to block this support
> from being integrated right now.

Indeed. I had hotplug support in the version of DPDK that I'm working
with for the last years. Don't get me wrong: I'm not arguing against
the inclusion of hotplug code. I just don't understand the reasoning
behind the implementation you proposed (with my original SOB)
especially since some of the things that you listed as going forward
are already there, are easy to fix or don't necessarily need to be
part of the DPDK EAL.

So lets try to get this right from the beginning. What is missing:
1. document and verify that existing scan() implementations are idempotent
2. example app with udev based hotplug
3. check that the symbols are in the correct libraries (bus, pci, vdev)

What am I missing?

> [1]: http://dpdk.org/ml/archives/dev/2017-June/069057.html
>
> --
> Gaėtan Rivet
> 6WIND

Re: [PATCH v7 15/15] ethdev: Use embedded rte_device to detach driver

[Thomas Monjalon]

29/06/2017 20:22, Jan Blunck:
> --- a/lib/librte_eal/common/eal_common_dev.c
> +++ b/lib/librte_eal/common/eal_common_dev.c
> +int rte_eal_device_detach(struct rte_device *dev)
[...]
>  int rte_eal_dev_detach(const char *name)

I would be in favor of breaking the API without notice,
and keep the name rte_eal_dev_detach for the function
taking a rte_device as parameter.
This function was introduced in 16.11 and has probably never been used,
except in ethdev and testpmd. Legacy apps probably use rte_eth_dev_detach.

Re: [PATCH v7 14/15] dev: use new hotplug API in attach / detach

[Thomas Monjalon]

29/06/2017 20:22, Jan Blunck:
> Using the new generic API allows attach and detach to be backwards
> compatible while decoupling from the concrete bus implementations.
> 
> Signed-off-by: Jan Blunck <jblunck@infradead.org>
> ---
>  int rte_eal_dev_attach(const char *name, const char *devargs)
>  {
> -	struct rte_pci_addr addr;
> +	int ret;
>  
>  	if (name == NULL || devargs == NULL) {
>  		RTE_LOG(ERR, EAL, "Invalid device or arguments provided\n");
>  		return -EINVAL;
>  	}
>  
> -	if (eal_parse_pci_DomBDF(name, &addr) == 0) {
> -		if (rte_pci_probe_one(&addr) < 0)
> -			goto err;
> +	ret = rte_eal_hotplug_add("PCI", name, devargs);
> +	if (ret && ret != -EINVAL)
> +		return ret;
>  
> -	} else {
> -		if (rte_vdev_init(name, devargs))
> -			goto err;
> -	}
> +	/*
> +	 * If we haven't found a bus device the user meant to "hotplug" a
> +	 * virtual device instead.
> +	 */
> +	ret = rte_vdev_init(name, devargs);
> +	if (ret)
> +		RTE_LOG(ERR, EAL, "Driver cannot attach the device (%s)\n",
> +			name);
> +	return ret;
> +}

I really don't like this function.
It is not really decoupled from the buses (cf "PCI" and rte_vdev_init).
However I think it is acceptable if it is explictly deprecated
and kept only for the legacy rte_eth_dev_attach() function.

Re: [PATCH v7 13/15] eal: add hotplug add/remove functions

[Thomas Monjalon]

29/06/2017 20:22, Jan Blunck:
>  /**
> + * Hotplug add a given device to a specific bus.
> + *
> + * @param busname
> + *   The bus name the device is added to.
> + * @param devname
> + *   The device name. Based on this device name, eal will identify a driver
> + *   capable of handling it and pass it to the driver probing function.
> + * @param devargs
> + *   Device arguments to be passed to the driver.
> + * @return
> + *   0 on success, negative on error.
> + */
> +int rte_eal_hotplug_add(const char *busname, const char *devname,
> +			const char *devargs);

After the hotplug, we may need to get the rte_device.
Should we add a struct **rte_device as parameter,
or should we add a helper function to get the rte_device
from busname and devname?

Re: [PATCH v7 13/15] eal: add hotplug add/remove functions

[Gaëtan Rivet]

On Fri, Jun 30, 2017 at 11:06:03AM +0200, Thomas Monjalon wrote:
> 29/06/2017 20:22, Jan Blunck:
> >  /**
> > + * Hotplug add a given device to a specific bus.
> > + *
> > + * @param busname
> > + *   The bus name the device is added to.
> > + * @param devname
> > + *   The device name. Based on this device name, eal will identify a driver
> > + *   capable of handling it and pass it to the driver probing function.
> > + * @param devargs
> > + *   Device arguments to be passed to the driver.
> > + * @return
> > + *   0 on success, negative on error.
> > + */
> > +int rte_eal_hotplug_add(const char *busname, const char *devname,
> > +			const char *devargs);
> 
> After the hotplug, we may need to get the rte_device.
> Should we add a struct **rte_device as parameter,
> or should we add a helper function to get the rte_device
> from busname and devname?

Also possible: return a struct *rte_device and set rte_errno on error.

-- 
Gaëtan Rivet
6WIND

Re: [PATCH v7 06/15] bus: add helper to find which bus holds a device

[Thomas Monjalon]

29/06/2017 20:21, Jan Blunck:
> +static int
> +bus_find_device(const struct rte_bus *bus, const void *_dev)
> +{
> +	struct rte_device *dev;
> +
> +	dev = bus->find_device(NULL, cmp_rte_device, _dev);
> +	return !dev;
> +}

The preferred code style is to make explicit the NULL comparisons:
	return dev == NULL;

Re: [PATCH v7 07/15] bus: add bus iterator to find a device

[Thomas Monjalon]

29/06/2017 20:21, Jan Blunck:
> +struct rte_device *
> +rte_bus_find_device(const struct rte_device *start, rte_dev_cmp_t cmp,
> +                   const void *data)
> +{
> +       struct rte_bus *bus;
> +       struct rte_device *dev = NULL;
> +
> +       TAILQ_FOREACH(bus, &rte_bus_list, next) {
> +               dev = bus->find_device(start, cmp, data);
> +               if (dev)

Same nit as previous patch,
better to make explicit dev != NULL.

> +                       break;
> +       }
> +       return dev;
> +}

Re: [PATCH v7 13/15] eal: add hotplug add/remove functions

[Bruce Richardson]

On Fri, Jun 30, 2017 at 11:11:42AM +0200, Gaëtan Rivet wrote:
> On Fri, Jun 30, 2017 at 11:06:03AM +0200, Thomas Monjalon wrote:
> > 29/06/2017 20:22, Jan Blunck:
> > >  /**
> > > + * Hotplug add a given device to a specific bus.
> > > + *
> > > + * @param busname
> > > + *   The bus name the device is added to.
> > > + * @param devname
> > > + *   The device name. Based on this device name, eal will identify a driver
> > > + *   capable of handling it and pass it to the driver probing function.
> > > + * @param devargs
> > > + *   Device arguments to be passed to the driver.
> > > + * @return
> > > + *   0 on success, negative on error.
> > > + */
> > > +int rte_eal_hotplug_add(const char *busname, const char *devname,
> > > +			const char *devargs);
> > 
> > After the hotplug, we may need to get the rte_device.
> > Should we add a struct **rte_device as parameter,
> > or should we add a helper function to get the rte_device
> > from busname and devname?
> 
> Also possible: return a struct *rte_device and set rte_errno on error.
> 
+1 for this option.

Re: [PATCH 00/15] bus attach/detach API and hotplug rework

[Thomas Monjalon]

29/06/2017 20:21, Jan Blunck:
> Gaetan Rivet (1):
>   pci: implement hotplug bus operation
> 
> Jan Blunck (14):
>   bus: add bus iterator to find a bus
>   bus: add find_device bus operation
>   vdev: implement find_device bus operation
>   pci: implement find_device bus operation
>   bus/fslmc: implement find_device bus operation
>   bus: add helper to find which bus holds a device
>   bus: add bus iterator to find a device
>   bus: require buses to implement find_device operation
>   bus: add rte_bus_find_by_name
>   bus: introduce device plug/unplug functionality
>   vdev: implement unplug bus operation
>   eal: add hotplug add/remove functions
>   dev: use new hotplug API in attach / detach
>   ethdev: Use embedded rte_device to detach driver

It seems you forgot this patch in the series:
	[PATCH v6 08/11] vdev: use standard bus registration function

Re: [PATCH v6 05/11] bus: introduce hotplug functionality

[Gaëtan Rivet]

On Thu, Jun 29, 2017 at 09:20:30PM +0200, Jan Blunck wrote:
> On Thu, Jun 29, 2017 at 2:59 PM, Gaëtan Rivet <gaetan.rivet@6wind.com> wrote:
> > What can be done to go forward:
> >
> > - Define the API for rte_bus interrupt sources, configuration and
> >   triggers to allow the development of the needed subsystems.
> >
> > - Each device event sources should offer an event parser to transform
> >   them in device descriptions. Depending on the specifics of the source,
> >   different levels of info are available.
> >
> > - Redefine the requirements for bus->scan() to allow hotplugging.
> >
> > - Reduce the scope of plug() from (scan-one + probe) to (probe), as
> >   everything is now in place.
> >
> 
> Also see the series that I send out today. From my point of view we
> are here already.
> 

Yes, your series is only a superficial departure from the v6. In the end,
I see that your critics were pretty much only on interfaces and that you
simply wanted it to be your way.

You did not expose your reason for disagreeing, thus I threw a few ideas
to at least make you either explicit your view or accept the current
proposal.

> > - Further hotplugging developments are then possible: add INTR_ADD
> >   support, with flexible device definition for example... A thing that
> >   is not yet possible with the current architecture but seemed to
> >   interest a few people.
> >
> > If we can agree that this is a way forward, do you think Jan that having
> > the current plug() API hinders this plan? We can reduce its scope later,
> > without changing current implementations as, as you said, a proper
> > scan should be idempotent. The future API as envisionned is already
> > respected, but right now the hotplug support in buses is a little more
> > involved. Applications that will start using plug() right now would not
> > have to be rewritten, as the requirements for plugging would still be
> > fullfilled.
> >
> > We can support already hotplugging in DPDK. We can refine this support
> > later to make it more generic and easier to implement for PMD
> > maintainers. But I do not see this as a reason to block this support
> > from being integrated right now.
> 
> Indeed. I had hotplug support in the version of DPDK that I'm working
> with for the last years. Don't get me wrong: I'm not arguing against
> the inclusion of hotplug code. I just don't understand the reasoning
> behind the implementation you proposed (with my original SOB)
> especially since some of the things that you listed as going forward
> are already there, are easy to fix or don't necessarily need to be
> part of the DPDK EAL.
> 

Don't get me wrong, I am not personally a proponent of these changes,
but I wanted the current implementation to at least leave things open as
I think a few people will push for this in a near future.

But as far as I'm concerned, the hotplug support in DPDK will be
sufficient in v17.08.

> So lets try to get this right from the beginning. What is missing:
> 1. document and verify that existing scan() implementations are idempotent
> 2. example app with udev based hotplug
> 3. check that the symbols are in the correct libraries (bus, pci, vdev)
> 
> What am I missing?
> 

Nothing on the technical side.

-- 
Gaëtan Rivet
6WIND

Re: [PATCH v7 13/15] eal: add hotplug add/remove functions

[Thomas Monjalon]

It seems a function is missing in this patch:

29/06/2017 20:22, Jan Blunck:
> +int rte_eal_hotplug_add(const char *busname, const char *devname,
> +			const char *devargs)
> +{
> +	struct rte_bus *bus;
> +	struct rte_device *dev;
> +	int ret;
> +
> +	bus = rte_bus_find_by_name(busname);
> +	if (!bus) {
> +		RTE_LOG(ERR, EAL, "Cannot find bus (%s)\n", busname);
> +		return -ENOENT;
> +	}
> +
> +	if (!bus->plug) {
> +		RTE_LOG(ERR, EAL, "Function plug not supported by bus (%s)\n",
> +			bus->name);
> +		return -ENOTSUP;
> +	}
> +
> +	ret = bus->scan();
> +	if (ret)
> +		return ret;
> +
> +	dev = bus->find_device(NULL, cmp_detached_dev_name, devname);

fatal error: use of undeclared identifier 'cmp_detached_dev_name'

Re: [PATCH v7 13/15] eal: add hotplug add/remove functions

[Jan Blunck]

On Fri, Jun 30, 2017 at 2:54 PM, Thomas Monjalon <thomas@monjalon.net> wrote:
> It seems a function is missing in this patch:
>
> 29/06/2017 20:22, Jan Blunck:
>> +int rte_eal_hotplug_add(const char *busname, const char *devname,
>> +                     const char *devargs)
>> +{
>> +     struct rte_bus *bus;
>> +     struct rte_device *dev;
>> +     int ret;
>> +
>> +     bus = rte_bus_find_by_name(busname);
>> +     if (!bus) {
>> +             RTE_LOG(ERR, EAL, "Cannot find bus (%s)\n", busname);
>> +             return -ENOENT;
>> +     }
>> +
>> +     if (!bus->plug) {
>> +             RTE_LOG(ERR, EAL, "Function plug not supported by bus (%s)\n",
>> +                     bus->name);
>> +             return -ENOTSUP;
>> +     }
>> +
>> +     ret = bus->scan();
>> +     if (ret)
>> +             return ret;
>> +
>> +     dev = bus->find_device(NULL, cmp_detached_dev_name, devname);
>
> fatal error: use of undeclared identifier 'cmp_detached_dev_name'
>

Sorry. I'll make them bisectable with the next drop today.

Re: [PATCH v7 13/15] eal: add hotplug add/remove functions

[Jan Blunck]

On Fri, Jun 30, 2017 at 11:20 AM, Bruce Richardson
<bruce.richardson@intel.com> wrote:
> On Fri, Jun 30, 2017 at 11:11:42AM +0200, Gaėtan Rivet wrote:
>> On Fri, Jun 30, 2017 at 11:06:03AM +0200, Thomas Monjalon wrote:
>> > 29/06/2017 20:22, Jan Blunck:
>> > >  /**
>> > > + * Hotplug add a given device to a specific bus.
>> > > + *
>> > > + * @param busname
>> > > + *   The bus name the device is added to.
>> > > + * @param devname
>> > > + *   The device name. Based on this device name, eal will identify a driver
>> > > + *   capable of handling it and pass it to the driver probing function.
>> > > + * @param devargs
>> > > + *   Device arguments to be passed to the driver.
>> > > + * @return
>> > > + *   0 on success, negative on error.
>> > > + */
>> > > +int rte_eal_hotplug_add(const char *busname, const char *devname,
>> > > +                 const char *devargs);
>> >
>> > After the hotplug, we may need to get the rte_device.
>> > Should we add a struct **rte_device as parameter,
>> > or should we add a helper function to get the rte_device
>> > from busname and devname?
>>
>> Also possible: return a struct *rte_device and set rte_errno on error.
>>
> +1 for this option.

Given that the caller of this is usually something that injects events
from the system I wonder what it is going to do with a rte_device
reference. Additionally to what the caller knows anyway (name,
numa_node, devargs) it can check if a driver got assigned. Sure the
caller could upcast conditionally based on the busname ...

At this point I guess the control plane would anyway want to get
access to a high-level object, e.g. the rte_ethdev. I believe it is
better to decouple this through callbacks that can get registered with
individual buses.

Re: [PATCH v7 13/15] eal: add hotplug add/remove functions

[Thomas Monjalon]

30/06/2017 17:44, Jan Blunck:
> On Fri, Jun 30, 2017 at 11:20 AM, Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> > On Fri, Jun 30, 2017 at 11:11:42AM +0200, Gaėtan Rivet wrote:
> >> On Fri, Jun 30, 2017 at 11:06:03AM +0200, Thomas Monjalon wrote:
> >> > 29/06/2017 20:22, Jan Blunck:
> >> > >  /**
> >> > > + * Hotplug add a given device to a specific bus.
> >> > > + *
> >> > > + * @param busname
> >> > > + *   The bus name the device is added to.
> >> > > + * @param devname
> >> > > + *   The device name. Based on this device name, eal will identify a driver
> >> > > + *   capable of handling it and pass it to the driver probing function.
> >> > > + * @param devargs
> >> > > + *   Device arguments to be passed to the driver.
> >> > > + * @return
> >> > > + *   0 on success, negative on error.
> >> > > + */
> >> > > +int rte_eal_hotplug_add(const char *busname, const char *devname,
> >> > > +                 const char *devargs);
> >> >
> >> > After the hotplug, we may need to get the rte_device.
> >> > Should we add a struct **rte_device as parameter,
> >> > or should we add a helper function to get the rte_device
> >> > from busname and devname?
> >>
> >> Also possible: return a struct *rte_device and set rte_errno on error.
> >>
> > +1 for this option.
> 
> Given that the caller of this is usually something that injects events
> from the system I wonder what it is going to do with a rte_device
> reference. Additionally to what the caller knows anyway (name,
> numa_node, devargs) it can check if a driver got assigned. Sure the
> caller could upcast conditionally based on the busname ...
> 
> At this point I guess the control plane would anyway want to get
> access to a high-level object, e.g. the rte_ethdev. I believe it is
> better to decouple this through callbacks that can get registered with
> individual buses.

I think Gaetan has an example of use of rte_device after plugging
with the failsafe PMD (managing slaves).
Anyway, it can be discussed later with a real example of use if needed.
We have a couple of weeks before freezing the API.

Re: [PATCH v7 13/15] eal: add hotplug add/remove functions

[Gaëtan Rivet]

On Fri, Jun 30, 2017 at 06:03:17PM +0200, Thomas Monjalon wrote:
> 30/06/2017 17:44, Jan Blunck:
> > On Fri, Jun 30, 2017 at 11:20 AM, Bruce Richardson
> > <bruce.richardson@intel.com> wrote:
> > > On Fri, Jun 30, 2017 at 11:11:42AM +0200, Gaėtan Rivet wrote:
> > >> On Fri, Jun 30, 2017 at 11:06:03AM +0200, Thomas Monjalon wrote:
> > >> > 29/06/2017 20:22, Jan Blunck:
> > >> > >  /**
> > >> > > + * Hotplug add a given device to a specific bus.
> > >> > > + *
> > >> > > + * @param busname
> > >> > > + *   The bus name the device is added to.
> > >> > > + * @param devname
> > >> > > + *   The device name. Based on this device name, eal will identify a driver
> > >> > > + *   capable of handling it and pass it to the driver probing function.
> > >> > > + * @param devargs
> > >> > > + *   Device arguments to be passed to the driver.
> > >> > > + * @return
> > >> > > + *   0 on success, negative on error.
> > >> > > + */
> > >> > > +int rte_eal_hotplug_add(const char *busname, const char *devname,
> > >> > > +                 const char *devargs);
> > >> >
> > >> > After the hotplug, we may need to get the rte_device.
> > >> > Should we add a struct **rte_device as parameter,
> > >> > or should we add a helper function to get the rte_device
> > >> > from busname and devname?
> > >>
> > >> Also possible: return a struct *rte_device and set rte_errno on error.
> > >>
> > > +1 for this option.
> > 
> > Given that the caller of this is usually something that injects events
> > from the system I wonder what it is going to do with a rte_device
> > reference. Additionally to what the caller knows anyway (name,
> > numa_node, devargs) it can check if a driver got assigned. Sure the
> > caller could upcast conditionally based on the busname ...
> > 
> > At this point I guess the control plane would anyway want to get
> > access to a high-level object, e.g. the rte_ethdev. I believe it is
> > better to decouple this through callbacks that can get registered with
> > individual buses.
> 
> I think Gaetan has an example of use of rte_device after plugging
> with the failsafe PMD (managing slaves).
> Anyway, it can be discussed later with a real example of use if needed.
> We have a couple of weeks before freezing the API.

The rte_device should be accessible from the rte_eth_dev anyway so it
does not make much difference. As long as a handle on the device is
available. It is of course possible to add yet another callback to
search the device just plugged, but I don't see a reason here not to do
it in one pass.

-- 
Gaëtan Rivet
6WIND

Re: [PATCH v7 13/15] eal: add hotplug add/remove functions

[Bruce Richardson]

On Fri, Jun 30, 2017 at 06:13:51PM +0200, Gaëtan Rivet wrote:
> On Fri, Jun 30, 2017 at 06:03:17PM +0200, Thomas Monjalon wrote:
> > 30/06/2017 17:44, Jan Blunck:
> > > On Fri, Jun 30, 2017 at 11:20 AM, Bruce Richardson
> > > <bruce.richardson@intel.com> wrote:
> > > > On Fri, Jun 30, 2017 at 11:11:42AM +0200, Gaėtan Rivet wrote:
> > > >> On Fri, Jun 30, 2017 at 11:06:03AM +0200, Thomas Monjalon wrote:
> > > >> > 29/06/2017 20:22, Jan Blunck:
> > > >> > >  /**
> > > >> > > + * Hotplug add a given device to a specific bus.
> > > >> > > + *
> > > >> > > + * @param busname
> > > >> > > + *   The bus name the device is added to.
> > > >> > > + * @param devname
> > > >> > > + *   The device name. Based on this device name, eal will identify a driver
> > > >> > > + *   capable of handling it and pass it to the driver probing function.
> > > >> > > + * @param devargs
> > > >> > > + *   Device arguments to be passed to the driver.
> > > >> > > + * @return
> > > >> > > + *   0 on success, negative on error.
> > > >> > > + */
> > > >> > > +int rte_eal_hotplug_add(const char *busname, const char *devname,
> > > >> > > +                 const char *devargs);
> > > >> >
> > > >> > After the hotplug, we may need to get the rte_device.
> > > >> > Should we add a struct **rte_device as parameter,
> > > >> > or should we add a helper function to get the rte_device
> > > >> > from busname and devname?
> > > >>
> > > >> Also possible: return a struct *rte_device and set rte_errno on error.
> > > >>
> > > > +1 for this option.
> > > 
> > > Given that the caller of this is usually something that injects events
> > > from the system I wonder what it is going to do with a rte_device
> > > reference. Additionally to what the caller knows anyway (name,
> > > numa_node, devargs) it can check if a driver got assigned. Sure the
> > > caller could upcast conditionally based on the busname ...
> > > 
> > > At this point I guess the control plane would anyway want to get
> > > access to a high-level object, e.g. the rte_ethdev. I believe it is
> > > better to decouple this through callbacks that can get registered with
> > > individual buses.
> > 
> > I think Gaetan has an example of use of rte_device after plugging
> > with the failsafe PMD (managing slaves).
> > Anyway, it can be discussed later with a real example of use if needed.
> > We have a couple of weeks before freezing the API.
> 
> The rte_device should be accessible from the rte_eth_dev anyway so it
> does not make much difference. As long as a handle on the device is
> available. It is of course possible to add yet another callback to
> search the device just plugged, but I don't see a reason here not to do
> it in one pass.
> 
At this point in the process we just need to get in what we can.

Given there is so much discussion I would suggest we apply what we have
now, but mark the new APIs as "experimental" at this point. That should
allow us to test them, and build upon them without holding us back if we
do need to change them in the next release. Once everyone is happy with
the final result, we can lock them down then. It seems premature
to do so now, with the current discussion, but on the other hand
it seems foolish to not put what work has been done thus far
into a release as a starting point.

my 2c.
/Bruce

Re: [PATCH v7 13/15] eal: add hotplug add/remove functions

[Thomas Monjalon]

30/06/2017 18:25, Bruce Richardson:
> On Fri, Jun 30, 2017 at 06:13:51PM +0200, Gaëtan Rivet wrote:
> > On Fri, Jun 30, 2017 at 06:03:17PM +0200, Thomas Monjalon wrote:
> > > 30/06/2017 17:44, Jan Blunck:
> > > > On Fri, Jun 30, 2017 at 11:20 AM, Bruce Richardson
> > > > <bruce.richardson@intel.com> wrote:
> > > > > On Fri, Jun 30, 2017 at 11:11:42AM +0200, Gaėtan Rivet wrote:
> > > > >> On Fri, Jun 30, 2017 at 11:06:03AM +0200, Thomas Monjalon wrote:
> > > > >> > 29/06/2017 20:22, Jan Blunck:
> > > > >> > >  /**
> > > > >> > > + * Hotplug add a given device to a specific bus.
> > > > >> > > + *
> > > > >> > > + * @param busname
> > > > >> > > + *   The bus name the device is added to.
> > > > >> > > + * @param devname
> > > > >> > > + *   The device name. Based on this device name, eal will identify a driver
> > > > >> > > + *   capable of handling it and pass it to the driver probing function.
> > > > >> > > + * @param devargs
> > > > >> > > + *   Device arguments to be passed to the driver.
> > > > >> > > + * @return
> > > > >> > > + *   0 on success, negative on error.
> > > > >> > > + */
> > > > >> > > +int rte_eal_hotplug_add(const char *busname, const char *devname,
> > > > >> > > +                 const char *devargs);
> > > > >> >
> > > > >> > After the hotplug, we may need to get the rte_device.
> > > > >> > Should we add a struct **rte_device as parameter,
> > > > >> > or should we add a helper function to get the rte_device
> > > > >> > from busname and devname?
> > > > >>
> > > > >> Also possible: return a struct *rte_device and set rte_errno on error.
> > > > >>
> > > > > +1 for this option.
> > > > 
> > > > Given that the caller of this is usually something that injects events
> > > > from the system I wonder what it is going to do with a rte_device
> > > > reference. Additionally to what the caller knows anyway (name,
> > > > numa_node, devargs) it can check if a driver got assigned. Sure the
> > > > caller could upcast conditionally based on the busname ...
> > > > 
> > > > At this point I guess the control plane would anyway want to get
> > > > access to a high-level object, e.g. the rte_ethdev. I believe it is
> > > > better to decouple this through callbacks that can get registered with
> > > > individual buses.
> > > 
> > > I think Gaetan has an example of use of rte_device after plugging
> > > with the failsafe PMD (managing slaves).
> > > Anyway, it can be discussed later with a real example of use if needed.
> > > We have a couple of weeks before freezing the API.
> > 
> > The rte_device should be accessible from the rte_eth_dev anyway so it
> > does not make much difference. As long as a handle on the device is
> > available. It is of course possible to add yet another callback to
> > search the device just plugged, but I don't see a reason here not to do
> > it in one pass.
> > 
> At this point in the process we just need to get in what we can.
> 
> Given there is so much discussion I would suggest we apply what we have
> now, but mark the new APIs as "experimental" at this point. That should
> allow us to test them, and build upon them without holding us back if we
> do need to change them in the next release. Once everyone is happy with
> the final result, we can lock them down then. It seems premature
> to do so now, with the current discussion, but on the other hand
> it seems foolish to not put what work has been done thus far
> into a release as a starting point.

I agree.
Please Jan, add EXPERIMENTAL in the doxygen of the new hotplug API
for your next version. Thanks

Re: [PATCH v7 06/15] bus: add helper to find which bus holds a device

[Jan Blunck]

On Fri, Jun 30, 2017 at 11:16 AM, Thomas Monjalon <thomas@monjalon.net> wrote:
> 29/06/2017 20:21, Jan Blunck:
>> +static int
>> +bus_find_device(const struct rte_bus *bus, const void *_dev)
>> +{
>> +     struct rte_device *dev;
>> +
>> +     dev = bus->find_device(NULL, cmp_rte_device, _dev);
>> +     return !dev;
>> +}
>
> The preferred code style is to make explicit the NULL comparisons:
>         return dev == NULL;

Oh, interesting ... not a lot of C++ programmers around here I guess.

Does this mean you also want me to make integer tests explicit again 0?

[PATCH v8 00/14] bus attach/detach API and hotplug rework

[Jan Blunck]

-- v7

* Revert removal of RTE_VERIFY on find_device operation since a bus that isn't
  able to even iterate its devices is pointless

* Revert plug() changes that have been introduced in v3:
  int (*rte_bus_plug_t)(struct rte_device *dev, const char *devargs);

* Revert pointless reordering of start argument for bus iterator functions

* Add fslmc bus find_device operation

* Add support to find a bus by its name

* Add EAL hotplug API:
  int rte_eal_hotplug_add(const char *busname, const char *devname,
                          const char *devargs);
  int rte_eal_hotplug_remove(const char *busname, const char *devname);

* Keep rte_eal_dev_attach() backwards compatible and support hotplug of PCI
  and virtual devices only

* Add back patch "ethdev: use embedded rte_device to detach driver" which is
  critical for correctly detaching the correct low-level device

-- v8

* Rework rte_eal_dev_detach to break API and remove rte_eal_device_detach

* Drop "bus: add bus iterator to find a device" since its not actively used

* Adjust coding style to explicitly check against NULL pointer

* Mark hotplug API EXPERIMENTAL

* Fix exporting of new bus function symbols


Gaetan Rivet (1):
  pci: implement hotplug bus operation

Jan Blunck (13):
  bus: add bus iterator to find a bus
  bus: add find_device bus operation
  vdev: implement find_device bus operation
  pci: implement find_device bus operation
  bus/fslmc: implement find_device bus operation
  bus: add helper to find which bus holds a device
  bus: require buses to implement find_device operation
  bus: add rte_bus_find_by_name
  bus: introduce device plug/unplug functionality
  vdev: implement unplug bus operation
  eal: add hotplug add/remove functions
  ethdev: Use embedded rte_device to detach driver
  dev: use new hotplug API in attach

 app/test-pmd/testpmd.c                          |   2 +-
 drivers/bus/fslmc/fslmc_bus.c                   |  23 ++++
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  17 +++
 lib/librte_eal/common/eal_common_bus.c          |  59 ++++++++++
 lib/librte_eal/common/eal_common_dev.c          | 143 +++++++++++++++++++-----
 lib/librte_eal/common/eal_common_pci.c          |  55 +++++++++
 lib/librte_eal/common/eal_common_vdev.c         |  35 ++++++
 lib/librte_eal/common/include/rte_bus.h         | 111 ++++++++++++++++++
 lib/librte_eal/common/include/rte_dev.h         |  62 +++++++++-
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |  17 +++
 lib/librte_ether/rte_ethdev.c                   |   3 +-
 11 files changed, 496 insertions(+), 31 deletions(-)

--
2.9.4

[PATCH v8 01/14] bus: add bus iterator to find a bus

[Jan Blunck]

This helper allows to iterate over all registered buses and find one
matching data used as parameter.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  7 +++++
 lib/librte_eal/common/eal_common_bus.c          | 19 +++++++++++
 lib/librte_eal/common/include/rte_bus.h         | 42 +++++++++++++++++++++++++
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |  7 +++++
 4 files changed, 75 insertions(+)

diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
index 2e48a73..edf9d7d 100644
--- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
@@ -193,3 +193,10 @@ DPDK_17.05 {
 	vfio_get_group_no;
 
 } DPDK_17.02;
+
+DPDK_17.08 {
+	global:
+
+	rte_bus_find;
+
+} DPDK_17.05;
diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
index 8f9baf8..3094daa 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -145,3 +145,22 @@ rte_bus_dump(FILE *f)
 		}
 	}
 }
+
+struct rte_bus *
+rte_bus_find(const struct rte_bus *start, rte_bus_cmp_t cmp,
+	     const void *data)
+{
+	struct rte_bus *bus = NULL;
+	bool start_found = !start;
+
+	TAILQ_FOREACH(bus, &rte_bus_list, next) {
+		if (!start_found) {
+			if (bus == start)
+				start_found = 1;
+			continue;
+		}
+		if (cmp(bus, data) == 0)
+			break;
+	}
+	return bus;
+}
diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
index 5f47b82..c52b65b 100644
--- a/lib/librte_eal/common/include/rte_bus.h
+++ b/lib/librte_eal/common/include/rte_bus.h
@@ -141,6 +141,48 @@ int rte_bus_probe(void);
 void rte_bus_dump(FILE *f);
 
 /**
+ * Bus comparison function.
+ *
+ * @param bus
+ *	Bus under test.
+ *
+ * @param data
+ *	Data to compare against.
+ *
+ * @return
+ *	0 if the bus matches the data.
+ *	!0 if the bus does not match.
+ *	<0 if ordering is possible and the bus is lower than the data.
+ *	>0 if ordering is possible and the bus is greater than the data.
+ */
+typedef int (*rte_bus_cmp_t)(const struct rte_bus *bus, const void *data);
+
+/**
+ * Bus iterator to find a particular bus.
+ *
+ * This function compares each registered bus to find one that matches
+ * the data passed as parameter.
+ *
+ * If the comparison function returns zero this function will stop iterating
+ * over any more buses. To continue a search the bus of a previous search can
+ * be passed via the start parameter.
+ *
+ * @param start
+ *	Starting point for the iteration.
+ *
+ * @param cmp
+ *	Comparison function.
+ *
+ * @param data
+ *	 Data to pass to comparison function.
+ *
+ * @return
+ *	 A pointer to a rte_bus structure or NULL in case no bus matches
+ */
+struct rte_bus *rte_bus_find(const struct rte_bus *start, rte_bus_cmp_t cmp,
+			     const void *data);
+
+/**
  * Helper for Bus registration.
  * The constructor has higher priority than PMD constructors.
  */
diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
index 670bab3..944416e 100644
--- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
@@ -198,3 +198,10 @@ DPDK_17.05 {
 	vfio_get_group_no;
 
 } DPDK_17.02;
+
+DPDK_17.08 {
+	global:
+
+	rte_bus_find;
+
+} DPDK_17.05;
-- 
2.9.4

[PATCH v8 02/14] bus: add find_device bus operation

[Jan Blunck]

This new method allows buses to expose their devices in a controlled
manner. A comparison function is provided by the user to discriminate
between devices, using arbitrary data as identifier.

It is possible to start an iteration from a specific point, in order to
continue a search.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/include/rte_bus.h | 27 +++++++++++++++++++++++++++
 lib/librte_eal/common/include/rte_dev.h | 21 +++++++++++++++++++++
 2 files changed, 48 insertions(+)

diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
index c52b65b..052ac8d 100644
--- a/lib/librte_eal/common/include/rte_bus.h
+++ b/lib/librte_eal/common/include/rte_bus.h
@@ -82,6 +82,32 @@ typedef int (*rte_bus_scan_t)(void);
 typedef int (*rte_bus_probe_t)(void);
 
 /**
+ * Device iterator to find a device on a bus.
+ *
+ * This function returns an rte_device if one of those held by the bus
+ * matches the data passed as parameter.
+ *
+ * If the comparison function returns zero this function should stop iterating
+ * over any more devices. To continue a search the device of a previous search
+ * can be passed via the start parameter.
+ *
+ * @param cmp
+ *	Comparison function.
+ *
+ * @param data
+ *	Data to compare each device against.
+ *
+ * @param start
+ *	starting point for the iteration
+ *
+ * @return
+ *	The first device matching the data, NULL if none exists.
+ */
+typedef struct rte_device *
+(*rte_bus_find_device_t)(const struct rte_device *start, rte_dev_cmp_t cmp,
+			 const void *data);
+
+/**
  * A structure describing a generic bus.
  */
 struct rte_bus {
@@ -89,6 +115,7 @@ struct rte_bus {
 	const char *name;            /**< Name of the bus */
 	rte_bus_scan_t scan;         /**< Scan for devices attached to bus */
 	rte_bus_probe_t probe;       /**< Probe devices on bus */
+	rte_bus_find_device_t find_device; /**< Find a device on the bus */
 };
 
 /**
diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
index de20c06..04d9c28 100644
--- a/lib/librte_eal/common/include/rte_dev.h
+++ b/lib/librte_eal/common/include/rte_dev.h
@@ -191,6 +191,27 @@ int rte_eal_dev_attach(const char *name, const char *devargs);
  */
 int rte_eal_dev_detach(const char *name);
 
+/**
+ * Device comparison function.
+ *
+ * This type of function is used to compare an rte_device with arbitrary
+ * data.
+ *
+ * @param dev
+ *   Device handle.
+ *
+ * @param data
+ *   Data to compare against. The type of this parameter is determined by
+ *   the kind of comparison performed by the function.
+ *
+ * @return
+ *   0 if the device matches the data.
+ *   !0 if the device does not match.
+ *   <0 if ordering is possible and the device is lower than the data.
+ *   >0 if ordering is possible and the device is greater than the data.
+ */
+typedef int (*rte_dev_cmp_t)(const struct rte_device *dev, const void *data);
+
 #define RTE_PMD_EXPORT_NAME_ARRAY(n, idx) n##idx[]
 
 #define RTE_PMD_EXPORT_NAME(name, idx) \
-- 
2.9.4

[PATCH v8 03/14] vdev: implement find_device bus operation

[Jan Blunck]

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/eal_common_vdev.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_vdev.c b/lib/librte_eal/common/eal_common_vdev.c
index b4db2be..9bb7427 100644
--- a/lib/librte_eal/common/eal_common_vdev.c
+++ b/lib/librte_eal/common/eal_common_vdev.c
@@ -35,6 +35,7 @@
 #include <stdio.h>
 #include <stdlib.h>
 #include <stdint.h>
+#include <stdbool.h>
 #include <sys/queue.h>
 
 #include <rte_eal.h>
@@ -334,9 +335,29 @@ vdev_probe(void)
 	return 0;
 }
 
+static struct rte_device *
+vdev_find_device(const struct rte_device *start, rte_dev_cmp_t cmp,
+		 const void *data)
+{
+	struct rte_vdev_device *dev;
+	bool start_found = !start;
+
+	TAILQ_FOREACH(dev, &vdev_device_list, next) {
+		if (start_found == 0) {
+			if (&dev->device == start)
+				start_found = 1;
+			continue;
+		}
+		if (cmp(&dev->device, data) == 0)
+			return &dev->device;
+	}
+	return NULL;
+}
+
 static struct rte_bus rte_vdev_bus = {
 	.scan = vdev_scan,
 	.probe = vdev_probe,
+	.find_device = vdev_find_device,
 };
 
 RTE_REGISTER_BUS(VIRTUAL_BUS_NAME, rte_vdev_bus);
-- 
2.9.4

[PATCH v8 04/14] pci: implement find_device bus operation

[Jan Blunck]

Signed-off-by: Jan Blunck <jblunck@infradead.org>
---
 lib/librte_eal/common/eal_common_pci.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c
index 479c62d..e449758 100644
--- a/lib/librte_eal/common/eal_common_pci.c
+++ b/lib/librte_eal/common/eal_common_pci.c
@@ -485,10 +485,31 @@ rte_pci_remove_device(struct rte_pci_device *pci_dev)
 	TAILQ_REMOVE(&rte_pci_bus.device_list, pci_dev, next);
 }
 
+static struct rte_device *
+pci_find_device(const struct rte_device *start, rte_dev_cmp_t cmp,
+		const void *data)
+{
+	struct rte_pci_device *dev;
+	bool start_found = !start;
+
+	FOREACH_DEVICE_ON_PCIBUS(dev) {
+		if (!start_found) {
+			if (&dev->device == start)
+				start_found = 1;
+			continue;
+		}
+		if (cmp(&dev->device, data) == 0)
+			return &dev->device;
+	}
+
+	return NULL;
+}
+
 struct rte_pci_bus rte_pci_bus = {
 	.bus = {
 		.scan = rte_pci_scan,
 		.probe = rte_pci_probe,
+		.find_device = pci_find_device,
 	},
 	.device_list = TAILQ_HEAD_INITIALIZER(rte_pci_bus.device_list),
 	.driver_list = TAILQ_HEAD_INITIALIZER(rte_pci_bus.driver_list),
-- 
2.9.4

[PATCH v8 05/14] bus/fslmc: implement find_device bus operation

[Jan Blunck]

Signed-off-by: Jan Blunck <jblunck@infradead.org>
---
 drivers/bus/fslmc/fslmc_bus.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/drivers/bus/fslmc/fslmc_bus.c b/drivers/bus/fslmc/fslmc_bus.c
index b24642d..1e3bbee 100644
--- a/drivers/bus/fslmc/fslmc_bus.c
+++ b/drivers/bus/fslmc/fslmc_bus.c
@@ -32,6 +32,7 @@
 
 #include <string.h>
 #include <dirent.h>
+#include <stdbool.h>
 
 #include <rte_log.h>
 #include <rte_bus.h>
@@ -105,6 +106,27 @@ rte_fslmc_probe(void)
 	return ret;
 }
 
+static struct rte_device *
+rte_fslmc_find_device(const struct rte_device *start, rte_dev_cmp_t cmp,
+		      const void *data)
+{
+	struct rte_dpaa2_device *dev;
+	bool start_found = !start;
+
+	TAILQ_FOREACH(dev, &rte_fslmc_bus.device_list, next) {
+		if (!start_found) {
+			if (&dev->device == start)
+				start_found = 1;
+			continue;
+		}
+
+		if (cmp(&dev->device, data) == 0)
+			return &dev->device;
+	}
+
+	return NULL;
+}
+
 /*register a fslmc bus based dpaa2 driver */
 void
 rte_fslmc_driver_register(struct rte_dpaa2_driver *driver)
@@ -133,6 +155,7 @@ struct rte_fslmc_bus rte_fslmc_bus = {
 	.bus = {
 		.scan = rte_fslmc_scan,
 		.probe = rte_fslmc_probe,
+		.find_device = rte_fslmc_find_device,
 	},
 	.device_list = TAILQ_HEAD_INITIALIZER(rte_fslmc_bus.device_list),
 	.driver_list = TAILQ_HEAD_INITIALIZER(rte_fslmc_bus.driver_list),
-- 
2.9.4

[PATCH v8 06/14] bus: add helper to find which bus holds a device

[Jan Blunck]

Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  1 +
 lib/librte_eal/common/eal_common_bus.c          | 23 +++++++++++++++++++++++
 lib/librte_eal/common/include/rte_bus.h         |  5 +++++
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |  1 +
 4 files changed, 30 insertions(+)

diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
index edf9d7d..458e3a6 100644
--- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
@@ -198,5 +198,6 @@ DPDK_17.08 {
 	global:
 
 	rte_bus_find;
+	rte_bus_find_by_device;
 
 } DPDK_17.05;
diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
index 3094daa..5d06f5d 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -164,3 +164,26 @@ rte_bus_find(const struct rte_bus *start, rte_bus_cmp_t cmp,
 	}
 	return bus;
 }
+
+static int
+cmp_rte_device(const struct rte_device *dev1, const void *_dev2)
+{
+	const struct rte_device *dev2 = _dev2;
+
+	return dev1 != dev2;
+}
+
+static int
+bus_find_device(const struct rte_bus *bus, const void *_dev)
+{
+	struct rte_device *dev;
+
+	dev = bus->find_device(NULL, cmp_rte_device, _dev);
+	return dev == NULL;
+}
+
+struct rte_bus *
+rte_bus_find_by_device(const struct rte_device *dev)
+{
+	return rte_bus_find(NULL, bus_find_device, (const void *)dev);
+}
diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
index 052ac8d..f8b3215 100644
--- a/lib/librte_eal/common/include/rte_bus.h
+++ b/lib/librte_eal/common/include/rte_bus.h
@@ -210,6 +210,11 @@ struct rte_bus *rte_bus_find(const struct rte_bus *start, rte_bus_cmp_t cmp,
 			     const void *data);
 
 /**
+ * Find the registered bus for a particular device.
+ */
+struct rte_bus *rte_bus_find_by_device(const struct rte_device *dev);
+
+/**
  * Helper for Bus registration.
  * The constructor has higher priority than PMD constructors.
  */
diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
index 944416e..3c193e4 100644
--- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
@@ -203,5 +203,6 @@ DPDK_17.08 {
 	global:
 
 	rte_bus_find;
+	rte_bus_find_by_device;
 
 } DPDK_17.05;
-- 
2.9.4

[PATCH v8 07/14] bus: require buses to implement find_device operation

[Jan Blunck]

Signed-off-by: Jan Blunck <jblunck@infradead.org>
---
 lib/librte_eal/common/eal_common_bus.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
index 5d06f5d..b8a9e30 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -50,6 +50,7 @@ rte_bus_register(struct rte_bus *bus)
 	/* A bus should mandatorily have the scan implemented */
 	RTE_VERIFY(bus->scan);
 	RTE_VERIFY(bus->probe);
+	RTE_VERIFY(bus->find_device);
 
 	TAILQ_INSERT_TAIL(&rte_bus_list, bus, next);
 	RTE_LOG(DEBUG, EAL, "Registered [%s] bus.\n", bus->name);
-- 
2.9.4

[PATCH v8 08/14] bus: add rte_bus_find_by_name

[Jan Blunck]

Signed-off-by: Jan Blunck <jblunck@infradead.org>
---
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  1 +
 lib/librte_eal/common/eal_common_bus.c          | 14 ++++++++++++++
 lib/librte_eal/common/include/rte_bus.h         |  5 +++++
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |  1 +
 4 files changed, 21 insertions(+)

diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
index 458e3a6..d138a96 100644
--- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
@@ -199,5 +199,6 @@ DPDK_17.08 {
 
 	rte_bus_find;
 	rte_bus_find_by_device;
+	rte_bus_find_by_name;
 
 } DPDK_17.05;
diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
index b8a9e30..bf2b138 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -188,3 +188,17 @@ rte_bus_find_by_device(const struct rte_device *dev)
 {
 	return rte_bus_find(NULL, bus_find_device, (const void *)dev);
 }
+
+static int
+cmp_bus_name(const struct rte_bus *bus, const void *_name)
+{
+	const char *name = _name;
+
+	return strcmp(bus->name, name);
+}
+
+struct rte_bus *
+rte_bus_find_by_name(const char *busname)
+{
+	return rte_bus_find(NULL, cmp_bus_name, (const void *)busname);
+}
diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
index f8b3215..509292d 100644
--- a/lib/librte_eal/common/include/rte_bus.h
+++ b/lib/librte_eal/common/include/rte_bus.h
@@ -215,6 +215,11 @@ struct rte_bus *rte_bus_find(const struct rte_bus *start, rte_bus_cmp_t cmp,
 struct rte_bus *rte_bus_find_by_device(const struct rte_device *dev);
 
 /**
+ * Find the registered bus for a given name.
+ */
+struct rte_bus *rte_bus_find_by_name(const char *busname);
+
+/**
  * Helper for Bus registration.
  * The constructor has higher priority than PMD constructors.
  */
diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
index 3c193e4..643ec1c 100644
--- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
@@ -204,5 +204,6 @@ DPDK_17.08 {
 
 	rte_bus_find;
 	rte_bus_find_by_device;
+	rte_bus_find_by_name;
 
 } DPDK_17.05;
-- 
2.9.4

[PATCH v8 09/14] bus: introduce device plug/unplug functionality

[Jan Blunck]

This allows the buses to plug and probe specific devices. This is meant to
be a building block for hotplug support.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
---
 lib/librte_eal/common/eal_common_bus.c  |  2 ++
 lib/librte_eal/common/include/rte_bus.h | 32 ++++++++++++++++++++++++++++++++
 2 files changed, 34 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
index bf2b138..87b0c6e 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -51,6 +51,8 @@ rte_bus_register(struct rte_bus *bus)
 	RTE_VERIFY(bus->scan);
 	RTE_VERIFY(bus->probe);
 	RTE_VERIFY(bus->find_device);
+	/* Buses supporting driver plug also require unplug. */
+	RTE_VERIFY(!bus->plug || bus->unplug);
 
 	TAILQ_INSERT_TAIL(&rte_bus_list, bus, next);
 	RTE_LOG(DEBUG, EAL, "Registered [%s] bus.\n", bus->name);
diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
index 509292d..34ea9d5 100644
--- a/lib/librte_eal/common/include/rte_bus.h
+++ b/lib/librte_eal/common/include/rte_bus.h
@@ -108,6 +108,36 @@ typedef struct rte_device *
 			 const void *data);
 
 /**
+ * Implementation specific probe function which is responsible for linking
+ * devices on that bus with applicable drivers.
+ *
+ * @param dev
+ *	Device pointer that was returned by a previous call to find_device.
+ *
+ * @param devargs
+ *	Device declaration.
+ *
+ * @return
+ *	The pointer to a valid rte_device usable by the bus on success.
+ *	NULL on error. rte_errno is then set.
+ */
+typedef int (*rte_bus_plug_t)(struct rte_device *dev,
+			      const char *devargs);
+
+/**
+ * Implementation specific remove function which is responsible for unlinking
+ * devices on that bus from assigned driver.
+ *
+ * @param dev
+ *	Device pointer that was returned by a previous call to find_device.
+ *
+ * @return
+ *	0 on success.
+ *	!0 on error. rte_errno is then set.
+ */
+typedef int (*rte_bus_unplug_t)(struct rte_device *dev);
+
+/**
  * A structure describing a generic bus.
  */
 struct rte_bus {
@@ -116,6 +146,8 @@ struct rte_bus {
 	rte_bus_scan_t scan;         /**< Scan for devices attached to bus */
 	rte_bus_probe_t probe;       /**< Probe devices on bus */
 	rte_bus_find_device_t find_device; /**< Find a device on the bus */
+	rte_bus_plug_t plug;         /**< Probe single device for drivers */
+	rte_bus_unplug_t unplug;     /**< Remove single device from driver */
 };
 
 /**
-- 
2.9.4

[PATCH v8 10/14] vdev: implement unplug bus operation

[Jan Blunck]

Signed-off-by: Jan Blunck <jblunck@infradead.org>
---
 lib/librte_eal/common/eal_common_vdev.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_vdev.c b/lib/librte_eal/common/eal_common_vdev.c
index 9bb7427..baf3c5b 100644
--- a/lib/librte_eal/common/eal_common_vdev.c
+++ b/lib/librte_eal/common/eal_common_vdev.c
@@ -39,11 +39,13 @@
 #include <sys/queue.h>
 
 #include <rte_eal.h>
+#include <rte_dev.h>
 #include <rte_bus.h>
 #include <rte_vdev.h>
 #include <rte_common.h>
 #include <rte_devargs.h>
 #include <rte_memory.h>
+#include <rte_errno.h>
 
 /** Double linked list of virtual device drivers. */
 TAILQ_HEAD(vdev_device_list, rte_vdev_device);
@@ -354,10 +356,22 @@ vdev_find_device(const struct rte_device *start, rte_dev_cmp_t cmp,
 	return NULL;
 }
 
+static int
+vdev_unplug(struct rte_device *dev)
+{
+	/*
+	 * The virtual bus doesn't support 'unattached' devices so this is
+	 * actually equal to hotplugging removal of it.
+	 */
+	return rte_vdev_uninit(dev->name);
+}
+
 static struct rte_bus rte_vdev_bus = {
 	.scan = vdev_scan,
 	.probe = vdev_probe,
 	.find_device = vdev_find_device,
+	/* .plug = NULL, see comment on vdev_unplug */
+	.unplug = vdev_unplug,
 };
 
 RTE_REGISTER_BUS(VIRTUAL_BUS_NAME, rte_vdev_bus);
-- 
2.9.4

[PATCH v8 11/14] pci: implement hotplug bus operation

[Jan Blunck]

From: Gaetan Rivet <gaetan.rivet@6wind.com>

Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---
 lib/librte_eal/common/eal_common_pci.c | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c
index e449758..5ee100e 100644
--- a/lib/librte_eal/common/eal_common_pci.c
+++ b/lib/librte_eal/common/eal_common_pci.c
@@ -47,6 +47,7 @@
 #include <rte_pci.h>
 #include <rte_per_lcore.h>
 #include <rte_memory.h>
+#include <rte_memcpy.h>
 #include <rte_memzone.h>
 #include <rte_eal.h>
 #include <rte_string_fns.h>
@@ -505,11 +506,44 @@ pci_find_device(const struct rte_device *start, rte_dev_cmp_t cmp,
 	return NULL;
 }
 
+static int
+pci_plug(struct rte_device *dev, const char *devargs __rte_unused)
+{
+	struct rte_pci_device *pdev;
+	struct rte_pci_addr *addr;
+
+	addr = &RTE_DEV_TO_PCI(dev)->addr;
+
+	/* Find the current device holding this address in the bus. */
+	FOREACH_DEVICE_ON_PCIBUS(pdev) {
+		if (rte_eal_compare_pci_addr(&pdev->addr, addr) == 0)
+			return rte_pci_probe_one(addr);
+	}
+
+	rte_errno = ENODEV;
+	return -1;
+}
+
+static int
+pci_unplug(struct rte_device *dev)
+{
+	struct rte_pci_device *pdev;
+
+	pdev = RTE_DEV_TO_PCI(dev);
+	if (rte_pci_detach(&pdev->addr) != 0) {
+		rte_errno = ENODEV;
+		return -1;
+	}
+	return 0;
+}
+
 struct rte_pci_bus rte_pci_bus = {
 	.bus = {
 		.scan = rte_pci_scan,
 		.probe = rte_pci_probe,
 		.find_device = pci_find_device,
+		.plug = pci_plug,
+		.unplug = pci_unplug,
 	},
 	.device_list = TAILQ_HEAD_INITIALIZER(rte_pci_bus.device_list),
 	.driver_list = TAILQ_HEAD_INITIALIZER(rte_pci_bus.driver_list),
-- 
2.9.4

[PATCH v8 12/14] eal: add hotplug add/remove functions

[Jan Blunck]

Signed-off-by: Jan Blunck <jblunck@infradead.org>
---
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  8 +++
 lib/librte_eal/common/eal_common_dev.c          | 88 +++++++++++++++++++++++++
 lib/librte_eal/common/include/rte_dev.h         | 34 ++++++++++
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |  8 +++
 4 files changed, 138 insertions(+)

diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
index d138a96..0295ea9 100644
--- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
@@ -202,3 +202,11 @@ DPDK_17.08 {
 	rte_bus_find_by_name;
 
 } DPDK_17.05;
+
+EXPERIMENTAL {
+	global:
+
+	rte_eal_hotplug_add;
+	rte_eal_hotplug_remove;
+
+} DPDK_17.08;
diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c
index a400ddd..3606332 100644
--- a/lib/librte_eal/common/eal_common_dev.c
+++ b/lib/librte_eal/common/eal_common_dev.c
@@ -37,6 +37,7 @@
 #include <inttypes.h>
 #include <sys/queue.h>
 
+#include <rte_bus.h>
 #include <rte_dev.h>
 #include <rte_devargs.h>
 #include <rte_debug.h>
@@ -45,6 +46,25 @@
 
 #include "eal_private.h"
 
+static int cmp_detached_dev_name(const struct rte_device *dev,
+	const void *_name)
+{
+	const char *name = _name;
+
+	/* skip attached devices */
+	if (dev->driver != NULL)
+		return 1;
+
+	return strcmp(dev->name, name);
+}
+
+static int cmp_dev_name(const struct rte_device *dev, const void *_name)
+{
+	const char *name = _name;
+
+	return strcmp(dev->name, name);
+}
+
 int rte_eal_dev_attach(const char *name, const char *devargs)
 {
 	struct rte_pci_addr addr;
@@ -92,3 +112,71 @@ int rte_eal_dev_detach(const char *name)
 	RTE_LOG(ERR, EAL, "Driver cannot detach the device (%s)\n", name);
 	return -EINVAL;
 }
+
+int rte_eal_hotplug_add(const char *busname, const char *devname,
+			const char *devargs)
+{
+	struct rte_bus *bus;
+	struct rte_device *dev;
+	int ret;
+
+	bus = rte_bus_find_by_name(busname);
+	if (bus == NULL) {
+		RTE_LOG(ERR, EAL, "Cannot find bus (%s)\n", busname);
+		return -ENOENT;
+	}
+
+	if (bus->plug == NULL) {
+		RTE_LOG(ERR, EAL, "Function plug not supported by bus (%s)\n",
+			bus->name);
+		return -ENOTSUP;
+	}
+
+	ret = bus->scan();
+	if (ret)
+		return ret;
+
+	dev = bus->find_device(NULL, cmp_detached_dev_name, devname);
+	if (dev == NULL) {
+		RTE_LOG(ERR, EAL, "Cannot find unplugged device (%s)\n",
+			devname);
+		return -EINVAL;
+	}
+
+	ret = bus->plug(dev, devargs);
+	if (ret)
+		RTE_LOG(ERR, EAL, "Driver cannot attach the device (%s)\n",
+			dev->name);
+	return ret;
+}
+
+int rte_eal_hotplug_remove(const char *busname, const char *devname)
+{
+	struct rte_bus *bus;
+	struct rte_device *dev;
+	int ret;
+
+	bus = rte_bus_find_by_name(busname);
+	if (bus == NULL) {
+		RTE_LOG(ERR, EAL, "Cannot find bus (%s)\n", busname);
+		return -ENOENT;
+	}
+
+	if (bus->unplug == NULL) {
+		RTE_LOG(ERR, EAL, "Function unplug not supported by bus (%s)\n",
+			bus->name);
+		return -ENOTSUP;
+	}
+
+	dev = bus->find_device(NULL, cmp_dev_name, devname);
+	if (dev == NULL) {
+		RTE_LOG(ERR, EAL, "Cannot find plugged device (%s)\n", devname);
+		return -EINVAL;
+	}
+
+	ret = bus->unplug(dev);
+	if (ret)
+		RTE_LOG(ERR, EAL, "Driver cannot detach the device (%s)\n",
+			dev->name);
+	return ret;
+}
diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
index 04d9c28..a0d67d0 100644
--- a/lib/librte_eal/common/include/rte_dev.h
+++ b/lib/librte_eal/common/include/rte_dev.h
@@ -192,6 +192,40 @@ int rte_eal_dev_attach(const char *name, const char *devargs);
 int rte_eal_dev_detach(const char *name);
 
 /**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Hotplug add a given device to a specific bus.
+ *
+ * @param busname
+ *   The bus name the device is added to.
+ * @param devname
+ *   The device name. Based on this device name, eal will identify a driver
+ *   capable of handling it and pass it to the driver probing function.
+ * @param devargs
+ *   Device arguments to be passed to the driver.
+ * @return
+ *   0 on success, negative on error.
+ */
+int rte_eal_hotplug_add(const char *busname, const char *devname,
+			const char *devargs);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Hotplug remove a given device from a specific bus.
+ *
+ * @param busname
+ *   The bus name the device is removed from.
+ * @param devname
+ *   The device name being removed.
+ * @return
+ *   0 on success, negative on error.
+ */
+int rte_eal_hotplug_remove(const char *busname, const char *devname);
+
+/**
  * Device comparison function.
  *
  * This type of function is used to compare an rte_device with arbitrary
diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
index 643ec1c..a118fb1 100644
--- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
@@ -207,3 +207,11 @@ DPDK_17.08 {
 	rte_bus_find_by_name;
 
 } DPDK_17.05;
+
+EXPERIMENTAL {
+	global:
+
+	rte_eal_hotplug_add;
+	rte_eal_hotplug_remove;
+
+} DPDK_17.08;
-- 
2.9.4

[PATCH v8 13/14] ethdev: Use embedded rte_device to detach driver

[Jan Blunck]

This is changing the API of rte_eal_dev_detach().

Signed-off-by: Jan Blunck <jblunck@infradead.org>
---
 app/test-pmd/testpmd.c                  |  2 +-
 lib/librte_eal/common/eal_common_dev.c  | 32 +++++++++++++++++++-------------
 lib/librte_eal/common/include/rte_dev.h |  7 +++----
 lib/librte_ether/rte_ethdev.c           |  3 ++-
 4 files changed, 25 insertions(+), 19 deletions(-)

diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index b29328a..86d494b 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -1824,7 +1824,7 @@ rmv_event_callback(void *arg)
 	else if (da->type == RTE_DEVTYPE_WHITELISTED_PCI)
 		rte_pci_device_name(&da->pci.addr, name, sizeof(name));
 	printf("removing device %s\n", name);
-	rte_eal_dev_detach(name);
+	rte_eal_dev_detach(dev->device);
 	dev->state = RTE_ETH_DEV_UNUSED;
 }
 
diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c
index 3606332..ede68e4 100644
--- a/lib/librte_eal/common/eal_common_dev.c
+++ b/lib/librte_eal/common/eal_common_dev.c
@@ -90,27 +90,33 @@ int rte_eal_dev_attach(const char *name, const char *devargs)
 	return -EINVAL;
 }
 
-int rte_eal_dev_detach(const char *name)
+int rte_eal_dev_detach(struct rte_device *dev)
 {
-	struct rte_pci_addr addr;
+	struct rte_bus *bus;
+	int ret;
 
-	if (name == NULL) {
+	if (dev == NULL) {
 		RTE_LOG(ERR, EAL, "Invalid device provided.\n");
 		return -EINVAL;
 	}
 
-	if (eal_parse_pci_DomBDF(name, &addr) == 0) {
-		if (rte_pci_detach(&addr) < 0)
-			goto err;
-	} else {
-		if (rte_vdev_uninit(name))
-			goto err;
+	bus = rte_bus_find_by_device(dev);
+	if (bus == NULL) {
+		RTE_LOG(ERR, EAL, "Cannot find bus for device (%s)\n",
+			dev->name);
+		return -EINVAL;
 	}
-	return 0;
 
-err:
-	RTE_LOG(ERR, EAL, "Driver cannot detach the device (%s)\n", name);
-	return -EINVAL;
+	if (bus->unplug == NULL) {
+		RTE_LOG(ERR, EAL, "Bus function not supported\n");
+		return -ENOTSUP;
+	}
+
+	ret = bus->unplug(dev);
+	if (ret)
+		RTE_LOG(ERR, EAL, "Driver cannot detach the device (%s)\n",
+			dev->name);
+	return ret;
 }
 
 int rte_eal_hotplug_add(const char *busname, const char *devname,
diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
index a0d67d0..9f2765d 100644
--- a/lib/librte_eal/common/include/rte_dev.h
+++ b/lib/librte_eal/common/include/rte_dev.h
@@ -183,13 +183,12 @@ int rte_eal_dev_attach(const char *name, const char *devargs);
 /**
  * Detach a device from its driver.
  *
- * @param name
- *   Same description as for rte_eal_dev_attach().
- *   Here, eal will call the driver detaching function.
+ * @param dev
+ *   A pointer to a rte_device structure.
  * @return
  *   0 on success, negative on error.
  */
-int rte_eal_dev_detach(const char *name);
+int rte_eal_dev_detach(struct rte_device *dev);
 
 /**
  * @warning
diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index 81a45c0..6554378 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -439,7 +439,8 @@ rte_eth_dev_detach(uint8_t port_id, char *name)
 
 	snprintf(name, sizeof(rte_eth_devices[port_id].data->name),
 		 "%s", rte_eth_devices[port_id].data->name);
-	ret = rte_eal_dev_detach(name);
+
+	ret = rte_eal_dev_detach(rte_eth_devices[0].device);
 	if (ret < 0)
 		goto err;
 
-- 
2.9.4

[PATCH v8 14/14] dev: use new hotplug API in attach

[Jan Blunck]

Using the new hotplug API allows attach to be backwards compatible while
decoupling it from the concrete bus implementations.

Signed-off-by: Jan Blunck <jblunck@infradead.org>
---
 lib/librte_eal/common/eal_common_dev.c | 27 +++++++++++++--------------
 1 file changed, 13 insertions(+), 14 deletions(-)

diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c
index ede68e4..4ee52fd 100644
--- a/lib/librte_eal/common/eal_common_dev.c
+++ b/lib/librte_eal/common/eal_common_dev.c
@@ -67,27 +67,26 @@ static int cmp_dev_name(const struct rte_device *dev, const void *_name)
 
 int rte_eal_dev_attach(const char *name, const char *devargs)
 {
-	struct rte_pci_addr addr;
+	int ret;
 
 	if (name == NULL || devargs == NULL) {
 		RTE_LOG(ERR, EAL, "Invalid device or arguments provided\n");
 		return -EINVAL;
 	}
 
-	if (eal_parse_pci_DomBDF(name, &addr) == 0) {
-		if (rte_pci_probe_one(&addr) < 0)
-			goto err;
-
-	} else {
-		if (rte_vdev_init(name, devargs))
-			goto err;
-	}
-
-	return 0;
+	ret = rte_eal_hotplug_add("PCI", name, devargs);
+	if (ret && ret != -EINVAL)
+		return ret;
 
-err:
-	RTE_LOG(ERR, EAL, "Driver cannot attach the device (%s)\n", name);
-	return -EINVAL;
+	/*
+	 * If we haven't found a bus device the user meant to "hotplug" a
+	 * virtual device instead.
+	 */
+	ret = rte_vdev_init(name, devargs);
+	if (ret)
+		RTE_LOG(ERR, EAL, "Driver cannot attach the device (%s)\n",
+			name);
+	return ret;
 }
 
 int rte_eal_dev_detach(struct rte_device *dev)
-- 
2.9.4

Re: [PATCH v7 06/15] bus: add helper to find which bus holds a device

[Thomas Monjalon]

30/06/2017 18:46, Jan Blunck:
> On Fri, Jun 30, 2017 at 11:16 AM, Thomas Monjalon <thomas@monjalon.net> wrote:
> > 29/06/2017 20:21, Jan Blunck:
> >> +static int
> >> +bus_find_device(const struct rte_bus *bus, const void *_dev)
> >> +{
> >> +     struct rte_device *dev;
> >> +
> >> +     dev = bus->find_device(NULL, cmp_rte_device, _dev);
> >> +     return !dev;
> >> +}
> >
> > The preferred code style is to make explicit the NULL comparisons:
> >         return dev == NULL;
> 
> Oh, interesting ... not a lot of C++ programmers around here I guess.
> 
> Does this mean you also want me to make integer tests explicit again 0?

Good question, I don't know.
I know only this part of the coding rules:
	http://dpdk.org/doc/guides/contributing/coding_style.html#null-pointers

Re: [PATCH v8 09/14] bus: introduce device plug/unplug functionality

[Gaëtan Rivet]

On Fri, Jun 30, 2017 at 08:19:38PM +0200, Jan Blunck wrote:
> This allows the buses to plug and probe specific devices. This is meant to
> be a building block for hotplug support.
> 
> Signed-off-by: Jan Blunck <jblunck@infradead.org>
> ---
>  lib/librte_eal/common/eal_common_bus.c  |  2 ++
>  lib/librte_eal/common/include/rte_bus.h | 32 ++++++++++++++++++++++++++++++++
>  2 files changed, 34 insertions(+)
> 
> diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
> index bf2b138..87b0c6e 100644
> --- a/lib/librte_eal/common/eal_common_bus.c
> +++ b/lib/librte_eal/common/eal_common_bus.c
> @@ -51,6 +51,8 @@ rte_bus_register(struct rte_bus *bus)
>  	RTE_VERIFY(bus->scan);
>  	RTE_VERIFY(bus->probe);
>  	RTE_VERIFY(bus->find_device);
> +	/* Buses supporting driver plug also require unplug. */
> +	RTE_VERIFY(!bus->plug || bus->unplug);
>  
>  	TAILQ_INSERT_TAIL(&rte_bus_list, bus, next);
>  	RTE_LOG(DEBUG, EAL, "Registered [%s] bus.\n", bus->name);
> diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
> index 509292d..34ea9d5 100644
> --- a/lib/librte_eal/common/include/rte_bus.h
> +++ b/lib/librte_eal/common/include/rte_bus.h
> @@ -108,6 +108,36 @@ typedef struct rte_device *
>  			 const void *data);
>  
>  /**
> + * Implementation specific probe function which is responsible for linking
> + * devices on that bus with applicable drivers.
> + *
> + * @param dev
> + *	Device pointer that was returned by a previous call to find_device.
> + *
> + * @param devargs
> + *	Device declaration.
> + *
> + * @return
> + *	The pointer to a valid rte_device usable by the bus on success.
> + *	NULL on error. rte_errno is then set.
> + */
> +typedef int (*rte_bus_plug_t)(struct rte_device *dev,

This typedef does not match its doc.
If it is the doc that is right, the PCI and vdev implementation should
be fixed as well.

> +			      const char *devargs);
> +
> +/**
> + * Implementation specific remove function which is responsible for unlinking
> + * devices on that bus from assigned driver.
> + *
> + * @param dev
> + *	Device pointer that was returned by a previous call to find_device.
> + *
> + * @return
> + *	0 on success.
> + *	!0 on error. rte_errno is then set.
> + */
> +typedef int (*rte_bus_unplug_t)(struct rte_device *dev);
> +
> +/**
>   * A structure describing a generic bus.
>   */
>  struct rte_bus {
> @@ -116,6 +146,8 @@ struct rte_bus {
>  	rte_bus_scan_t scan;         /**< Scan for devices attached to bus */
>  	rte_bus_probe_t probe;       /**< Probe devices on bus */
>  	rte_bus_find_device_t find_device; /**< Find a device on the bus */
> +	rte_bus_plug_t plug;         /**< Probe single device for drivers */
> +	rte_bus_unplug_t unplug;     /**< Remove single device from driver */
>  };
>  
>  /**
> -- 
> 2.9.4
> 

-- 
Gaëtan Rivet
6WIND

Re: [PATCH v8 09/14] bus: introduce device plug/unplug functionality

[Jan Blunck]

On Fri, Jun 30, 2017 at 8:38 PM, Gaëtan Rivet <gaetan.rivet@6wind.com> wrote:
> On Fri, Jun 30, 2017 at 08:19:38PM +0200, Jan Blunck wrote:
>> This allows the buses to plug and probe specific devices. This is meant to
>> be a building block for hotplug support.
>>
>> Signed-off-by: Jan Blunck <jblunck@infradead.org>
>> ---
>>  lib/librte_eal/common/eal_common_bus.c  |  2 ++
>>  lib/librte_eal/common/include/rte_bus.h | 32 ++++++++++++++++++++++++++++++++
>>  2 files changed, 34 insertions(+)
>>
>> diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
>> index bf2b138..87b0c6e 100644
>> --- a/lib/librte_eal/common/eal_common_bus.c
>> +++ b/lib/librte_eal/common/eal_common_bus.c
>> @@ -51,6 +51,8 @@ rte_bus_register(struct rte_bus *bus)
>>       RTE_VERIFY(bus->scan);
>>       RTE_VERIFY(bus->probe);
>>       RTE_VERIFY(bus->find_device);
>> +     /* Buses supporting driver plug also require unplug. */
>> +     RTE_VERIFY(!bus->plug || bus->unplug);
>>
>>       TAILQ_INSERT_TAIL(&rte_bus_list, bus, next);
>>       RTE_LOG(DEBUG, EAL, "Registered [%s] bus.\n", bus->name);
>> diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h
>> index 509292d..34ea9d5 100644
>> --- a/lib/librte_eal/common/include/rte_bus.h
>> +++ b/lib/librte_eal/common/include/rte_bus.h
>> @@ -108,6 +108,36 @@ typedef struct rte_device *
>>                        const void *data);
>>
>>  /**
>> + * Implementation specific probe function which is responsible for linking
>> + * devices on that bus with applicable drivers.
>> + *
>> + * @param dev
>> + *   Device pointer that was returned by a previous call to find_device.
>> + *
>> + * @param devargs
>> + *   Device declaration.
>> + *
>> + * @return
>> + *   The pointer to a valid rte_device usable by the bus on success.
>> + *   NULL on error. rte_errno is then set.
>> + */
>> +typedef int (*rte_bus_plug_t)(struct rte_device *dev,
>
> This typedef does not match its doc.
> If it is the doc that is right, the PCI and vdev implementation should
> be fixed as well.
>

No, its the doc that needs fixing.

diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/inc
lude/rte_bus.h
index 34ea9d5..2f1c911 100644
--- a/lib/librte_eal/common/include/rte_bus.h
+++ b/lib/librte_eal/common/include/rte_bus.h
@@ -118,8 +118,8 @@ typedef struct rte_device *
  *     Device declaration.
  *
  * @return
- *     The pointer to a valid rte_device usable by the bus on success.
- *     NULL on error. rte_errno is then set.
+ *     0 on success.
+ *     !0 on error.
  */
 typedef int (*rte_bus_plug_t)(struct rte_device *dev,
                              const char *devargs);
@@ -133,7 +133,7 @@ typedef int (*rte_bus_plug_t)(struct rte_device *dev,
  *
  * @return
  *     0 on success.
- *     !0 on error. rte_errno is then set.
+ *     !0 on error.
  */
 typedef int (*rte_bus_unplug_t)(struct rte_device *dev);

Re: [PATCH v7 06/15] bus: add helper to find which bus holds a device

[Bruce Richardson]

On Fri, Jun 30, 2017 at 08:29:06PM +0200, Thomas Monjalon wrote:
> 30/06/2017 18:46, Jan Blunck:
> > On Fri, Jun 30, 2017 at 11:16 AM, Thomas Monjalon <thomas@monjalon.net> wrote:
> > > 29/06/2017 20:21, Jan Blunck:
> > >> +static int
> > >> +bus_find_device(const struct rte_bus *bus, const void *_dev)
> > >> +{
> > >> +     struct rte_device *dev;
> > >> +
> > >> +     dev = bus->find_device(NULL, cmp_rte_device, _dev);
> > >> +     return !dev;
> > >> +}
> > >
> > > The preferred code style is to make explicit the NULL comparisons:
> > >         return dev == NULL;
> > 
> > Oh, interesting ... not a lot of C++ programmers around here I guess.
> > 
> > Does this mean you also want me to make integer tests explicit again 0?
> 
> Good question, I don't know.
> I know only this part of the coding rules:
> 	http://dpdk.org/doc/guides/contributing/coding_style.html#null-pointers
> 
Yes, I noticed that gap the other day. IMHO for consistency the integers
should similarly be compared to 0/non-zero explicitly rather than using
"!" operator. The exception I would allow is where a function is named
in such a way that is clearly returns a boolean value as int e.g. a
function "int is_computer_on()".

/Bruce

Re: [PATCH v7 06/15] bus: add helper to find which bus holds a device

[Bruce Richardson]

On Fri, Jun 30, 2017 at 06:46:31PM +0200, Jan Blunck wrote:
> On Fri, Jun 30, 2017 at 11:16 AM, Thomas Monjalon <thomas@monjalon.net> wrote:
> > 29/06/2017 20:21, Jan Blunck:
> >> +static int
> >> +bus_find_device(const struct rte_bus *bus, const void *_dev)
> >> +{
> >> +     struct rte_device *dev;
> >> +
> >> +     dev = bus->find_device(NULL, cmp_rte_device, _dev);
> >> +     return !dev;
> >> +}
> >
> > The preferred code style is to make explicit the NULL comparisons:
> >         return dev == NULL;
> 
> Oh, interesting ... not a lot of C++ programmers around here I guess.
> 
> Does this mean you also want me to make integer tests explicit again 0?

Please do.

Re: [PATCH v6 08/11] vdev: use standard bus registration function

[Thomas Monjalon]

27/06/2017 18:11, Gaetan Rivet:
> Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>

Applied as a prerequisite for v8, thanks

Re: [PATCH v8 00/14] bus attach/detach API and hotplug rework

[Thomas Monjalon]

> Gaetan Rivet (1):
>   pci: implement hotplug bus operation
> 
> Jan Blunck (13):
>   bus: add bus iterator to find a bus
>   bus: add find_device bus operation
>   vdev: implement find_device bus operation
>   pci: implement find_device bus operation
>   bus/fslmc: implement find_device bus operation
>   bus: add helper to find which bus holds a device
>   bus: require buses to implement find_device operation
>   bus: add rte_bus_find_by_name
>   bus: introduce device plug/unplug functionality
>   vdev: implement unplug bus operation
>   eal: add hotplug add/remove functions
>   ethdev: Use embedded rte_device to detach driver
>   dev: use new hotplug API in attach

Applied, thanks Jan and Gaetan

Re: [PATCH v8 13/14] ethdev: Use embedded rte_device to detach driver

[Thomas Monjalon]

30/06/2017 20:19, Jan Blunck:
> --- a/lib/librte_ether/rte_ethdev.c
> +++ b/lib/librte_ether/rte_ethdev.c
> @@ -439,7 +439,8 @@ rte_eth_dev_detach(uint8_t port_id, char *name)
>  
>         snprintf(name, sizeof(rte_eth_devices[port_id].data->name),
>                  "%s", rte_eth_devices[port_id].data->name);
> -       ret = rte_eal_dev_detach(name);
> +
> +       ret = rte_eal_dev_detach(rte_eth_devices[0].device);

Obviously, I change rte_eth_devices[0] to rte_eth_devices[port_id].