[PATCH 0/7] NIC port restoration

[PATCH 0/7] NIC port restoration

[Wei Dai]

    Sometimes configuration and run time environment in PMD layer of a
running port is changed and the port has to stop all bi-directional
traffic and initialize the port device and restore its configurations
and traffic again. Such this procedure can be regarded as restoration. 
    Some customers found that in some VFN scenario a running port with
i40e VF DPDK PMD has to be restored when the host PF with kernel driver
need to reset the whole PF due to some reasons. For example, i40e HW
may need a reset after reconfiguring a few key registers. Then PF host
needs to Inform all VFs generated by that PF with the event of 'PF
reset'. After VF driver see this event, VF driver needs to restore
the VF port.

    In order to make restoration as a common functionality of all PMD,
based on current implementation of rte_ethdev, most of restoration work
can be done in rte_ethdev layer with the settings stored in data 
structures in rte_ethdev layer. If some settings is not stored in
rte_ethdev after they are configured before, they should be stored for
restoration by adding data structure in rte_ethdev layer. Ethdev should
also add a API like dev_restore( ) for PMD possible specific work in
restoration process.
   The outline of restoration procedure is as follows.
1. rte_eth_dev_stop(port_id);	
2. (*dev->dev_ops->dev_uninit)(dev);
3. (*dev->dev_ops->dev_init)(dev);
4. rte_eth_dev_configure(...);
5. rte_eth_dev_rx_queue_config(...) invoked for all Rx queues.
6. rte_eth_dev_tx_queue_config(...) invoked for all Tx queues;
7. rte_eth_start(port) 
7.1  rte_eth_dev_config_restore(port); // need to be enhanced
8.  (*dev->dev_ops->dev_restore)(dev); // doing PMD specific restoration


Wei Dai (7):
  ethdev: add support of NIC restoration
  ethdev: add support of restoration of queue state
  ethdev: add support of restoration of port status
  ethdev: add support of MTU restoration
  ethdev: add support of restoration of multicast addr
  net/ixgbe: add support of restoration
  net/i40e: add support of restoration

 drivers/net/i40e/i40e_ethdev.c         |   2 +
 drivers/net/i40e/i40e_ethdev_vf.c      |   5 +
 drivers/net/ixgbe/ixgbe_ethdev.c       |   4 +
 lib/librte_ether/rte_ethdev.c          | 280 +++++++++++++++++++++++++++++++--
 lib/librte_ether/rte_ethdev.h          |  45 +++++-
 lib/librte_ether/rte_ether_version.map |   6 +
 6 files changed, 326 insertions(+), 16 deletions(-)

-- 
2.7.4

[PATCH 1/7] ethdev: add support of NIC restoration

[Wei Dai]

The steps of NIC restoration process include following items in order:
dev_stop, dev_uninit, dev_init, dev_configure with stored configuration,
setup each Rx and Tx queue with previous configurations and dev_start.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 lib/librte_ether/rte_ethdev.c          | 102 ++++++++++++++++++++++++++++++++-
 lib/librte_ether/rte_ethdev.h          |  36 ++++++++++++
 lib/librte_ether/rte_ether_version.map |   6 ++
 3 files changed, 142 insertions(+), 2 deletions(-)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index 83898a8..a5a9519 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -795,17 +795,39 @@ rte_eth_dev_configure(uint8_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q,
 	/*
 	 * Setup new number of RX/TX queues and reconfigure device.
 	 */
+	if (dev->data->rxq_conf == NULL) {
+		dev->data->rxq_conf = rte_zmalloc("ethdev->rxq_conf",
+				sizeof(struct rte_eth_rx_queue_conf) * nb_rx_q,
+				RTE_CACHE_LINE_SIZE);
+		if (dev->data->rxq_conf == NULL)
+			return -ENOMEM;
+	}
+
 	diag = rte_eth_dev_rx_queue_config(dev, nb_rx_q);
 	if (diag != 0) {
 		RTE_PMD_DEBUG_TRACE("port%d rte_eth_dev_rx_queue_config = %d\n",
 				port_id, diag);
+		rte_free(dev->data->rxq_conf);
 		return diag;
 	}
 
+	if (dev->data->txq_conf == NULL) {
+		dev->data->txq_conf = rte_zmalloc("ethdev->txq_conf",
+				sizeof(struct rte_eth_tx_queue_conf) * nb_tx_q,
+				RTE_CACHE_LINE_SIZE);
+		if (dev->data->txq_conf == NULL) {
+			rte_free(dev->data->rxq_conf);
+			rte_eth_dev_rx_queue_config(dev, 0);
+			return -ENOMEM;
+		}
+	}
+
 	diag = rte_eth_dev_tx_queue_config(dev, nb_tx_q);
 	if (diag != 0) {
 		RTE_PMD_DEBUG_TRACE("port%d rte_eth_dev_tx_queue_config = %d\n",
 				port_id, diag);
+		rte_free(dev->data->rxq_conf);
+		rte_free(dev->data->txq_conf);
 		rte_eth_dev_rx_queue_config(dev, 0);
 		return diag;
 	}
@@ -814,6 +836,8 @@ rte_eth_dev_configure(uint8_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q,
 	if (diag != 0) {
 		RTE_PMD_DEBUG_TRACE("port%d dev_configure = %d\n",
 				port_id, diag);
+		rte_free(dev->data->rxq_conf);
+		rte_free(dev->data->txq_conf);
 		rte_eth_dev_rx_queue_config(dev, 0);
 		rte_eth_dev_tx_queue_config(dev, 0);
 		return diag;
@@ -1005,6 +1029,7 @@ rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id,
 	struct rte_eth_dev *dev;
 	struct rte_eth_dev_info dev_info;
 	void **rxq;
+	struct rte_eth_rx_queue_conf *rxq_conf;
 
 	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
 
@@ -1080,6 +1105,12 @@ rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id,
 		if (!dev->data->min_rx_buf_size ||
 		    dev->data->min_rx_buf_size > mbp_buf_size)
 			dev->data->min_rx_buf_size = mbp_buf_size;
+
+		rxq_conf = &dev->data->rxq_conf[rx_queue_id];
+		rxq_conf->nb_rx_desc = nb_rx_desc;
+		rxq_conf->socket_id = socket_id;
+		rxq_conf->rx_conf = *rx_conf;
+		rxq_conf->mp = mp;
 	}
 
 	return ret;
@@ -1093,6 +1124,8 @@ rte_eth_tx_queue_setup(uint8_t port_id, uint16_t tx_queue_id,
 	struct rte_eth_dev *dev;
 	struct rte_eth_dev_info dev_info;
 	void **txq;
+	int ret;
+	struct rte_eth_tx_queue_conf *txq_conf;
 
 	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
 
@@ -1136,8 +1169,16 @@ rte_eth_tx_queue_setup(uint8_t port_id, uint16_t tx_queue_id,
 	if (tx_conf == NULL)
 		tx_conf = &dev_info.default_txconf;
 
-	return (*dev->dev_ops->tx_queue_setup)(dev, tx_queue_id, nb_tx_desc,
-					       socket_id, tx_conf);
+	ret = (*dev->dev_ops->tx_queue_setup)(dev, tx_queue_id, nb_tx_desc,
+					      socket_id, tx_conf);
+	if (!ret) {
+		txq_conf = &dev->data->txq_conf[tx_queue_id];
+		txq_conf->nb_tx_desc = nb_tx_desc;
+		txq_conf->socket_id = socket_id;
+		txq_conf->tx_conf = *tx_conf;
+	}
+
+	return ret;
 }
 
 void
@@ -3472,3 +3513,60 @@ rte_eth_dev_l2_tunnel_offload_set(uint8_t port_id,
 				-ENOTSUP);
 	return (*dev->dev_ops->l2_tunnel_offload_set)(dev, l2_tunnel, mask, en);
 }
+
+int
+rte_eth_dev_restore(uint8_t port_id)
+{
+	struct rte_eth_dev *dev;
+	int ret;
+	uint16_t q;
+	struct rte_eth_rx_queue_conf *rxq_conf;
+	struct rte_eth_tx_queue_conf *txq_conf;
+
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
+	dev = &rte_eth_devices[port_id];
+
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_uninit, -ENOTSUP);
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_init, -ENOTSUP);
+
+	rte_eth_dev_stop(port_id);
+
+	ret = dev->dev_ops->dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = dev->dev_ops->dev_init(dev);
+	if (ret)
+		return ret;
+
+	ret = rte_eth_dev_configure(port_id, dev->data->nb_rx_queues,
+			dev->data->nb_tx_queues, &dev->data->dev_conf);
+	if (ret)
+		return ret;
+
+	for (q = 0; q < dev->data->nb_rx_queues; q++) {
+		rxq_conf = &dev->data->rxq_conf[q];
+		ret = rte_eth_rx_queue_setup(port_id, q, rxq_conf->nb_rx_desc,
+					     rxq_conf->socket_id,
+					     &rxq_conf->rx_conf,
+					     rxq_conf->mp);
+		if (!ret)
+			return ret;
+	}
+
+	for (q = 0; q < dev->data->nb_tx_queues; q++) {
+		txq_conf = &dev->data->txq_conf[q];
+		ret = rte_eth_tx_queue_setup(port_id, q, txq_conf->nb_tx_desc,
+					     txq_conf->socket_id,
+					     &txq_conf->tx_conf);
+		if (!ret)
+			return ret;
+	}
+
+	ret = rte_eth_dev_start(port_id);
+
+	if (dev->dev_ops->dev_restore)
+		ret = dev->dev_ops->dev_restore(dev);
+
+	return ret;
+}
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 0f38b45..0298a1f 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1085,6 +1085,15 @@ typedef int  (*eth_dev_set_link_down_t)(struct rte_eth_dev *dev);
 typedef void (*eth_dev_close_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to close a configured Ethernet device. */
 
+typedef int  (*eth_dev_init_t)(struct rte_eth_dev *dev);
+/** < @internal Function used to initialize a configured Ethernet device. */
+
+typedef int  (*eth_dev_uninit_t)(struct rte_eth_dev *dev);
+/** < @internal Function used to uninit a configured Ethernet device. */
+
+typedef int (*eth_dev_restore_t)(struct rte_eth_dev *dev);
+/**< @internal Function used to restore a configured Ethernet device. */
+
 typedef void (*eth_promiscuous_enable_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to enable the RX promiscuous mode of an Ethernet device. */
 
@@ -1455,6 +1464,9 @@ struct eth_dev_ops {
 	eth_dev_set_link_up_t      dev_set_link_up;   /**< Device link up. */
 	eth_dev_set_link_down_t    dev_set_link_down; /**< Device link down. */
 	eth_dev_close_t            dev_close;     /**< Close device. */
+	eth_dev_init_t             dev_init;      /**< Initialize device */
+	eth_dev_uninit_t           dev_uninit;    /**< Uninit device */
+	eth_dev_restore_t          dev_restore;   /**< Restore device */
 	eth_link_update_t          link_update;   /**< Get device link state. */
 
 	eth_promiscuous_enable_t   promiscuous_enable; /**< Promiscuous ON. */
@@ -1689,6 +1701,19 @@ struct rte_eth_dev_sriov {
 
 #define RTE_ETH_NAME_MAX_LEN (32)
 
+struct rte_eth_rx_queue_conf {
+	uint16_t nb_rx_desc;
+	unsigned int socket_id;
+	struct rte_eth_rxconf rx_conf;
+	struct rte_mempool *mp;
+};
+
+struct rte_eth_tx_queue_conf {
+	uint16_t nb_tx_desc;
+	unsigned int socket_id;
+	struct rte_eth_txconf tx_conf;
+};
+
 /**
  * @internal
  * The data part, with no function pointers, associated with each ethernet device.
@@ -1738,6 +1763,9 @@ struct rte_eth_dev_data {
 	enum rte_kernel_driver kdrv;    /**< Kernel driver passthrough */
 	int numa_node;  /**< NUMA node connection */
 	const char *drv_name;   /**< Driver name */
+
+	struct rte_eth_rx_queue_conf *rxq_conf;
+	struct rte_eth_tx_queue_conf *txq_conf;
 };
 
 /** Device supports hotplug detach */
@@ -2168,6 +2196,14 @@ int rte_eth_dev_set_link_down(uint8_t port_id);
 void rte_eth_dev_close(uint8_t port_id);
 
 /**
+ * Resotre a Ethernet device.
+ *
+ * @param port_id
+ *   The port identifier of the Ethernet device.
+ */
+int rte_eth_dev_restore(uint8_t port_id);
+
+/**
  * Enable receipt in promiscuous mode for an Ethernet device.
  *
  * @param port_id
diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map
index d6726bb..b867c4c 100644
--- a/lib/librte_ether/rte_ether_version.map
+++ b/lib/librte_ether/rte_ether_version.map
@@ -156,3 +156,9 @@ DPDK_17.05 {
 	rte_eth_xstats_get_names_by_id;
 
 } DPDK_17.02;
+
+DPDK_17.08 {
+	global:
+
+	rte_eth_dev_restore;
+} DPDK_17.05;
-- 
2.7.4

[PATCH 2/7] ethdev: add support of restoration of queue state

[Wei Dai]

As dev->dev_ops->dev_start may change dev->data->rx_queue_state[]
and dev->data->tx_queue_state[], this patch adds rxq_restore_state[]
and txq_restore_state[ ] for restoration.
In the restoration process, PMD should start or stop each Rx or Tx
queue according to dev->data->rx_restore_state[] or
dev->data->tx_restore_state[].

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 lib/librte_ether/rte_ethdev.c | 87 +++++++++++++++++++++++++++++++++++++++----
 lib/librte_ether/rte_ethdev.h |  5 ++-
 2 files changed, 83 insertions(+), 9 deletions(-)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index a5a9519..97c0044 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -504,6 +504,7 @@ int
 rte_eth_dev_rx_queue_start(uint8_t port_id, uint16_t rx_queue_id)
 {
 	struct rte_eth_dev *dev;
+	int ret;
 
 	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
 
@@ -522,14 +523,18 @@ rte_eth_dev_rx_queue_start(uint8_t port_id, uint16_t rx_queue_id)
 		return 0;
 	}
 
-	return dev->dev_ops->rx_queue_start(dev, rx_queue_id);
-
+	ret = dev->dev_ops->rx_queue_start(dev, rx_queue_id);
+	if (!ret)
+		dev->data->rxq_restore_state[rx_queue_id] =
+			RTE_ETH_QUEUE_STATE_STARTED;
+	return ret;
 }
 
 int
 rte_eth_dev_rx_queue_stop(uint8_t port_id, uint16_t rx_queue_id)
 {
 	struct rte_eth_dev *dev;
+	int ret;
 
 	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
 
@@ -548,14 +553,18 @@ rte_eth_dev_rx_queue_stop(uint8_t port_id, uint16_t rx_queue_id)
 		return 0;
 	}
 
-	return dev->dev_ops->rx_queue_stop(dev, rx_queue_id);
-
+	ret = dev->dev_ops->rx_queue_stop(dev, rx_queue_id);
+	if (!ret)
+		dev->data->rxq_restore_state[rx_queue_id] =
+			RTE_ETH_QUEUE_STATE_STOPPED;
+	return ret;
 }
 
 int
 rte_eth_dev_tx_queue_start(uint8_t port_id, uint16_t tx_queue_id)
 {
 	struct rte_eth_dev *dev;
+	int ret;
 
 	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
 
@@ -574,14 +583,18 @@ rte_eth_dev_tx_queue_start(uint8_t port_id, uint16_t tx_queue_id)
 		return 0;
 	}
 
-	return dev->dev_ops->tx_queue_start(dev, tx_queue_id);
-
+	ret = dev->dev_ops->tx_queue_start(dev, tx_queue_id);
+	if (!ret)
+		dev->data->txq_restore_state[tx_queue_id] =
+			RTE_ETH_QUEUE_STATE_STARTED;
+	return ret;
 }
 
 int
 rte_eth_dev_tx_queue_stop(uint8_t port_id, uint16_t tx_queue_id)
 {
 	struct rte_eth_dev *dev;
+	int ret;
 
 	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
 
@@ -600,8 +613,11 @@ rte_eth_dev_tx_queue_stop(uint8_t port_id, uint16_t tx_queue_id)
 		return 0;
 	}
 
-	return dev->dev_ops->tx_queue_stop(dev, tx_queue_id);
-
+	ret = dev->dev_ops->tx_queue_stop(dev, tx_queue_id);
+	if (!ret)
+		dev->data->txq_restore_state[tx_queue_id] =
+			RTE_ETH_QUEUE_STATE_STOPPED;
+	return ret;
 }
 
 static int
@@ -863,6 +879,50 @@ _rte_eth_dev_reset(struct rte_eth_dev *dev)
 }
 
 static void
+rte_eth_dev_rx_queue_restore(uint8_t port_id, uint16_t queue_id)
+{
+	struct rte_eth_dev *dev;
+	uint16_t q = queue_id;
+
+	dev = &rte_eth_devices[port_id];
+
+	if (dev->data->in_restoration == 0) {
+		dev->data->rxq_restore_state[q] = dev->data->rx_queue_state[q];
+		return;
+	}
+
+	if (dev->data->rxq_restore_state[q] != dev->data->rx_queue_state[q]) {
+		if (dev->data->rxq_restore_state[q]
+		    == RTE_ETH_QUEUE_STATE_STARTED)
+			rte_eth_dev_rx_queue_start(port_id, q);
+		else
+			rte_eth_dev_rx_queue_stop(port_id, q);
+	}
+}
+
+static void
+rte_eth_dev_tx_queue_restore(uint8_t port_id, uint16_t queue_id)
+{
+	struct rte_eth_dev *dev;
+	uint16_t q = queue_id;
+
+	dev = &rte_eth_devices[port_id];
+
+	if (dev->data->in_restoration == 0) {
+		dev->data->txq_restore_state[q] = dev->data->tx_queue_state[q];
+		return;
+	}
+
+	if (dev->data->txq_restore_state[q] != dev->data->tx_queue_state[q]) {
+		if (dev->data->txq_restore_state[q]
+		    == RTE_ETH_QUEUE_STATE_STARTED)
+			rte_eth_dev_tx_queue_start(port_id, q);
+		else
+			rte_eth_dev_tx_queue_stop(port_id, q);
+	}
+}
+
+static void
 rte_eth_dev_config_restore(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
@@ -871,6 +931,7 @@ rte_eth_dev_config_restore(uint8_t port_id)
 	uint16_t i;
 	uint32_t pool = 0;
 	uint64_t pool_mask;
+	uint16_t q;
 
 	dev = &rte_eth_devices[port_id];
 
@@ -915,6 +976,12 @@ rte_eth_dev_config_restore(uint8_t port_id)
 		rte_eth_allmulticast_enable(port_id);
 	else if (rte_eth_allmulticast_get(port_id) == 0)
 		rte_eth_allmulticast_disable(port_id);
+
+	for (q = 0; q < dev->data->nb_rx_queues; q++)
+		rte_eth_dev_rx_queue_restore(port_id, q);
+	for (q = 0; q < dev->data->nb_tx_queues; q++)
+		rte_eth_dev_tx_queue_restore(port_id, q);
+
 }
 
 int
@@ -3531,6 +3598,8 @@ rte_eth_dev_restore(uint8_t port_id)
 
 	rte_eth_dev_stop(port_id);
 
+	dev->data->in_restoration = 1;
+
 	ret = dev->dev_ops->dev_uninit(dev);
 	if (ret)
 		return ret;
@@ -3568,5 +3637,7 @@ rte_eth_dev_restore(uint8_t port_id)
 	if (dev->dev_ops->dev_restore)
 		ret = dev->dev_ops->dev_restore(dev);
 
+	dev->data->in_restoration = 0;
+
 	return ret;
 }
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 0298a1f..7a2ce07 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1754,10 +1754,13 @@ struct rte_eth_dev_data {
 		scattered_rx : 1,  /**< RX of scattered packets is ON(1) / OFF(0) */
 		all_multicast : 1, /**< RX all multicast mode ON(1) / OFF(0). */
 		dev_started : 1,   /**< Device state: STARTED(1) / STOPPED(0). */
-		lro         : 1;   /**< RX LRO is ON(1) / OFF(0) */
+		lro         : 1,   /**< RX LRO is ON(1) / OFF(0) */
+		in_restoration : 1; /**< In Restoration Yes(1) / NO(0) */
 	uint8_t rx_queue_state[RTE_MAX_QUEUES_PER_PORT];
+	uint8_t rxq_restore_state[RTE_MAX_QUEUES_PER_PORT];
 	/** Queues state: STARTED(1) / STOPPED(0) */
 	uint8_t tx_queue_state[RTE_MAX_QUEUES_PER_PORT];
+	uint8_t txq_restore_state[RTE_MAX_QUEUES_PER_PORT];
 	/** Queues state: STARTED(1) / STOPPED(0) */
 	uint32_t dev_flags; /**< Capabilities */
 	enum rte_kernel_driver kdrv;    /**< Kernel driver passthrough */
-- 
2.7.4

[PATCH 3/7] ethdev: add support of restoration of port status

[Wei Dai]

As dev->data->dev_link.link_status may change when the port is
initialized again, this patch adds dev->data->restore_link for
restoration.
In the restoration process, ethdev layer can restore link status
as up or down by comparing dev->data->restore_link.link_status
and dev->data->dev_link.link_status.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 lib/librte_ether/rte_ethdev.c | 37 +++++++++++++++++++++++++++++++++++--
 lib/librte_ether/rte_ethdev.h |  1 +
 2 files changed, 36 insertions(+), 2 deletions(-)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index 97c0044..af8ccf6 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -923,6 +923,28 @@ rte_eth_dev_tx_queue_restore(uint8_t port_id, uint16_t queue_id)
 }
 
 static void
+rte_eth_dev_link_status_restore(uint8_t port_id)
+{
+	struct rte_eth_dev *dev;
+
+	dev = &rte_eth_devices[port_id];
+
+	if (dev->data->in_restoration == 0) {
+		dev->data->restore_link.link_status =
+			dev->data->dev_link.link_status;
+		return;
+	}
+
+	if (dev->data->restore_link.link_status
+	    != dev->data->dev_link.link_status) {
+		if (dev->data->restore_link.link_status != 0)
+			rte_eth_dev_set_link_up(port_id);
+		else
+			rte_eth_dev_set_link_down(port_id);
+	}
+}
+
+static void
 rte_eth_dev_config_restore(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
@@ -982,6 +1004,7 @@ rte_eth_dev_config_restore(uint8_t port_id)
 	for (q = 0; q < dev->data->nb_tx_queues; q++)
 		rte_eth_dev_tx_queue_restore(port_id, q);
 
+	rte_eth_dev_link_status_restore(port_id);
 }
 
 int
@@ -1014,6 +1037,8 @@ rte_eth_dev_start(uint8_t port_id)
 	if (dev->data->dev_conf.intr_conf.lsc == 0) {
 		RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->link_update, -ENOTSUP);
 		(*dev->dev_ops->link_update)(dev, 0);
+		dev->data->restore_link.link_status =
+			dev->data->dev_link.link_status;
 	}
 	return 0;
 }
@@ -1043,26 +1068,34 @@ int
 rte_eth_dev_set_link_up(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
+	int ret;
 
 	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
 
 	dev = &rte_eth_devices[port_id];
 
 	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_up, -ENOTSUP);
-	return (*dev->dev_ops->dev_set_link_up)(dev);
+	ret = (*dev->dev_ops->dev_set_link_up)(dev);
+	if (!ret)
+		dev->data->restore_link.link_status = 1;
+	return ret;
 }
 
 int
 rte_eth_dev_set_link_down(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
+	int ret;
 
 	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
 
 	dev = &rte_eth_devices[port_id];
 
 	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_set_link_down, -ENOTSUP);
-	return (*dev->dev_ops->dev_set_link_down)(dev);
+	ret = (*dev->dev_ops->dev_set_link_down)(dev);
+	if (!ret)
+		dev->data->restore_link.link_status = 0;
+	return ret;
 }
 
 void
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 7a2ce07..9428f57 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1734,6 +1734,7 @@ struct rte_eth_dev_data {
 	void *dev_private;              /**< PMD-specific private data */
 
 	struct rte_eth_link dev_link;
+	struct rte_eth_link restore_link;
 	/**< Link-level information & status */
 
 	struct rte_eth_conf dev_conf;   /**< Configuration applied to device. */
-- 
2.7.4

[PATCH 4/7] ethdev: add support of MTU restoration

[Wei Dai]

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 lib/librte_ether/rte_ethdev.c | 23 +++++++++++++++++++++--
 lib/librte_ether/rte_ethdev.h |  1 +
 2 files changed, 22 insertions(+), 2 deletions(-)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index af8ccf6..0d9544c 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -945,6 +945,22 @@ rte_eth_dev_link_status_restore(uint8_t port_id)
 }
 
 static void
+rte_eth_dev_mtu_restore(uint8_t port_id)
+{
+	struct rte_eth_dev *dev;
+
+	dev = &rte_eth_devices[port_id];
+
+	if (dev->data->in_restoration == 0) {
+		dev->data->restore_mtu = dev->data->mtu;
+		return;
+	}
+
+	if (dev->data->restore_mtu != dev->data->mtu)
+		rte_eth_dev_set_mtu(port_id, dev->data->restore_mtu);
+}
+
+static void
 rte_eth_dev_config_restore(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
@@ -1005,6 +1021,8 @@ rte_eth_dev_config_restore(uint8_t port_id)
 		rte_eth_dev_tx_queue_restore(port_id, q);
 
 	rte_eth_dev_link_status_restore(port_id);
+
+	rte_eth_dev_mtu_restore(port_id);
 }
 
 int
@@ -2106,9 +2124,10 @@ rte_eth_dev_set_mtu(uint8_t port_id, uint16_t mtu)
 	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mtu_set, -ENOTSUP);
 
 	ret = (*dev->dev_ops->mtu_set)(dev, mtu);
-	if (!ret)
+	if (!ret) {
 		dev->data->mtu = mtu;
-
+		dev->data->restore_mtu = mtu;
+	}
 	return ret;
 }
 
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 9428f57..aca8510 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1739,6 +1739,7 @@ struct rte_eth_dev_data {
 
 	struct rte_eth_conf dev_conf;   /**< Configuration applied to device. */
 	uint16_t mtu;                   /**< Maximum Transmission Unit. */
+	uint16_t restore_mtu;
 
 	uint32_t min_rx_buf_size;
 	/**< Common rx buffer size handled by all queues */
-- 
2.7.4

[PATCH 5/7] ethdev: add support of restoration of multicast addr

[Wei Dai]

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 lib/librte_ether/rte_ethdev.c | 31 ++++++++++++++++++++++++++++++-
 lib/librte_ether/rte_ethdev.h |  2 ++
 2 files changed, 32 insertions(+), 1 deletion(-)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index 0d9544c..78609f5 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -961,6 +961,20 @@ rte_eth_dev_mtu_restore(uint8_t port_id)
 }
 
 static void
+rte_eth_dev_mc_addr_list_restore(uint8_t port_id)
+{
+	struct rte_eth_dev *dev;
+
+	dev = &rte_eth_devices[port_id];
+
+	if (dev->data->mc_addr_count == 0)
+		return;
+
+	rte_eth_dev_set_mc_addr_list(port_id, dev->data->mc_addr_list,
+		dev->data->mc_addr_count);
+}
+
+static void
 rte_eth_dev_config_restore(uint8_t port_id)
 {
 	struct rte_eth_dev *dev;
@@ -1023,6 +1037,8 @@ rte_eth_dev_config_restore(uint8_t port_id)
 	rte_eth_dev_link_status_restore(port_id);
 
 	rte_eth_dev_mtu_restore(port_id);
+
+	rte_eth_dev_mc_addr_list_restore(port_id);
 }
 
 int
@@ -3423,12 +3439,25 @@ rte_eth_dev_set_mc_addr_list(uint8_t port_id,
 			     uint32_t nb_mc_addr)
 {
 	struct rte_eth_dev *dev;
+	int ret;
 
 	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
 
 	dev = &rte_eth_devices[port_id];
 	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->set_mc_addr_list, -ENOTSUP);
-	return dev->dev_ops->set_mc_addr_list(dev, mc_addr_set, nb_mc_addr);
+	ret = dev->dev_ops->set_mc_addr_list(dev, mc_addr_set, nb_mc_addr);
+	if (!ret) {
+		if (dev->data->mc_addr_list != NULL)
+			rte_free(dev->data->mc_addr_list);
+		dev->data->mc_addr_list = rte_zmalloc("dev->mc_addr_list",
+				sizeof(struct ether_addr) * nb_mc_addr, 0);
+		if (dev->data->mc_addr_list != NULL) {
+			rte_memcpy(dev->data->mc_addr_list, mc_addr_set,
+				   sizeof(struct ether_addr) * nb_mc_addr);
+			dev->data->mc_addr_count = nb_mc_addr;
+		}
+	}
+	return ret;
 }
 
 int
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index aca8510..8f7e772 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1771,6 +1771,8 @@ struct rte_eth_dev_data {
 
 	struct rte_eth_rx_queue_conf *rxq_conf;
 	struct rte_eth_tx_queue_conf *txq_conf;
+	uint32_t mc_addr_count;
+	struct ether_addr *mc_addr_list;
 };
 
 /** Device supports hotplug detach */
-- 
2.7.4

[PATCH 6/7] net/ixgbe: add support of restoration

[Wei Dai]

export dev_uninit and dev_init for restoration to
reset port but keep same port id.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 drivers/net/ixgbe/ixgbe_ethdev.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 2083cde..64b8a78 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -525,6 +525,8 @@ static const struct eth_dev_ops ixgbe_eth_dev_ops = {
 	.dev_set_link_up    = ixgbe_dev_set_link_up,
 	.dev_set_link_down  = ixgbe_dev_set_link_down,
 	.dev_close            = ixgbe_dev_close,
+	.dev_init             = eth_ixgbe_dev_init,
+	.dev_uninit           = eth_ixgbe_dev_uninit,
 	.promiscuous_enable   = ixgbe_dev_promiscuous_enable,
 	.promiscuous_disable  = ixgbe_dev_promiscuous_disable,
 	.allmulticast_enable  = ixgbe_dev_allmulticast_enable,
@@ -625,6 +627,8 @@ static const struct eth_dev_ops ixgbevf_eth_dev_ops = {
 	.xstats_reset         = ixgbevf_dev_stats_reset,
 	.xstats_get_names     = ixgbevf_dev_xstats_get_names,
 	.dev_close            = ixgbevf_dev_close,
+	.dev_init             = eth_ixgbevf_dev_init,
+	.dev_uninit           = eth_ixgbevf_dev_uninit,
 	.allmulticast_enable  = ixgbevf_dev_allmulticast_enable,
 	.allmulticast_disable = ixgbevf_dev_allmulticast_disable,
 	.dev_infos_get        = ixgbevf_dev_info_get,
-- 
2.7.4

[PATCH 7/7] net/i40e: add support of restoration

[Wei Dai]

export dev_uninit and dev_init for restoration to
reset port but keep same port id.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 drivers/net/i40e/i40e_ethdev.c    | 2 ++
 drivers/net/i40e/i40e_ethdev_vf.c | 5 +++++
 2 files changed, 7 insertions(+)

diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 4c49673..9512ca4 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -449,6 +449,8 @@ static const struct eth_dev_ops i40e_eth_dev_ops = {
 	.dev_start                    = i40e_dev_start,
 	.dev_stop                     = i40e_dev_stop,
 	.dev_close                    = i40e_dev_close,
+	.dev_init                     = eth_i40e_dev_init,
+	.dev_uninit                   = eth_i40e_dev_uninit,
 	.promiscuous_enable           = i40e_dev_promiscuous_enable,
 	.promiscuous_disable          = i40e_dev_promiscuous_disable,
 	.allmulticast_enable          = i40e_dev_allmulticast_enable,
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 859b5e8..9405888 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -163,6 +163,9 @@ static void i40evf_handle_pf_event(__rte_unused struct rte_eth_dev *dev,
 				   uint8_t *msg,
 				   uint16_t msglen);
 
+static int i40evf_dev_init(struct rte_eth_dev *eth_dev);
+static int i40evf_dev_uninit(struct rte_eth_dev *eth_dev);
+
 /* Default hash key buffer for RSS */
 static uint32_t rss_key_default[I40E_VFQF_HKEY_MAX_INDEX + 1];
 
@@ -194,6 +197,8 @@ static const struct eth_dev_ops i40evf_eth_dev_ops = {
 	.dev_configure        = i40evf_dev_configure,
 	.dev_start            = i40evf_dev_start,
 	.dev_stop             = i40evf_dev_stop,
+	.dev_init             = i40evf_dev_init,
+	.dev_uninit           = i40evf_dev_uninit,
 	.promiscuous_enable   = i40evf_dev_promiscuous_enable,
 	.promiscuous_disable  = i40evf_dev_promiscuous_disable,
 	.allmulticast_enable  = i40evf_dev_allmulticast_enable,
-- 
2.7.4

Re: [PATCH 7/7] net/i40e: add support of restoration

[Wu, Jingjing]

> -----Original Message-----
> From: Dai, Wei
> Sent: Saturday, May 27, 2017 4:22 PM
> To: Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Zhang, Helin <helin.zhang@intel.com>; Wu, Jingjing
> <jingjing.wu@intel.com>
> Cc: dev@dpdk.org; Dai, Wei <wei.dai@intel.com>
> Subject: [PATCH 7/7] net/i40e: add support of restoration
> 
> export dev_uninit and dev_init for restoration to
> reset port but keep same port id.
> 
> Signed-off-by: Wei Dai <wei.dai@intel.com>
> ---
>  drivers/net/i40e/i40e_ethdev.c    | 2 ++
>  drivers/net/i40e/i40e_ethdev_vf.c | 5 +++++
>  2 files changed, 7 insertions(+)
> 
> diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
> index 4c49673..9512ca4 100644
> --- a/drivers/net/i40e/i40e_ethdev.c
> +++ b/drivers/net/i40e/i40e_ethdev.c
> @@ -449,6 +449,8 @@ static const struct eth_dev_ops i40e_eth_dev_ops = {
>  	.dev_start                    = i40e_dev_start,
>  	.dev_stop                     = i40e_dev_stop,
>  	.dev_close                    = i40e_dev_close,
> +	.dev_init                     = eth_i40e_dev_init,
> +	.dev_uninit                   = eth_i40e_dev_uninit,
>  	.promiscuous_enable           = i40e_dev_promiscuous_enable,
>  	.promiscuous_disable          = i40e_dev_promiscuous_disable,
>  	.allmulticast_enable          = i40e_dev_allmulticast_enable,


you can just get int and unint func point by eth_dev->driver.eth_dev_init and eth_dev->driver.eth_dev_init

No need to define ops here.

Re: [PATCH 7/7] net/i40e: add support of restoration

[Dai, Wei]

> -----Original Message-----
> From: Wu, Jingjing
> Sent: Wednesday, June 7, 2017 3:50 PM
> To: Dai, Wei <wei.dai@intel.com>; Lu, Wenzhuo <wenzhuo.lu@intel.com>;
> Ananyev, Konstantin <konstantin.ananyev@intel.com>; Zhang, Helin
> <helin.zhang@intel.com>
> Cc: dev@dpdk.org
> Subject: RE: [PATCH 7/7] net/i40e: add support of restoration
> 
> 
> 
> > -----Original Message-----
> > From: Dai, Wei
> > Sent: Saturday, May 27, 2017 4:22 PM
> > To: Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev, Konstantin
> > <konstantin.ananyev@intel.com>; Zhang, Helin <helin.zhang@intel.com>;
> > Wu, Jingjing <jingjing.wu@intel.com>
> > Cc: dev@dpdk.org; Dai, Wei <wei.dai@intel.com>
> > Subject: [PATCH 7/7] net/i40e: add support of restoration
> >
> > export dev_uninit and dev_init for restoration to reset port but keep
> > same port id.
> >
> > Signed-off-by: Wei Dai <wei.dai@intel.com>
> > ---
> >  drivers/net/i40e/i40e_ethdev.c    | 2 ++
> >  drivers/net/i40e/i40e_ethdev_vf.c | 5 +++++
> >  2 files changed, 7 insertions(+)
> >
> > diff --git a/drivers/net/i40e/i40e_ethdev.c
> > b/drivers/net/i40e/i40e_ethdev.c index 4c49673..9512ca4 100644
> > --- a/drivers/net/i40e/i40e_ethdev.c
> > +++ b/drivers/net/i40e/i40e_ethdev.c
> > @@ -449,6 +449,8 @@ static const struct eth_dev_ops i40e_eth_dev_ops =
> {
> >  	.dev_start                    = i40e_dev_start,
> >  	.dev_stop                     = i40e_dev_stop,
> >  	.dev_close                    = i40e_dev_close,
> > +	.dev_init                     = eth_i40e_dev_init,
> > +	.dev_uninit                   = eth_i40e_dev_uninit,
> >  	.promiscuous_enable           = i40e_dev_promiscuous_enable,
> >  	.promiscuous_disable          = i40e_dev_promiscuous_disable,
> >  	.allmulticast_enable          = i40e_dev_allmulticast_enable,
> 
> 
> you can just get int and unint func point by eth_dev->driver.eth_dev_init and
> eth_dev->driver.eth_dev_init
> 
> No need to define ops here.

I'm sorry that struct rte_eth_dev doesn't have member driver.
The struct rte_eth_dev_data, struct rte_pci_device and struct rte_pci_device don't have
Any member to refer dev_init() and dev_uninit.

[PATCH v2 0/5] Support NIC reset and keep same port id

[Wei Dai]

Sometimes a port have to be reset. After reset, if the port goes
through PCI remove() and then PCI probe() for restoration, its
port id may be changed and this is not expected by some customer
DPDK application. 
Normally, PCI probe() includes two parts: one is in rte_ethdev layer
and the other is calling PMD dev_init(). PCI remove( ) release all 
resource allocated from rte_ethdev layer in PCI probe( ) and calls
PMD dev_unit( ).
To keep same port id and reset the port, only dev_uninit() and
dev_init( ) in PMD can be called and keep all resources allocated
from rte_ethdev layer poart in PCI probe( ).

New rte_eth_dev_reset( ) calls rte_eth_dev_stop( ), PMD dev_uninit( )
and then PMD dev_init( ) to reset a port and keep same port id.
And then application can go through rte_eth_dev_configure( ), 
rte_eth_rx_queue_setup( ), rte_eth_tx_queue_setup( ) and
rte_eth_dev_start( ) again to restore its previous settings or
to reconfigure itself with different settings.

To test this new feature, 2 testpmd commands are introduced.
The first command "reset_port port_id" calls rte_eth_dev_reset( ).
The second command "reconfig_port port_id" is used to test if the
port can be reconfigured successfully. This command configure the port
with the simplest settings include only 1 Rx queue, only 1 Tx queue,
Rx mode = None and Tx mode = None. It check port by receving at least
100 packets and then forward these packets from itself.

A typical test steps are listed as follows:
For example, run "ifconfig PF-name down" will trigger a reset to its VF.
1. run testpmd with ixgbe PF
2. testpmd > set verbose 1 //to observe VF working
3. ifconfig name-of-PF down
4. A message is shown in testmd to indicate PF reset
5. ifconfig name-of-PF up
6. testpmd > stop // stop forwarding to avoid crash during reset
7. testpmd > reset_port port_id_of_VF
8. testpmd > reconfig_port port_id_of_VF


Wei Dai (5):
  ethdev: add support of NIC reset
  net/ixgbe: add support of reset
  net/i40e: add support of reset
  app/testpmd: add command to test NIC reset
  app/testpmd: add command to test NIC restoration

 app/test-pmd/cmdline.c                 |  62 ++++++++++++++++++
 app/test-pmd/config.c                  | 111 +++++++++++++++++++++++++++++++++
 app/test-pmd/testpmd.h                 |   2 +
 drivers/net/i40e/i40e_ethdev.c         |  16 +++++
 drivers/net/i40e/i40e_ethdev_vf.c      |  16 +++++
 drivers/net/ixgbe/ixgbe_ethdev.c       |  38 +++++++++++
 lib/librte_ether/rte_ethdev.c          |  17 +++++
 lib/librte_ether/rte_ethdev.h          |  12 ++++
 lib/librte_ether/rte_ether_version.map |   2 +-
 9 files changed, 275 insertions(+), 1 deletion(-)

-- 
2.7.4

[PATCH v2 1/5] ethdev: add support of NIC reset

[Wei Dai]

A DPDK application can reset a NIC and keep its port id afterwards.
It means that all SW resources allocated in ethdev layer should be
kept and  SW and HW resources of the NIC in PMD need to be reset
in similar way that it runs PCI dev_uninit() and then dev_init().
The sequence of dev_uninit() and dev_init() can be packed into a
single interface API dev_reset().

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 lib/librte_ether/rte_ethdev.c          | 17 +++++++++++++++++
 lib/librte_ether/rte_ethdev.h          | 12 ++++++++++++
 lib/librte_ether/rte_ether_version.map |  2 +-
 3 files changed, 30 insertions(+), 1 deletion(-)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index 81a45c0..5e240f5 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -3349,3 +3349,20 @@ rte_eth_dev_l2_tunnel_offload_set(uint8_t port_id,
 				-ENOTSUP);
 	return (*dev->dev_ops->l2_tunnel_offload_set)(dev, l2_tunnel, mask, en);
 }
+
+int
+rte_eth_dev_reset(uint8_t port_id)
+{
+	struct rte_eth_dev *dev;
+	int ret;
+
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
+	dev = &rte_eth_devices[port_id];
+
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_reset, -ENOTSUP);
+
+	rte_eth_dev_stop(port_id);
+	ret = dev->dev_ops->dev_reset(dev);
+	return ret;
+}
+
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index f6e6c74..3b82dd6 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1087,6 +1087,9 @@ typedef int  (*eth_dev_set_link_down_t)(struct rte_eth_dev *dev);
 typedef void (*eth_dev_close_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to close a configured Ethernet device. */
 
+typedef int (*eth_dev_reset_t)(struct rte_eth_dev *dev);
+/** <@internal Function used to reset a configured Ethernet device. */
+
 typedef void (*eth_promiscuous_enable_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to enable the RX promiscuous mode of an Ethernet device. */
 
@@ -1404,6 +1407,7 @@ struct eth_dev_ops {
 	eth_dev_set_link_up_t      dev_set_link_up;   /**< Device link up. */
 	eth_dev_set_link_down_t    dev_set_link_down; /**< Device link down. */
 	eth_dev_close_t            dev_close;     /**< Close device. */
+	eth_dev_reset_t            dev_reset;     /**< Reset device. */
 	eth_link_update_t          link_update;   /**< Get device link state. */
 
 	eth_promiscuous_enable_t   promiscuous_enable; /**< Promiscuous ON. */
@@ -2104,6 +2108,14 @@ int rte_eth_dev_set_link_down(uint8_t port_id);
 void rte_eth_dev_close(uint8_t port_id);
 
 /**
++ * Reset a Ethernet device.
++ *
++ * @param port_id
++ *   The port identifier of the Ethernet device.
++ */
+int rte_eth_dev_reset(uint8_t port_id);
+
+/**
  * Enable receipt in promiscuous mode for an Ethernet device.
  *
  * @param port_id
diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map
index 805e6de..8c0b2c9 100644
--- a/lib/librte_ether/rte_ether_version.map
+++ b/lib/librte_ether/rte_ether_version.map
@@ -152,5 +152,5 @@ DPDK_17.08 {
 	global:
 
 	rte_flow_isolate;
-
+	rte_eth_dev_reset;
 } DPDK_17.05;
-- 
2.7.4

[PATCH v2 2/5] net/ixgbe: add support of reset

[Wei Dai]

Reset a NIC by calling dev_uninit and then dev_init.
Go through same way in NIC PCI remove without release of
ethdev resource and then NIC PCI probe function without
ethdev resource allocation.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 drivers/net/ixgbe/ixgbe_ethdev.c | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index ebc5467..5ff8fbe 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -170,6 +170,7 @@ static void ixgbe_dev_stop(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_up(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_down(struct rte_eth_dev *dev);
 static void ixgbe_dev_close(struct rte_eth_dev *dev);
+static int  ixgbe_dev_reset(struct rte_eth_dev *dev);
 static void ixgbe_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void ixgbe_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void ixgbe_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -264,6 +265,7 @@ static int  ixgbevf_dev_configure(struct rte_eth_dev *dev);
 static int  ixgbevf_dev_start(struct rte_eth_dev *dev);
 static void ixgbevf_dev_stop(struct rte_eth_dev *dev);
 static void ixgbevf_dev_close(struct rte_eth_dev *dev);
+static int  ixgbevf_dev_reset(struct rte_eth_dev *dev);
 static void ixgbevf_intr_disable(struct ixgbe_hw *hw);
 static void ixgbevf_intr_enable(struct ixgbe_hw *hw);
 static void ixgbevf_dev_stats_get(struct rte_eth_dev *dev,
@@ -525,6 +527,7 @@ static const struct eth_dev_ops ixgbe_eth_dev_ops = {
 	.dev_set_link_up    = ixgbe_dev_set_link_up,
 	.dev_set_link_down  = ixgbe_dev_set_link_down,
 	.dev_close            = ixgbe_dev_close,
+	.dev_reset            = ixgbe_dev_reset,
 	.promiscuous_enable   = ixgbe_dev_promiscuous_enable,
 	.promiscuous_disable  = ixgbe_dev_promiscuous_disable,
 	.allmulticast_enable  = ixgbe_dev_allmulticast_enable,
@@ -614,6 +617,7 @@ static const struct eth_dev_ops ixgbevf_eth_dev_ops = {
 	.xstats_reset         = ixgbevf_dev_stats_reset,
 	.xstats_get_names     = ixgbevf_dev_xstats_get_names,
 	.dev_close            = ixgbevf_dev_close,
+	.dev_reset            = ixgbevf_dev_reset,
 	.allmulticast_enable  = ixgbevf_dev_allmulticast_enable,
 	.allmulticast_disable = ixgbevf_dev_allmulticast_disable,
 	.dev_infos_get        = ixgbevf_dev_info_get,
@@ -2838,6 +2842,23 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 	ixgbe_set_rar(hw, 0, hw->mac.addr, 0, IXGBE_RAH_AV);
 }
 
+/*
+ * Reest PF device.
+ */
+static int
+ixgbe_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	ret = eth_ixgbe_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_ixgbe_dev_init(dev);
+
+	return ret;
+}
+
 static void
 ixgbe_read_stats_registers(struct ixgbe_hw *hw,
 			   struct ixgbe_hw_stats *hw_stats,
@@ -4902,6 +4923,23 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	ixgbevf_remove_mac_addr(dev, 0);
 }
 
+/*
+ * Reset VF device
+ */
+static int
+ixgbevf_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	ret = eth_ixgbevf_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_ixgbevf_dev_init(dev);
+
+	return ret;
+}
+
 static void ixgbevf_set_vfta_all(struct rte_eth_dev *dev, bool on)
 {
 	struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-- 
2.7.4

[PATCH v2 3/5] net/i40e: add support of reset

[Wei Dai]

Reset a NIC by calling dev_uninit() and then dev_init().
Go through the same way in NIC PCI remove without release
of ethdev resource and then NIC PCI probe function without
ethdev resource allocation.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 drivers/net/i40e/i40e_ethdev.c    | 16 ++++++++++++++++
 drivers/net/i40e/i40e_ethdev_vf.c | 16 ++++++++++++++++
 2 files changed, 32 insertions(+)

diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 4ee1113..7694957 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -250,6 +250,7 @@ static int i40e_dev_configure(struct rte_eth_dev *dev);
 static int i40e_dev_start(struct rte_eth_dev *dev);
 static void i40e_dev_stop(struct rte_eth_dev *dev);
 static void i40e_dev_close(struct rte_eth_dev *dev);
+static int  i40e_dev_reset(struct rte_eth_dev *dev);
 static void i40e_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void i40e_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void i40e_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -449,6 +450,7 @@ static const struct eth_dev_ops i40e_eth_dev_ops = {
 	.dev_start                    = i40e_dev_start,
 	.dev_stop                     = i40e_dev_stop,
 	.dev_close                    = i40e_dev_close,
+	.dev_reset                    = i40e_dev_reset,
 	.promiscuous_enable           = i40e_dev_promiscuous_enable,
 	.promiscuous_disable          = i40e_dev_promiscuous_disable,
 	.allmulticast_enable          = i40e_dev_allmulticast_enable,
@@ -2135,6 +2137,20 @@ i40e_dev_close(struct rte_eth_dev *dev)
 	I40E_WRITE_FLUSH(hw);
 }
 
+static int
+i40e_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	ret = eth_i40e_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_i40e_dev_init(dev);
+
+	return ret;
+}
+
 static void
 i40e_dev_promiscuous_enable(struct rte_eth_dev *dev)
 {
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 2d5a9b5..bf287a0 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -123,6 +123,7 @@ static void i40evf_vlan_offload_set(struct rte_eth_dev *dev, int mask);
 static int i40evf_vlan_pvid_set(struct rte_eth_dev *dev, uint16_t pvid,
 				int on);
 static void i40evf_dev_close(struct rte_eth_dev *dev);
+static int  i40evf_dev_reset(struct rte_eth_dev *dev);
 static void i40evf_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void i40evf_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void i40evf_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -204,6 +205,7 @@ static const struct eth_dev_ops i40evf_eth_dev_ops = {
 	.xstats_get_names     = i40evf_dev_xstats_get_names,
 	.xstats_reset         = i40evf_dev_xstats_reset,
 	.dev_close            = i40evf_dev_close,
+	.dev_reset            = i40evf_dev_reset,
 	.dev_infos_get        = i40evf_dev_info_get,
 	.dev_supported_ptypes_get = i40e_dev_supported_ptypes_get,
 	.vlan_filter_set      = i40evf_vlan_filter_set,
@@ -2347,6 +2349,20 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 }
 
 static int
+i40evf_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	ret = i40evf_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = i40evf_dev_init(dev);
+
+	return ret;
+}
+
+static int
 i40evf_get_rss_lut(struct i40e_vsi *vsi, uint8_t *lut, uint16_t lut_size)
 {
 	struct i40e_vf *vf = I40E_VSI_TO_VF(vsi);
-- 
2.7.4

[PATCH v2 4/5] app/testpmd: add command to test NIC reset

[Wei Dai]

When a NIC is reset, a message will show it.
And then user can run the command "reset_port port_id"
to process it.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 app/test-pmd/cmdline.c | 31 +++++++++++++++++++++++++++++++
 app/test-pmd/config.c  | 13 +++++++++++++
 app/test-pmd/testpmd.h |  1 +
 3 files changed, 45 insertions(+)

diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
index ff8ffd2..90f6bde 100644
--- a/app/test-pmd/cmdline.c
+++ b/app/test-pmd/cmdline.c
@@ -2588,6 +2588,36 @@ cmdline_parse_inst_t cmd_stop = {
 	},
 };
 
+/* *** reset a port *** */
+struct cmd_reset_port_result {
+	cmdline_fixed_string_t command;
+	uint8_t port_id;
+};
+
+static void cmd_reset_port_parsed(__attribute__((unused)) void *parsed_result,
+			    __attribute__((unused)) struct cmdline *cl,
+			    __attribute__((unused)) void *data)
+{
+	struct cmd_reset_port_result *res = parsed_result;
+	reset_port(res->port_id);
+}
+
+cmdline_parse_token_string_t cmd_reset_port_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_reset_port_result, command, "reset_port");
+cmdline_parse_token_num_t cmd_reset_port_id =
+	TOKEN_NUM_INITIALIZER(struct cmd_reset_port_result, port_id, UINT8);
+
+cmdline_parse_inst_t cmd_reset_port = {
+	.f = cmd_reset_port_parsed,
+	.data = NULL,
+	.help_str = "reset_port <port_id>",
+	.tokens = {
+		(void *)&cmd_reset_port_cmd,
+		(void *)&cmd_reset_port_id,
+		NULL,
+	},
+};
+
 /* *** SET CORELIST and PORTLIST CONFIGURATION *** */
 
 unsigned int
@@ -13751,6 +13781,7 @@ cmdline_parse_ctx_t main_ctx[] = {
 	(cmdline_parse_inst_t *)&cmd_write_reg_bit,
 	(cmdline_parse_inst_t *)&cmd_read_rxd_txd,
 	(cmdline_parse_inst_t *)&cmd_stop,
+	(cmdline_parse_inst_t *)&cmd_reset_port,
 	(cmdline_parse_inst_t *)&cmd_mac_addr,
 	(cmdline_parse_inst_t *)&cmd_set_qmap,
 	(cmdline_parse_inst_t *)&cmd_operate_port,
diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
index b0b340e..da3b525 100644
--- a/app/test-pmd/config.c
+++ b/app/test-pmd/config.c
@@ -3318,3 +3318,16 @@ close_ddp_package_file(uint8_t *buf)
 
 	return -1;
 }
+
+void
+reset_port(portid_t port_id)
+{
+	int diag;
+
+	if (port_id_is_invalid(port_id, ENABLED_WARN))
+		return;
+	diag = rte_eth_dev_reset(port_id);
+	if (diag == 0)
+		return;
+	printf("Reset port %d failed. diag=%d\n", port_id, diag);	
+}
diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
index 364502d..956eec5 100644
--- a/app/test-pmd/testpmd.h
+++ b/app/test-pmd/testpmd.h
@@ -598,6 +598,7 @@ void stop_port(portid_t pid);
 void close_port(portid_t pid);
 void attach_port(char *identifier);
 void detach_port(uint8_t port_id);
+void reset_port(portid_t port_id);
 int all_ports_stopped(void);
 int port_is_started(portid_t port_id);
 void pmd_test_exit(void);
-- 
2.7.4

Re: [PATCH v2 4/5] app/testpmd: add command to test NIC reset

[Wu, Jingjing]

> -----Original Message-----
> From: Dai, Wei
> Sent: Tuesday, June 27, 2017 10:07 PM
> To: thomas@monjalon.net; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev,
> Konstantin <konstantin.ananyev@intel.com>; Zhang, Helin
> <helin.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>
> Cc: dev@dpdk.org; Dai, Wei <wei.dai@intel.com>
> Subject: [PATCH v2 4/5] app/testpmd: add command to test NIC reset
> 
> When a NIC is reset, a message will show it.
> And then user can run the command "reset_port port_id"
> to process it.
> 
> Signed-off-by: Wei Dai <wei.dai@intel.com>
> ---
>  app/test-pmd/cmdline.c | 31 +++++++++++++++++++++++++++++++  app/test-
> pmd/config.c  | 13 +++++++++++++  app/test-pmd/testpmd.h |  1 +
>  3 files changed, 45 insertions(+)
> 
> diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index
> ff8ffd2..90f6bde 100644
> --- a/app/test-pmd/cmdline.c
> +++ b/app/test-pmd/cmdline.c
> @@ -2588,6 +2588,36 @@ cmdline_parse_inst_t cmd_stop = {
>  	},
>  };
> 
> +/* *** reset a port *** */
> +struct cmd_reset_port_result {
> +	cmdline_fixed_string_t command;
> +	uint8_t port_id;
> +};
> +
> +static void cmd_reset_port_parsed(__attribute__((unused)) void
> *parsed_result,
> +			    __attribute__((unused)) struct cmdline *cl,
> +			    __attribute__((unused)) void *data) {
> +	struct cmd_reset_port_result *res = parsed_result;
> +	reset_port(res->port_id);
> +}
> +
> +cmdline_parse_token_string_t cmd_reset_port_cmd =
> +	TOKEN_STRING_INITIALIZER(struct cmd_reset_port_result, command,
> +"reset_port"); cmdline_parse_token_num_t cmd_reset_port_id =
> +	TOKEN_NUM_INITIALIZER(struct cmd_reset_port_result, port_id,
> UINT8);
> +
> +cmdline_parse_inst_t cmd_reset_port = {
> +	.f = cmd_reset_port_parsed,
> +	.data = NULL,
> +	.help_str = "reset_port <port_id>",

You can just add one more option in the commands
"port start|stop|close <port_id>" and " port start|stop|close all"
To "port start|stop|close|reset <port_id>" and " port start|stop|close|reset all" instead of defining a new one.

Re: [PATCH v2 4/5] app/testpmd: add command to test NIC reset

[Dai, Wei]

> -----Original Message-----
> From: Wu, Jingjing
> Sent: Wednesday, June 28, 2017 5:11 PM
> To: Dai, Wei <wei.dai@intel.com>; thomas@monjalon.net; Lu, Wenzhuo
> <wenzhuo.lu@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Zhang, Helin <helin.zhang@intel.com>
> Cc: dev@dpdk.org
> Subject: RE: [PATCH v2 4/5] app/testpmd: add command to test NIC reset
> 
> 
> 
> > -----Original Message-----
> > From: Dai, Wei
> > Sent: Tuesday, June 27, 2017 10:07 PM
> > To: thomas@monjalon.net; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev,
> > Konstantin <konstantin.ananyev@intel.com>; Zhang, Helin
> > <helin.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>
> > Cc: dev@dpdk.org; Dai, Wei <wei.dai@intel.com>
> > Subject: [PATCH v2 4/5] app/testpmd: add command to test NIC reset
> >
> > When a NIC is reset, a message will show it.
> > And then user can run the command "reset_port port_id"
> > to process it.
> >
> > Signed-off-by: Wei Dai <wei.dai@intel.com>
> > ---
> >  app/test-pmd/cmdline.c | 31 +++++++++++++++++++++++++++++++
> > app/test- pmd/config.c  | 13 +++++++++++++  app/test-pmd/testpmd.h |
> > 1 +
> >  3 files changed, 45 insertions(+)
> >
> > diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index
> > ff8ffd2..90f6bde 100644
> > --- a/app/test-pmd/cmdline.c
> > +++ b/app/test-pmd/cmdline.c
> > @@ -2588,6 +2588,36 @@ cmdline_parse_inst_t cmd_stop = {
> >  	},
> >  };
> >
> > +/* *** reset a port *** */
> > +struct cmd_reset_port_result {
> > +	cmdline_fixed_string_t command;
> > +	uint8_t port_id;
> > +};
> > +
> > +static void cmd_reset_port_parsed(__attribute__((unused)) void
> > *parsed_result,
> > +			    __attribute__((unused)) struct cmdline *cl,
> > +			    __attribute__((unused)) void *data) {
> > +	struct cmd_reset_port_result *res = parsed_result;
> > +	reset_port(res->port_id);
> > +}
> > +
> > +cmdline_parse_token_string_t cmd_reset_port_cmd =
> > +	TOKEN_STRING_INITIALIZER(struct cmd_reset_port_result, command,
> > +"reset_port"); cmdline_parse_token_num_t cmd_reset_port_id =
> > +	TOKEN_NUM_INITIALIZER(struct cmd_reset_port_result, port_id,
> > UINT8);
> > +
> > +cmdline_parse_inst_t cmd_reset_port = {
> > +	.f = cmd_reset_port_parsed,
> > +	.data = NULL,
> > +	.help_str = "reset_port <port_id>",
> 
> You can just add one more option in the commands "port start|stop|close
> <port_id>" and " port start|stop|close all"
> To "port start|stop|close|reset <port_id>" and " port start|stop|close|reset
> all" instead of defining a new one.
Thanks for your feedback.
V3 patch set will follow your guide.

[PATCH v2 5/5] app/testpmd: add command to test NIC restoration

[Wei Dai]

When a NIC is reset, a message will show it.
And then user can run the command "reset_port port_id"
to reset the port and to keep same port id without any
without any configuration.
This patch adds a testpmd command "reconfig_port port_id"
to test whether the port can be reconfigured to have
success Rx and Tx function.
The new command will configure the port with the simplest
setting which includes only 1 Rx queue, only 1 Tx queue,
Rx mode = None and Tx mode = None.
It check if the port can receive and forward some packets.
For example 100 packets in this new command.
Before testing with "reset_port port_id" and then
"reconfig_port port_id", current forwarding should be stopped
first to avoid crash.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 app/test-pmd/cmdline.c | 31 ++++++++++++++++
 app/test-pmd/config.c  | 98 ++++++++++++++++++++++++++++++++++++++++++++++++++
 app/test-pmd/testpmd.h |  1 +
 3 files changed, 130 insertions(+)

diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
index 90f6bde..1038cee 100644
--- a/app/test-pmd/cmdline.c
+++ b/app/test-pmd/cmdline.c
@@ -2618,6 +2618,36 @@ cmdline_parse_inst_t cmd_reset_port = {
 	},
 };
 
+/* *** reconfig a port with simplest settings only for test *** */
+struct cmd_reconfig_port_result {
+	cmdline_fixed_string_t command;
+	uint8_t port_id;
+};
+
+static void cmd_reconfig_port_parsed(__attribute__((unused)) void *parsed_result,
+			    __attribute__((unused)) struct cmdline *cl,
+			    __attribute__((unused)) void *data)
+{
+	struct cmd_reconfig_port_result *res = parsed_result;
+	reconfig_port(res->port_id);
+}
+
+cmdline_parse_token_string_t cmd_reconfig_port_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_reconfig_port_result, command, "reconfig_port");
+cmdline_parse_token_num_t cmd_reconfig_port_id =
+	TOKEN_NUM_INITIALIZER(struct cmd_reconfig_port_result, port_id, UINT8);
+
+cmdline_parse_inst_t cmd_reconfig_port = {
+	.f = cmd_reconfig_port_parsed,
+	.data = NULL,
+	.help_str = "reconfig_port <port_id>",
+	.tokens = {
+		(void *)&cmd_reconfig_port_cmd,
+		(void *)&cmd_reconfig_port_id,
+		NULL,
+	},
+};
+
 /* *** SET CORELIST and PORTLIST CONFIGURATION *** */
 
 unsigned int
@@ -13782,6 +13812,7 @@ cmdline_parse_ctx_t main_ctx[] = {
 	(cmdline_parse_inst_t *)&cmd_read_rxd_txd,
 	(cmdline_parse_inst_t *)&cmd_stop,
 	(cmdline_parse_inst_t *)&cmd_reset_port,
+	(cmdline_parse_inst_t *)&cmd_reconfig_port,
 	(cmdline_parse_inst_t *)&cmd_mac_addr,
 	(cmdline_parse_inst_t *)&cmd_set_qmap,
 	(cmdline_parse_inst_t *)&cmd_operate_port,
diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
index da3b525..1fd6a54 100644
--- a/app/test-pmd/config.c
+++ b/app/test-pmd/config.c
@@ -3331,3 +3331,101 @@ reset_port(portid_t port_id)
 		return;
 	printf("Reset port %d failed. diag=%d\n", port_id, diag);	
 }
+
+static void
+test_simplest_rxtx(portid_t port)
+{
+#define BURST_SIZE 32
+#define NUM_PACKETS 100
+
+	struct rte_mbuf *bufs[BURST_SIZE];
+	uint16_t nb_rx, nb_tx, total;
+
+	printf("Begin to forward at least %d packets to test reconfiguration\n", NUM_PACKETS);
+	total = 0;
+	while (1) {
+		nb_rx = rte_eth_rx_burst(port, 0, bufs, BURST_SIZE);
+		if (nb_rx == 0)
+			continue;
+		nb_tx = rte_eth_tx_burst(port, 0, bufs, nb_rx);
+		total += nb_tx;
+		/* Free any unsent packets. */
+		if (unlikely(nb_tx < nb_rx)) {
+			uint16_t buf;
+			for (buf = nb_tx; buf < nb_rx; buf++)
+				rte_pktmbuf_free(bufs[buf]);
+		}
+		if (total >= NUM_PACKETS)
+			break;
+	}
+	printf("Finish forwarding %u packets to test reconfiguration\n", total);
+	return;
+}
+
+int
+reconfig_port(portid_t port)
+{
+#define RX_RING_SIZE 128
+#define TX_RING_SIZE 512
+
+#define NUM_MBUFS 8191
+#define MBUF_CACHE_SIZE 250
+
+	struct rte_mempool *mbuf_pool;
+	struct rte_eth_conf dev_conf = {
+		.rxmode = { .max_rx_pkt_len = ETHER_MAX_LEN }
+	};
+	const uint16_t rx_rings = 1, tx_rings = 1;
+	int retval;
+	uint16_t q;
+
+	if (port_id_is_invalid(port, ENABLED_WARN))
+		return -1;
+
+	/* Creates a new mempool in memory to hold the mbufs. */
+	mbuf_pool = rte_pktmbuf_pool_create("MBUF_POOL", NUM_MBUFS,
+		MBUF_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE, rte_eth_dev_socket_id(port));
+
+	/* Configure the Ethernet device. */
+	retval = rte_eth_dev_configure(port, rx_rings, tx_rings, &dev_conf);
+	if (retval != 0)
+		return retval;
+
+	/* Allocate and set up 1 RX queue per Ethernet port. */
+	for (q = 0; q < rx_rings; q++) {
+		retval = rte_eth_rx_queue_setup(port, q, RX_RING_SIZE,
+				rte_eth_dev_socket_id(port), NULL, mbuf_pool);
+		if (retval < 0)
+			return retval;
+	}
+
+	/* Allocate and set up 1 TX queue per Ethernet port. */
+	for (q = 0; q < tx_rings; q++) {
+		retval = rte_eth_tx_queue_setup(port, q, TX_RING_SIZE,
+				rte_eth_dev_socket_id(port), NULL);
+		if (retval < 0)
+			return retval;
+	}
+
+	/* Start the Ethernet port. */
+	retval = rte_eth_dev_start(port);
+	if (retval < 0)
+		return retval;
+
+	/* Display the port MAC address. */
+	struct ether_addr addr;
+	rte_eth_macaddr_get(port, &addr);
+	printf("Port %u MAC: %02" PRIx8 " %02" PRIx8 " %02" PRIx8
+			   " %02" PRIx8 " %02" PRIx8 " %02" PRIx8 "\n",
+			(unsigned)port,
+			addr.addr_bytes[0], addr.addr_bytes[1],
+			addr.addr_bytes[2], addr.addr_bytes[3],
+			addr.addr_bytes[4], addr.addr_bytes[5]);
+
+	/* Enable RX in promiscuous mode for the Ethernet device. */
+	rte_eth_promiscuous_enable(port);
+
+	test_simplest_rxtx(port);
+
+	return 0;
+}
diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
index 956eec5..c4c2e59 100644
--- a/app/test-pmd/testpmd.h
+++ b/app/test-pmd/testpmd.h
@@ -599,6 +599,7 @@ void close_port(portid_t pid);
 void attach_port(char *identifier);
 void detach_port(uint8_t port_id);
 void reset_port(portid_t port_id);
+int reconfig_port(portid_t port_id);
 int all_ports_stopped(void);
 int port_is_started(portid_t port_id);
 void pmd_test_exit(void);
-- 
2.7.4

Re: [PATCH v2 5/5] app/testpmd: add command to test NIC restoration

[Wu, Jingjing]

> -----Original Message-----
> From: Dai, Wei
> Sent: Tuesday, June 27, 2017 10:07 PM
> To: thomas@monjalon.net; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev,
> Konstantin <konstantin.ananyev@intel.com>; Zhang, Helin
> <helin.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>
> Cc: dev@dpdk.org; Dai, Wei <wei.dai@intel.com>
> Subject: [PATCH v2 5/5] app/testpmd: add command to test NIC restoration
> 
> When a NIC is reset, a message will show it.
> And then user can run the command "reset_port port_id"
> to reset the port and to keep same port id without any without any
> configuration.
> This patch adds a testpmd command "reconfig_port port_id"
> to test whether the port can be reconfigured to have success Rx and Tx function.
> The new command will configure the port with the simplest setting which
> includes only 1 Rx queue, only 1 Tx queue, Rx mode = None and Tx mode =
> None.
> It check if the port can receive and forward some packets.
> For example 100 packets in this new command.
> Before testing with "reset_port port_id" and then "reconfig_port port_id",
> current forwarding should be stopped first to avoid crash.
> 

Testpmd has mechanism to support reconfigure, you can check the need_reconfig
Field in port struct.
No need to define a new command.


Thanks
Jingjing

Re: [PATCH v2 5/5] app/testpmd: add command to test NIC restoration

[Dai, Wei]

Thanks for your feedback.
V3 patch set will follow your guide.

> -----Original Message-----
> From: Wu, Jingjing
> Sent: Wednesday, June 28, 2017 5:05 PM
> To: Dai, Wei <wei.dai@intel.com>; thomas@monjalon.net; Lu, Wenzhuo
> <wenzhuo.lu@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Zhang, Helin <helin.zhang@intel.com>
> Cc: dev@dpdk.org
> Subject: RE: [PATCH v2 5/5] app/testpmd: add command to test NIC restoration
> 
> 
> 
> > -----Original Message-----
> > From: Dai, Wei
> > Sent: Tuesday, June 27, 2017 10:07 PM
> > To: thomas@monjalon.net; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev,
> > Konstantin <konstantin.ananyev@intel.com>; Zhang, Helin
> > <helin.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>
> > Cc: dev@dpdk.org; Dai, Wei <wei.dai@intel.com>
> > Subject: [PATCH v2 5/5] app/testpmd: add command to test NIC
> > restoration
> >
> > When a NIC is reset, a message will show it.
> > And then user can run the command "reset_port port_id"
> > to reset the port and to keep same port id without any without any
> > configuration.
> > This patch adds a testpmd command "reconfig_port port_id"
> > to test whether the port can be reconfigured to have success Rx and Tx
> function.
> > The new command will configure the port with the simplest setting
> > which includes only 1 Rx queue, only 1 Tx queue, Rx mode = None and Tx
> > mode = None.
> > It check if the port can receive and forward some packets.
> > For example 100 packets in this new command.
> > Before testing with "reset_port port_id" and then "reconfig_port
> > port_id", current forwarding should be stopped first to avoid crash.
> >
> 
> Testpmd has mechanism to support reconfigure, you can check the
> need_reconfig Field in port struct.
> No need to define a new command.
> 
> 
> Thanks
> Jingjing

Re: [PATCH v2 0/5] Support NIC reset and keep same port id

[Peng, Yuan]

Tested-by: Peng, Yuan <yuan.peng@intel.com>

- Tested commit c1923afc0999b5b6f109190bc5b69b6c3d334635+the 5 patches.
- OS: 4.5.5-300.fc24.x86_64
- GCC: gcc version 6.1.1 20160510 (Red Hat 6.1.1-2) (GCC)
- CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz
- NIC: Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
- Default x86_64-native-linuxapp-gcc configuration
- Prerequisites:
- Total 1cases, 1 passed, 0 failed

Steps:
DUT:
1. run testpmd with ixgbe VF
echo 1 >/sys/bus/pci/devices/0000:07:00.0/sriov_numvfs
ip link set ens786f0 vf 0 mac 00:11:22:33:44:11
./usertools/dpdk-devbind.py -b vfio-pci 07:10.0
./x86_64-native-linuxapp-gcc/app/testpmd -c 0x3 -n 4 -- -i

2. testpmd > set verbose 1 
  testpmd> set fwd rxonly
  testpmd> start
  tester:
  scapy
  >>> pkt1 = Ether(dst="00:11:22:33:44:11")/IP()/UDP()/Raw('x' * 20)
  >>> sendp(pkt1, iface="ens786f1", count=1)
  Vf can receive the packet correctly.

3. ifconfig ens786f0 down 
  A message is shown in testmd to indicate PF reset:
  Port 0: Interrupt reset event

4. ifconfig ens786f0 up

5. testpmd > stop 

6. testpmd > reset_port 0

7. testpmd> reconfig_port 0
PMD: ixgbevf_dev_configure(): VF can't disable HW CRC Strip
Port 0 MAC: 00 11 22 33 44 11
Begin to forward at least 100 packets to test reconfiguration

8. tester:
>>> sendp(pkt1, iface="ens786f1", count=100)

9. Finish forwarding 100 packets to test reconfiguration
testpmd> show port stats all

  ######################## NIC statistics for port 0  ########################
  RX-packets: 100        RX-missed: 0          RX-bytes:  6200
  RX-errors: 0
  RX-nombuf:  0
  TX-packets: 0          TX-errors: 0          TX-bytes:  0

  Throughput (since last show)
  Rx-pps:            0
  Tx-pps:            0
  ############################################################################

The vf can be reconfigured successfully after pf reset.

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wei Dai
Sent: Tuesday, June 27, 2017 10:07 PM
To: thomas@monjalon.net; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Zhang, Helin <helin.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>
Cc: dev@dpdk.org; Dai, Wei <wei.dai@intel.com>
Subject: [dpdk-dev] [PATCH v2 0/5] Support NIC reset and keep same port id

Sometimes a port have to be reset. After reset, if the port goes through PCI remove() and then PCI probe() for restoration, its port id may be changed and this is not expected by some customer DPDK application. 
Normally, PCI probe() includes two parts: one is in rte_ethdev layer and the other is calling PMD dev_init(). PCI remove( ) release all resource allocated from rte_ethdev layer in PCI probe( ) and calls PMD dev_unit( ).
To keep same port id and reset the port, only dev_uninit() and dev_init( ) in PMD can be called and keep all resources allocated from rte_ethdev layer poart in PCI probe( ).

New rte_eth_dev_reset( ) calls rte_eth_dev_stop( ), PMD dev_uninit( ) and then PMD dev_init( ) to reset a port and keep same port id.
And then application can go through rte_eth_dev_configure( ), rte_eth_rx_queue_setup( ), rte_eth_tx_queue_setup( ) and rte_eth_dev_start( ) again to restore its previous settings or to reconfigure itself with different settings.

To test this new feature, 2 testpmd commands are introduced.
The first command "reset_port port_id" calls rte_eth_dev_reset( ).
The second command "reconfig_port port_id" is used to test if the port can be reconfigured successfully. This command configure the port with the simplest settings include only 1 Rx queue, only 1 Tx queue, Rx mode = None and Tx mode = None. It check port by receving at least
100 packets and then forward these packets from itself.

A typical test steps are listed as follows:
For example, run "ifconfig PF-name down" will trigger a reset to its VF.
1. run testpmd with ixgbe vF
2. testpmd > set verbose 1 //to observe VF working 3. ifconfig name-of-PF down 4. A message is shown in testmd to indicate PF reset 5. ifconfig name-of-PF up 6. testpmd > stop // stop forwarding to avoid crash during reset 7. testpmd > reset_port port_id_of_VF 8. testpmd > reconfig_port port_id_of_VF


Wei Dai (5):
  ethdev: add support of NIC reset
  net/ixgbe: add support of reset
  net/i40e: add support of reset
  app/testpmd: add command to test NIC reset
  app/testpmd: add command to test NIC restoration

 app/test-pmd/cmdline.c                 |  62 ++++++++++++++++++
 app/test-pmd/config.c                  | 111 +++++++++++++++++++++++++++++++++
 app/test-pmd/testpmd.h                 |   2 +
 drivers/net/i40e/i40e_ethdev.c         |  16 +++++
 drivers/net/i40e/i40e_ethdev_vf.c      |  16 +++++
 drivers/net/ixgbe/ixgbe_ethdev.c       |  38 +++++++++++
 lib/librte_ether/rte_ethdev.c          |  17 +++++
 lib/librte_ether/rte_ethdev.h          |  12 ++++
 lib/librte_ether/rte_ether_version.map |   2 +-
 9 files changed, 275 insertions(+), 1 deletion(-)

--
2.7.4

Re: [PATCH v2 0/5] Support NIC reset and keep same port id

[Dai, Wei]

Many thanks to Yuan for your test support.
Your test show the code changes for rte_ether and driver/net can work as expected.
I'll update a v3 patch set according to feedback from Wu Jingjing and your test.
In v3 patch, only code in testpmd will be changed but with similar test method.

> -----Original Message-----
> From: Peng, Yuan
> Sent: Thursday, June 29, 2017 11:52 AM
> To: Dai, Wei <wei.dai@intel.com>; thomas@monjalon.net; Lu, Wenzhuo
> <wenzhuo.lu@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Zhang, Helin <helin.zhang@intel.com>; Wu,
> Jingjing <jingjing.wu@intel.com>
> Cc: dev@dpdk.org; Dai, Wei <wei.dai@intel.com>
> Subject: RE: [dpdk-dev] [PATCH v2 0/5] Support NIC reset and keep same port id
> 
> Tested-by: Peng, Yuan <yuan.peng@intel.com>
> 
> - Tested commit c1923afc0999b5b6f109190bc5b69b6c3d334635+the 5
> patches.
> - OS: 4.5.5-300.fc24.x86_64
> - GCC: gcc version 6.1.1 20160510 (Red Hat 6.1.1-2) (GCC)
> - CPU: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz
> - NIC: Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+
> Network Connection [8086:10fb] (rev 01)
> - Default x86_64-native-linuxapp-gcc configuration
> - Prerequisites:
> - Total 1cases, 1 passed, 0 failed
> 
> Steps:
> DUT:
> 1. run testpmd with ixgbe VF
> echo 1 >/sys/bus/pci/devices/0000:07:00.0/sriov_numvfs
> ip link set ens786f0 vf 0 mac 00:11:22:33:44:11 ./usertools/dpdk-devbind.py -b
> vfio-pci 07:10.0 ./x86_64-native-linuxapp-gcc/app/testpmd -c 0x3 -n 4 -- -i
> 
> 2. testpmd > set verbose 1
>   testpmd> set fwd rxonly
>   testpmd> start
>   tester:
>   scapy
>   >>> pkt1 = Ether(dst="00:11:22:33:44:11")/IP()/UDP()/Raw('x' * 20)
>   >>> sendp(pkt1, iface="ens786f1", count=1)
>   Vf can receive the packet correctly.
> 
> 3. ifconfig ens786f0 down
>   A message is shown in testmd to indicate PF reset:
>   Port 0: Interrupt reset event
> 
> 4. ifconfig ens786f0 up
> 
> 5. testpmd > stop
> 
> 6. testpmd > reset_port 0
> 
> 7. testpmd> reconfig_port 0
> PMD: ixgbevf_dev_configure(): VF can't disable HW CRC Strip Port 0 MAC: 00
> 11 22 33 44 11 Begin to forward at least 100 packets to test reconfiguration
> 
> 8. tester:
> >>> sendp(pkt1, iface="ens786f1", count=100)
> 
> 9. Finish forwarding 100 packets to test reconfiguration
> testpmd> show port stats all
> 
>   ######################## NIC statistics for port 0
> ########################
>   RX-packets: 100        RX-missed: 0          RX-bytes:  6200
>   RX-errors: 0
>   RX-nombuf:  0
>   TX-packets: 0          TX-errors: 0          TX-bytes:  0
> 
>   Throughput (since last show)
>   Rx-pps:            0
>   Tx-pps:            0
> 
> ################################################################
> ############
> 
> The vf can be reconfigured successfully after pf reset.
> 
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wei Dai
> Sent: Tuesday, June 27, 2017 10:07 PM
> To: thomas@monjalon.net; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev,
> Konstantin <konstantin.ananyev@intel.com>; Zhang, Helin
> <helin.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>
> Cc: dev@dpdk.org; Dai, Wei <wei.dai@intel.com>
> Subject: [dpdk-dev] [PATCH v2 0/5] Support NIC reset and keep same port id
> 
> Sometimes a port have to be reset. After reset, if the port goes through PCI
> remove() and then PCI probe() for restoration, its port id may be changed and
> this is not expected by some customer DPDK application.
> Normally, PCI probe() includes two parts: one is in rte_ethdev layer and the
> other is calling PMD dev_init(). PCI remove( ) release all resource allocated
> from rte_ethdev layer in PCI probe( ) and calls PMD dev_unit( ).
> To keep same port id and reset the port, only dev_uninit() and dev_init( ) in
> PMD can be called and keep all resources allocated from rte_ethdev layer poart
> in PCI probe( ).
> 
> New rte_eth_dev_reset( ) calls rte_eth_dev_stop( ), PMD dev_uninit( ) and
> then PMD dev_init( ) to reset a port and keep same port id.
> And then application can go through rte_eth_dev_configure( ),
> rte_eth_rx_queue_setup( ), rte_eth_tx_queue_setup( ) and
> rte_eth_dev_start( ) again to restore its previous settings or to reconfigure
> itself with different settings.
> 
> To test this new feature, 2 testpmd commands are introduced.
> The first command "reset_port port_id" calls rte_eth_dev_reset( ).
> The second command "reconfig_port port_id" is used to test if the port can be
> reconfigured successfully. This command configure the port with the simplest
> settings include only 1 Rx queue, only 1 Tx queue, Rx mode = None and Tx mode
> = None. It check port by receving at least
> 100 packets and then forward these packets from itself.
> 
> A typical test steps are listed as follows:
> For example, run "ifconfig PF-name down" will trigger a reset to its VF.
> 1. run testpmd with ixgbe vF
> 2. testpmd > set verbose 1 //to observe VF working 3. ifconfig name-of-PF down
> 4. A message is shown in testmd to indicate PF reset 5. ifconfig name-of-PF up 6.
> testpmd > stop // stop forwarding to avoid crash during reset 7. testpmd >
> reset_port port_id_of_VF 8. testpmd > reconfig_port port_id_of_VF
> 
> 
> Wei Dai (5):
>   ethdev: add support of NIC reset
>   net/ixgbe: add support of reset
>   net/i40e: add support of reset
>   app/testpmd: add command to test NIC reset
>   app/testpmd: add command to test NIC restoration
> 
>  app/test-pmd/cmdline.c                 |  62 ++++++++++++++++++
>  app/test-pmd/config.c                  | 111
> +++++++++++++++++++++++++++++++++
>  app/test-pmd/testpmd.h                 |   2 +
>  drivers/net/i40e/i40e_ethdev.c         |  16 +++++
>  drivers/net/i40e/i40e_ethdev_vf.c      |  16 +++++
>  drivers/net/ixgbe/ixgbe_ethdev.c       |  38 +++++++++++
>  lib/librte_ether/rte_ethdev.c          |  17 +++++
>  lib/librte_ether/rte_ethdev.h          |  12 ++++
>  lib/librte_ether/rte_ether_version.map |   2 +-
>  9 files changed, 275 insertions(+), 1 deletion(-)
> 
> --
> 2.7.4

[PATCH v3 0/4] Support NIC reset and keep same port id

[Wei Dai]

Sometimes a port have to be reset. After reset, if the port goes
through PCI remove() and then PCI probe() for restoration, its
port id may be changed and this is not expected by some customer
DPDK application.
Normally, PCI probe() includes two parts: one is in rte_ethdev layer
and the other is calling PMD dev_init(). PCI remove( ) release all
resource allocated from rte_ethdev layer in PCI probe( ) and calls
PMD dev_unit( ).
To keep same port id and reset the port, only dev_uninit() and
dev_init( ) in PMD can be called and keep all resources allocated
from rte_ethdev layer poart in PCI probe( ).

New rte_eth_dev_reset( ) calls rte_eth_dev_stop( ), PMD dev_uninit( )
and then PMD dev_init( ) to reset a port and keep same port id.
And then application can go through rte_eth_dev_configure( ),
rte_eth_rx_queue_setup( ), rte_eth_tx_queue_setup( ) and
rte_eth_dev_start( ) again to restore its previous settings or
to reconfigure itself with different settings.

To test this new feature, a testpmd command "port reset port_id" is added.
 
A typical test steps are listed as follows:
For example, run "ifconfig PF-name down" will trigger a reset to its VF.
1. run testpmd with ixgbe VF
2. testpmd > set verbose 1 //to observe VF working
3. ifconfig name-of-PF down
4. A message is shown in testmd to indicate PF reset
5. ifconfig name-of-PF up
6. testpmd > stop // stop forwarding to avoid crash during reset
7. testpmd > port reset port_id_of_VF
8. testpmd > start // restore forwarding

chagnes:
v3: 
  update testpmd command
v2:
  only reset PMD layer resource and keep same port id, but
  not restore settings

Wei Dai (4):
  ethdev: add support of NIC reset
  net/ixgbe: add support of reset
  net/i40e: add support of reset
  app/testpmd: enhance command to test NIC reset

 app/test-pmd/cmdline.c                 | 10 ++++++---
 app/test-pmd/testpmd.c                 | 39 ++++++++++++++++++++++++++++++++++
 app/test-pmd/testpmd.h                 |  1 +
 drivers/net/i40e/i40e_ethdev.c         | 16 ++++++++++++++
 drivers/net/i40e/i40e_ethdev_vf.c      | 16 ++++++++++++++
 drivers/net/ixgbe/ixgbe_ethdev.c       | 38 +++++++++++++++++++++++++++++++++
 lib/librte_ether/rte_ethdev.c          | 16 ++++++++++++++
 lib/librte_ether/rte_ethdev.h          | 12 +++++++++++
 lib/librte_ether/rte_ether_version.map |  2 +-
 9 files changed, 146 insertions(+), 4 deletions(-)

-- 
2.9.3

[PATCH v3 1/4] ethdev: add support of NIC reset

[Wei Dai]

A DPDK application can reset a NIC and keep its port id afterwards.
It means that all SW resources allocated in ethdev layer should be
kept and  SW and HW resources of the NIC in PMD need to be reset
in similar way that it runs PCI dev_uninit() and then dev_init().
The sequence of dev_uninit() and dev_init() can be packed into a
single interface API rte_eth_dev_reset().

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 lib/librte_ether/rte_ethdev.c          | 16 ++++++++++++++++
 lib/librte_ether/rte_ethdev.h          | 12 ++++++++++++
 lib/librte_ether/rte_ether_version.map |  2 +-
 3 files changed, 29 insertions(+), 1 deletion(-)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index 81a45c0..06b8839 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -3349,3 +3349,19 @@ rte_eth_dev_l2_tunnel_offload_set(uint8_t port_id,
 				-ENOTSUP);
 	return (*dev->dev_ops->l2_tunnel_offload_set)(dev, l2_tunnel, mask, en);
 }
+
+int
+rte_eth_dev_reset(uint8_t port_id)
+{
+	struct rte_eth_dev *dev;
+	int ret;
+
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
+	dev = &rte_eth_devices[port_id];
+
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_reset, -ENOTSUP);
+
+	rte_eth_dev_stop(port_id);
+	ret = dev->dev_ops->dev_reset(dev);
+	return ret;
+}
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index f6e6c74..4922b36 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1087,6 +1087,9 @@ typedef int  (*eth_dev_set_link_down_t)(struct rte_eth_dev *dev);
 typedef void (*eth_dev_close_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to close a configured Ethernet device. */
 
+typedef int (*eth_dev_reset_t)(struct rte_eth_dev *dev);
+/** <@internal Function used to reset a configured Ethernet device. */
+
 typedef void (*eth_promiscuous_enable_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to enable the RX promiscuous mode of an Ethernet device. */
 
@@ -1404,6 +1407,7 @@ struct eth_dev_ops {
 	eth_dev_set_link_up_t      dev_set_link_up;   /**< Device link up. */
 	eth_dev_set_link_down_t    dev_set_link_down; /**< Device link down. */
 	eth_dev_close_t            dev_close;     /**< Close device. */
+	eth_dev_reset_t		   dev_reset;	   /**<	Reset device. */
 	eth_link_update_t          link_update;   /**< Get device link state. */
 
 	eth_promiscuous_enable_t   promiscuous_enable; /**< Promiscuous ON. */
@@ -2104,6 +2108,14 @@ int rte_eth_dev_set_link_down(uint8_t port_id);
 void rte_eth_dev_close(uint8_t port_id);
 
 /**
+ * Reset a Ethernet device.
+ *
+ * @param port_id
+ *   The port identifier of the Ethernet device.
+ */
+int rte_eth_dev_reset(uint8_t port_id);
+
+/**
  * Enable receipt in promiscuous mode for an Ethernet device.
  *
  * @param port_id
diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map
index e0881f0..d569bcb 100644
--- a/lib/librte_ether/rte_ether_version.map
+++ b/lib/librte_ether/rte_ether_version.map
@@ -153,5 +153,5 @@ DPDK_17.08 {
 	global:
 
 	rte_flow_isolate;
-
+	rte_eth_dev_reset;
 } DPDK_17.05;
-- 
2.9.3

[PATCH v3 2/4] net/ixgbe: add support of reset

[Wei Dai]

Reset a NIC by calling dev_uninit and then dev_init.
Go through same way in NIC PCI remove without release of
ethdev resource and then NIC PCI probe function without
ethdev resource allocation.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 drivers/net/ixgbe/ixgbe_ethdev.c | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index ebc5467..5ff8fbe 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -170,6 +170,7 @@ static void ixgbe_dev_stop(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_up(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_down(struct rte_eth_dev *dev);
 static void ixgbe_dev_close(struct rte_eth_dev *dev);
+static int  ixgbe_dev_reset(struct rte_eth_dev *dev);
 static void ixgbe_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void ixgbe_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void ixgbe_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -264,6 +265,7 @@ static int  ixgbevf_dev_configure(struct rte_eth_dev *dev);
 static int  ixgbevf_dev_start(struct rte_eth_dev *dev);
 static void ixgbevf_dev_stop(struct rte_eth_dev *dev);
 static void ixgbevf_dev_close(struct rte_eth_dev *dev);
+static int  ixgbevf_dev_reset(struct rte_eth_dev *dev);
 static void ixgbevf_intr_disable(struct ixgbe_hw *hw);
 static void ixgbevf_intr_enable(struct ixgbe_hw *hw);
 static void ixgbevf_dev_stats_get(struct rte_eth_dev *dev,
@@ -525,6 +527,7 @@ static const struct eth_dev_ops ixgbe_eth_dev_ops = {
 	.dev_set_link_up    = ixgbe_dev_set_link_up,
 	.dev_set_link_down  = ixgbe_dev_set_link_down,
 	.dev_close            = ixgbe_dev_close,
+	.dev_reset            = ixgbe_dev_reset,
 	.promiscuous_enable   = ixgbe_dev_promiscuous_enable,
 	.promiscuous_disable  = ixgbe_dev_promiscuous_disable,
 	.allmulticast_enable  = ixgbe_dev_allmulticast_enable,
@@ -614,6 +617,7 @@ static const struct eth_dev_ops ixgbevf_eth_dev_ops = {
 	.xstats_reset         = ixgbevf_dev_stats_reset,
 	.xstats_get_names     = ixgbevf_dev_xstats_get_names,
 	.dev_close            = ixgbevf_dev_close,
+	.dev_reset            = ixgbevf_dev_reset,
 	.allmulticast_enable  = ixgbevf_dev_allmulticast_enable,
 	.allmulticast_disable = ixgbevf_dev_allmulticast_disable,
 	.dev_infos_get        = ixgbevf_dev_info_get,
@@ -2838,6 +2842,23 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 	ixgbe_set_rar(hw, 0, hw->mac.addr, 0, IXGBE_RAH_AV);
 }
 
+/*
+ * Reest PF device.
+ */
+static int
+ixgbe_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	ret = eth_ixgbe_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_ixgbe_dev_init(dev);
+
+	return ret;
+}
+
 static void
 ixgbe_read_stats_registers(struct ixgbe_hw *hw,
 			   struct ixgbe_hw_stats *hw_stats,
@@ -4902,6 +4923,23 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	ixgbevf_remove_mac_addr(dev, 0);
 }
 
+/*
+ * Reset VF device
+ */
+static int
+ixgbevf_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	ret = eth_ixgbevf_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_ixgbevf_dev_init(dev);
+
+	return ret;
+}
+
 static void ixgbevf_set_vfta_all(struct rte_eth_dev *dev, bool on)
 {
 	struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-- 
2.9.3

[PATCH v3 3/4] net/i40e: add support of reset

[Wei Dai]

Reset a NIC by calling dev_uninit() and then dev_init().
Go through the same way in NIC PCI remove without release
of ethdev resource and then NIC PCI probe function without
ethdev resource allocation.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 drivers/net/i40e/i40e_ethdev.c    | 16 ++++++++++++++++
 drivers/net/i40e/i40e_ethdev_vf.c | 16 ++++++++++++++++
 2 files changed, 32 insertions(+)

diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 4ee1113..7694957 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -250,6 +250,7 @@ static int i40e_dev_configure(struct rte_eth_dev *dev);
 static int i40e_dev_start(struct rte_eth_dev *dev);
 static void i40e_dev_stop(struct rte_eth_dev *dev);
 static void i40e_dev_close(struct rte_eth_dev *dev);
+static int  i40e_dev_reset(struct rte_eth_dev *dev);
 static void i40e_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void i40e_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void i40e_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -449,6 +450,7 @@ static const struct eth_dev_ops i40e_eth_dev_ops = {
 	.dev_start                    = i40e_dev_start,
 	.dev_stop                     = i40e_dev_stop,
 	.dev_close                    = i40e_dev_close,
+	.dev_reset                    = i40e_dev_reset,
 	.promiscuous_enable           = i40e_dev_promiscuous_enable,
 	.promiscuous_disable          = i40e_dev_promiscuous_disable,
 	.allmulticast_enable          = i40e_dev_allmulticast_enable,
@@ -2135,6 +2137,20 @@ i40e_dev_close(struct rte_eth_dev *dev)
 	I40E_WRITE_FLUSH(hw);
 }
 
+static int
+i40e_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	ret = eth_i40e_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_i40e_dev_init(dev);
+
+	return ret;
+}
+
 static void
 i40e_dev_promiscuous_enable(struct rte_eth_dev *dev)
 {
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 2d5a9b5..bf287a0 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -123,6 +123,7 @@ static void i40evf_vlan_offload_set(struct rte_eth_dev *dev, int mask);
 static int i40evf_vlan_pvid_set(struct rte_eth_dev *dev, uint16_t pvid,
 				int on);
 static void i40evf_dev_close(struct rte_eth_dev *dev);
+static int  i40evf_dev_reset(struct rte_eth_dev *dev);
 static void i40evf_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void i40evf_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void i40evf_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -204,6 +205,7 @@ static const struct eth_dev_ops i40evf_eth_dev_ops = {
 	.xstats_get_names     = i40evf_dev_xstats_get_names,
 	.xstats_reset         = i40evf_dev_xstats_reset,
 	.dev_close            = i40evf_dev_close,
+	.dev_reset            = i40evf_dev_reset,
 	.dev_infos_get        = i40evf_dev_info_get,
 	.dev_supported_ptypes_get = i40e_dev_supported_ptypes_get,
 	.vlan_filter_set      = i40evf_vlan_filter_set,
@@ -2347,6 +2349,20 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 }
 
 static int
+i40evf_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	ret = i40evf_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = i40evf_dev_init(dev);
+
+	return ret;
+}
+
+static int
 i40evf_get_rss_lut(struct i40e_vsi *vsi, uint8_t *lut, uint16_t lut_size)
 {
 	struct i40e_vf *vf = I40E_VSI_TO_VF(vsi);
-- 
2.9.3

[PATCH v3 4/4] app/testpmd: enhance command to test NIC reset

[Wei Dai]

When PF is reset, a message will show it.
User can run the command "port reset port_id"
to reset the VF port and to keep same port id without any
without any configuration.
And then user can run "start port_id" to reconfigure
its forwarding mode and parmaters as previous.
To avoid crash, current forwarding should be stopped.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 app/test-pmd/cmdline.c | 10 +++++++---
 app/test-pmd/testpmd.c | 39 +++++++++++++++++++++++++++++++++++++++
 app/test-pmd/testpmd.h |  1 +
 3 files changed, 47 insertions(+), 3 deletions(-)

diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
index ff8ffd2..58ba6e4 100644
--- a/app/test-pmd/cmdline.c
+++ b/app/test-pmd/cmdline.c
@@ -950,6 +950,8 @@ static void cmd_operate_port_parsed(void *parsed_result,
 		stop_port(RTE_PORT_ALL);
 	else if (!strcmp(res->name, "close"))
 		close_port(RTE_PORT_ALL);
+	else if (!strcmp(res->name, "reset"))
+		reset_port(RTE_PORT_ALL);
 	else
 		printf("Unknown parameter\n");
 }
@@ -959,7 +961,7 @@ cmdline_parse_token_string_t cmd_operate_port_all_cmd =
 								"port");
 cmdline_parse_token_string_t cmd_operate_port_all_port =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_port_result, name,
-						"start#stop#close");
+						"start#stop#close#reset");
 cmdline_parse_token_string_t cmd_operate_port_all_all =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_port_result, value, "all");
 
@@ -994,6 +996,8 @@ static void cmd_operate_specific_port_parsed(void *parsed_result,
 		stop_port(res->value);
 	else if (!strcmp(res->name, "close"))
 		close_port(res->value);
+	else if (!strcmp(res->name, "reset"))
+		reset_port(res->value);
 	else
 		printf("Unknown parameter\n");
 }
@@ -1003,7 +1007,7 @@ cmdline_parse_token_string_t cmd_operate_specific_port_cmd =
 							keyword, "port");
 cmdline_parse_token_string_t cmd_operate_specific_port_port =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_specific_port_result,
-						name, "start#stop#close");
+						name, "start#stop#close#reset");
 cmdline_parse_token_num_t cmd_operate_specific_port_id =
 	TOKEN_NUM_INITIALIZER(struct cmd_operate_specific_port_result,
 							value, UINT8);
@@ -1011,7 +1015,7 @@ cmdline_parse_token_num_t cmd_operate_specific_port_id =
 cmdline_parse_inst_t cmd_operate_specific_port = {
 	.f = cmd_operate_specific_port_parsed,
 	.data = NULL,
-	.help_str = "port start|stop|close <port_id>: Start/Stop/Close port_id",
+	.help_str = "port start|stop|close|reset <port_id>: Start/Stop/Close/Reset port_id",
 	.tokens = {
 		(void *)&cmd_operate_specific_port_cmd,
 		(void *)&cmd_operate_specific_port_port,
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index b29328a..77a517f 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -1665,6 +1665,45 @@ close_port(portid_t pid)
 }
 
 void
+reset_port(portid_t pid)
+{
+	int diag;
+	portid_t pi;
+	struct rte_port *port;
+
+	if (port_id_is_invalid(pid, ENABLED_WARN))
+		return;
+
+	printf("Reseting ports...\n");
+
+	RTE_ETH_FOREACH_DEV(pi) {
+		if (pid != pi && pid != (portid_t)RTE_PORT_ALL)
+			continue;
+
+		if (port_is_forwarding(pi) != 0 && test_done == 0) {
+			printf("Please remove port %d from forwarding configuration.\n", pi);
+			continue;
+		}
+
+		if (port_is_bonding_slave(pi)) {
+			printf("Please remove port %d from bonded device.\n", pi);
+			continue;
+		}
+
+		diag = rte_eth_dev_reset(pi);
+		if (diag == 0) {
+			port = &ports[pi];
+			port->need_reconfig = 1;
+			port->need_reconfig_queues = 1;
+		}
+		else
+			printf("Failed to reset port %d. diag=%d\n", pi, diag);
+	}
+
+	printf("Done\n");
+}
+
+void
 attach_port(char *identifier)
 {
 	portid_t pi = 0;
diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
index 364502d..e4c704a 100644
--- a/app/test-pmd/testpmd.h
+++ b/app/test-pmd/testpmd.h
@@ -596,6 +596,7 @@ int init_port_dcb_config(portid_t pid, enum dcb_mode_enable dcb_mode,
 int start_port(portid_t pid);
 void stop_port(portid_t pid);
 void close_port(portid_t pid);
+void reset_port(portid_t pid);
 void attach_port(char *identifier);
 void detach_port(uint8_t port_id);
 int all_ports_stopped(void);
-- 
2.9.3

[PATCH v4 0/5] Support NIC reset and keep same port id

[Wei Dai]

Sometimes a port have to be reset. For example PF is reset, all its
VF should also be reset. After reset, if the port goes
through PCI remove() and then PCI probe() for restoration, its
port id may be changed and this is not expected by some customer
DPDK application.
Normally, PCI probe() includes two parts: one is in rte_ethdev layer
and the other is calling PMD dev_init(). PCI remove( ) release all
resource allocated from rte_ethdev layer in PCI probe( ) and calls
PMD dev_unit( ).
To keep same port id and reset the port, only dev_uninit() and
dev_init( ) in PMD can be called and keep all resources allocated
from rte_ethdev layer poart in PCI probe( ).

New rte_eth_dev_reset( ) calls rte_eth_dev_stop( ), PMD dev_uninit( )
and then PMD dev_init( ) to reset a port and keep same port id.
And then application can go through rte_eth_dev_configure( ),
rte_eth_rx_queue_setup( ), rte_eth_tx_queue_setup( ) and
rte_eth_dev_start( ) again to restore its previous settings or
to reconfigure itself with different settings.

To test this new feature, a testpmd command "port reset port_id" is added.
The mapping between port number and its PCI address can be monitored to
confirm its port number is kept.
And following test case can also be used to confirm the port can work
again after reset.

A typical test steps are listed as follows:
For example, run "ifconfig PF-name down" will trigger a reset to VF.
1.  run testpmd with 2 ixgbe VF ports belonging to same PF
2.  testpmd > set verbose 1 //to observe VF working
3.  testpmd > show port info all //show port number, PCI addr and MAC addr
4.  testpmd > start
5.  let all ports forwarding work for a while
6.  testpmd > show port stats all
7.  ifconfig name-of-PF down
8.  A message is shown in testmd to indicate PF reset
9.  ifconfig name-of-PF up
10. testpmd > stop // stop forwarding to avoid crash during reset
11. testpmd > port reset all
12. testpmd > port stop all
13. testpmd > port start all //recofnig all ports
14. testpmd > show port info all
    //confirm same mapping of port id and PCI addr
15. testpmd > start // restore forwarding
14. let all ports forwarding work for a while
15. testpmd > show port stats all //confirm all port can work again
16. repeat above step 7 - 15

chagnes:
v4:
  add PCI address to confirm its port number keep same
  correct test method in cover letter
v3:
  update testpmd command
v2:
  only reset PMD layer resource and keep same port id, but
  not restore settings


Wei Dai (5):
  ethdev: add support of NIC reset
  net/ixgbe: add support of reset
  net/i40e: add support of reset
  app/testpmd: display PCI address in port info
  app/testpmd: enhance command to test NIC reset

 app/test-pmd/cmdline.c                 | 10 ++++--
 app/test-pmd/config.c                  |  5 +++
 app/test-pmd/testpmd.c                 | 61 ++++++++++++++++++++++++++++++++--
 app/test-pmd/testpmd.h                 |  1 +
 drivers/net/i40e/i40e_ethdev.c         | 16 +++++++++
 drivers/net/i40e/i40e_ethdev_vf.c      | 16 +++++++++
 drivers/net/ixgbe/ixgbe_ethdev.c       | 38 +++++++++++++++++++++
 lib/librte_ether/rte_ethdev.c          | 16 +++++++++
 lib/librte_ether/rte_ethdev.h          | 12 +++++++
 lib/librte_ether/rte_ether_version.map |  2 +-
 10 files changed, 170 insertions(+), 7 deletions(-)

-- 
2.7.4

[PATCH v4 1/5] ethdev: add support of NIC reset

[Wei Dai]

A DPDK application can reset a NIC and keep its port id afterwards.
It means that all SW resources allocated in ethdev layer should be
kept and  SW and HW resources of the NIC in PMD need to be reset
in similar way that it runs PCI dev_uninit() and then dev_init().
The sequence of dev_uninit() and dev_init() can be packed into a
single interface API rte_eth_dev_reset().

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 lib/librte_ether/rte_ethdev.c          | 16 ++++++++++++++++
 lib/librte_ether/rte_ethdev.h          | 12 ++++++++++++
 lib/librte_ether/rte_ether_version.map |  2 +-
 3 files changed, 29 insertions(+), 1 deletion(-)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index 81a45c0..06b8839 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -3349,3 +3349,19 @@ rte_eth_dev_l2_tunnel_offload_set(uint8_t port_id,
 				-ENOTSUP);
 	return (*dev->dev_ops->l2_tunnel_offload_set)(dev, l2_tunnel, mask, en);
 }
+
+int
+rte_eth_dev_reset(uint8_t port_id)
+{
+	struct rte_eth_dev *dev;
+	int ret;
+
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
+	dev = &rte_eth_devices[port_id];
+
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_reset, -ENOTSUP);
+
+	rte_eth_dev_stop(port_id);
+	ret = dev->dev_ops->dev_reset(dev);
+	return ret;
+}
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index f6e6c74..4922b36 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1087,6 +1087,9 @@ typedef int  (*eth_dev_set_link_down_t)(struct rte_eth_dev *dev);
 typedef void (*eth_dev_close_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to close a configured Ethernet device. */
 
+typedef int (*eth_dev_reset_t)(struct rte_eth_dev *dev);
+/** <@internal Function used to reset a configured Ethernet device. */
+
 typedef void (*eth_promiscuous_enable_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to enable the RX promiscuous mode of an Ethernet device. */
 
@@ -1404,6 +1407,7 @@ struct eth_dev_ops {
 	eth_dev_set_link_up_t      dev_set_link_up;   /**< Device link up. */
 	eth_dev_set_link_down_t    dev_set_link_down; /**< Device link down. */
 	eth_dev_close_t            dev_close;     /**< Close device. */
+	eth_dev_reset_t		   dev_reset;	   /**<	Reset device. */
 	eth_link_update_t          link_update;   /**< Get device link state. */
 
 	eth_promiscuous_enable_t   promiscuous_enable; /**< Promiscuous ON. */
@@ -2104,6 +2108,14 @@ int rte_eth_dev_set_link_down(uint8_t port_id);
 void rte_eth_dev_close(uint8_t port_id);
 
 /**
+ * Reset a Ethernet device.
+ *
+ * @param port_id
+ *   The port identifier of the Ethernet device.
+ */
+int rte_eth_dev_reset(uint8_t port_id);
+
+/**
  * Enable receipt in promiscuous mode for an Ethernet device.
  *
  * @param port_id
diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map
index e0881f0..d569bcb 100644
--- a/lib/librte_ether/rte_ether_version.map
+++ b/lib/librte_ether/rte_ether_version.map
@@ -153,5 +153,5 @@ DPDK_17.08 {
 	global:
 
 	rte_flow_isolate;
-
+	rte_eth_dev_reset;
 } DPDK_17.05;
-- 
2.7.4

[PATCH v4 2/5] net/ixgbe: add support of reset

[Wei Dai]

Reset a NIC by calling dev_uninit and then dev_init.
Go through same way in NIC PCI remove without release of
ethdev resource and then NIC PCI probe function without
ethdev resource allocation.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 drivers/net/ixgbe/ixgbe_ethdev.c | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index ebc5467..5ff8fbe 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -170,6 +170,7 @@ static void ixgbe_dev_stop(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_up(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_down(struct rte_eth_dev *dev);
 static void ixgbe_dev_close(struct rte_eth_dev *dev);
+static int  ixgbe_dev_reset(struct rte_eth_dev *dev);
 static void ixgbe_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void ixgbe_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void ixgbe_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -264,6 +265,7 @@ static int  ixgbevf_dev_configure(struct rte_eth_dev *dev);
 static int  ixgbevf_dev_start(struct rte_eth_dev *dev);
 static void ixgbevf_dev_stop(struct rte_eth_dev *dev);
 static void ixgbevf_dev_close(struct rte_eth_dev *dev);
+static int  ixgbevf_dev_reset(struct rte_eth_dev *dev);
 static void ixgbevf_intr_disable(struct ixgbe_hw *hw);
 static void ixgbevf_intr_enable(struct ixgbe_hw *hw);
 static void ixgbevf_dev_stats_get(struct rte_eth_dev *dev,
@@ -525,6 +527,7 @@ static const struct eth_dev_ops ixgbe_eth_dev_ops = {
 	.dev_set_link_up    = ixgbe_dev_set_link_up,
 	.dev_set_link_down  = ixgbe_dev_set_link_down,
 	.dev_close            = ixgbe_dev_close,
+	.dev_reset            = ixgbe_dev_reset,
 	.promiscuous_enable   = ixgbe_dev_promiscuous_enable,
 	.promiscuous_disable  = ixgbe_dev_promiscuous_disable,
 	.allmulticast_enable  = ixgbe_dev_allmulticast_enable,
@@ -614,6 +617,7 @@ static const struct eth_dev_ops ixgbevf_eth_dev_ops = {
 	.xstats_reset         = ixgbevf_dev_stats_reset,
 	.xstats_get_names     = ixgbevf_dev_xstats_get_names,
 	.dev_close            = ixgbevf_dev_close,
+	.dev_reset            = ixgbevf_dev_reset,
 	.allmulticast_enable  = ixgbevf_dev_allmulticast_enable,
 	.allmulticast_disable = ixgbevf_dev_allmulticast_disable,
 	.dev_infos_get        = ixgbevf_dev_info_get,
@@ -2838,6 +2842,23 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 	ixgbe_set_rar(hw, 0, hw->mac.addr, 0, IXGBE_RAH_AV);
 }
 
+/*
+ * Reest PF device.
+ */
+static int
+ixgbe_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	ret = eth_ixgbe_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_ixgbe_dev_init(dev);
+
+	return ret;
+}
+
 static void
 ixgbe_read_stats_registers(struct ixgbe_hw *hw,
 			   struct ixgbe_hw_stats *hw_stats,
@@ -4902,6 +4923,23 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	ixgbevf_remove_mac_addr(dev, 0);
 }
 
+/*
+ * Reset VF device
+ */
+static int
+ixgbevf_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	ret = eth_ixgbevf_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_ixgbevf_dev_init(dev);
+
+	return ret;
+}
+
 static void ixgbevf_set_vfta_all(struct rte_eth_dev *dev, bool on)
 {
 	struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-- 
2.7.4

[PATCH v4 3/5] net/i40e: add support of reset

[Wei Dai]

Reset a NIC by calling dev_uninit() and then dev_init().
Go through the same way in NIC PCI remove without release
of ethdev resource and then NIC PCI probe function without
ethdev resource allocation.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 drivers/net/i40e/i40e_ethdev.c    | 16 ++++++++++++++++
 drivers/net/i40e/i40e_ethdev_vf.c | 16 ++++++++++++++++
 2 files changed, 32 insertions(+)

diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 4ee1113..7694957 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -250,6 +250,7 @@ static int i40e_dev_configure(struct rte_eth_dev *dev);
 static int i40e_dev_start(struct rte_eth_dev *dev);
 static void i40e_dev_stop(struct rte_eth_dev *dev);
 static void i40e_dev_close(struct rte_eth_dev *dev);
+static int  i40e_dev_reset(struct rte_eth_dev *dev);
 static void i40e_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void i40e_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void i40e_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -449,6 +450,7 @@ static const struct eth_dev_ops i40e_eth_dev_ops = {
 	.dev_start                    = i40e_dev_start,
 	.dev_stop                     = i40e_dev_stop,
 	.dev_close                    = i40e_dev_close,
+	.dev_reset                    = i40e_dev_reset,
 	.promiscuous_enable           = i40e_dev_promiscuous_enable,
 	.promiscuous_disable          = i40e_dev_promiscuous_disable,
 	.allmulticast_enable          = i40e_dev_allmulticast_enable,
@@ -2135,6 +2137,20 @@ i40e_dev_close(struct rte_eth_dev *dev)
 	I40E_WRITE_FLUSH(hw);
 }
 
+static int
+i40e_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	ret = eth_i40e_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_i40e_dev_init(dev);
+
+	return ret;
+}
+
 static void
 i40e_dev_promiscuous_enable(struct rte_eth_dev *dev)
 {
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 2d5a9b5..bf287a0 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -123,6 +123,7 @@ static void i40evf_vlan_offload_set(struct rte_eth_dev *dev, int mask);
 static int i40evf_vlan_pvid_set(struct rte_eth_dev *dev, uint16_t pvid,
 				int on);
 static void i40evf_dev_close(struct rte_eth_dev *dev);
+static int  i40evf_dev_reset(struct rte_eth_dev *dev);
 static void i40evf_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void i40evf_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void i40evf_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -204,6 +205,7 @@ static const struct eth_dev_ops i40evf_eth_dev_ops = {
 	.xstats_get_names     = i40evf_dev_xstats_get_names,
 	.xstats_reset         = i40evf_dev_xstats_reset,
 	.dev_close            = i40evf_dev_close,
+	.dev_reset            = i40evf_dev_reset,
 	.dev_infos_get        = i40evf_dev_info_get,
 	.dev_supported_ptypes_get = i40e_dev_supported_ptypes_get,
 	.vlan_filter_set      = i40evf_vlan_filter_set,
@@ -2347,6 +2349,20 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 }
 
 static int
+i40evf_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	ret = i40evf_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = i40evf_dev_init(dev);
+
+	return ret;
+}
+
+static int
 i40evf_get_rss_lut(struct i40e_vsi *vsi, uint8_t *lut, uint16_t lut_size)
 {
 	struct i40e_vf *vf = I40E_VSI_TO_VF(vsi);
-- 
2.7.4

Re: [PATCH v4 3/5] net/i40e: add support of reset

[Wu, Jingjing]

>  }
> 
> +static int
> +i40e_dev_reset(struct rte_eth_dev *dev)
> +{
> +	int ret;
> +
> +	ret = eth_i40e_dev_uninit(dev);
> +	if (ret)
> +		return ret;
> +
> +	ret = eth_i40e_dev_init(dev);
> +
> +	return ret;
Have you thought about if DPDK is working as host driver, and there
Are VFs rely on it? If it doesn't support, at least a check should be added.
Or you can add those check in Rte level.

Thanks
Jingjing

Re: [PATCH v4 3/5] net/i40e: add support of reset

[Dai, Wei]

> -----Original Message-----
> From: Wu, Jingjing
> Sent: Friday, June 30, 2017 4:54 PM
> To: Dai, Wei <wei.dai@intel.com>; thomas@monjalon.net; Lu, Wenzhuo
> <wenzhuo.lu@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Zhang, Helin <helin.zhang@intel.com>;
> Peng, Yuan <yuan.peng@intel.com>
> Cc: dev@dpdk.org
> Subject: RE: [PATCH v4 3/5] net/i40e: add support of reset
> 
> >  }
> >
> > +static int
> > +i40e_dev_reset(struct rte_eth_dev *dev) {
> > +	int ret;
> > +
> > +	ret = eth_i40e_dev_uninit(dev);
> > +	if (ret)
> > +		return ret;
> > +
> > +	ret = eth_i40e_dev_init(dev);
> > +
> > +	return ret;
> Have you thought about if DPDK is working as host driver, and there Are VFs
> rely on it? If it doesn't support, at least a check should be added.
> Or you can add those check in Rte level.
If there is any VF, PF reset need sync with VF, but sync method may be much different with different NIC type.
So, we'd like first reject PF reset if its VF exist.
I'll update it in v5 patch set.

> 
> Thanks
> Jingjing

[PATCH v4 4/5] app/testpmd: display PCI address in port info

[Wei Dai]

Add the PCI address when running "show port info port_id".

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 app/test-pmd/config.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
index b0b340e..c2d5c03 100644
--- a/app/test-pmd/config.c
+++ b/app/test-pmd/config.c
@@ -438,6 +438,11 @@ port_infos_display(portid_t port_id)
 	rte_eth_dev_info_get(port_id, &dev_info);
 	printf("\n%s Infos for port %-2d %s\n",
 	       info_border, port_id, info_border);
+	printf("PCI Address: " PCI_PRI_FMT "\n",
+		dev_info.pci_dev->addr.domain,
+		dev_info.pci_dev->addr.bus,
+		dev_info.pci_dev->addr.devid,
+		dev_info.pci_dev->addr.function);
 	rte_eth_macaddr_get(port_id, &mac_addr);
 	print_ethaddr("MAC address: ", &mac_addr);
 	printf("\nDriver name: %s", dev_info.driver_name);
-- 
2.7.4

Re: [PATCH v4 4/5] app/testpmd: display PCI address in port info

[Wu, Jingjing]

> -----Original Message-----
> From: Dai, Wei
> Sent: Thursday, June 29, 2017 10:58 PM
> To: thomas@monjalon.net; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Zhang, Helin <helin.zhang@intel.com>; Wu, Jingjing
> <jingjing.wu@intel.com>; yuan.pntel.com
> Cc: dev@dpdk.org; Dai, Wei <wei.dai@intel.com>
> Subject: [PATCH v4 4/5] app/testpmd: display PCI address in port info
> 
> Add the PCI address when running "show port info port_id".
> 
> Signed-off-by: Wei Dai <wei.dai@intel.com>
> ---
>  app/test-pmd/config.c | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
> index b0b340e..c2d5c03 100644
> --- a/app/test-pmd/config.c
> +++ b/app/test-pmd/config.c
> @@ -438,6 +438,11 @@ port_infos_display(portid_t port_id)
>  	rte_eth_dev_info_get(port_id, &dev_info);
>  	printf("\n%s Infos for port %-2d %s\n",
>  	       info_border, port_id, info_border);
> +	printf("PCI Address: " PCI_PRI_FMT "\n",
> +		dev_info.pci_dev->addr.domain,
> +		dev_info.pci_dev->addr.bus,
> +		dev_info.pci_dev->addr.devid,
> +		dev_info.pci_dev->addr.function);
>  	rte_eth_macaddr_get(port_id, &mac_addr);
>  	print_ethaddr("MAC address: ", &mac_addr);
>  	printf("\nDriver name: %s", dev_info.driver_name);
> --
> 2.7.4

The field pci_dev will probably be removed.
Yulong already submit a patch like this and rejected.
http://www.dpdk.org/dev/patchwork/patch/23899/

So, you can add more info when the Bus refine work is done.
You can drop this change from your patch set.

Thanks
Jingjing

[PATCH v4 5/5] app/testpmd: enhance command to test NIC reset

[Wei Dai]

When PF is reset, a message will show it and all its
VF need to be reset.
User can run the command "port reset port_id"
to reset the VF port and to keep same port id without
any configuration. Then user can run "port stop port_id"
and "port start port_id" to reconfigure its forwarding
mode and parmaters as previous ones.
To avoid crash, current forwarding should be stopped
before running "port reset port_id".

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 app/test-pmd/cmdline.c | 10 ++++++---
 app/test-pmd/testpmd.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++---
 app/test-pmd/testpmd.h |  1 +
 3 files changed, 66 insertions(+), 6 deletions(-)

diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
index ff8ffd2..58ba6e4 100644
--- a/app/test-pmd/cmdline.c
+++ b/app/test-pmd/cmdline.c
@@ -950,6 +950,8 @@ static void cmd_operate_port_parsed(void *parsed_result,
 		stop_port(RTE_PORT_ALL);
 	else if (!strcmp(res->name, "close"))
 		close_port(RTE_PORT_ALL);
+	else if (!strcmp(res->name, "reset"))
+		reset_port(RTE_PORT_ALL);
 	else
 		printf("Unknown parameter\n");
 }
@@ -959,7 +961,7 @@ cmdline_parse_token_string_t cmd_operate_port_all_cmd =
 								"port");
 cmdline_parse_token_string_t cmd_operate_port_all_port =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_port_result, name,
-						"start#stop#close");
+						"start#stop#close#reset");
 cmdline_parse_token_string_t cmd_operate_port_all_all =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_port_result, value, "all");
 
@@ -994,6 +996,8 @@ static void cmd_operate_specific_port_parsed(void *parsed_result,
 		stop_port(res->value);
 	else if (!strcmp(res->name, "close"))
 		close_port(res->value);
+	else if (!strcmp(res->name, "reset"))
+		reset_port(res->value);
 	else
 		printf("Unknown parameter\n");
 }
@@ -1003,7 +1007,7 @@ cmdline_parse_token_string_t cmd_operate_specific_port_cmd =
 							keyword, "port");
 cmdline_parse_token_string_t cmd_operate_specific_port_port =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_specific_port_result,
-						name, "start#stop#close");
+						name, "start#stop#close#reset");
 cmdline_parse_token_num_t cmd_operate_specific_port_id =
 	TOKEN_NUM_INITIALIZER(struct cmd_operate_specific_port_result,
 							value, UINT8);
@@ -1011,7 +1015,7 @@ cmdline_parse_token_num_t cmd_operate_specific_port_id =
 cmdline_parse_inst_t cmd_operate_specific_port = {
 	.f = cmd_operate_specific_port_parsed,
 	.data = NULL,
-	.help_str = "port start|stop|close <port_id>: Start/Stop/Close port_id",
+	.help_str = "port start|stop|close|reset <port_id>: Start/Stop/Close/Reset port_id",
 	.tokens = {
 		(void *)&cmd_operate_specific_port_cmd,
 		(void *)&cmd_operate_specific_port_port,
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index b29328a..7773879 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -1403,6 +1403,7 @@ start_port(portid_t pid)
 	queueid_t qi;
 	struct rte_port *port;
 	struct ether_addr mac_addr;
+	struct rte_eth_dev_info dev_info;
 	enum rte_eth_event_type event_type;
 
 	if (port_id_is_invalid(pid, ENABLED_WARN))
@@ -1424,9 +1425,14 @@ start_port(portid_t pid)
 
 		if (port->need_reconfig > 0) {
 			port->need_reconfig = 0;
-
-			printf("Configuring Port %d (socket %u)\n", pi,
-					port->socket_id);
+			rte_eth_dev_info_get(pi, &dev_info);
+			printf("Configuring Port %d (socket %u) with "
+				"PCI Address: " PCI_PRI_FMT "\n",
+				pi, port->socket_id,
+				dev_info.pci_dev->addr.domain,
+				dev_info.pci_dev->addr.bus,
+				dev_info.pci_dev->addr.devid,
+				dev_info.pci_dev->addr.function);
 			/* configure port */
 			diag = rte_eth_dev_configure(pi, nb_rxq, nb_txq,
 						&(port->dev_conf));
@@ -1665,6 +1671,55 @@ close_port(portid_t pid)
 }
 
 void
+reset_port(portid_t pid)
+{
+	int diag;
+	portid_t pi;
+	struct rte_port *port;
+	struct rte_eth_dev_info dev_info;
+
+	if (port_id_is_invalid(pid, ENABLED_WARN))
+		return;
+
+	printf("Resetting ports...\n");
+
+	RTE_ETH_FOREACH_DEV(pi) {
+		if (pid != pi && pid != (portid_t)RTE_PORT_ALL)
+			continue;
+
+		if (port_is_forwarding(pi) != 0 && test_done == 0) {
+			printf("Please remove port %d from forwarding "
+			       "configuration.\n", pi);
+			continue;
+		}
+
+		if (port_is_bonding_slave(pi)) {
+			printf("Please remove port %d from bonded device.\n",
+			       pi);
+			continue;
+		}
+
+		diag = rte_eth_dev_reset(pi);
+		if (diag == 0) {
+			port = &ports[pi];
+			port->need_reconfig = 1;
+			port->need_reconfig_queues = 1;
+			rte_eth_dev_info_get(pi, &dev_info);
+			printf("Finish resetting Port %d with PCI Address: "
+			       PCI_PRI_FMT "\n", pi,
+			       dev_info.pci_dev->addr.domain,
+			       dev_info.pci_dev->addr.bus,
+			       dev_info.pci_dev->addr.devid,
+			       dev_info.pci_dev->addr.function);
+		} else {
+			printf("Failed to reset port %d. diag=%d\n", pi, diag);
+		}
+	}
+
+	printf("Done\n");
+}
+
+void
 attach_port(char *identifier)
 {
 	portid_t pi = 0;
diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
index 364502d..e4c704a 100644
--- a/app/test-pmd/testpmd.h
+++ b/app/test-pmd/testpmd.h
@@ -596,6 +596,7 @@ int init_port_dcb_config(portid_t pid, enum dcb_mode_enable dcb_mode,
 int start_port(portid_t pid);
 void stop_port(portid_t pid);
 void close_port(portid_t pid);
+void reset_port(portid_t pid);
 void attach_port(char *identifier);
 void detach_port(uint8_t port_id);
 int all_ports_stopped(void);
-- 
2.7.4

Re: [PATCH v4 5/5] app/testpmd: enhance command to test NIC reset

[Peng, Yuan]

Tested-by: Peng, Yuan <yuan.peng@intel.com>

- Tested commit bbe569daa7e99b36d44b12bb3d23ddfbc26d383c+the 5 patches.
- OS: 4.8.6-300.fc25.x86_64
- GCC: gcc version 6.2.1 20160916 (Red Hat 6.2.1-2) (GCC)
- CPU: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz
- NIC: Ethernet controller [0200]: Intel Corporation Ethernet Controller 10G X550T [8086:1563] (rev 01)
- Default x86_64-native-linuxapp-gcc configuration
- Prerequisites:
- Total 1cases, 1 passed, 0 failed

Steps:
DUT:
1. echo 2 >/sys/bus/pci/devices/0000:83:00.0/sriov_numvfs
./usertools/dpdk-devbind.py -b vfio-pci 83:10.0 83:10.2 ./x86_64-native-linuxapp-gcc/app/testpmd -c 0x3 -n 4 -- -i

2.testpmd> set verbose 1
testpmd> set fwd mac
testpmd> start
tester:
scapy
sendp([Ether(dst="02:09:C0:63:DA:4B")/IP()/UDP()], iface="ens6f0", count=1)
dut:
testpmd>show port stats all
port0 can fwd the packet normally
testpmd>show port info all
show port number, PCI addr and MAC addr

3. ifconfig ens801f0 down
Port 0: Interrupt reset event
Port 1: Interrupt reset event
ifconfig ens801f0 up

4.testpmd> stop

5. testpmd> port reset all
Resetting ports...
Finish resetting Port 0 with PCI Address: 0000:83:10.0 Finish resetting Port 1 with PCI Address: 0000:83:10.2 Done

6. testpmd> port stop all
Stopping ports...
Checking link statuses...
Done
testpmd> port start all
Configuring Port 0 (socket 1) with PCI Address: 0000:83:10.0 Port 0: 02:09:C0:63:DA:4B Configuring Port 1 (socket 1) with PCI Address: 0000:83:10.2 Port 1: 02:09:C0:37:93:6F Checking link statuses...
Done

7. testpmd> show port info all
confirm same mapping of port id and PCI address.

8.testpmd> start
Tester:
scapy
sendp([Ether(dst="02:09:C0:63:DA:4B")/IP()/UDP()], iface="ens6f0", count=1)
dut:
testpmd>show port stats all
port0 can fwd the packet normally

9.repeat step3 to step8, the same result.

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wei Dai
Sent: Thursday, June 29, 2017 10:58 PM
To: thomas@monjalon.net; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Zhang, Helin <helin.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; yuan.pntel.com@dpdk.org
Cc: dev@dpdk.org; Dai, Wei <wei.dai@intel.com>
Subject: [dpdk-dev] [PATCH v4 5/5] app/testpmd: enhance command to test NIC reset

When PF is reset, a message will show it and all its VF need to be reset.
User can run the command "port reset port_id"
to reset the VF port and to keep same port id without any configuration. Then user can run "port stop port_id"
and "port start port_id" to reconfigure its forwarding mode and parmaters as previous ones.
To avoid crash, current forwarding should be stopped before running "port reset port_id".

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 app/test-pmd/cmdline.c | 10 ++++++---
 app/test-pmd/testpmd.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++---
 app/test-pmd/testpmd.h |  1 +
 3 files changed, 66 insertions(+), 6 deletions(-)

diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index ff8ffd2..58ba6e4 100644
--- a/app/test-pmd/cmdline.c
+++ b/app/test-pmd/cmdline.c
@@ -950,6 +950,8 @@ static void cmd_operate_port_parsed(void *parsed_result,
 		stop_port(RTE_PORT_ALL);
 	else if (!strcmp(res->name, "close"))
 		close_port(RTE_PORT_ALL);
+	else if (!strcmp(res->name, "reset"))
+		reset_port(RTE_PORT_ALL);
 	else
 		printf("Unknown parameter\n");
 }
@@ -959,7 +961,7 @@ cmdline_parse_token_string_t cmd_operate_port_all_cmd =
 								"port");
 cmdline_parse_token_string_t cmd_operate_port_all_port =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_port_result, name,
-						"start#stop#close");
+						"start#stop#close#reset");
 cmdline_parse_token_string_t cmd_operate_port_all_all =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_port_result, value, "all");
 
@@ -994,6 +996,8 @@ static void cmd_operate_specific_port_parsed(void *parsed_result,
 		stop_port(res->value);
 	else if (!strcmp(res->name, "close"))
 		close_port(res->value);
+	else if (!strcmp(res->name, "reset"))
+		reset_port(res->value);
 	else
 		printf("Unknown parameter\n");
 }
@@ -1003,7 +1007,7 @@ cmdline_parse_token_string_t cmd_operate_specific_port_cmd =
 							keyword, "port");
 cmdline_parse_token_string_t cmd_operate_specific_port_port =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_specific_port_result,
-						name, "start#stop#close");
+						name, "start#stop#close#reset");
 cmdline_parse_token_num_t cmd_operate_specific_port_id =
 	TOKEN_NUM_INITIALIZER(struct cmd_operate_specific_port_result,
 							value, UINT8);
@@ -1011,7 +1015,7 @@ cmdline_parse_token_num_t cmd_operate_specific_port_id =  cmdline_parse_inst_t cmd_operate_specific_port = {
 	.f = cmd_operate_specific_port_parsed,
 	.data = NULL,
-	.help_str = "port start|stop|close <port_id>: Start/Stop/Close port_id",
+	.help_str = "port start|stop|close|reset <port_id>: 
+Start/Stop/Close/Reset port_id",
 	.tokens = {
 		(void *)&cmd_operate_specific_port_cmd,
 		(void *)&cmd_operate_specific_port_port,
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index b29328a..7773879 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -1403,6 +1403,7 @@ start_port(portid_t pid)
 	queueid_t qi;
 	struct rte_port *port;
 	struct ether_addr mac_addr;
+	struct rte_eth_dev_info dev_info;
 	enum rte_eth_event_type event_type;
 
 	if (port_id_is_invalid(pid, ENABLED_WARN)) @@ -1424,9 +1425,14 @@ start_port(portid_t pid)
 
 		if (port->need_reconfig > 0) {
 			port->need_reconfig = 0;
-
-			printf("Configuring Port %d (socket %u)\n", pi,
-					port->socket_id);
+			rte_eth_dev_info_get(pi, &dev_info);
+			printf("Configuring Port %d (socket %u) with "
+				"PCI Address: " PCI_PRI_FMT "\n",
+				pi, port->socket_id,
+				dev_info.pci_dev->addr.domain,
+				dev_info.pci_dev->addr.bus,
+				dev_info.pci_dev->addr.devid,
+				dev_info.pci_dev->addr.function);
 			/* configure port */
 			diag = rte_eth_dev_configure(pi, nb_rxq, nb_txq,
 						&(port->dev_conf));
@@ -1665,6 +1671,55 @@ close_port(portid_t pid)  }
 
 void
+reset_port(portid_t pid)
+{
+	int diag;
+	portid_t pi;
+	struct rte_port *port;
+	struct rte_eth_dev_info dev_info;
+
+	if (port_id_is_invalid(pid, ENABLED_WARN))
+		return;
+
+	printf("Resetting ports...\n");
+
+	RTE_ETH_FOREACH_DEV(pi) {
+		if (pid != pi && pid != (portid_t)RTE_PORT_ALL)
+			continue;
+
+		if (port_is_forwarding(pi) != 0 && test_done == 0) {
+			printf("Please remove port %d from forwarding "
+			       "configuration.\n", pi);
+			continue;
+		}
+
+		if (port_is_bonding_slave(pi)) {
+			printf("Please remove port %d from bonded device.\n",
+			       pi);
+			continue;
+		}
+
+		diag = rte_eth_dev_reset(pi);
+		if (diag == 0) {
+			port = &ports[pi];
+			port->need_reconfig = 1;
+			port->need_reconfig_queues = 1;
+			rte_eth_dev_info_get(pi, &dev_info);
+			printf("Finish resetting Port %d with PCI Address: "
+			       PCI_PRI_FMT "\n", pi,
+			       dev_info.pci_dev->addr.domain,
+			       dev_info.pci_dev->addr.bus,
+			       dev_info.pci_dev->addr.devid,
+			       dev_info.pci_dev->addr.function);
+		} else {
+			printf("Failed to reset port %d. diag=%d\n", pi, diag);
+		}
+	}
+
+	printf("Done\n");
+}
+
+void
 attach_port(char *identifier)
 {
 	portid_t pi = 0;
diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 364502d..e4c704a 100644
--- a/app/test-pmd/testpmd.h
+++ b/app/test-pmd/testpmd.h
@@ -596,6 +596,7 @@ int init_port_dcb_config(portid_t pid, enum dcb_mode_enable dcb_mode,  int start_port(portid_t pid);  void stop_port(portid_t pid);  void close_port(portid_t pid);
+void reset_port(portid_t pid);
 void attach_port(char *identifier);
 void detach_port(uint8_t port_id);
 int all_ports_stopped(void);
--
2.7.4

Re: [PATCH v4 5/5] app/testpmd: enhance command to test NIC reset

[Wu, Jingjing]

> -----Original Message-----
> From: Dai, Wei
> Sent: Thursday, June 29, 2017 10:58 PM
> To: thomas@monjalon.net; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Zhang, Helin <helin.zhang@intel.com>; Wu, Jingjing
> <jingjing.wu@intel.com>; yuan.pntel.com
> Cc: dev@dpdk.org; Dai, Wei <wei.dai@intel.com>
> Subject: [PATCH v4 5/5] app/testpmd: enhance command to test NIC reset
> 
> When PF is reset, a message will show it and all its
> VF need to be reset.
> User can run the command "port reset port_id"
> to reset the VF port and to keep same port id without
> any configuration. Then user can run "port stop port_id"
> and "port start port_id" to reconfigure its forwarding
> mode and parmaters as previous ones.
> To avoid crash, current forwarding should be stopped
> before running "port reset port_id".
> 
> Signed-off-by: Wei Dai <wei.dai@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>

Re: [PATCH v4 5/5] app/testpmd: enhance command to test NIC reset

[Wu, Jingjing]

> -----Original Message-----
> From: Dai, Wei
> Sent: Thursday, June 29, 2017 10:58 PM
> To: thomas@monjalon.net; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Zhang, Helin <helin.zhang@intel.com>; Wu, Jingjing
> <jingjing.wu@intel.com>; yuan.pntel.com
> Cc: dev@dpdk.org; Dai, Wei <wei.dai@intel.com>
> Subject: [PATCH v4 5/5] app/testpmd: enhance command to test NIC reset
> 
> When PF is reset, a message will show it and all its
> VF need to be reset.
> User can run the command "port reset port_id"
> to reset the VF port and to keep same port id without
> any configuration. Then user can run "port stop port_id"
> and "port start port_id" to reconfigure its forwarding
> mode and parmaters as previous ones.
> To avoid crash, current forwarding should be stopped
> before running "port reset port_id".
> 
> Signed-off-by: Wei Dai <wei.dai@intel.com>
> ---
>  app/test-pmd/cmdline.c | 10 ++++++---
>  app/test-pmd/testpmd.c | 61
> +++++++++++++++++++++++++++++++++++++++++++++++---
>  app/test-pmd/testpmd.h |  1 +
>  3 files changed, 66 insertions(+), 6 deletions(-)
> 
> diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
> index ff8ffd2..58ba6e4 100644
> --- a/app/test-pmd/cmdline.c
> +++ b/app/test-pmd/cmdline.c
> @@ -950,6 +950,8 @@ static void cmd_operate_port_parsed(void *parsed_result,
>  		stop_port(RTE_PORT_ALL);
>  	else if (!strcmp(res->name, "close"))
>  		close_port(RTE_PORT_ALL);
> +	else if (!strcmp(res->name, "reset"))
> +		reset_port(RTE_PORT_ALL);
>  	else
>  		printf("Unknown parameter\n");
>  }
> @@ -959,7 +961,7 @@ cmdline_parse_token_string_t cmd_operate_port_all_cmd =
>  								"port");
>  cmdline_parse_token_string_t cmd_operate_port_all_port =
>  	TOKEN_STRING_INITIALIZER(struct cmd_operate_port_result, name,
> -						"start#stop#close");
> +						"start#stop#close#reset");
>  cmdline_parse_token_string_t cmd_operate_port_all_all =
>  	TOKEN_STRING_INITIALIZER(struct cmd_operate_port_result, value, "all");
> 
> @@ -994,6 +996,8 @@ static void cmd_operate_specific_port_parsed(void *parsed_result,
>  		stop_port(res->value);
>  	else if (!strcmp(res->name, "close"))
>  		close_port(res->value);
> +	else if (!strcmp(res->name, "reset"))
> +		reset_port(res->value);
>  	else
>  		printf("Unknown parameter\n");
>  }
> @@ -1003,7 +1007,7 @@ cmdline_parse_token_string_t cmd_operate_specific_port_cmd =
>  							keyword, "port");
>  cmdline_parse_token_string_t cmd_operate_specific_port_port =
>  	TOKEN_STRING_INITIALIZER(struct cmd_operate_specific_port_result,
> -						name, "start#stop#close");
> +						name, "start#stop#close#reset");
>  cmdline_parse_token_num_t cmd_operate_specific_port_id =
>  	TOKEN_NUM_INITIALIZER(struct cmd_operate_specific_port_result,
>  							value, UINT8);
> @@ -1011,7 +1015,7 @@ cmdline_parse_token_num_t cmd_operate_specific_port_id =
>  cmdline_parse_inst_t cmd_operate_specific_port = {
>  	.f = cmd_operate_specific_port_parsed,
>  	.data = NULL,
> -	.help_str = "port start|stop|close <port_id>: Start/Stop/Close port_id",
> +	.help_str = "port start|stop|close|reset <port_id>: Start/Stop/Close/Reset port_id",
>  	.tokens = {
>  		(void *)&cmd_operate_specific_port_cmd,
>  		(void *)&cmd_operate_specific_port_port,
> diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
> index b29328a..7773879 100644
> --- a/app/test-pmd/testpmd.c
> +++ b/app/test-pmd/testpmd.c
> @@ -1403,6 +1403,7 @@ start_port(portid_t pid)
>  	queueid_t qi;
>  	struct rte_port *port;
>  	struct ether_addr mac_addr;
> +	struct rte_eth_dev_info dev_info;
>  	enum rte_eth_event_type event_type;
> 
>  	if (port_id_is_invalid(pid, ENABLED_WARN))
> @@ -1424,9 +1425,14 @@ start_port(portid_t pid)
> 
>  		if (port->need_reconfig > 0) {
>  			port->need_reconfig = 0;
> -
> -			printf("Configuring Port %d (socket %u)\n", pi,
> -					port->socket_id);
> +			rte_eth_dev_info_get(pi, &dev_info);
> +			printf("Configuring Port %d (socket %u) with "
> +				"PCI Address: " PCI_PRI_FMT "\n",
> +				pi, port->socket_id,
> +				dev_info.pci_dev->addr.domain,
> +				dev_info.pci_dev->addr.bus,
> +				dev_info.pci_dev->addr.devid,
> +				dev_info.pci_dev->addr.function);
I'm OK with the command change, but could you remove the
PCI print from here. The same reason as your patch 4/5.

Thanks
Jingjing

Re: [PATCH v4 5/5] app/testpmd: enhance command to test NIC reset

[Dai, Wei]

Adding PCI address info here is to let user confirm mapping between PCI address and port id is kept after reset.
As this is only for test purpose,  are you OK for other part except these PCI address printf ?

> -----Original Message-----
> From: Wu, Jingjing
> Sent: Friday, June 30, 2017 5:10 PM
> To: Dai, Wei <wei.dai@intel.com>; thomas@monjalon.net; Lu, Wenzhuo
> <wenzhuo.lu@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Zhang, Helin <helin.zhang@intel.com>;
> Peng, Yuan <yuan.peng@intel.com>
> Cc: dev@dpdk.org
> Subject: RE: [PATCH v4 5/5] app/testpmd: enhance command to test NIC reset
> 
> 
> 
> > -----Original Message-----
> > From: Dai, Wei
> > Sent: Thursday, June 29, 2017 10:58 PM
> > To: thomas@monjalon.net; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev,
> > Konstantin <konstantin.ananyev@intel.com>; Zhang, Helin
> > <helin.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>;
> > yuan.pntel.com
> > Cc: dev@dpdk.org; Dai, Wei <wei.dai@intel.com>
> > Subject: [PATCH v4 5/5] app/testpmd: enhance command to test NIC reset
> >
> > When PF is reset, a message will show it and all its VF need to be
> > reset.
> > User can run the command "port reset port_id"
> > to reset the VF port and to keep same port id without any
> > configuration. Then user can run "port stop port_id"
> > and "port start port_id" to reconfigure its forwarding mode and
> > parmaters as previous ones.
> > To avoid crash, current forwarding should be stopped before running
> > "port reset port_id".
> >
> > Signed-off-by: Wei Dai <wei.dai@intel.com>
> > ---
> >  app/test-pmd/cmdline.c | 10 ++++++---  app/test-pmd/testpmd.c | 61
> > +++++++++++++++++++++++++++++++++++++++++++++++---
> >  app/test-pmd/testpmd.h |  1 +
> >  3 files changed, 66 insertions(+), 6 deletions(-)
> >
> > diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index
> > ff8ffd2..58ba6e4 100644
> > --- a/app/test-pmd/cmdline.c
> > +++ b/app/test-pmd/cmdline.c
> > @@ -950,6 +950,8 @@ static void cmd_operate_port_parsed(void
> *parsed_result,
> >  		stop_port(RTE_PORT_ALL);
> >  	else if (!strcmp(res->name, "close"))
> >  		close_port(RTE_PORT_ALL);
> > +	else if (!strcmp(res->name, "reset"))
> > +		reset_port(RTE_PORT_ALL);
> >  	else
> >  		printf("Unknown parameter\n");
> >  }
> > @@ -959,7 +961,7 @@ cmdline_parse_token_string_t
> cmd_operate_port_all_cmd =
> >  								"port");
> >  cmdline_parse_token_string_t cmd_operate_port_all_port =
> >  	TOKEN_STRING_INITIALIZER(struct cmd_operate_port_result, name,
> > -						"start#stop#close");
> > +						"start#stop#close#reset");
> >  cmdline_parse_token_string_t cmd_operate_port_all_all =
> >  	TOKEN_STRING_INITIALIZER(struct cmd_operate_port_result, value,
> > "all");
> >
> > @@ -994,6 +996,8 @@ static void cmd_operate_specific_port_parsed(void
> *parsed_result,
> >  		stop_port(res->value);
> >  	else if (!strcmp(res->name, "close"))
> >  		close_port(res->value);
> > +	else if (!strcmp(res->name, "reset"))
> > +		reset_port(res->value);
> >  	else
> >  		printf("Unknown parameter\n");
> >  }
> > @@ -1003,7 +1007,7 @@ cmdline_parse_token_string_t
> cmd_operate_specific_port_cmd =
> >  							keyword, "port");
> >  cmdline_parse_token_string_t cmd_operate_specific_port_port =
> >  	TOKEN_STRING_INITIALIZER(struct cmd_operate_specific_port_result,
> > -						name, "start#stop#close");
> > +						name, "start#stop#close#reset");
> >  cmdline_parse_token_num_t cmd_operate_specific_port_id =
> >  	TOKEN_NUM_INITIALIZER(struct cmd_operate_specific_port_result,
> >  							value, UINT8);
> > @@ -1011,7 +1015,7 @@ cmdline_parse_token_num_t
> > cmd_operate_specific_port_id =  cmdline_parse_inst_t
> cmd_operate_specific_port = {
> >  	.f = cmd_operate_specific_port_parsed,
> >  	.data = NULL,
> > -	.help_str = "port start|stop|close <port_id>: Start/Stop/Close port_id",
> > +	.help_str = "port start|stop|close|reset <port_id>:
> > +Start/Stop/Close/Reset port_id",
> >  	.tokens = {
> >  		(void *)&cmd_operate_specific_port_cmd,
> >  		(void *)&cmd_operate_specific_port_port,
> > diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index
> > b29328a..7773879 100644
> > --- a/app/test-pmd/testpmd.c
> > +++ b/app/test-pmd/testpmd.c
> > @@ -1403,6 +1403,7 @@ start_port(portid_t pid)
> >  	queueid_t qi;
> >  	struct rte_port *port;
> >  	struct ether_addr mac_addr;
> > +	struct rte_eth_dev_info dev_info;
> >  	enum rte_eth_event_type event_type;
> >
> >  	if (port_id_is_invalid(pid, ENABLED_WARN)) @@ -1424,9 +1425,14 @@
> > start_port(portid_t pid)
> >
> >  		if (port->need_reconfig > 0) {
> >  			port->need_reconfig = 0;
> > -
> > -			printf("Configuring Port %d (socket %u)\n", pi,
> > -					port->socket_id);
> > +			rte_eth_dev_info_get(pi, &dev_info);
> > +			printf("Configuring Port %d (socket %u) with "
> > +				"PCI Address: " PCI_PRI_FMT "\n",
> > +				pi, port->socket_id,
> > +				dev_info.pci_dev->addr.domain,
> > +				dev_info.pci_dev->addr.bus,
> > +				dev_info.pci_dev->addr.devid,
> > +				dev_info.pci_dev->addr.function);
> I'm OK with the command change, but could you remove the PCI print from
> here. The same reason as your patch 4/5.
> 
> Thanks
> Jingjing

Re: [PATCH v4 0/5] Support NIC reset and keep same port id

[Peng, Yuan]

Tested-by: Peng, Yuan <yuan.peng@intel.com>

- Tested commit bbe569daa7e99b36d44b12bb3d23ddfbc26d383c+the 5 patches.
- OS: 4.8.6-300.fc25.x86_64
- GCC: gcc version 6.2.1 20160916 (Red Hat 6.2.1-2) (GCC)
- CPU: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz
- NIC: Ethernet controller [0200]: Intel Corporation Ethernet Controller 10G X550T [8086:1563] (rev 01)
- Default x86_64-native-linuxapp-gcc configuration
- Prerequisites:
- Total 1cases, 1 passed, 0 failed

Steps:
DUT:
1. echo 2 >/sys/bus/pci/devices/0000:83:00.0/sriov_numvfs
./usertools/dpdk-devbind.py -b vfio-pci 83:10.0 83:10.2
./x86_64-native-linuxapp-gcc/app/testpmd -c 0x3 -n 4 -- -i

2.testpmd> set verbose 1
testpmd> set fwd mac
testpmd> start
tester:
scapy
sendp([Ether(dst="02:09:C0:63:DA:4B")/IP()/UDP()], iface="ens6f0", count=1)
dut:
testpmd>show port stats all
port0 can fwd the packet normally
testpmd>show port info all
show port number, PCI addr and MAC addr

3. ifconfig ens801f0 down
Port 0: Interrupt reset event
Port 1: Interrupt reset event
ifconfig ens801f0 up

4.testpmd> stop

5. testpmd> port reset all
Resetting ports...
Finish resetting Port 0 with PCI Address: 0000:83:10.0
Finish resetting Port 1 with PCI Address: 0000:83:10.2
Done

6. testpmd> port stop all
Stopping ports...
Checking link statuses...
Done
testpmd> port start all
Configuring Port 0 (socket 1) with PCI Address: 0000:83:10.0
Port 0: 02:09:C0:63:DA:4B
Configuring Port 1 (socket 1) with PCI Address: 0000:83:10.2
Port 1: 02:09:C0:37:93:6F
Checking link statuses...
Done

7. testpmd> show port info all
confirm same mapping of port id and PCI address.

8.testpmd> start
Tester:
scapy
sendp([Ether(dst="02:09:C0:63:DA:4B")/IP()/UDP()], iface="ens6f0", count=1)
dut:
testpmd>show port stats all
port0 can fwd the packet normally

9.repeat step3 to step8, the same result.



-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wei Dai
Sent: Thursday, June 29, 2017 10:58 PM
To: thomas@monjalon.net; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Zhang, Helin <helin.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; yuan.pntel.com@dpdk.org
Cc: dev@dpdk.org; Dai, Wei <wei.dai@intel.com>
Subject: [dpdk-dev] [PATCH v4 0/5] Support NIC reset and keep same port id

Sometimes a port have to be reset. For example PF is reset, all its VF should also be reset. After reset, if the port goes through PCI remove() and then PCI probe() for restoration, its port id may be changed and this is not expected by some customer DPDK application.
Normally, PCI probe() includes two parts: one is in rte_ethdev layer and the other is calling PMD dev_init(). PCI remove( ) release all resource allocated from rte_ethdev layer in PCI probe( ) and calls PMD dev_unit( ).
To keep same port id and reset the port, only dev_uninit() and dev_init( ) in PMD can be called and keep all resources allocated from rte_ethdev layer poart in PCI probe( ).

New rte_eth_dev_reset( ) calls rte_eth_dev_stop( ), PMD dev_uninit( ) and then PMD dev_init( ) to reset a port and keep same port id.
And then application can go through rte_eth_dev_configure( ), rte_eth_rx_queue_setup( ), rte_eth_tx_queue_setup( ) and rte_eth_dev_start( ) again to restore its previous settings or to reconfigure itself with different settings.

To test this new feature, a testpmd command "port reset port_id" is added.
The mapping between port number and its PCI address can be monitored to confirm its port number is kept.
And following test case can also be used to confirm the port can work again after reset.

A typical test steps are listed as follows:
For example, run "ifconfig PF-name down" will trigger a reset to VF.
1.  run testpmd with 2 ixgbe VF ports belonging to same PF 2.  testpmd > set verbose 1 //to observe VF working 3.  testpmd > show port info all //show port number, PCI addr and MAC addr 4.  testpmd > start 5.  let all ports forwarding work for a while 6.  testpmd > show port stats all 7.  ifconfig name-of-PF down 8.  A message is shown in testmd to indicate PF reset 9.  ifconfig name-of-PF up 10. testpmd > stop // stop forwarding to avoid crash during reset 11. testpmd > port reset all 12. testpmd > port stop all 13. testpmd > port start all //recofnig all ports 14. testpmd > show port info all
    //confirm same mapping of port id and PCI addr 15. testpmd > start // restore forwarding 14. let all ports forwarding work for a while 15. testpmd > show port stats all //confirm all port can work again 16. repeat above step 7 - 15

chagnes:
v4:
  add PCI address to confirm its port number keep same
  correct test method in cover letter
v3:
  update testpmd command
v2:
  only reset PMD layer resource and keep same port id, but
  not restore settings


Wei Dai (5):
  ethdev: add support of NIC reset
  net/ixgbe: add support of reset
  net/i40e: add support of reset
  app/testpmd: display PCI address in port info
  app/testpmd: enhance command to test NIC reset

 app/test-pmd/cmdline.c                 | 10 ++++--
 app/test-pmd/config.c                  |  5 +++
 app/test-pmd/testpmd.c                 | 61 ++++++++++++++++++++++++++++++++--
 app/test-pmd/testpmd.h                 |  1 +
 drivers/net/i40e/i40e_ethdev.c         | 16 +++++++++
 drivers/net/i40e/i40e_ethdev_vf.c      | 16 +++++++++
 drivers/net/ixgbe/ixgbe_ethdev.c       | 38 +++++++++++++++++++++
 lib/librte_ether/rte_ethdev.c          | 16 +++++++++
 lib/librte_ether/rte_ethdev.h          | 12 +++++++
 lib/librte_ether/rte_ether_version.map |  2 +-
 10 files changed, 170 insertions(+), 7 deletions(-)

--
2.7.4

[PATCH v5 0/4] Support NIC reset and keep same port id

[Wei Dai]

Sometimes a port have to be reset. For example PF is reset, all its
VF should also be reset. After reset, if the port goes
through PCI remove() and then PCI probe() for restoration, its
port id may be changed and this is not expected by some customer
DPDK application.
Normally, PCI probe() includes two parts: one is in rte_ethdev layer
and the other is calling PMD dev_init(). PCI remove( ) release all
resource allocated from rte_ethdev layer in PCI probe( ) and calls
PMD dev_unit( ).
To keep same port id and reset the port, only dev_uninit() and
dev_init( ) in PMD can be called and keep all resources allocated
from rte_ethdev layer poart in PCI probe( ).

New rte_eth_dev_reset( ) calls rte_eth_dev_stop( ), PMD dev_uninit( )
and then PMD dev_init( ) to reset a port and keep same port id.
And then application can go through rte_eth_dev_configure( ),
rte_eth_rx_queue_setup( ), rte_eth_tx_queue_setup( ) and
rte_eth_dev_start( ) again to restore its previous settings or
to reconfigure itself with different settings.

To test this new feature, a testpmd command "port reset port_id" is added.
The mapping between port number and its PCI address can be monitored to
confirm its port number is kept.
And following test case can also be used to confirm the port can work
again after reset.

A typical test steps are listed as follows:
For example, run "ifconfig PF-name down" will trigger a reset to VF.
1.  run testpmd with 2 ixgbe VF ports belonging to same PF
2.  testpmd > set verbose 1 //to observe VF working
3.  testpmd > show port info all //show port number and MAC addr
4.  testpmd > start
5.  let all ports forwarding work for a while
6.  testpmd > show port stats all
7.  ifconfig name-of-PF down
8.  A message is shown in testmd to indicate PF reset
9.  ifconfig name-of-PF up
10. testpmd > stop // stop forwarding to avoid crash during reset
11. testpmd > port reset all
12. testpmd > port stop all
13. testpmd > port start all //recofnig all ports
14. testpmd > show port info all
    //get mapping of port id and MAC addr for forwarding
15. testpmd > start // restore forwarding
14. let all ports forwarding work for a while
15. testpmd > show port stats all //confirm all port can work again
16. repeat above step 7 - 15

chagnes:
v5:
  remove PCI address output to align with other modification which
    will output it in other way
  disable PF reset when its VF is ative to avoid unexpected VF behavior
v4:
  add PCI address to confirm its port number keep same
  correct test method in cover letter
v3:
  update testpmd command
v2:
  only reset PMD layer resource and keep same port id, but
  not restore settings

Signed-off-by: Wei Dai <wei.dai@intel.com>
Tested-by: Yuan Peng <yuan.peng@intel.com>

Wei Dai (4):
  ethdev: add support of NIC reset
  net/ixgbe: add support of reset
  net/i40e: add support of reset
  app/testpmd: enhance command to test NIC reset

 app/test-pmd/cmdline.c                 | 10 +++++---
 app/test-pmd/testpmd.c                 | 44 +++++++++++++++++++++++++++++++---
 app/test-pmd/testpmd.h                 |  1 +
 drivers/net/i40e/i40e_ethdev.c         | 20 ++++++++++++++++
 drivers/net/i40e/i40e_ethdev_vf.c      | 16 +++++++++++++
 drivers/net/ixgbe/ixgbe_ethdev.c       | 42 ++++++++++++++++++++++++++++++++
 lib/librte_ether/rte_ethdev.c          | 16 +++++++++++++
 lib/librte_ether/rte_ethdev.h          | 12 ++++++++++
 lib/librte_ether/rte_ether_version.map |  2 +-
 9 files changed, 156 insertions(+), 7 deletions(-)

-- 
2.7.4

[PATCH v5 1/4] ethdev: add support of NIC reset

[Wei Dai]

A DPDK application can reset a NIC and keep its port id afterwards.
It means that all SW resources allocated in ethdev layer should be
kept and  SW and HW resources of the NIC in PMD need to be reset
in similar way that it runs PCI dev_uninit() and then dev_init().
The sequence of dev_uninit() and dev_init() can be packed into a
single interface API rte_eth_dev_reset().

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 lib/librte_ether/rte_ethdev.c          | 16 ++++++++++++++++
 lib/librte_ether/rte_ethdev.h          | 12 ++++++++++++
 lib/librte_ether/rte_ether_version.map |  2 +-
 3 files changed, 29 insertions(+), 1 deletion(-)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index 81a45c0..06b8839 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -3349,3 +3349,19 @@ rte_eth_dev_l2_tunnel_offload_set(uint8_t port_id,
 				-ENOTSUP);
 	return (*dev->dev_ops->l2_tunnel_offload_set)(dev, l2_tunnel, mask, en);
 }
+
+int
+rte_eth_dev_reset(uint8_t port_id)
+{
+	struct rte_eth_dev *dev;
+	int ret;
+
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
+	dev = &rte_eth_devices[port_id];
+
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_reset, -ENOTSUP);
+
+	rte_eth_dev_stop(port_id);
+	ret = dev->dev_ops->dev_reset(dev);
+	return ret;
+}
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index f6e6c74..4922b36 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1087,6 +1087,9 @@ typedef int  (*eth_dev_set_link_down_t)(struct rte_eth_dev *dev);
 typedef void (*eth_dev_close_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to close a configured Ethernet device. */
 
+typedef int (*eth_dev_reset_t)(struct rte_eth_dev *dev);
+/** <@internal Function used to reset a configured Ethernet device. */
+
 typedef void (*eth_promiscuous_enable_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to enable the RX promiscuous mode of an Ethernet device. */
 
@@ -1404,6 +1407,7 @@ struct eth_dev_ops {
 	eth_dev_set_link_up_t      dev_set_link_up;   /**< Device link up. */
 	eth_dev_set_link_down_t    dev_set_link_down; /**< Device link down. */
 	eth_dev_close_t            dev_close;     /**< Close device. */
+	eth_dev_reset_t		   dev_reset;	   /**<	Reset device. */
 	eth_link_update_t          link_update;   /**< Get device link state. */
 
 	eth_promiscuous_enable_t   promiscuous_enable; /**< Promiscuous ON. */
@@ -2104,6 +2108,14 @@ int rte_eth_dev_set_link_down(uint8_t port_id);
 void rte_eth_dev_close(uint8_t port_id);
 
 /**
+ * Reset a Ethernet device.
+ *
+ * @param port_id
+ *   The port identifier of the Ethernet device.
+ */
+int rte_eth_dev_reset(uint8_t port_id);
+
+/**
  * Enable receipt in promiscuous mode for an Ethernet device.
  *
  * @param port_id
diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map
index e0881f0..d569bcb 100644
--- a/lib/librte_ether/rte_ether_version.map
+++ b/lib/librte_ether/rte_ether_version.map
@@ -153,5 +153,5 @@ DPDK_17.08 {
 	global:
 
 	rte_flow_isolate;
-
+	rte_eth_dev_reset;
 } DPDK_17.05;
-- 
2.7.4

[PATCH v5 2/4] net/ixgbe: add support of reset

[Wei Dai]

Reset a NIC by calling dev_uninit and then dev_init.
Go through same way in NIC PCI remove without release of
ethdev resource and then NIC PCI probe function without
ethdev resource allocation.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 drivers/net/ixgbe/ixgbe_ethdev.c | 42 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index ebc5467..e2f1da0 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -170,6 +170,7 @@ static void ixgbe_dev_stop(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_up(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_down(struct rte_eth_dev *dev);
 static void ixgbe_dev_close(struct rte_eth_dev *dev);
+static int  ixgbe_dev_reset(struct rte_eth_dev *dev);
 static void ixgbe_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void ixgbe_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void ixgbe_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -264,6 +265,7 @@ static int  ixgbevf_dev_configure(struct rte_eth_dev *dev);
 static int  ixgbevf_dev_start(struct rte_eth_dev *dev);
 static void ixgbevf_dev_stop(struct rte_eth_dev *dev);
 static void ixgbevf_dev_close(struct rte_eth_dev *dev);
+static int  ixgbevf_dev_reset(struct rte_eth_dev *dev);
 static void ixgbevf_intr_disable(struct ixgbe_hw *hw);
 static void ixgbevf_intr_enable(struct ixgbe_hw *hw);
 static void ixgbevf_dev_stats_get(struct rte_eth_dev *dev,
@@ -525,6 +527,7 @@ static const struct eth_dev_ops ixgbe_eth_dev_ops = {
 	.dev_set_link_up    = ixgbe_dev_set_link_up,
 	.dev_set_link_down  = ixgbe_dev_set_link_down,
 	.dev_close            = ixgbe_dev_close,
+	.dev_reset            = ixgbe_dev_reset,
 	.promiscuous_enable   = ixgbe_dev_promiscuous_enable,
 	.promiscuous_disable  = ixgbe_dev_promiscuous_disable,
 	.allmulticast_enable  = ixgbe_dev_allmulticast_enable,
@@ -614,6 +617,7 @@ static const struct eth_dev_ops ixgbevf_eth_dev_ops = {
 	.xstats_reset         = ixgbevf_dev_stats_reset,
 	.xstats_get_names     = ixgbevf_dev_xstats_get_names,
 	.dev_close            = ixgbevf_dev_close,
+	.dev_reset            = ixgbevf_dev_reset,
 	.allmulticast_enable  = ixgbevf_dev_allmulticast_enable,
 	.allmulticast_disable = ixgbevf_dev_allmulticast_disable,
 	.dev_infos_get        = ixgbevf_dev_info_get,
@@ -2838,6 +2842,27 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 	ixgbe_set_rar(hw, 0, hw->mac.addr, 0, IXGBE_RAH_AV);
 }
 
+/*
+ * Reest PF device.
+ */
+static int
+ixgbe_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	/* To avoid unexpected behavior in VF, disable PF reset */
+	if (dev->data->sriov.active)
+		return -ENOTSUP;
+
+	ret = eth_ixgbe_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_ixgbe_dev_init(dev);
+
+	return ret;
+}
+
 static void
 ixgbe_read_stats_registers(struct ixgbe_hw *hw,
 			   struct ixgbe_hw_stats *hw_stats,
@@ -4902,6 +4927,23 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	ixgbevf_remove_mac_addr(dev, 0);
 }
 
+/*
+ * Reset VF device
+ */
+static int
+ixgbevf_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	ret = eth_ixgbevf_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_ixgbevf_dev_init(dev);
+
+	return ret;
+}
+
 static void ixgbevf_set_vfta_all(struct rte_eth_dev *dev, bool on)
 {
 	struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-- 
2.7.4

Re: [PATCH v5 2/4] net/ixgbe: add support of reset

[Thomas Monjalon]

Hi,

30/06/2017 12:12, Wei Dai:
> +/*
> + * Reest PF device.
> + */
> +static int
> +ixgbe_dev_reset(struct rte_eth_dev *dev)
> +{
> +       int ret;
> +
> +       /* To avoid unexpected behavior in VF, disable PF reset */
> +       if (dev->data->sriov.active)
> +               return -ENOTSUP;
> +
> +       ret = eth_ixgbe_dev_uninit(dev);
> +       if (ret)
> +               return ret;
> +
> +       ret = eth_ixgbe_dev_init(dev);
> +
> +       return ret;
> +}

rte_eth_dev_reset() just do
+       rte_eth_dev_stop(port_id);
+       ret = dev->dev_ops->dev_reset(dev);

and dev_reset() just do
+       ret = eth_ixgbe_dev_uninit(dev);
+       ret = eth_ixgbe_dev_init(dev);

It is doing one more thing, the check of SR-IOV.
Unfortunately, this restriction is not documented.

This is the documentation of the new API:

 /**
+ * Reset a Ethernet device.
+ *
+ * @param port_id
+ *   The port identifier of the Ethernet device.
+ */
+int rte_eth_dev_reset(uint8_t port_id);

It is really really too short.
>From the beginning of this proposal we are asking you to better explain
why this API is needed. It still does not appear in the doc.
Are you adding it to offer a new service to DPDK application developpers?
Or is it just a secret sauce that you will explain only to your customers?

This is what is expected to be documented:
- why/when this API must be used
- what the API will do
- what is needed to do after

Re: [PATCH v5 2/4] net/ixgbe: add support of reset

[Thomas Monjalon]

07/07/2017 10:25, Thomas Monjalon:
> Hi,
> 
> 30/06/2017 12:12, Wei Dai:
> > +/*
> > + * Reest PF device.
> > + */
> > +static int
> > +ixgbe_dev_reset(struct rte_eth_dev *dev)
> > +{
> > +       int ret;
> > +
> > +       /* To avoid unexpected behavior in VF, disable PF reset */
> > +       if (dev->data->sriov.active)
> > +               return -ENOTSUP;
> > +
> > +       ret = eth_ixgbe_dev_uninit(dev);
> > +       if (ret)
> > +               return ret;
> > +
> > +       ret = eth_ixgbe_dev_init(dev);
> > +
> > +       return ret;
> > +}
> 
> rte_eth_dev_reset() just do
> +       rte_eth_dev_stop(port_id);
> +       ret = dev->dev_ops->dev_reset(dev);
> 
> and dev_reset() just do
> +       ret = eth_ixgbe_dev_uninit(dev);
> +       ret = eth_ixgbe_dev_init(dev);
> 
> It is doing one more thing, the check of SR-IOV.
> Unfortunately, this restriction is not documented.
> 
> This is the documentation of the new API:
> 
>  /**
> + * Reset a Ethernet device.
> + *
> + * @param port_id
> + *   The port identifier of the Ethernet device.
> + */
> +int rte_eth_dev_reset(uint8_t port_id);
> 
> It is really really too short.
> From the beginning of this proposal we are asking you to better explain
> why this API is needed. It still does not appear in the doc.
> Are you adding it to offer a new service to DPDK application developpers?
> Or is it just a secret sauce that you will explain only to your customers?
> 
> This is what is expected to be documented:
> - why/when this API must be used
> - what the API will do
> - what is needed to do after

I would like to add that the description of the API must also help
other PMD maintainers to implement it.
Adding a new op means more work for PMD maintainers, that's why
they should understand the benefit and acknowledge it.
Ferruh, as the maintainer of next-net, please could you ask for feedbacks
from other PMD maintainers?

Re: [PATCH v5 2/4] net/ixgbe: add support of reset

[Dai, Wei]

Thanks, Thomas
I have just sent out my v6 patch set which includes more details to explain why/what/when.

-----Original Message-----
From: Thomas Monjalon [mailto:thomas@monjalon.net] 
Sent: Friday, July 7, 2017 4:36 PM
To: Dai, Wei <wei.dai@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>
Cc: dev@dpdk.org; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Zhang, Helin <helin.zhang@intel.com>; Peng, Yuan <yuan.peng@intel.com>
Subject: Re: [dpdk-dev] [PATCH v5 2/4] net/ixgbe: add support of reset

07/07/2017 10:25, Thomas Monjalon:
> Hi,
> 
> 30/06/2017 12:12, Wei Dai:
> > +/*
> > + * Reest PF device.
> > + */
> > +static int
> > +ixgbe_dev_reset(struct rte_eth_dev *dev) {
> > +       int ret;
> > +
> > +       /* To avoid unexpected behavior in VF, disable PF reset */
> > +       if (dev->data->sriov.active)
> > +               return -ENOTSUP;
> > +
> > +       ret = eth_ixgbe_dev_uninit(dev);
> > +       if (ret)
> > +               return ret;
> > +
> > +       ret = eth_ixgbe_dev_init(dev);
> > +
> > +       return ret;
> > +}
> 
> rte_eth_dev_reset() just do
> +       rte_eth_dev_stop(port_id);
> +       ret = dev->dev_ops->dev_reset(dev);
> 
> and dev_reset() just do
> +       ret = eth_ixgbe_dev_uninit(dev);
> +       ret = eth_ixgbe_dev_init(dev);
> 
> It is doing one more thing, the check of SR-IOV.
> Unfortunately, this restriction is not documented.
> 
> This is the documentation of the new API:
> 
>  /**
> + * Reset a Ethernet device.
> + *
> + * @param port_id
> + *   The port identifier of the Ethernet device.
> + */
> +int rte_eth_dev_reset(uint8_t port_id);
> 
> It is really really too short.
> From the beginning of this proposal we are asking you to better 
> explain why this API is needed. It still does not appear in the doc.
> Are you adding it to offer a new service to DPDK application developpers?
> Or is it just a secret sauce that you will explain only to your customers?
> 
> This is what is expected to be documented:
> - why/when this API must be used
> - what the API will do
> - what is needed to do after

I would like to add that the description of the API must also help other PMD maintainers to implement it.
Adding a new op means more work for PMD maintainers, that's why they should understand the benefit and acknowledge it.
Ferruh, as the maintainer of next-net, please could you ask for feedbacks from other PMD maintainers?

[PATCH v5 3/4] net/i40e: add support of reset

[Wei Dai]

Reset a NIC by calling dev_uninit() and then dev_init().
Go through the same way in NIC PCI remove without release
of ethdev resource and then NIC PCI probe function without
ethdev resource allocation.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 drivers/net/i40e/i40e_ethdev.c    | 20 ++++++++++++++++++++
 drivers/net/i40e/i40e_ethdev_vf.c | 16 ++++++++++++++++
 2 files changed, 36 insertions(+)

diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 4ee1113..261a5d0 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -250,6 +250,7 @@ static int i40e_dev_configure(struct rte_eth_dev *dev);
 static int i40e_dev_start(struct rte_eth_dev *dev);
 static void i40e_dev_stop(struct rte_eth_dev *dev);
 static void i40e_dev_close(struct rte_eth_dev *dev);
+static int  i40e_dev_reset(struct rte_eth_dev *dev);
 static void i40e_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void i40e_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void i40e_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -449,6 +450,7 @@ static const struct eth_dev_ops i40e_eth_dev_ops = {
 	.dev_start                    = i40e_dev_start,
 	.dev_stop                     = i40e_dev_stop,
 	.dev_close                    = i40e_dev_close,
+	.dev_reset                    = i40e_dev_reset,
 	.promiscuous_enable           = i40e_dev_promiscuous_enable,
 	.promiscuous_disable          = i40e_dev_promiscuous_disable,
 	.allmulticast_enable          = i40e_dev_allmulticast_enable,
@@ -2135,6 +2137,24 @@ i40e_dev_close(struct rte_eth_dev *dev)
 	I40E_WRITE_FLUSH(hw);
 }
 
+static int
+i40e_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	/* To avoid unexpected behavior in VF, disable PF reset */
+	if (dev->data->sriov.active)
+		return -ENOTSUP;
+
+	ret = eth_i40e_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_i40e_dev_init(dev);
+
+	return ret;
+}
+
 static void
 i40e_dev_promiscuous_enable(struct rte_eth_dev *dev)
 {
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 2d5a9b5..bf287a0 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -123,6 +123,7 @@ static void i40evf_vlan_offload_set(struct rte_eth_dev *dev, int mask);
 static int i40evf_vlan_pvid_set(struct rte_eth_dev *dev, uint16_t pvid,
 				int on);
 static void i40evf_dev_close(struct rte_eth_dev *dev);
+static int  i40evf_dev_reset(struct rte_eth_dev *dev);
 static void i40evf_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void i40evf_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void i40evf_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -204,6 +205,7 @@ static const struct eth_dev_ops i40evf_eth_dev_ops = {
 	.xstats_get_names     = i40evf_dev_xstats_get_names,
 	.xstats_reset         = i40evf_dev_xstats_reset,
 	.dev_close            = i40evf_dev_close,
+	.dev_reset            = i40evf_dev_reset,
 	.dev_infos_get        = i40evf_dev_info_get,
 	.dev_supported_ptypes_get = i40e_dev_supported_ptypes_get,
 	.vlan_filter_set      = i40evf_vlan_filter_set,
@@ -2347,6 +2349,20 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 }
 
 static int
+i40evf_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	ret = i40evf_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = i40evf_dev_init(dev);
+
+	return ret;
+}
+
+static int
 i40evf_get_rss_lut(struct i40e_vsi *vsi, uint8_t *lut, uint16_t lut_size)
 {
 	struct i40e_vf *vf = I40E_VSI_TO_VF(vsi);
-- 
2.7.4

[PATCH v5 4/4] app/testpmd: enhance command to test NIC reset

[Wei Dai]

When PF is reset, a message will show it and all its
VF need to be reset.
User can run the command "port reset port_id"
to reset the VF port and to keep same port id without
any configuration. Then user can run "port stop port_id"
and "port start port_id" to reconfigure its forwarding
mode and parmaters as previous ones.
To avoid crash, current forwarding should be stopped
before running "port reset port_id".

Signed-off-by: Wei Dai <wei.dai@intel.com>
Tested-by: Yuan Peng <yuan.peng@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>
---
 app/test-pmd/cmdline.c | 10 +++++++---
 app/test-pmd/testpmd.c | 44 +++++++++++++++++++++++++++++++++++++++++---
 app/test-pmd/testpmd.h |  1 +
 3 files changed, 49 insertions(+), 6 deletions(-)

diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
index ff8ffd2..58ba6e4 100644
--- a/app/test-pmd/cmdline.c
+++ b/app/test-pmd/cmdline.c
@@ -950,6 +950,8 @@ static void cmd_operate_port_parsed(void *parsed_result,
 		stop_port(RTE_PORT_ALL);
 	else if (!strcmp(res->name, "close"))
 		close_port(RTE_PORT_ALL);
+	else if (!strcmp(res->name, "reset"))
+		reset_port(RTE_PORT_ALL);
 	else
 		printf("Unknown parameter\n");
 }
@@ -959,7 +961,7 @@ cmdline_parse_token_string_t cmd_operate_port_all_cmd =
 								"port");
 cmdline_parse_token_string_t cmd_operate_port_all_port =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_port_result, name,
-						"start#stop#close");
+						"start#stop#close#reset");
 cmdline_parse_token_string_t cmd_operate_port_all_all =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_port_result, value, "all");
 
@@ -994,6 +996,8 @@ static void cmd_operate_specific_port_parsed(void *parsed_result,
 		stop_port(res->value);
 	else if (!strcmp(res->name, "close"))
 		close_port(res->value);
+	else if (!strcmp(res->name, "reset"))
+		reset_port(res->value);
 	else
 		printf("Unknown parameter\n");
 }
@@ -1003,7 +1007,7 @@ cmdline_parse_token_string_t cmd_operate_specific_port_cmd =
 							keyword, "port");
 cmdline_parse_token_string_t cmd_operate_specific_port_port =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_specific_port_result,
-						name, "start#stop#close");
+						name, "start#stop#close#reset");
 cmdline_parse_token_num_t cmd_operate_specific_port_id =
 	TOKEN_NUM_INITIALIZER(struct cmd_operate_specific_port_result,
 							value, UINT8);
@@ -1011,7 +1015,7 @@ cmdline_parse_token_num_t cmd_operate_specific_port_id =
 cmdline_parse_inst_t cmd_operate_specific_port = {
 	.f = cmd_operate_specific_port_parsed,
 	.data = NULL,
-	.help_str = "port start|stop|close <port_id>: Start/Stop/Close port_id",
+	.help_str = "port start|stop|close|reset <port_id>: Start/Stop/Close/Reset port_id",
 	.tokens = {
 		(void *)&cmd_operate_specific_port_cmd,
 		(void *)&cmd_operate_specific_port_port,
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index b29328a..17eaa31 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -1424,9 +1424,6 @@ start_port(portid_t pid)
 
 		if (port->need_reconfig > 0) {
 			port->need_reconfig = 0;
-
-			printf("Configuring Port %d (socket %u)\n", pi,
-					port->socket_id);
 			/* configure port */
 			diag = rte_eth_dev_configure(pi, nb_rxq, nb_txq,
 						&(port->dev_conf));
@@ -1665,6 +1662,47 @@ close_port(portid_t pid)
 }
 
 void
+reset_port(portid_t pid)
+{
+	int diag;
+	portid_t pi;
+	struct rte_port *port;
+
+	if (port_id_is_invalid(pid, ENABLED_WARN))
+		return;
+
+	printf("Resetting ports...\n");
+
+	RTE_ETH_FOREACH_DEV(pi) {
+		if (pid != pi && pid != (portid_t)RTE_PORT_ALL)
+			continue;
+
+		if (port_is_forwarding(pi) != 0 && test_done == 0) {
+			printf("Please remove port %d from forwarding "
+			       "configuration.\n", pi);
+			continue;
+		}
+
+		if (port_is_bonding_slave(pi)) {
+			printf("Please remove port %d from bonded device.\n",
+			       pi);
+			continue;
+		}
+
+		diag = rte_eth_dev_reset(pi);
+		if (diag == 0) {
+			port = &ports[pi];
+			port->need_reconfig = 1;
+			port->need_reconfig_queues = 1;
+		} else {
+			printf("Failed to reset port %d. diag=%d\n", pi, diag);
+		}
+	}
+
+	printf("Done\n");
+}
+
+void
 attach_port(char *identifier)
 {
 	portid_t pi = 0;
diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
index 364502d..e4c704a 100644
--- a/app/test-pmd/testpmd.h
+++ b/app/test-pmd/testpmd.h
@@ -596,6 +596,7 @@ int init_port_dcb_config(portid_t pid, enum dcb_mode_enable dcb_mode,
 int start_port(portid_t pid);
 void stop_port(portid_t pid);
 void close_port(portid_t pid);
+void reset_port(portid_t pid);
 void attach_port(char *identifier);
 void detach_port(uint8_t port_id);
 int all_ports_stopped(void);
-- 
2.7.4

Re: [PATCH v5 0/4] Support NIC reset and keep same port id

[Wu, Jingjing]

> -----Original Message-----
> From: Dai, Wei
> Sent: Friday, June 30, 2017 6:13 PM
> To: thomas@monjalon.net; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Zhang, Helin
> <helin.zhang@intel.com>; Peng, Yuan <yuan.peng@intel.com>
> Cc: dev@dpdk.org; Dai, Wei <wei.dai@intel.com>
> Subject: [PATCH v5 0/4] Support NIC reset and keep same port id
> 
> Sometimes a port have to be reset. For example PF is reset, all its
> VF should also be reset. After reset, if the port goes
> through PCI remove() and then PCI probe() for restoration, its
> port id may be changed and this is not expected by some customer
> DPDK application.
> Normally, PCI probe() includes two parts: one is in rte_ethdev layer
> and the other is calling PMD dev_init(). PCI remove( ) release all
> resource allocated from rte_ethdev layer in PCI probe( ) and calls
> PMD dev_unit( ).
> To keep same port id and reset the port, only dev_uninit() and
> dev_init( ) in PMD can be called and keep all resources allocated
> from rte_ethdev layer poart in PCI probe( ).
> 
> New rte_eth_dev_reset( ) calls rte_eth_dev_stop( ), PMD dev_uninit( )
> and then PMD dev_init( ) to reset a port and keep same port id.
> And then application can go through rte_eth_dev_configure( ),
> rte_eth_rx_queue_setup( ), rte_eth_tx_queue_setup( ) and
> rte_eth_dev_start( ) again to restore its previous settings or
> to reconfigure itself with different settings.
> 
> To test this new feature, a testpmd command "port reset port_id" is added.
> The mapping between port number and its PCI address can be monitored to
> confirm its port number is kept.
> And following test case can also be used to confirm the port can work
> again after reset.
> 
> A typical test steps are listed as follows:
> For example, run "ifconfig PF-name down" will trigger a reset to VF.
> 1.  run testpmd with 2 ixgbe VF ports belonging to same PF
> 2.  testpmd > set verbose 1 //to observe VF working
> 3.  testpmd > show port info all //show port number and MAC addr
> 4.  testpmd > start
> 5.  let all ports forwarding work for a while
> 6.  testpmd > show port stats all
> 7.  ifconfig name-of-PF down
> 8.  A message is shown in testmd to indicate PF reset
> 9.  ifconfig name-of-PF up
> 10. testpmd > stop // stop forwarding to avoid crash during reset
> 11. testpmd > port reset all
> 12. testpmd > port stop all
> 13. testpmd > port start all //recofnig all ports
> 14. testpmd > show port info all
>     //get mapping of port id and MAC addr for forwarding
> 15. testpmd > start // restore forwarding
> 14. let all ports forwarding work for a while
> 15. testpmd > show port stats all //confirm all port can work again
> 16. repeat above step 7 - 15
> 
> chagnes:
> v5:
>   remove PCI address output to align with other modification which
>     will output it in other way
>   disable PF reset when its VF is ative to avoid unexpected VF behavior
> v4:
>   add PCI address to confirm its port number keep same
>   correct test method in cover letter
> v3:
>   update testpmd command
> v2:
>   only reset PMD layer resource and keep same port id, but
>   not restore settings
> 
> Signed-off-by: Wei Dai <wei.dai@intel.com>
> Tested-by: Yuan Peng <yuan.peng@intel.com>


Acked-by: Jingjing Wu <jingjing.wu@intel.com>

Re: [PATCH v5 0/4] Support NIC reset and keep same port id

[Dai, Wei]

Hi, Thomas

Would you please have a look at this patch set ?
If any question,  please let me know .

Thanks
-Wei


-----Original Message-----
From: Wu, Jingjing 
Sent: Friday, June 30, 2017 6:50 PM
To: Dai, Wei <wei.dai@intel.com>; thomas@monjalon.net; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Zhang, Helin <helin.zhang@intel.com>; Peng, Yuan <yuan.peng@intel.com>
Cc: dev@dpdk.org
Subject: RE: [PATCH v5 0/4] Support NIC reset and keep same port id



> -----Original Message-----
> From: Dai, Wei
> Sent: Friday, June 30, 2017 6:13 PM
> To: thomas@monjalon.net; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev, 
> Konstantin <konstantin.ananyev@intel.com>; Wu, Jingjing 
> <jingjing.wu@intel.com>; Zhang, Helin <helin.zhang@intel.com>; Peng, 
> Yuan <yuan.peng@intel.com>
> Cc: dev@dpdk.org; Dai, Wei <wei.dai@intel.com>
> Subject: [PATCH v5 0/4] Support NIC reset and keep same port id
> 
> Sometimes a port have to be reset. For example PF is reset, all its VF 
> should also be reset. After reset, if the port goes through PCI 
> remove() and then PCI probe() for restoration, its port id may be 
> changed and this is not expected by some customer DPDK application.
> Normally, PCI probe() includes two parts: one is in rte_ethdev layer 
> and the other is calling PMD dev_init(). PCI remove( ) release all 
> resource allocated from rte_ethdev layer in PCI probe( ) and calls PMD 
> dev_unit( ).
> To keep same port id and reset the port, only dev_uninit() and 
> dev_init( ) in PMD can be called and keep all resources allocated from 
> rte_ethdev layer poart in PCI probe( ).
> 
> New rte_eth_dev_reset( ) calls rte_eth_dev_stop( ), PMD dev_uninit( ) 
> and then PMD dev_init( ) to reset a port and keep same port id.
> And then application can go through rte_eth_dev_configure( ), 
> rte_eth_rx_queue_setup( ), rte_eth_tx_queue_setup( ) and 
> rte_eth_dev_start( ) again to restore its previous settings or to 
> reconfigure itself with different settings.
> 
> To test this new feature, a testpmd command "port reset port_id" is added.
> The mapping between port number and its PCI address can be monitored 
> to confirm its port number is kept.
> And following test case can also be used to confirm the port can work 
> again after reset.
> 
> A typical test steps are listed as follows:
> For example, run "ifconfig PF-name down" will trigger a reset to VF.
> 1.  run testpmd with 2 ixgbe VF ports belonging to same PF 2.  testpmd 
> > set verbose 1 //to observe VF working 3.  testpmd > show port info 
> all //show port number and MAC addr 4.  testpmd > start 5.  let all 
> ports forwarding work for a while 6.  testpmd > show port stats all 7.  
> ifconfig name-of-PF down 8.  A message is shown in testmd to indicate 
> PF reset 9.  ifconfig name-of-PF up 10. testpmd > stop // stop 
> forwarding to avoid crash during reset 11. testpmd > port reset all 
> 12. testpmd > port stop all 13. testpmd > port start all //recofnig 
> all ports 14. testpmd > show port info all
>     //get mapping of port id and MAC addr for forwarding 15. testpmd > 
> start // restore forwarding 14. let all ports forwarding work for a 
> while 15. testpmd > show port stats all //confirm all port can work 
> again 16. repeat above step 7 - 15
> 
> chagnes:
> v5:
>   remove PCI address output to align with other modification which
>     will output it in other way
>   disable PF reset when its VF is ative to avoid unexpected VF 
> behavior
> v4:
>   add PCI address to confirm its port number keep same
>   correct test method in cover letter
> v3:
>   update testpmd command
> v2:
>   only reset PMD layer resource and keep same port id, but
>   not restore settings
> 
> Signed-off-by: Wei Dai <wei.dai@intel.com>
> Tested-by: Yuan Peng <yuan.peng@intel.com>


Acked-by: Jingjing Wu <jingjing.wu@intel.com>

[PATCH v6 0/4] Support NIC reset and keep same port id

[Wei Dai]

This patch set adds a function rte_eth_dev_reset( ) in rte_ethdev layer.
Sometimes a port have to be reset passively. For example a PF is reset,
all its VFs should also be reset by application itself to align with 
the PF.
A DPDK application also can call this function to trigger an initative
port reset.

When processing reset, if the port goes through PCI remove() and then
PCI probe() for restoration, its port id may be changed and this is not
expected by some DPDK application.

Normally, PCI probe() includes two parts: one is in rte_ethdev layer
to allocate resource in rte_ethdev layer and the other is calling PMD
specific dev_init() to allocate and initialize resource in PMD layer.
PCI remove( ) releases all resource allocated from rte_ethdev layer
in PCI probe( ) and calls PMD specific dev_uninit( ) to releaase
resource allocated by dev_init( ) in PMD layer.

To keep same port id and reset the port, only dev_uninit() and
dev_init( ) in PMD can be called and keep all resources allocated
from rte_ethdev layer poart in PCI probe( ). All these are what
rte_eth_dev_reset() does.

The rte_eth_dev_reset( ) calls rte_eth_dev_stop( ), PMD dev_uninit( )
and then PMD dev_init( ) to reset a port and keep same port id.

After calling rte_eth_dev_reset( ), the application can go through
rte_eth_dev_configure( ), rte_eth_rx_queue_setup( ),
rte_eth_tx_queue_setup( ) and rte_eth_dev_start( ) again to restore
its previous settings or to reconfigure itself with different settings.

To test this new feature, a testpmd command "port reset port_id" is added.
The mapping between port number and its PCI address can be monitored to
confirm its port number is kept.
And following test case can also be used to confirm the port can work
again after reset.

A typical test steps are listed as follows:
For example, run "ifconfig PF-name down" will trigger a reset to VF.
1.  run testpmd with 2 ixgbe VF ports belonging to same PF
2.  testpmd > set verbose 1 //to observe VF working
3.  testpmd > show port info all //show port number and MAC addr
4.  testpmd > start
5.  let all ports forwarding work for a while
6.  testpmd > show port stats all
7.  ifconfig name-of-PF down
8.  A message is shown in testmd to indicate PF reset
9.  ifconfig name-of-PF up
10. testpmd > stop // stop forwarding to avoid crash during reset
11. testpmd > port reset all
12. testpmd > port stop all
13. testpmd > port start all //recofnig all ports
14. testpmd > show port info all
    //get mapping of port id and MAC addr for forwarding
15. testpmd > start // restore forwarding
14. let all ports forwarding work for a while
15. testpmd > show port stats all //confirm all port can work again
16. repeat above step 7 - 15

chagnes:
v6:
  add more comments to explain why the rte_eth_dev_reset( ) is needeed,
  when it is called and what the application should do after calling it.
  add more comments to explain why reset of PF with SR-IOV is not
  supported currently.
 
v5:
  remove PCI address output to align with other modification which
    will output it in other way
  disable PF reset when its VF is ative to avoid unexpected VF behavior
v4:
  add PCI address to confirm its port number keep same
  correct test method in cover letter
v3:
  update testpmd command
v2:
  only reset PMD layer resource and keep same port id, but
  not restore settings

Signed-off-by: Wei Dai <wei.dai@intel.com>
Tested-by: Yuan Peng <yuan.peng@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>

Wei Dai (4):
  ethdev: add support of NIC reset
  net/ixgbe: add support of reset
  net/i40e: add support of reset
  app/testpmd: enhance command to test NIC reset

 app/test-pmd/cmdline.c                 | 10 +++++---
 app/test-pmd/testpmd.c                 | 41 +++++++++++++++++++++++++++++
 app/test-pmd/testpmd.h                 |  1 +
 drivers/net/i40e/i40e_ethdev.c         | 28 ++++++++++++++++++++
 drivers/net/i40e/i40e_ethdev_vf.c      | 19 ++++++++++++++
 drivers/net/ixgbe/ixgbe_ethdev.c       | 47 ++++++++++++++++++++++++++++++++++
 lib/librte_ether/rte_ethdev.c          | 16 ++++++++++++
 lib/librte_ether/rte_ethdev.h          | 45 ++++++++++++++++++++++++++++++++
 lib/librte_ether/rte_ether_version.map |  2 +-
 9 files changed, 205 insertions(+), 4 deletions(-)

-- 
2.7.5

[PATCH v6 1/4] ethdev: add support of NIC reset

[Wei Dai]

This patch adds a new eth_dev layer API function rte_eth_dev_reset().
A DPDK application can call this function to reset a NIC and keep
its port id afterwards.
It means that all SW resources allocated in ethdev layer should be
kept and SW and HW resources of the NIC in PMD need to be reset in
similar way that it runs PCI dev_uninit() and then dev_init().
The sequence of dev_uninit() and dev_init() can be packed into a
single interface API rte_eth_dev_reset().
Please See the comments before the declaration of rte_eht_dev_reset()
in lib/librte_ether/rte_ethdev.h to get more details on why this
function is needed, what this function does and what an application
should do after calling this function.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 lib/librte_ether/rte_ethdev.c          | 16 ++++++++++++
 lib/librte_ether/rte_ethdev.h          | 45 ++++++++++++++++++++++++++++++++++
 lib/librte_ether/rte_ether_version.map |  2 +-
 3 files changed, 62 insertions(+), 1 deletion(-)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index 76179fd..21ea5bb 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -3414,3 +3414,19 @@ rte_eth_dev_adjust_nb_rx_tx_desc(uint8_t port_id,
 
 	return 0;
 }
+
+int
+rte_eth_dev_reset(uint8_t port_id)
+{
+	struct rte_eth_dev *dev;
+	int ret;
+
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
+	dev = &rte_eth_devices[port_id];
+
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_reset, -ENOTSUP);
+
+	rte_eth_dev_stop(port_id);
+	ret = dev->dev_ops->dev_reset(dev);
+	return ret;
+}
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index fd6baf3..f5fd047 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1113,6 +1113,9 @@ typedef int  (*eth_dev_set_link_down_t)(struct rte_eth_dev *dev);
 typedef void (*eth_dev_close_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to close a configured Ethernet device. */
 
+typedef int (*eth_dev_reset_t)(struct rte_eth_dev *dev);
+/** <@internal Function used to reset a configured Ethernet device. */
+
 typedef void (*eth_promiscuous_enable_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to enable the RX promiscuous mode of an Ethernet device. */
 
@@ -1430,6 +1433,7 @@ struct eth_dev_ops {
 	eth_dev_set_link_up_t      dev_set_link_up;   /**< Device link up. */
 	eth_dev_set_link_down_t    dev_set_link_down; /**< Device link down. */
 	eth_dev_close_t            dev_close;     /**< Close device. */
+	eth_dev_reset_t		   dev_reset;	  /**< Reset device. */
 	eth_link_update_t          link_update;   /**< Get device link state. */
 
 	eth_promiscuous_enable_t   promiscuous_enable; /**< Promiscuous ON. */
@@ -2132,6 +2136,47 @@ int rte_eth_dev_set_link_down(uint8_t port_id);
 void rte_eth_dev_close(uint8_t port_id);
 
 /**
+ * Reset a Ethernet device and keep its port id.
+ *
+ * A DPDK application calls this function to do an initiative or passive
+ * reset of a port.
+ *
+ * Sometimes a port have to be reset passively. For example a PF is reset,
+ * all its VFs should also be reset by application itself to align with the
+ * PF.
+ * A DPDK application also can call this function to trigger an initative
+ * port reset.
+ *
+ * When processing reset, if the port goes through PCI remove() and then
+ * PCI probe() for restoration, its port id may be changed and this is not
+ * expected by some DPDK application.
+ *
+ * Normally, PCI probe() includes two parts: one is in rte_ethdev layer
+ * to allocate resource in rte_ethdev layer and the other is calling PMD
+ * specific dev_init() to allocate and initialize resource in PMD layer.
+ * PCI remove( ) releases all resource allocated from rte_ethdev layer
+ * in PCI probe( ) and calls PMD specific dev_uninit( ) to releaase
+ * resource allocated by dev_init( ) in PMD layer.
+ *
+ * To keep same port id and reset the port, only dev_uninit() and
+ * dev_init( ) in PMD can be called and keep all resources allocated
+ * from rte_ethdev layer poart in PCI probe( ). All these are what
+ * rte_eth_dev_reset() does.
+ *
+ * The rte_eth_dev_reset( ) calls rte_eth_dev_stop( ), PMD dev_uninit( )
+ * and then PMD dev_init( ) to reset a port and keep same port id.
+ *
+ * After calling rte_eth_dev_reset( ), the application can go through
+ * rte_eth_dev_configure( ), rte_eth_rx_queue_setup( ),
+ * rte_eth_tx_queue_setup( ) and rte_eth_dev_start( ) again to restore
+ * its previous settings or to reconfigure itself with different settings.
+ *
+ * @param port_id
+ *   The port identifier of the Ethernet device.
+ */
+int rte_eth_dev_reset(uint8_t port_id);
+
+/**
  * Enable receipt in promiscuous mode for an Ethernet device.
  *
  * @param port_id
diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map
index 08deb1f..d89a101 100644
--- a/lib/librte_ether/rte_ether_version.map
+++ b/lib/librte_ether/rte_ether_version.map
@@ -155,5 +155,5 @@ DPDK_17.08 {
 	rte_eth_dev_adjust_nb_rx_tx_desc;
 	rte_flow_copy;
 	rte_flow_isolate;
-
+	rte_eth_dev_reset;
 } DPDK_17.05;
-- 
2.7.5

Re: [PATCH v6 1/4] ethdev: add support of NIC reset

[Jerin Jacob]

-----Original Message-----
> Date: Mon, 10 Jul 2017 18:05:41 +0800
> From: Wei Dai <wei.dai@intel.com>
> To: thomas@monjalon.net, wenzhuo.lu@intel.com,
>  konstantin.ananyev@intel.com, jingjing.wu@intel.com, beilei.xing@intel.com
> CC: dev@dpdk.org, Wei Dai <wei.dai@intel.com>
> Subject: [dpdk-dev] [PATCH v6 1/4] ethdev: add support of NIC reset
> X-Mailer: git-send-email 2.7.5
> 
> This patch adds a new eth_dev layer API function rte_eth_dev_reset().
> A DPDK application can call this function to reset a NIC and keep
> its port id afterwards.
> It means that all SW resources allocated in ethdev layer should be
> kept and SW and HW resources of the NIC in PMD need to be reset in
> similar way that it runs PCI dev_uninit() and then dev_init().
> The sequence of dev_uninit() and dev_init() can be packed into a
> single interface API rte_eth_dev_reset().
> Please See the comments before the declaration of rte_eht_dev_reset()
> in lib/librte_ether/rte_ethdev.h to get more details on why this
> function is needed, what this function does and what an application
> should do after calling this function.
> 
> Signed-off-by: Wei Dai <wei.dai@intel.com>
> ---
>  /**
> + * Reset a Ethernet device and keep its port id.
> + *
> + * A DPDK application calls this function to do an initiative or passive
> + * reset of a port.
> + *
> + * Sometimes a port have to be reset passively. For example a PF is reset,
> + * all its VFs should also be reset by application itself to align with the
> + * PF.

May be I didn't understood this use case properly,But, PF can always send mailbox
message to VF on PF reset. Right?
On such message from PF, VF can transparently reset without application
knowledge(i.e rx and/or tx burst return zero)

> + * A DPDK application also can call this function to trigger an initative
> + * port reset.

When apart from the above use case? Even if it is above case, we should have
some event to notify application to initiate the reset.Right? Without the
event,  When or on what basics application needs to initiate reset?

> + *

Re: [PATCH v6 1/4] ethdev: add support of NIC reset

[Dai, Wei]

-----Original Message-----
From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com] 
Sent: Monday, July 10, 2017 7:35 PM
To: Dai, Wei <wei.dai@intel.com>
Cc: thomas@monjalon.net; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei <beilei.xing@intel.com>; dev@dpdk.org
Subject: Re: [dpdk-dev] [PATCH v6 1/4] ethdev: add support of NIC reset

-----Original Message-----
> Date: Mon, 10 Jul 2017 18:05:41 +0800
> From: Wei Dai <wei.dai@intel.com>
> To: thomas@monjalon.net, wenzhuo.lu@intel.com,  
> konstantin.ananyev@intel.com, jingjing.wu@intel.com, 
> beilei.xing@intel.com
> CC: dev@dpdk.org, Wei Dai <wei.dai@intel.com>
> Subject: [dpdk-dev] [PATCH v6 1/4] ethdev: add support of NIC reset
> X-Mailer: git-send-email 2.7.5
> 
> This patch adds a new eth_dev layer API function rte_eth_dev_reset().
> A DPDK application can call this function to reset a NIC and keep its 
> port id afterwards.
> It means that all SW resources allocated in ethdev layer should be 
> kept and SW and HW resources of the NIC in PMD need to be reset in 
> similar way that it runs PCI dev_uninit() and then dev_init().
> The sequence of dev_uninit() and dev_init() can be packed into a 
> single interface API rte_eth_dev_reset().
> Please See the comments before the declaration of rte_eht_dev_reset() 
> in lib/librte_ether/rte_ethdev.h to get more details on why this 
> function is needed, what this function does and what an application 
> should do after calling this function.
> 
> Signed-off-by: Wei Dai <wei.dai@intel.com>
> ---
>  /**
> + * Reset a Ethernet device and keep its port id.
> + *
> + * A DPDK application calls this function to do an initiative or 
> + passive
> + * reset of a port.
> + *
> + * Sometimes a port have to be reset passively. For example a PF is 
> + reset,
> + * all its VFs should also be reset by application itself to align 
> + with the
> + * PF.

May be I didn't understood this use case properly,But, PF can always send mailbox message to VF on PF reset. Right?
[Wei: As to ixgbe, PF kernel driver always send mailbox message to VF when PF is reset. For other NICs, there are maybe 
other mechanism to notify VF of PF reset.]
On such message from PF, VF can transparently reset without application knowledge(i.e rx and/or tx burst return zero)
[Wei: VF reset is handling many HW registers which may have effect on working Rx/Tx path and may cause some unexpected
behavior like crash. So application should make some operations like stopping Rx/Tx path before it begin VF reset. This is why
application should handle VF reset itself.]

> + * A DPDK application also can call this function to trigger an 
> + initative
> + * port reset.

When apart from the above use case? Even if it is above case, we should have some event to notify application to initiate the reset.Right? Without the event,  When or on what basics application needs to initiate reset?
[Wei: Indeed, until now we didn't see any use case which DPDK application initiative port reset itself. 
The most usual case is that PF is working with kernel driver and VFs are working with DPDK PMD.
Some operations on kernel PF lead to a PF reset which causes its VF reset.
Anyway this new function provides a possibility for application to trigger an initiative port reset.]

> + *

Re: [PATCH v6 1/4] ethdev: add support of NIC reset

[Jerin Jacob]

-----Original Message-----
> Date: Tue, 11 Jul 2017 01:57:15 +0000
> From: "Dai, Wei" <wei.dai@intel.com>
> To: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> CC: "thomas@monjalon.net" <thomas@monjalon.net>, "Lu, Wenzhuo"
>  <wenzhuo.lu@intel.com>, "Ananyev, Konstantin"
>  <konstantin.ananyev@intel.com>, "Wu, Jingjing" <jingjing.wu@intel.com>,
>  "Xing, Beilei" <beilei.xing@intel.com>, "dev@dpdk.org" <dev@dpdk.org>
> Subject: RE: [dpdk-dev] [PATCH v6 1/4] ethdev: add support of NIC reset
> 
> 
> > + * A DPDK application also can call this function to trigger an 
> > + initative
> > + * port reset.
> 
> When apart from the above use case? Even if it is above case, we should have some event to notify application to initiate the reset.Right? Without the event,  When or on what basics application needs to initiate reset?
> [Wei: Indeed, until now we didn't see any use case which DPDK application initiative port reset itself. 
> The most usual case is that PF is working with kernel driver and VFs are working with DPDK PMD.
> Some operations on kernel PF lead to a PF reset which causes its VF reset.
> Anyway this new function provides a possibility for application to trigger an initiative port reset.]

That's fine. The only concern part is when to call reset API from
application. Can we say on RTE_ETH_EVENT_INTR_RESET event, application
needs to call the reset API? I think, it is important to specify when
application need to call this API, otherwise this api behavior will be
tightly coupled with underneath PMD. Side effect is, a new, non portable PMD specific API.
If a PMD wishes to fixup some overflow case(as an example), by invoking the reset API from
the application BUT same case may not valid for another PMD hence it
will create unexpected behavior from application based on the underneath PMD.

if RTE_ETH_EVENT_INTR_RESET event is not expected event to call the
reset API then create a new event or if it needs to be called in
RTE_ETH_EVENT_INTR_RESET then update the API documentation.

> 
> > + *

Re: [PATCH v6 1/4] ethdev: add support of NIC reset

[Dai, Wei]

> -----Original Message-----
> From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com]
> Sent: Tuesday, July 11, 2017 1:17 PM
> To: Dai, Wei <wei.dai@intel.com>
> Cc: thomas@monjalon.net; Lu, Wenzhuo <wenzhuo.lu@intel.com>;
> Ananyev, Konstantin <konstantin.ananyev@intel.com>; Wu, Jingjing
> <jingjing.wu@intel.com>; Xing, Beilei <beilei.xing@intel.com>;
> dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v6 1/4] ethdev: add support of NIC reset
> 
> -----Original Message-----
> > Date: Tue, 11 Jul 2017 01:57:15 +0000
> > From: "Dai, Wei" <wei.dai@intel.com>
> > To: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> > CC: "thomas@monjalon.net" <thomas@monjalon.net>, "Lu, Wenzhuo"
> >  <wenzhuo.lu@intel.com>, "Ananyev, Konstantin"
> >  <konstantin.ananyev@intel.com>, "Wu, Jingjing"
> > <jingjing.wu@intel.com>,  "Xing, Beilei" <beilei.xing@intel.com>,
> > "dev@dpdk.org" <dev@dpdk.org>
> > Subject: RE: [dpdk-dev] [PATCH v6 1/4] ethdev: add support of NIC
> > reset
> >
> >
> > > + * A DPDK application also can call this function to trigger an
> > > + initative
> > > + * port reset.
> >
> > When apart from the above use case? Even if it is above case, we should
> have some event to notify application to initiate the reset.Right? Without
> the event,  When or on what basics application needs to initiate reset?
> > [Wei: Indeed, until now we didn't see any use case which DPDK application
> initiative port reset itself.
> > The most usual case is that PF is working with kernel driver and VFs are
> working with DPDK PMD.
> > Some operations on kernel PF lead to a PF reset which causes its VF reset.
> > Anyway this new function provides a possibility for application to
> > trigger an initiative port reset.]
> 
> That's fine. The only concern part is when to call reset API from application.
> Can we say on RTE_ETH_EVENT_INTR_RESET event, application needs to
> call the reset API? I think, it is important to specify when application need to
> call this API, otherwise this api behavior will be tightly coupled with
> underneath PMD. Side effect is, a new, non portable PMD specific API.
> If a PMD wishes to fixup some overflow case(as an example), by invoking the
> reset API from the application BUT same case may not valid for another
> PMD hence it will create unexpected behavior from application based on the
> underneath PMD.
It is duty of PMD to trigger RTE_ETH_EVENT_INTR_RESET event and application
should also register some callback function to handle this event. 
When PMD wants to trigger a reset, it can trigger RTE_ETH_EVENT_INTR_RESET.
On the received event of RTE_ETH_EVENT_INTR_RESET, application can begin to
handle it:  stop working queues,  make rx and tx function not be called 
and then call rte_eth_dev_reset( ).
For thread safety, all these controlling operations had better be made in same thread.
For example, when ixgbe PF is reset, PF send a message to notify VF this event and 
also trigger an interrupt to VF.  And then in the interrupt service routine DPDK VF 
detect this notification message and calls 
_rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_RESET, NULL, NULL).
So it means that PF reset trigger RTE_ETH_EVENT_INTR_RESET event in VF.
The function _rte_eth_dev_callback_process( ) will call the registered callback function.
The callback function can trigger application to handle all operations of VF reset including
something like stopping working Rx/Tx queues and call this rte_eth_dev_reset().
The rte_eth_dev_reset( ) itself is generic function which only does some HW reset operations
through calling dev_unint() and dev_init(). And itself doesn't handle above synchronization which
is handled by application.
PMD itself should not call rte_eth_dev_reset( ). PMD can trigger application to handle reset event.
It is duty of application to handle all synchronization befort it calls rte_eth_dev_reset( ).

> 
> if RTE_ETH_EVENT_INTR_RESET event is not expected event to call the reset
> API then create a new event or if it needs to be called in
> RTE_ETH_EVENT_INTR_RESET then update the API documentation.
> 
Of course, when PMD wants to trigger a reset event, it can trigger other event other than
RTE_ETH_EVENT_INTR_RESET. So the application should know which the alternate event is.
This make application more complex. So it is suggested that only RTE_ETH_EVENT_INTR_RESET
can be used to trigger a port reset.

> >
> > > + *

Re: [PATCH v6 1/4] ethdev: add support of NIC reset

[Jerin Jacob]

-----Original Message-----
> Date: Tue, 11 Jul 2017 14:36:57 +0000
> From: "Dai, Wei" <wei.dai@intel.com>
> To: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> CC: "thomas@monjalon.net" <thomas@monjalon.net>, "Lu, Wenzhuo"
>  <wenzhuo.lu@intel.com>, "Ananyev, Konstantin"
>  <konstantin.ananyev@intel.com>, "Wu, Jingjing" <jingjing.wu@intel.com>,
>  "Xing, Beilei" <beilei.xing@intel.com>, "dev@dpdk.org" <dev@dpdk.org>
> Subject: RE: [dpdk-dev] [PATCH v6 1/4] ethdev: add support of NIC reset
> 
> > -----Original Message-----
> > From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com]
> > Sent: Tuesday, July 11, 2017 1:17 PM
> > To: Dai, Wei <wei.dai@intel.com>
> > Cc: thomas@monjalon.net; Lu, Wenzhuo <wenzhuo.lu@intel.com>;
> > Ananyev, Konstantin <konstantin.ananyev@intel.com>; Wu, Jingjing
> > <jingjing.wu@intel.com>; Xing, Beilei <beilei.xing@intel.com>;
> > dev@dpdk.org
> > Subject: Re: [dpdk-dev] [PATCH v6 1/4] ethdev: add support of NIC reset
> > 
> > -----Original Message-----
> > > Date: Tue, 11 Jul 2017 01:57:15 +0000
> > > From: "Dai, Wei" <wei.dai@intel.com>
> > > To: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> > > CC: "thomas@monjalon.net" <thomas@monjalon.net>, "Lu, Wenzhuo"
> > >  <wenzhuo.lu@intel.com>, "Ananyev, Konstantin"
> > >  <konstantin.ananyev@intel.com>, "Wu, Jingjing"
> > > <jingjing.wu@intel.com>,  "Xing, Beilei" <beilei.xing@intel.com>,
> > > "dev@dpdk.org" <dev@dpdk.org>
> > > Subject: RE: [dpdk-dev] [PATCH v6 1/4] ethdev: add support of NIC
> > > reset
> > >
> > >
> > > > + * A DPDK application also can call this function to trigger an
> > > > + initative
> > > > + * port reset.
> > >
> > > When apart from the above use case? Even if it is above case, we should
> > have some event to notify application to initiate the reset.Right? Without
> > the event,  When or on what basics application needs to initiate reset?
> > > [Wei: Indeed, until now we didn't see any use case which DPDK application
> > initiative port reset itself.
> > > The most usual case is that PF is working with kernel driver and VFs are
> > working with DPDK PMD.
> > > Some operations on kernel PF lead to a PF reset which causes its VF reset.
> > > Anyway this new function provides a possibility for application to
> > > trigger an initiative port reset.]
> > 
> > That's fine. The only concern part is when to call reset API from application.
> > Can we say on RTE_ETH_EVENT_INTR_RESET event, application needs to
> > call the reset API? I think, it is important to specify when application need to
> > call this API, otherwise this api behavior will be tightly coupled with
> > underneath PMD. Side effect is, a new, non portable PMD specific API.
> > If a PMD wishes to fixup some overflow case(as an example), by invoking the
> > reset API from the application BUT same case may not valid for another
> > PMD hence it will create unexpected behavior from application based on the
> > underneath PMD.
> It is duty of PMD to trigger RTE_ETH_EVENT_INTR_RESET event and application
> should also register some callback function to handle this event. 
> When PMD wants to trigger a reset, it can trigger RTE_ETH_EVENT_INTR_RESET.
> On the received event of RTE_ETH_EVENT_INTR_RESET, application can begin to
> handle it:  stop working queues,  make rx and tx function not be called 
> and then call rte_eth_dev_reset( ).
> For thread safety, all these controlling operations had better be made in same thread.
> For example, when ixgbe PF is reset, PF send a message to notify VF this event and 
> also trigger an interrupt to VF.  And then in the interrupt service routine DPDK VF 
> detect this notification message and calls 
> _rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_RESET, NULL, NULL).
> So it means that PF reset trigger RTE_ETH_EVENT_INTR_RESET event in VF.
> The function _rte_eth_dev_callback_process( ) will call the registered callback function.
> The callback function can trigger application to handle all operations of VF reset including
> something like stopping working Rx/Tx queues and call this rte_eth_dev_reset().
> The rte_eth_dev_reset( ) itself is generic function which only does some HW reset operations
> through calling dev_unint() and dev_init(). And itself doesn't handle above synchronization which
> is handled by application.
> PMD itself should not call rte_eth_dev_reset( ). PMD can trigger application to handle reset event.
> It is duty of application to handle all synchronization befort it calls rte_eth_dev_reset( ).

No disagreement on the expected behavior.


> 
> > 
> > if RTE_ETH_EVENT_INTR_RESET event is not expected event to call the reset
> > API then create a new event or if it needs to be called in
> > RTE_ETH_EVENT_INTR_RESET then update the API documentation.
> > 
> Of course, when PMD wants to trigger a reset event, it can trigger other event other than
> RTE_ETH_EVENT_INTR_RESET. So the application should know which the alternate event is.
> This make application more complex. So it is suggested that only RTE_ETH_EVENT_INTR_RESET
> can be used to trigger a port reset.

Yes. I suggest to add this info on documentation. ie "application invokes the
reset API on RTE_ETH_EVENT_INTR_RESET event". That will answer "when"
application need to invoke this API.

> 
> > >
> > > > + *

Re: [PATCH v6 1/4] ethdev: add support of NIC reset

[Dai, Wei]

> > Subject: RE: [dpdk-dev] [PATCH v6 1/4] ethdev: add support of NIC
> > reset
> >
> >
> > > + * A DPDK application also can call this function to trigger an
> > > + initative
> > > + * port reset.
> >
> > When apart from the above use case? Even if it is above case, we should
> have some event to notify application to initiate the reset.Right? Without
> the event,  When or on what basics application needs to initiate reset?
> > [Wei: Indeed, until now we didn't see any use case which DPDK application
> initiative port reset itself.
> > The most usual case is that PF is working with kernel driver and VFs are
> working with DPDK PMD.
> > Some operations on kernel PF lead to a PF reset which causes its VF reset.
> > Anyway this new function provides a possibility for application to
> > trigger an initiative port reset.]
> 
> That's fine. The only concern part is when to call reset API from application.
> Can we say on RTE_ETH_EVENT_INTR_RESET event, application needs to
> call the reset API? I think, it is important to specify when application need to
> call this API, otherwise this api behavior will be tightly coupled with
> underneath PMD. Side effect is, a new, non portable PMD specific API.
> If a PMD wishes to fixup some overflow case(as an example), by invoking the
> reset API from the application BUT same case may not valid for another
> PMD hence it will create unexpected behavior from application based on the
> underneath PMD.
> 
> if RTE_ETH_EVENT_INTR_RESET event is not expected event to call the reset
> API then create a new event or if it needs to be called in
> RTE_ETH_EVENT_INTR_RESET then update the API documentation.
Thanks for your feedback.
I'll add description of this function in API doc in my v7 patch set.
> 
> >
> > > + *

[PATCH v6 2/4] net/ixgbe: add support of reset

[Wei Dai]

Reset a NIC by calling dev_uninit and then dev_init.
Go through same way in NIC PCI remove without release of
ethdev resource and then NIC PCI probe function without
ethdev resource allocation.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 drivers/net/ixgbe/ixgbe_ethdev.c | 47 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)

diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index a0ae089..280f4bb 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -170,6 +170,7 @@ static void ixgbe_dev_stop(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_up(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_down(struct rte_eth_dev *dev);
 static void ixgbe_dev_close(struct rte_eth_dev *dev);
+static int  ixgbe_dev_reset(struct rte_eth_dev *dev);
 static void ixgbe_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void ixgbe_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void ixgbe_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -266,6 +267,7 @@ static int ixgbevf_dev_link_update(struct rte_eth_dev *dev,
 				   int wait_to_complete);
 static void ixgbevf_dev_stop(struct rte_eth_dev *dev);
 static void ixgbevf_dev_close(struct rte_eth_dev *dev);
+static int  ixgbevf_dev_reset(struct rte_eth_dev *dev);
 static void ixgbevf_intr_disable(struct ixgbe_hw *hw);
 static void ixgbevf_intr_enable(struct ixgbe_hw *hw);
 static void ixgbevf_dev_stats_get(struct rte_eth_dev *dev,
@@ -527,6 +529,7 @@ static const struct eth_dev_ops ixgbe_eth_dev_ops = {
 	.dev_set_link_up    = ixgbe_dev_set_link_up,
 	.dev_set_link_down  = ixgbe_dev_set_link_down,
 	.dev_close            = ixgbe_dev_close,
+	.dev_reset	      = ixgbe_dev_reset,
 	.promiscuous_enable   = ixgbe_dev_promiscuous_enable,
 	.promiscuous_disable  = ixgbe_dev_promiscuous_disable,
 	.allmulticast_enable  = ixgbe_dev_allmulticast_enable,
@@ -616,6 +619,7 @@ static const struct eth_dev_ops ixgbevf_eth_dev_ops = {
 	.xstats_reset         = ixgbevf_dev_stats_reset,
 	.xstats_get_names     = ixgbevf_dev_xstats_get_names,
 	.dev_close            = ixgbevf_dev_close,
+	.dev_reset	      = ixgbevf_dev_reset,
 	.allmulticast_enable  = ixgbevf_dev_allmulticast_enable,
 	.allmulticast_disable = ixgbevf_dev_allmulticast_disable,
 	.dev_infos_get        = ixgbevf_dev_info_get,
@@ -2840,6 +2844,32 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 	ixgbe_set_rar(hw, 0, hw->mac.addr, 0, IXGBE_RAH_AV);
 }
 
+/*
+ * Reest PF device.
+ */
+static int
+ixgbe_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	/* When a DPDK PMD PF begin to reset PF port, it should notify all
+	 * its VF to make them align with it. The detailed notification
+	 * mechanism is PMD specific. As to ixgbe PF, it is rather complex.
+	 * To avoid unexpected behavior in VF, currently reset of PF with
+	 * SR-IOV activation is not supported. It might be supported later.
+	 */
+	if (dev->data->sriov.active)
+		return -ENOTSUP;
+
+	ret = eth_ixgbe_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_ixgbe_dev_init(dev);
+
+	return ret;
+}
+
 static void
 ixgbe_read_stats_registers(struct ixgbe_hw *hw,
 			   struct ixgbe_hw_stats *hw_stats,
@@ -5036,6 +5066,23 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	ixgbevf_remove_mac_addr(dev, 0);
 }
 
+/*
+ * Reset VF device
+ */
+static int
+ixgbevf_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	ret = eth_ixgbevf_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_ixgbevf_dev_init(dev);
+
+	return ret;
+}
+
 static void ixgbevf_set_vfta_all(struct rte_eth_dev *dev, bool on)
 {
 	struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-- 
2.7.5

[PATCH v6 3/4] net/i40e: add support of reset

[Wei Dai]

Reset a NIC by calling dev_uninit() and then dev_init().
Go through the same way in NIC PCI remove without release
of ethdev resource and then NIC PCI probe function without
ethdev resource allocation.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 drivers/net/i40e/i40e_ethdev.c    | 28 ++++++++++++++++++++++++++++
 drivers/net/i40e/i40e_ethdev_vf.c | 19 +++++++++++++++++++
 2 files changed, 47 insertions(+)

diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 71cb7d3..1b00717 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -250,6 +250,7 @@ static int i40e_dev_configure(struct rte_eth_dev *dev);
 static int i40e_dev_start(struct rte_eth_dev *dev);
 static void i40e_dev_stop(struct rte_eth_dev *dev);
 static void i40e_dev_close(struct rte_eth_dev *dev);
+static int  i40e_dev_reset(struct rte_eth_dev *dev);
 static void i40e_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void i40e_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void i40e_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -449,6 +450,7 @@ static const struct eth_dev_ops i40e_eth_dev_ops = {
 	.dev_start                    = i40e_dev_start,
 	.dev_stop                     = i40e_dev_stop,
 	.dev_close                    = i40e_dev_close,
+	.dev_reset		      = i40e_dev_reset,
 	.promiscuous_enable           = i40e_dev_promiscuous_enable,
 	.promiscuous_disable          = i40e_dev_promiscuous_disable,
 	.allmulticast_enable          = i40e_dev_allmulticast_enable,
@@ -2136,6 +2138,32 @@ i40e_dev_close(struct rte_eth_dev *dev)
 	I40E_WRITE_FLUSH(hw);
 }
 
+/*
+ * Reest PF device only to re-initialize resources in PMD layer
+ */
+static int
+i40e_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	/* When a DPDK PMD PF begin to reset PF port, it should notify all
+	 * its VF to make them align with it. The detailed notification
+	 * mechanism is PMD specific. As to i40e PF, it is rather complex.
+	 * To avoid unexpected behavior in VF, currently reset of PF with
+	 * SR-IOV activation is not supported. It might be supported later.
+	 */
+	if (dev->data->sriov.active)
+		return -ENOTSUP;
+
+	ret = eth_i40e_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_i40e_dev_init(dev);
+
+	return ret;
+}
+
 static void
 i40e_dev_promiscuous_enable(struct rte_eth_dev *dev)
 {
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index bab09f8..858d2a5 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -123,6 +123,7 @@ static void i40evf_vlan_offload_set(struct rte_eth_dev *dev, int mask);
 static int i40evf_vlan_pvid_set(struct rte_eth_dev *dev, uint16_t pvid,
 				int on);
 static void i40evf_dev_close(struct rte_eth_dev *dev);
+static int  i40evf_dev_reset(struct rte_eth_dev *dev);
 static void i40evf_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void i40evf_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void i40evf_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -204,6 +205,7 @@ static const struct eth_dev_ops i40evf_eth_dev_ops = {
 	.xstats_get_names     = i40evf_dev_xstats_get_names,
 	.xstats_reset         = i40evf_dev_xstats_reset,
 	.dev_close            = i40evf_dev_close,
+	.dev_reset	      = i40evf_dev_reset,
 	.dev_infos_get        = i40evf_dev_info_get,
 	.dev_supported_ptypes_get = i40e_dev_supported_ptypes_get,
 	.vlan_filter_set      = i40evf_vlan_filter_set,
@@ -2356,6 +2358,23 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	i40evf_disable_irq0(hw);
 }
 
+/*
+ * Reest VF device only to re-initialize resources in PMD layer
+ */
+static int
+i40evf_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	ret = i40evf_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = i40evf_dev_init(dev);
+
+	return ret;
+}
+
 static int
 i40evf_get_rss_lut(struct i40e_vsi *vsi, uint8_t *lut, uint16_t lut_size)
 {
-- 
2.7.5

[PATCH v6 4/4] app/testpmd: enhance command to test NIC reset

[Wei Dai]

When PF is reset, a message will show it and all its
VF need to be reset.
User can run the command "port reset port_id"
to reset the VF port and to keep same port id without
any configuration. Then user can run "port stop port_id"
and "port start port_id" to reconfigure its forwarding
mode and parmaters as previous ones.
To avoid crash, current forwarding should be stopped
before running "port reset port_id".

Signed-off-by: Wei Dai <wei.dai@intel.com>
Tested-by: Yuan Peng <yuan.peng@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>
---
 app/test-pmd/cmdline.c | 10 +++++++---
 app/test-pmd/testpmd.c | 41 +++++++++++++++++++++++++++++++++++++++++
 app/test-pmd/testpmd.h |  1 +
 3 files changed, 49 insertions(+), 3 deletions(-)

diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
index d4ff608..d096adc 100644
--- a/app/test-pmd/cmdline.c
+++ b/app/test-pmd/cmdline.c
@@ -970,6 +970,8 @@ static void cmd_operate_port_parsed(void *parsed_result,
 		stop_port(RTE_PORT_ALL);
 	else if (!strcmp(res->name, "close"))
 		close_port(RTE_PORT_ALL);
+	else if (!strcmp(res->name, "reset"))
+		reset_port(RTE_PORT_ALL);
 	else
 		printf("Unknown parameter\n");
 }
@@ -979,7 +981,7 @@ cmdline_parse_token_string_t cmd_operate_port_all_cmd =
 								"port");
 cmdline_parse_token_string_t cmd_operate_port_all_port =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_port_result, name,
-						"start#stop#close");
+						"start#stop#close#reset");
 cmdline_parse_token_string_t cmd_operate_port_all_all =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_port_result, value, "all");
 
@@ -1014,6 +1016,8 @@ static void cmd_operate_specific_port_parsed(void *parsed_result,
 		stop_port(res->value);
 	else if (!strcmp(res->name, "close"))
 		close_port(res->value);
+	else if (!strcmp(res->name, "reset"))
+		reset_port(res->value);
 	else
 		printf("Unknown parameter\n");
 }
@@ -1023,7 +1027,7 @@ cmdline_parse_token_string_t cmd_operate_specific_port_cmd =
 							keyword, "port");
 cmdline_parse_token_string_t cmd_operate_specific_port_port =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_specific_port_result,
-						name, "start#stop#close");
+						name, "start#stop#close#reset");
 cmdline_parse_token_num_t cmd_operate_specific_port_id =
 	TOKEN_NUM_INITIALIZER(struct cmd_operate_specific_port_result,
 							value, UINT8);
@@ -1031,7 +1035,7 @@ cmdline_parse_token_num_t cmd_operate_specific_port_id =
 cmdline_parse_inst_t cmd_operate_specific_port = {
 	.f = cmd_operate_specific_port_parsed,
 	.data = NULL,
-	.help_str = "port start|stop|close <port_id>: Start/Stop/Close port_id",
+	.help_str = "port start|stop|close|reset <port_id>: Start/Stop/Close/Reset port_id",
 	.tokens = {
 		(void *)&cmd_operate_specific_port_cmd,
 		(void *)&cmd_operate_specific_port_port,
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index e09b803..8eb4292 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -1669,6 +1669,47 @@ close_port(portid_t pid)
 }
 
 void
+reset_port(portid_t pid)
+{
+	int diag;
+	portid_t pi;
+	struct rte_port *port;
+
+	if (port_id_is_invalid(pid, ENABLED_WARN))
+		return;
+
+	printf("Resetting ports...\n");
+
+	RTE_ETH_FOREACH_DEV(pi) {
+		if (pid != pi && pid != (portid_t)RTE_PORT_ALL)
+			continue;
+
+		if (port_is_forwarding(pi) != 0 && test_done == 0) {
+			printf("Please remove port %d from forwarding "
+			       "configuration.\n", pi);
+			continue;
+		}
+
+		if (port_is_bonding_slave(pi)) {
+			printf("Please remove port %d from bonded device.\n",
+			       pi);
+			continue;
+		}
+
+		diag = rte_eth_dev_reset(pi);
+		if (diag == 0) {
+			port = &ports[pi];
+			port->need_reconfig = 1;
+			port->need_reconfig_queues = 1;
+		} else {
+			printf("Failed to reset port %d. diag=%d\n", pi, diag);
+		}
+	}
+
+	printf("Done\n");
+}
+
+void
 attach_port(char *identifier)
 {
 	portid_t pi = 0;
diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
index 73985c3..fb44fdd 100644
--- a/app/test-pmd/testpmd.h
+++ b/app/test-pmd/testpmd.h
@@ -609,6 +609,7 @@ int init_port_dcb_config(portid_t pid, enum dcb_mode_enable dcb_mode,
 int start_port(portid_t pid);
 void stop_port(portid_t pid);
 void close_port(portid_t pid);
+void reset_port(portid_t pid);
 void attach_port(char *identifier);
 void detach_port(uint8_t port_id);
 int all_ports_stopped(void);
-- 
2.7.5

[PATCH v7 0/5] Support NIC reset and keep same port id

[Wei Dai]

This patch set adds a function rte_eth_dev_reset( ) in rte_ethdev layer.
Sometimes a port have to be reset passively. For example a PF is reset,
all its VFs should also be reset by application itself to align with 
the PF.
A DPDK application also can call this function to trigger an initative
port reset.

When processing reset, if the port goes through PCI remove() and then
PCI probe() for restoration, its port id may be changed and this is not
expected by some DPDK application.

Normally, PCI probe() includes two parts: one is in rte_ethdev layer
to allocate resource in rte_ethdev layer and the other is calling PMD
specific dev_init() to allocate and initialize resource in PMD layer.
PCI remove( ) releases all resource allocated from rte_ethdev layer
in PCI probe( ) and calls PMD specific dev_uninit( ) to releaase
resource allocated by dev_init( ) in PMD layer.

To keep same port id and reset the port, only dev_uninit() and
dev_init( ) in PMD can be called and keep all resources allocated
from rte_ethdev layer poart in PCI probe( ). All these are what
rte_eth_dev_reset() does.

The rte_eth_dev_reset( ) calls rte_eth_dev_stop( ), PMD dev_uninit( )
and then PMD dev_init( ) to reset a port and keep same port id.

After calling rte_eth_dev_reset( ), the application can go through
rte_eth_dev_configure( ), rte_eth_rx_queue_setup( ),
rte_eth_tx_queue_setup( ) and rte_eth_dev_start( ) again to restore
its previous settings or to reconfigure itself with different settings.

To test this new feature, a testpmd command "port reset port_id" is added.
The mapping between port number and its PCI address can be monitored to
confirm its port number is kept.
And following test case can also be used to confirm the port can work
again after reset.

A typical test steps are listed as follows:
For example, run "ifconfig PF-name down" will trigger a reset to VF.
1.  run testpmd with 2 ixgbe VF ports belonging to same PF
2.  testpmd > set verbose 1 //to observe VF working
3.  testpmd > show port info all //show port number and MAC addr
4.  testpmd > start
5.  let all ports forwarding work for a while
6.  testpmd > show port stats all
7.  ifconfig name-of-PF down
8.  A message is shown in testmd to indicate PF reset
9.  ifconfig name-of-PF up
10. testpmd > stop // stop forwarding to avoid crash during reset
11. testpmd > port reset all
12. testpmd > port stop all
13. testpmd > port start all //recofnig all ports
14. testpmd > show port info all
    //get mapping of port id and MAC addr for forwarding
15. testpmd > start // restore forwarding
14. let all ports forwarding work for a while
15. testpmd > show port stats all //confirm all port can work again
16. repeat above step 7 - 15

chagnes:
v7:
  add description of NIC reset in doc/poll_mode_drv.rst

v6:
  add more comments to explain why the rte_eth_dev_reset( ) is needeed,
  when it is called and what the application should do after calling it.
  add more comments to explain why reset of PF with SR-IOV is not
  supported currently.
 
v5:
  remove PCI address output to align with other modification which
    will output it in other way
  disable PF reset when its VF is ative to avoid unexpected VF behavior
v4:
  add PCI address to confirm its port number keep same
  correct test method in cover letter
v3:
  update testpmd command
v2:
  only reset PMD layer resource and keep same port id, but
  not restore settings

Signed-off-by: Wei Dai <wei.dai@intel.com>
Tested-by: Yuan Peng <yuan.peng@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>

Wei Dai (5):
  ethdev: add support of NIC reset
  net/ixgbe: add support of reset
  net/i40e: add support of reset
  app/testpmd: enhance command to test NIC reset
  doc: add description of the NIC reset API

 app/test-pmd/cmdline.c                  | 12 ++++--
 app/test-pmd/testpmd.c                  | 41 ++++++++++++++++++
 app/test-pmd/testpmd.h                  |  1 +
 doc/guides/prog_guide/poll_mode_drv.rst | 40 ++++++++++++++++++
 drivers/net/i40e/i40e_ethdev.c          | 28 +++++++++++++
 drivers/net/i40e/i40e_ethdev_vf.c       | 19 +++++++++
 drivers/net/ixgbe/ixgbe_ethdev.c        | 47 +++++++++++++++++++++
 lib/librte_ether/rte_ethdev.c           | 17 ++++++++
 lib/librte_ether/rte_ethdev.h           | 73 +++++++++++++++++++++++++++++++++
 lib/librte_ether/rte_ether_version.map  |  1 +
 10 files changed, 275 insertions(+), 4 deletions(-)

-- 
2.7.5

[PATCH v7 1/5] ethdev: add support of NIC reset

[Wei Dai]

This patch adds a new eth_dev layer API function rte_eth_dev_reset().
A DPDK application can call this function to reset a NIC and keep
its port id afterwards.
It means that all SW resources allocated in ethdev layer should be
kept and SW and HW resources of the NIC in PMD need to be reset in
similar way that it runs PCI dev_uninit() and then dev_init().
The sequence of dev_uninit() and dev_init() can be packed into a
single interface API rte_eth_dev_reset().
Please See the comments before the declaration of rte_eht_dev_reset()
in lib/librte_ether/rte_ethdev.h to get more details on why this
function is needed, what it does, when it should be called
and what an application should do after calling this function.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 lib/librte_ether/rte_ethdev.c          | 17 ++++++++
 lib/librte_ether/rte_ethdev.h          | 73 ++++++++++++++++++++++++++++++++++
 lib/librte_ether/rte_ether_version.map |  1 +
 3 files changed, 91 insertions(+)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index a1b7447..4172d82 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -3415,3 +3415,20 @@ rte_eth_dev_adjust_nb_rx_tx_desc(uint8_t port_id,
 
 	return 0;
 }
+
+int
+rte_eth_dev_reset(uint8_t port_id)
+{
+	struct rte_eth_dev *dev;
+	int ret;
+
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
+	dev = &rte_eth_devices[port_id];
+
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_reset, -ENOTSUP);
+
+	rte_eth_dev_stop(port_id);
+	ret = dev->dev_ops->dev_reset(dev);
+
+	return ret;
+}
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index f683727..c458c52 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1113,6 +1113,9 @@ typedef int  (*eth_dev_set_link_down_t)(struct rte_eth_dev *dev);
 typedef void (*eth_dev_close_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to close a configured Ethernet device. */
 
+typedef int (*eth_dev_reset_t)(struct rte_eth_dev *dev);
+/** <@internal Function used to reset a configured Ethernet device. */
+
 typedef void (*eth_promiscuous_enable_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to enable the RX promiscuous mode of an Ethernet device. */
 
@@ -1433,6 +1436,7 @@ struct eth_dev_ops {
 	eth_dev_set_link_up_t      dev_set_link_up;   /**< Device link up. */
 	eth_dev_set_link_down_t    dev_set_link_down; /**< Device link down. */
 	eth_dev_close_t            dev_close;     /**< Close device. */
+	eth_dev_reset_t		   dev_reset;	  /**< Reset device. */
 	eth_link_update_t          link_update;   /**< Get device link state. */
 
 	eth_promiscuous_enable_t   promiscuous_enable; /**< Promiscuous ON. */
@@ -2138,6 +2142,75 @@ int rte_eth_dev_set_link_down(uint8_t port_id);
 void rte_eth_dev_close(uint8_t port_id);
 
 /**
+ * Reset a Ethernet device and keep its port id.
+ *
+ * A DPDK application calls this function to do an initiative or passive
+ * reset of a port.
+ *
+ * Sometimes a port have to be reset passively. For example a PF is reset,
+ * all its VFs should also be reset by application itself to align with the
+ * PF. A DPDK application also can call this function to trigger an initative
+ * port reset.
+ *
+ * When processing reset, if the port goes through PCI remove() and then
+ * PCI probe() for restoration, its port id may be changed and this is not
+ * expected by some DPDK application.
+ *
+ * Normally, PCI probe() includes two parts: one is in rte_ethdev layer
+ * to allocate resource in rte_ethdev layer and the other is calling PMD
+ * specific dev_init() to allocate and initialize resource in PMD layer.
+ * PCI remove( ) releases all resource allocated from rte_ethdev layer
+ * in PCI probe( ) and calls PMD specific dev_uninit( ) to releaase
+ * resource allocated by dev_init( ) in PMD layer.
+ *
+ * To keep same port id and reset the port, only dev_uninit() and
+ * dev_init( ) in PMD can be called and keep all resources allocated
+ * from rte_ethdev layer poart in PCI probe( ). All these are what
+ * rte_eth_dev_reset() does.
+ *
+ * The rte_eth_dev_reset( ) calls rte_eth_dev_stop( ), PMD dev_uninit( )
+ * and then PMD dev_init( ) to reset a port and keep same port id.
+ *
+ * Normally, a DPDK application can invoke this function when
+ * RTE_ETH_EVENT_INTR_RESET event is detected.
+
+ * It is duty of PMD to trigger RTE_ETH_EVENT_INTR_RESET event and application
+ * should also register some callback function to handle this event.
+ * When PMD needs to trigger a reset, it can trigger RTE_ETH_EVENT_INTR_RESET.
+ * On the received event of RTE_ETH_EVENT_INTR_RESET, application can begin to
+ * handle it:  stop working queues,  make rx and tx function not be called and
+ * then call rte_eth_dev_reset( ).For thread safety, all these controlling
+ * operations had better be made in same thread.
+ *
+ * For example, when PF is reset, PF send a message to notify VF this event and
+ * also trigger an interrupt to VF.  And then in the interrupt service routine
+ * DPDK VF detect this notification message and calls
+ * _rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_RESET, NULL, NULL).
+ * So it means that PF reset trigger RTE_ETH_EVENT_INTR_RESET event in VF.
+ * The function _rte_eth_dev_callback_process( ) will call the registered
+ * callback function. The callback function can trigger application to handle
+ * all operations of VF reset including something like stopping working Rx/Tx
+ * queues and call this rte_eth_dev_reset().
+ *
+ * The rte_eth_dev_reset( ) itself is generic function which only does some HW
+ * reset operations through calling dev_unint() and dev_init(). And itself
+ * doesn't handle above synchronization which is handled by application.
+ *
+ * PMD itself should not call rte_eth_dev_reset( ). PMD can trigger application
+ * to handle reset event. It is duty of application to handle all
+ * synchronizations befort it calls rte_eth_dev_reset( ).
+ *
+ * After calling rte_eth_dev_reset( ), the application can go through
+ * rte_eth_dev_configure( ), rte_eth_rx_queue_setup( ),
+ * rte_eth_tx_queue_setup( ) and rte_eth_dev_start( ) again to restore
+ * its previous settings or to reconfigure itself with different settings.
+ *
+ * @param port_id
+ *   The port identifier of the Ethernet device.
+ */
+int rte_eth_dev_reset(uint8_t port_id);
+
+/**
  * Enable receipt in promiscuous mode for an Ethernet device.
  *
  * @param port_id
diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map
index 4283728..e86d51e 100644
--- a/lib/librte_ether/rte_ether_version.map
+++ b/lib/librte_ether/rte_ether_version.map
@@ -155,6 +155,7 @@ DPDK_17.08 {
 	rte_eth_dev_adjust_nb_rx_tx_desc;
 	rte_flow_copy;
 	rte_flow_isolate;
+	rte_eth_dev_reset;
 	rte_tm_capabilities_get;
 	rte_tm_get_leaf_nodes;
 	rte_tm_hierarchy_commit;
-- 
2.7.5

[PATCH v7 2/5] net/ixgbe: add support of reset

[Wei Dai]

Reset a NIC by calling dev_uninit and then dev_init.
Go through same way in NIC PCI remove without release of
ethdev resource and then NIC PCI probe function without
ethdev resource allocation.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 drivers/net/ixgbe/ixgbe_ethdev.c | 47 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)

diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index c54d325..b5df073 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -170,6 +170,7 @@ static void ixgbe_dev_stop(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_up(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_down(struct rte_eth_dev *dev);
 static void ixgbe_dev_close(struct rte_eth_dev *dev);
+static int  ixgbe_dev_reset(struct rte_eth_dev *dev);
 static void ixgbe_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void ixgbe_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void ixgbe_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -266,6 +267,7 @@ static int ixgbevf_dev_link_update(struct rte_eth_dev *dev,
 				   int wait_to_complete);
 static void ixgbevf_dev_stop(struct rte_eth_dev *dev);
 static void ixgbevf_dev_close(struct rte_eth_dev *dev);
+static int  ixgbevf_dev_reset(struct rte_eth_dev *dev);
 static void ixgbevf_intr_disable(struct ixgbe_hw *hw);
 static void ixgbevf_intr_enable(struct ixgbe_hw *hw);
 static void ixgbevf_dev_stats_get(struct rte_eth_dev *dev,
@@ -524,6 +526,7 @@ static const struct eth_dev_ops ixgbe_eth_dev_ops = {
 	.dev_set_link_up    = ixgbe_dev_set_link_up,
 	.dev_set_link_down  = ixgbe_dev_set_link_down,
 	.dev_close            = ixgbe_dev_close,
+	.dev_reset	      = ixgbe_dev_reset,
 	.promiscuous_enable   = ixgbe_dev_promiscuous_enable,
 	.promiscuous_disable  = ixgbe_dev_promiscuous_disable,
 	.allmulticast_enable  = ixgbe_dev_allmulticast_enable,
@@ -614,6 +617,7 @@ static const struct eth_dev_ops ixgbevf_eth_dev_ops = {
 	.xstats_reset         = ixgbevf_dev_stats_reset,
 	.xstats_get_names     = ixgbevf_dev_xstats_get_names,
 	.dev_close            = ixgbevf_dev_close,
+	.dev_reset	      = ixgbevf_dev_reset,
 	.allmulticast_enable  = ixgbevf_dev_allmulticast_enable,
 	.allmulticast_disable = ixgbevf_dev_allmulticast_disable,
 	.dev_infos_get        = ixgbevf_dev_info_get,
@@ -2856,6 +2860,32 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 	ixgbe_set_rar(hw, 0, hw->mac.addr, 0, IXGBE_RAH_AV);
 }
 
+/*
+ * Reest PF device.
+ */
+static int
+ixgbe_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	/* When a DPDK PMD PF begin to reset PF port, it should notify all
+	 * its VF to make them align with it. The detailed notification
+	 * mechanism is PMD specific. As to ixgbe PF, it is rather complex.
+	 * To avoid unexpected behavior in VF, currently reset of PF with
+	 * SR-IOV activation is not supported. It might be supported later.
+	 */
+	if (dev->data->sriov.active)
+		return -ENOTSUP;
+
+	ret = eth_ixgbe_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_ixgbe_dev_init(dev);
+
+	return ret;
+}
+
 static void
 ixgbe_read_stats_registers(struct ixgbe_hw *hw,
 			   struct ixgbe_hw_stats *hw_stats,
@@ -5052,6 +5082,23 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	ixgbevf_remove_mac_addr(dev, 0);
 }
 
+/*
+ * Reset VF device
+ */
+static int
+ixgbevf_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	ret = eth_ixgbevf_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_ixgbevf_dev_init(dev);
+
+	return ret;
+}
+
 static void ixgbevf_set_vfta_all(struct rte_eth_dev *dev, bool on)
 {
 	struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-- 
2.7.5

[PATCH v7 3/5] net/i40e: add support of reset

[Wei Dai]

Reset a NIC by calling dev_uninit() and then dev_init().
Go through the same way in NIC PCI remove without release
of ethdev resource and then NIC PCI probe function without
ethdev resource allocation.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 drivers/net/i40e/i40e_ethdev.c    | 28 ++++++++++++++++++++++++++++
 drivers/net/i40e/i40e_ethdev_vf.c | 19 +++++++++++++++++++
 2 files changed, 47 insertions(+)

diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 97a73e1..4ed4cf0 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -250,6 +250,7 @@ static int i40e_dev_configure(struct rte_eth_dev *dev);
 static int i40e_dev_start(struct rte_eth_dev *dev);
 static void i40e_dev_stop(struct rte_eth_dev *dev);
 static void i40e_dev_close(struct rte_eth_dev *dev);
+static int  i40e_dev_reset(struct rte_eth_dev *dev);
 static void i40e_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void i40e_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void i40e_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -449,6 +450,7 @@ static const struct eth_dev_ops i40e_eth_dev_ops = {
 	.dev_start                    = i40e_dev_start,
 	.dev_stop                     = i40e_dev_stop,
 	.dev_close                    = i40e_dev_close,
+	.dev_reset		      = i40e_dev_reset,
 	.promiscuous_enable           = i40e_dev_promiscuous_enable,
 	.promiscuous_disable          = i40e_dev_promiscuous_disable,
 	.allmulticast_enable          = i40e_dev_allmulticast_enable,
@@ -2151,6 +2153,32 @@ i40e_dev_close(struct rte_eth_dev *dev)
 	I40E_WRITE_FLUSH(hw);
 }
 
+/*
+ * Reest PF device only to re-initialize resources in PMD layer
+ */
+static int
+i40e_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	/* When a DPDK PMD PF begin to reset PF port, it should notify all
+	 * its VF to make them align with it. The detailed notification
+	 * mechanism is PMD specific. As to i40e PF, it is rather complex.
+	 * To avoid unexpected behavior in VF, currently reset of PF with
+	 * SR-IOV activation is not supported. It might be supported later.
+	 */
+	if (dev->data->sriov.active)
+		return -ENOTSUP;
+
+	ret = eth_i40e_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_i40e_dev_init(dev);
+
+	return ret;
+}
+
 static void
 i40e_dev_promiscuous_enable(struct rte_eth_dev *dev)
 {
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index bab09f8..858d2a5 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -123,6 +123,7 @@ static void i40evf_vlan_offload_set(struct rte_eth_dev *dev, int mask);
 static int i40evf_vlan_pvid_set(struct rte_eth_dev *dev, uint16_t pvid,
 				int on);
 static void i40evf_dev_close(struct rte_eth_dev *dev);
+static int  i40evf_dev_reset(struct rte_eth_dev *dev);
 static void i40evf_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void i40evf_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void i40evf_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -204,6 +205,7 @@ static const struct eth_dev_ops i40evf_eth_dev_ops = {
 	.xstats_get_names     = i40evf_dev_xstats_get_names,
 	.xstats_reset         = i40evf_dev_xstats_reset,
 	.dev_close            = i40evf_dev_close,
+	.dev_reset	      = i40evf_dev_reset,
 	.dev_infos_get        = i40evf_dev_info_get,
 	.dev_supported_ptypes_get = i40e_dev_supported_ptypes_get,
 	.vlan_filter_set      = i40evf_vlan_filter_set,
@@ -2356,6 +2358,23 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	i40evf_disable_irq0(hw);
 }
 
+/*
+ * Reest VF device only to re-initialize resources in PMD layer
+ */
+static int
+i40evf_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	ret = i40evf_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = i40evf_dev_init(dev);
+
+	return ret;
+}
+
 static int
 i40evf_get_rss_lut(struct i40e_vsi *vsi, uint8_t *lut, uint16_t lut_size)
 {
-- 
2.7.5

[PATCH v7 4/5] app/testpmd: enhance command to test NIC reset

[Wei Dai]

When PF is reset, a message will show it and all its
VF need to be reset.
User can run the command "port reset port_id"
to reset the VF port and to keep same port id without
any configuration. Then user can run "port stop port_id"
and "port start port_id" to reconfigure its forwarding
mode and parmaters as previous ones.
To avoid crash, current forwarding should be stopped
before running "port reset port_id".

Signed-off-by: Wei Dai <wei.dai@intel.com>
Tested-by: Yuan Peng <yuan.peng@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>
---
 app/test-pmd/cmdline.c | 12 ++++++++----
 app/test-pmd/testpmd.c | 41 +++++++++++++++++++++++++++++++++++++++++
 app/test-pmd/testpmd.h |  1 +
 3 files changed, 50 insertions(+), 4 deletions(-)

diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
index d4ff608..8784aff 100644
--- a/app/test-pmd/cmdline.c
+++ b/app/test-pmd/cmdline.c
@@ -970,6 +970,8 @@ static void cmd_operate_port_parsed(void *parsed_result,
 		stop_port(RTE_PORT_ALL);
 	else if (!strcmp(res->name, "close"))
 		close_port(RTE_PORT_ALL);
+	else if (!strcmp(res->name, "reset"))
+		reset_port(RTE_PORT_ALL);
 	else
 		printf("Unknown parameter\n");
 }
@@ -979,14 +981,14 @@ cmdline_parse_token_string_t cmd_operate_port_all_cmd =
 								"port");
 cmdline_parse_token_string_t cmd_operate_port_all_port =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_port_result, name,
-						"start#stop#close");
+						"start#stop#close#reset");
 cmdline_parse_token_string_t cmd_operate_port_all_all =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_port_result, value, "all");
 
 cmdline_parse_inst_t cmd_operate_port = {
 	.f = cmd_operate_port_parsed,
 	.data = NULL,
-	.help_str = "port start|stop|close all: Start/Stop/Close all ports",
+	.help_str = "port start|stop|close all: Start/Stop/Close/Reset all ports",
 	.tokens = {
 		(void *)&cmd_operate_port_all_cmd,
 		(void *)&cmd_operate_port_all_port,
@@ -1014,6 +1016,8 @@ static void cmd_operate_specific_port_parsed(void *parsed_result,
 		stop_port(res->value);
 	else if (!strcmp(res->name, "close"))
 		close_port(res->value);
+	else if (!strcmp(res->name, "reset"))
+		reset_port(res->value);
 	else
 		printf("Unknown parameter\n");
 }
@@ -1023,7 +1027,7 @@ cmdline_parse_token_string_t cmd_operate_specific_port_cmd =
 							keyword, "port");
 cmdline_parse_token_string_t cmd_operate_specific_port_port =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_specific_port_result,
-						name, "start#stop#close");
+						name, "start#stop#close#reset");
 cmdline_parse_token_num_t cmd_operate_specific_port_id =
 	TOKEN_NUM_INITIALIZER(struct cmd_operate_specific_port_result,
 							value, UINT8);
@@ -1031,7 +1035,7 @@ cmdline_parse_token_num_t cmd_operate_specific_port_id =
 cmdline_parse_inst_t cmd_operate_specific_port = {
 	.f = cmd_operate_specific_port_parsed,
 	.data = NULL,
-	.help_str = "port start|stop|close <port_id>: Start/Stop/Close port_id",
+	.help_str = "port start|stop|close <port_id>: Start/Stop/Close/Reset port_id",
 	.tokens = {
 		(void *)&cmd_operate_specific_port_cmd,
 		(void *)&cmd_operate_specific_port_port,
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index e754d12..59fc441 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -1683,6 +1683,47 @@ close_port(portid_t pid)
 }
 
 void
+reset_port(portid_t pid)
+{
+	int diag;
+	portid_t pi;
+	struct rte_port *port;
+
+	if (port_id_is_invalid(pid, ENABLED_WARN))
+		return;
+
+	printf("Resetting ports...\n");
+
+	RTE_ETH_FOREACH_DEV(pi) {
+		if (pid != pi && pid != (portid_t)RTE_PORT_ALL)
+			continue;
+
+		if (port_is_forwarding(pi) != 0 && test_done == 0) {
+			printf("Please remove port %d from forwarding "
+			       "configuration.\n", pi);
+			continue;
+		}
+
+		if (port_is_bonding_slave(pi)) {
+			printf("Please remove port %d from bonded device.\n",
+			       pi);
+			continue;
+		}
+
+		diag = rte_eth_dev_reset(pi);
+		if (diag == 0) {
+			port = &ports[pi];
+			port->need_reconfig = 1;
+			port->need_reconfig_queues = 1;
+		} else {
+			printf("Failed to reset port %d. diag=%d\n", pi, diag);
+		}
+	}
+
+	printf("Done\n");
+}
+
+void
 attach_port(char *identifier)
 {
 	portid_t pi = 0;
diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
index c9d7739..1d1ee75 100644
--- a/app/test-pmd/testpmd.h
+++ b/app/test-pmd/testpmd.h
@@ -610,6 +610,7 @@ int init_port_dcb_config(portid_t pid, enum dcb_mode_enable dcb_mode,
 int start_port(portid_t pid);
 void stop_port(portid_t pid);
 void close_port(portid_t pid);
+void reset_port(portid_t pid);
 void attach_port(char *identifier);
 void detach_port(uint8_t port_id);
 int all_ports_stopped(void);
-- 
2.7.5

[PATCH v7 5/5] doc: add description of the NIC reset API

[Wei Dai]

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 doc/guides/prog_guide/poll_mode_drv.rst | 40 +++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 4987f70..60518fc 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -525,3 +525,43 @@ call. As an end result, the application is able to achieve its goal of
 monitoring a single statistic ("rx_errors" in this case), and if that shows
 packets being dropped, it can easily retrieve a "set" of statistics using the
 IDs array parameter to ``rte_eth_xstats_get_by_id`` function.
+
+NIC Reset API
+~~~~~~~~~~~~~
+
+.. code-block:: c
+
+    int rte_eth_dev_reset(uint8_t port_id);
+
+Sometimes a port have to be reset passively. For example a PF is reset, all its
+VFs should also be reset by application itself to align with the PF. A DPDK
+application also can call this function to trigger an initative port reset.
+
+Normally, a DPDK application can invake this function when
+RTE_ETH_EVENT_INTR_RESET event is detected.
+
+It is duty of PMD to trigger RTE_ETH_EVENT_INTR_RESET event and application
+should also register some callback function to handle this event.
+When PMD needs to trigger a reset, it can trigger RTE_ETH_EVENT_INTR_RESET.
+On the received event of RTE_ETH_EVENT_INTR_RESET, application can begin to
+handle it:  stop working queues,  make rx and tx function not be called and
+then call rte_eth_dev_reset( ).For thread safety, all these controlling
+operations had better be made in same thread.
+
+For example, when PF is reset, PF send a message to notify VF this event and
+also trigger an interrupt to VF.  And then in the interrupt service routine
+DPDK VF detect this notification message and calls
+_rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_RESET, NULL, NULL).
+So it means that PF reset trigger RTE_ETH_EVENT_INTR_RESET event in VF.
+The function _rte_eth_dev_callback_process( ) will call the registered
+callback function. The callback function can trigger application to handle
+all operations of VF reset including something like stopping working Rx/Tx
+queues and call this rte_eth_dev_reset().
+
+The rte_eth_dev_reset( ) itself is generic function which only does some HW
+reset operations through calling dev_unint() and dev_init(). And itself doesn't
+handle above synchronization which is handled by application.
+
+PMD itself should not call rte_eth_dev_reset( ). PMD can trigger application to
+handle reset event. It is duty of application to handle all synchronization
+befort it calls rte_eth_dev_reset( ).
-- 
2.7.5

Re: [PATCH v7 0/5] Support NIC reset and keep same port id

[Thomas Monjalon]

13/07/2017 17:53, Wei Dai:
> chagnes:
> v7:
>   add description of NIC reset in doc/poll_mode_drv.rst

Thanks for bringing more explanations.
The texts have to be reviewed by a native english speaker.

About the organization of the doc, it is good to explain the VF use case
in the prog guide as you do. But you should not add it to the doxygen.
The doxygen must be concise enough and explains only the API:
	- when it can be called
	- what a driver is doing when calling reset
	- what is the expected state of the device after reset

Re: [PATCH v7 0/5] Support NIC reset and keep same port id

[Dai, Wei]

Hi, Thomas
Thanks for your expert guidance.
I'll rework out v8 patch set to follow your guide and rebase to master brach.

> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas@monjalon.net]
> Sent: Monday, July 17, 2017 3:26 AM
> To: Dai, Wei <wei.dai@intel.com>
> Cc: dev@dpdk.org; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev,
> Konstantin <konstantin.ananyev@intel.com>; Wu, Jingjing
> <jingjing.wu@intel.com>; Xing, Beilei <beilei.xing@intel.com>; Mcnamara,
> John <john.mcnamara@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v7 0/5] Support NIC reset and keep same
> port id
> 
> 13/07/2017 17:53, Wei Dai:
> > chagnes:
> > v7:
> >   add description of NIC reset in doc/poll_mode_drv.rst
> 
> Thanks for bringing more explanations.
> The texts have to be reviewed by a native english speaker.
> 
> About the organization of the doc, it is good to explain the VF use case in the
> prog guide as you do. But you should not add it to the doxygen.
> The doxygen must be concise enough and explains only the API:
> 	- when it can be called
> 	- what a driver is doing when calling reset
> 	- what is the expected state of the device after reset

Re: [PATCH v7 0/5] Support NIC reset and keep same port id

[Dai, Wei]

Remy is a Native English speaker.
He will help me review my v8 patch set.
Thank you, Remy.

> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas@monjalon.net]
> Sent: Monday, July 17, 2017 3:26 AM
> To: Dai, Wei <wei.dai@intel.com>
> Cc: dev@dpdk.org; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev,
> Konstantin <konstantin.ananyev@intel.com>; Wu, Jingjing
> <jingjing.wu@intel.com>; Xing, Beilei <beilei.xing@intel.com>; Mcnamara,
> John <john.mcnamara@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v7 0/5] Support NIC reset and keep same
> port id
> 
> 13/07/2017 17:53, Wei Dai:
> > chagnes:
> > v7:
> >   add description of NIC reset in doc/poll_mode_drv.rst
> 
> Thanks for bringing more explanations.
> The texts have to be reviewed by a native english speaker.
> 
> About the organization of the doc, it is good to explain the VF use case in the
> prog guide as you do. But you should not add it to the doxygen.
> The doxygen must be concise enough and explains only the API:
> 	- when it can be called
> 	- what a driver is doing when calling reset
> 	- what is the expected state of the device after reset

[PATCH v8 0/5] Support of NIC reset and keep same port id

[Wei Dai]

s patch set adds a function rte_eth_dev_reset( ) in rte_ethdev layer.
Sometimes a port have to be reset passively. For example a PF is reset,
all its VFs should also be reset by application itself to align with 
the PF.
A DPDK application also can call this function to trigger an initative
port reset.

When processing reset, if the port goes through PCI remove() and then
PCI probe() for restoration, its port id may be changed and this is not
expected by some DPDK application.

Normally, PCI probe() includes two parts: one is in rte_ethdev layer
to allocate resource in rte_ethdev layer and the other is calling PMD
specific dev_init() to allocate and initialize resource in PMD layer.
PCI remove( ) releases all resource allocated from rte_ethdev layer
in PCI probe( ) and calls PMD specific dev_uninit( ) to releaase
resource allocated by dev_init( ) in PMD layer.

To keep same port id and reset the port, only dev_uninit() and
dev_init( ) in PMD can be called and keep all resources allocated
from rte_ethdev layer poart in PCI probe( ). All these are what
rte_eth_dev_reset() does.

The rte_eth_dev_reset( ) calls rte_eth_dev_stop( ), PMD dev_uninit( )
and then PMD dev_init( ) to reset a port and keep same port id.

After calling rte_eth_dev_reset( ), the application can go through
rte_eth_dev_configure( ), rte_eth_rx_queue_setup( ),
rte_eth_tx_queue_setup( ) and rte_eth_dev_start( ) again to restore
its previous settings or to reconfigure itself with different settings.

To test this new feature, a testpmd command "port reset port_id" is added.
The mapping between port number and its PCI address can be monitored to
confirm its port number is kept.
And following test case can also be used to confirm the port can work
again after reset.

A typical test steps are listed as follows:
For example, run "ifconfig PF-name down" will trigger a reset to VF.
1.  run testpmd with 2 ixgbe VF ports belonging to same PF
2.  testpmd > set verbose 1 //to observe VF working
3.  testpmd > show port info all //show port number and MAC addr
4.  testpmd > start
5.  let all ports forwarding work for a while
6.  testpmd > show port stats all
7.  ifconfig name-of-PF down
8.  A message is shown in testmd to indicate PF reset
9.  ifconfig name-of-PF up
10. testpmd > stop // stop forwarding to avoid crash during reset
11. testpmd > port reset all
12. testpmd > port stop all
13. testpmd > port start all //recofnig all ports
14. testpmd > show port info all
    //get mapping of port id and MAC addr for forwarding
15. testpmd > start // restore forwarding
14. let all ports forwarding work for a while
15. testpmd > show port stats all //confirm all port can work again
16. repeat above step 7 - 15

chagnes:
v8:
  modify the comments before the declaration of rte_eth_dev_reset( ) to
  make doc generated from doxygen be concise.
 
v7:
  add description of NIC reset in doc/poll_mode_drv.rst

v6:
  add more comments to explain why the rte_eth_dev_reset( ) is needeed,
  when it is called and what the application should do after calling it.
  add more comments to explain why reset of PF with SR-IOV is not
  supported currently.
 
v5:
  remove PCI address output to align with other modification which
    will output it in other way
  disable PF reset when its VF is ative to avoid unexpected VF behavior
v4:
  add PCI address to confirm its port number keep same
  correct test method in cover letter
v3:
  update testpmd command
v2:
  only reset PMD layer resource and keep same port id, but
  not restore settings

Signed-off-by: Wei Dai <wei.dai@intel.com>
Tested-by: Yuan Peng <yuan.peng@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>

Wei Dai (5):
  ethdev: add support of NIC reset
  net/ixgbe: add support of reset
  net/i40e: add support of reset
  app/testpmd: enhance command to test NIC reset
  doc: add description of the NIC reset API

 app/test-pmd/cmdline.c                  | 12 ++++++---
 app/test-pmd/testpmd.c                  | 41 ++++++++++++++++++++++++++++
 app/test-pmd/testpmd.h                  |  1 +
 doc/guides/prog_guide/poll_mode_drv.rst | 40 ++++++++++++++++++++++++++++
 drivers/net/i40e/i40e_ethdev.c          | 28 ++++++++++++++++++++
 drivers/net/i40e/i40e_ethdev_vf.c       | 19 +++++++++++++
 drivers/net/ixgbe/ixgbe_ethdev.c        | 47 +++++++++++++++++++++++++++++++++
 lib/librte_ether/rte_ethdev.c           | 17 ++++++++++++
 lib/librte_ether/rte_ethdev.h           | 33 +++++++++++++++++++++++
 lib/librte_ether/rte_ether_version.map  |  1 +
 10 files changed, 235 insertions(+), 4 deletions(-)

-- 
2.7.5

[PATCH v8 1/5] ethdev: add support of NIC reset

[Wei Dai]

This patch adds a new eth_dev layer API function rte_eth_dev_reset().
A DPDK application can call this function to reset a NIC and keep
its port id afterwards.
It means that all SW resources allocated in ethdev layer should be
kept and SW and HW resources of the NIC in PMD need to be reset in
similar way that it runs PCI dev_uninit() and then dev_init().
The sequence of dev_uninit() and dev_init() can be packed into a
single interface API rte_eth_dev_reset().
Please See the comments before the declaration of rte_eht_dev_reset()
in lib/librte_ether/rte_ethdev.h to get more details on why this
function is needed, what it does, when it should be called
and what an application should do after calling this function.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 lib/librte_ether/rte_ethdev.c          | 17 +++++++++++++++++
 lib/librte_ether/rte_ethdev.h          | 33 +++++++++++++++++++++++++++++++++
 lib/librte_ether/rte_ether_version.map |  1 +
 3 files changed, 51 insertions(+)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index a1b7447..2448bde 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -1001,6 +1001,23 @@ rte_eth_dev_close(uint8_t port_id)
 }
 
 int
+rte_eth_dev_reset(uint8_t port_id)
+{
+	struct rte_eth_dev *dev;
+	int ret;
+
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
+	dev = &rte_eth_devices[port_id];
+
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_reset, -ENOTSUP);
+
+	rte_eth_dev_stop(port_id);
+	ret = dev->dev_ops->dev_reset(dev);
+
+	return ret;
+}
+
+int
 rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id,
 		       uint16_t nb_rx_desc, unsigned int socket_id,
 		       const struct rte_eth_rxconf *rx_conf,
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 224f485..bd74d15 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1111,6 +1111,9 @@ typedef int  (*eth_dev_set_link_down_t)(struct rte_eth_dev *dev);
 typedef void (*eth_dev_close_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to close a configured Ethernet device. */
 
+typedef int (*eth_dev_reset_t)(struct rte_eth_dev *dev);
+/** <@internal Function used to reset a configured Ethernet device. */
+
 typedef void (*eth_promiscuous_enable_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to enable the RX promiscuous mode of an Ethernet device. */
 
@@ -1431,6 +1434,7 @@ struct eth_dev_ops {
 	eth_dev_set_link_up_t      dev_set_link_up;   /**< Device link up. */
 	eth_dev_set_link_down_t    dev_set_link_down; /**< Device link down. */
 	eth_dev_close_t            dev_close;     /**< Close device. */
+	eth_dev_reset_t		   dev_reset;	  /**< Reset device. */
 	eth_link_update_t          link_update;   /**< Get device link state. */
 
 	eth_promiscuous_enable_t   promiscuous_enable; /**< Promiscuous ON. */
@@ -2136,6 +2140,35 @@ int rte_eth_dev_set_link_down(uint8_t port_id);
 void rte_eth_dev_close(uint8_t port_id);
 
 /**
+ * Reset a Ethernet device and keep its port id.
+ *
+ * When a port has to be reset passively, the DPDK application can invoke this
+ * function. For example a PF is reset, all its VFs should also be reset.
+ * Normally, a DPDK application can invoke this function when
+ * RTE_ETH_EVENT_INTR_RESET event is detected. A DPDK application can also call
+ * this function to start an initiative port reset.
+ *
+ * When this function is called, it first stops the port and then call PMD
+ * specific dev_uninit( ) and  dev_init( ) to makes the port return to initial
+ * status in which no any Tx queue and no Rx queue are setup and the port has
+ * just be reset and not started. And the port keeps its port id after calling
+ * this function.
+ *
+ * After calling rte_eth_dev_reset( ), the application should go through
+ * rte_eth_dev_configure( ), rte_eth_rx_queue_setup( ),
+ * rte_eth_tx_queue_setup( ) and rte_eth_dev_start( ) again to restore
+ * its previous settings or to reconfigure itself with different settings.
+ *
+ * Note: to avoid unexpected behaviour, the application should stop calling Rx
+ *       and Rx function before calling rte_eth_dev_reset( ).For thread safety,
+ *       all these controlling operations had better be made in same thread.
+ *
+ * @param port_id
+ *   The port identifier of the Ethernet device.
+ */
+int rte_eth_dev_reset(uint8_t port_id);
+
+/**
  * Enable receipt in promiscuous mode for an Ethernet device.
  *
  * @param port_id
diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map
index 4283728..e86d51e 100644
--- a/lib/librte_ether/rte_ether_version.map
+++ b/lib/librte_ether/rte_ether_version.map
@@ -155,6 +155,7 @@ DPDK_17.08 {
 	rte_eth_dev_adjust_nb_rx_tx_desc;
 	rte_flow_copy;
 	rte_flow_isolate;
+	rte_eth_dev_reset;
 	rte_tm_capabilities_get;
 	rte_tm_get_leaf_nodes;
 	rte_tm_hierarchy_commit;
-- 
2.7.5

Re: [PATCH v8 1/5] ethdev: add support of NIC reset

[Remy Horton]

See suggested wording inline

On 17/07/2017 16:14, Wei Dai wrote:
> This patch adds a new eth_dev layer API function rte_eth_dev_reset().
> A DPDK application can call this function to reset a NIC and keep
> its port id afterwards.
> It means that all SW resources allocated in ethdev layer should be
> kept and SW and HW resources of the NIC in PMD need to be reset in
> similar way that it runs PCI dev_uninit() and then dev_init().
> The sequence of dev_uninit() and dev_init() can be packed into a
> single interface API rte_eth_dev_reset().
> Please See the comments before the declaration of rte_eht_dev_reset()
> in lib/librte_ether/rte_ethdev.h to get more details on why this
> function is needed, what it does, when it should be called
> and what an application should do after calling this function.

This patch adds a new eth_dev layer API function rte_eth_dev_reset(),
which a DPDK application can call to reset a NIC and keep its port id
afterwards. It means that all software resources allocated in the ethdev 
layer are kept, and software & hardware resources of the NIC within the 
NIC's PMD are reset to a state simular to that obtained by calling the 
PCI dev_uninit() and then dev_init(). This effective sequence of 
dev_uninit() and dev_init() is packed into a single API function 
rte_eth_dev_reset().

Please see the comments before the declaration of rte_eht_dev_reset()
in lib/librte_ether/rte_ethdev.h to get more details on why this
function is needed, what it does, when it should be called
and what an application should do after calling this function.


> Signed-off-by: Wei Dai <wei.dai@intel.com>
> ---
>  lib/librte_ether/rte_ethdev.c          | 17 +++++++++++++++++
>  lib/librte_ether/rte_ethdev.h          | 33 +++++++++++++++++++++++++++++++++
>  lib/librte_ether/rte_ether_version.map |  1 +
>  3 files changed, 51 insertions(+)

Reviewed-by: Remy Horton <remy.horton@intel.com>



>  /**
> + * Reset a Ethernet device and keep its port id.
> + *
> + * When a port has to be reset passively, the DPDK application can invoke this
> + * function. For example a PF is reset, all its VFs should also be reset.
> + * Normally, a DPDK application can invoke this function when
> + * RTE_ETH_EVENT_INTR_RESET event is detected. A DPDK application can also call
> + * this function to start an initiative port reset.

When a port has to be reset passively, the DPDK application can invoke 
this function. For example when a PF is reset, all its VFs should also 
be reset. Normally a DPDK application can invoke this function when 
RTE_ETH_EVENT_INTR_RESET event is detected, but can also use it to start 
a port reset in other circumstances.

> + * When this function is called, it first stops the port and then call PMD
> + * specific dev_uninit( ) and  dev_init( ) to makes the port return to initial
> + * status in which no any Tx queue and no Rx queue are setup and the port has
> + * just be reset and not started. And the port keeps its port id after calling
> + * this function.

When this function is called, it first stops the port and then calls the 
PMD specific dev_uninit( ) and dev_init( ) to return the port to initial 
state, in which no Tx and Rx queues are setup, as if the port has been 
reset and not started. The port keeps the port id it had before the 
function call.

> + * After calling rte_eth_dev_reset( ), the application should go through
> + * rte_eth_dev_configure( ), rte_eth_rx_queue_setup( ),
> + * rte_eth_tx_queue_setup( ) and rte_eth_dev_start( ) again to restore
> + * its previous settings or to reconfigure itself with different settings.

After calling rte_eth_dev_reset( ), the application should use
rte_eth_dev_configure( ), rte_eth_rx_queue_setup( ),
rte_eth_tx_queue_setup( ), and rte_eth_dev_start( )
to reconfigure the device as appropriate.


> + * Note: to avoid unexpected behaviour, the application should stop calling Rx
> + *       and Rx function before calling rte_eth_dev_reset( ).For thread safety,
> + *       all these controlling operations had better be made in same thread.

Note: To avoid unexpected behavior, the application should stop calling 
Tx and Rx functions before calling rte_eth_dev_reset( ). For thread 
safety, all these controlling functions should be called from the same 
thread.

Re: [PATCH v8 1/5] ethdev: add support of NIC reset

[Dai, Wei]

Many thanks, Remy.
My v9 patch set has followed your guidance.

> -----Original Message-----
> From: Horton, Remy
> Sent: Thursday, July 20, 2017 9:22 PM
> To: Dai, Wei <wei.dai@intel.com>; dev@dpdk.org
> Cc: thomas@monjalon.net; Lu, Wenzhuo <wenzhuo.lu@intel.com>;
> Ananyev, Konstantin <konstantin.ananyev@intel.com>; Xing, Beilei
> <beilei.xing@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Mcnamara,
> John <john.mcnamara@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v8 1/5] ethdev: add support of NIC reset
> 
> See suggested wording inline
> 
> On 17/07/2017 16:14, Wei Dai wrote:
> > This patch adds a new eth_dev layer API function rte_eth_dev_reset().
> > A DPDK application can call this function to reset a NIC and keep
> > its port id afterwards.
> > It means that all SW resources allocated in ethdev layer should be
> > kept and SW and HW resources of the NIC in PMD need to be reset in
> > similar way that it runs PCI dev_uninit() and then dev_init().
> > The sequence of dev_uninit() and dev_init() can be packed into a
> > single interface API rte_eth_dev_reset().
> > Please See the comments before the declaration of rte_eht_dev_reset()
> > in lib/librte_ether/rte_ethdev.h to get more details on why this
> > function is needed, what it does, when it should be called
> > and what an application should do after calling this function.
> 
> This patch adds a new eth_dev layer API function rte_eth_dev_reset(),
> which a DPDK application can call to reset a NIC and keep its port id
> afterwards. It means that all software resources allocated in the ethdev
> layer are kept, and software & hardware resources of the NIC within the
> NIC's PMD are reset to a state simular to that obtained by calling the
> PCI dev_uninit() and then dev_init(). This effective sequence of
> dev_uninit() and dev_init() is packed into a single API function
> rte_eth_dev_reset().
> 
> Please see the comments before the declaration of rte_eht_dev_reset()
> in lib/librte_ether/rte_ethdev.h to get more details on why this
> function is needed, what it does, when it should be called
> and what an application should do after calling this function.
> 
> 
> > Signed-off-by: Wei Dai <wei.dai@intel.com>
> > ---
> >  lib/librte_ether/rte_ethdev.c          | 17 +++++++++++++++++
> >  lib/librte_ether/rte_ethdev.h          | 33
> +++++++++++++++++++++++++++++++++
> >  lib/librte_ether/rte_ether_version.map |  1 +
> >  3 files changed, 51 insertions(+)
> 
> Reviewed-by: Remy Horton <remy.horton@intel.com>
> 
> 
> 
> >  /**
> > + * Reset a Ethernet device and keep its port id.
> > + *
> > + * When a port has to be reset passively, the DPDK application can invoke
> this
> > + * function. For example a PF is reset, all its VFs should also be reset.
> > + * Normally, a DPDK application can invoke this function when
> > + * RTE_ETH_EVENT_INTR_RESET event is detected. A DPDK application
> can also call
> > + * this function to start an initiative port reset.
> 
> When a port has to be reset passively, the DPDK application can invoke
> this function. For example when a PF is reset, all its VFs should also
> be reset. Normally a DPDK application can invoke this function when
> RTE_ETH_EVENT_INTR_RESET event is detected, but can also use it to start
> a port reset in other circumstances.
> 
> > + * When this function is called, it first stops the port and then call PMD
> > + * specific dev_uninit( ) and  dev_init( ) to makes the port return to initial
> > + * status in which no any Tx queue and no Rx queue are setup and the port
> has
> > + * just be reset and not started. And the port keeps its port id after calling
> > + * this function.
> 
> When this function is called, it first stops the port and then calls the
> PMD specific dev_uninit( ) and dev_init( ) to return the port to initial
> state, in which no Tx and Rx queues are setup, as if the port has been
> reset and not started. The port keeps the port id it had before the
> function call.
> 
> > + * After calling rte_eth_dev_reset( ), the application should go through
> > + * rte_eth_dev_configure( ), rte_eth_rx_queue_setup( ),
> > + * rte_eth_tx_queue_setup( ) and rte_eth_dev_start( ) again to restore
> > + * its previous settings or to reconfigure itself with different settings.
> 
> After calling rte_eth_dev_reset( ), the application should use
> rte_eth_dev_configure( ), rte_eth_rx_queue_setup( ),
> rte_eth_tx_queue_setup( ), and rte_eth_dev_start( )
> to reconfigure the device as appropriate.
> 
> 
> > + * Note: to avoid unexpected behaviour, the application should stop
> calling Rx
> > + *       and Rx function before calling rte_eth_dev_reset( ).For thread
> safety,
> > + *       all these controlling operations had better be made in same
> thread.
> 
> Note: To avoid unexpected behavior, the application should stop calling
> Tx and Rx functions before calling rte_eth_dev_reset( ). For thread
> safety, all these controlling functions should be called from the same
> thread.
> 

[PATCH v8 2/5] net/ixgbe: add support of reset

[Wei Dai]

Reset a NIC by calling dev_uninit and then dev_init.
Go through same way in NIC PCI remove without release of
ethdev resource and then NIC PCI probe function without
ethdev resource allocation.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 drivers/net/ixgbe/ixgbe_ethdev.c | 47 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)

diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 9b06ac1..f27c34e 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -169,6 +169,7 @@ static void ixgbe_dev_stop(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_up(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_down(struct rte_eth_dev *dev);
 static void ixgbe_dev_close(struct rte_eth_dev *dev);
+static int  ixgbe_dev_reset(struct rte_eth_dev *dev);
 static void ixgbe_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void ixgbe_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void ixgbe_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -265,6 +266,7 @@ static int ixgbevf_dev_link_update(struct rte_eth_dev *dev,
 				   int wait_to_complete);
 static void ixgbevf_dev_stop(struct rte_eth_dev *dev);
 static void ixgbevf_dev_close(struct rte_eth_dev *dev);
+static int  ixgbevf_dev_reset(struct rte_eth_dev *dev);
 static void ixgbevf_intr_disable(struct ixgbe_hw *hw);
 static void ixgbevf_intr_enable(struct ixgbe_hw *hw);
 static void ixgbevf_dev_stats_get(struct rte_eth_dev *dev,
@@ -523,6 +525,7 @@ static const struct eth_dev_ops ixgbe_eth_dev_ops = {
 	.dev_set_link_up    = ixgbe_dev_set_link_up,
 	.dev_set_link_down  = ixgbe_dev_set_link_down,
 	.dev_close            = ixgbe_dev_close,
+	.dev_reset	      = ixgbe_dev_reset,
 	.promiscuous_enable   = ixgbe_dev_promiscuous_enable,
 	.promiscuous_disable  = ixgbe_dev_promiscuous_disable,
 	.allmulticast_enable  = ixgbe_dev_allmulticast_enable,
@@ -613,6 +616,7 @@ static const struct eth_dev_ops ixgbevf_eth_dev_ops = {
 	.xstats_reset         = ixgbevf_dev_stats_reset,
 	.xstats_get_names     = ixgbevf_dev_xstats_get_names,
 	.dev_close            = ixgbevf_dev_close,
+	.dev_reset	      = ixgbevf_dev_reset,
 	.allmulticast_enable  = ixgbevf_dev_allmulticast_enable,
 	.allmulticast_disable = ixgbevf_dev_allmulticast_disable,
 	.dev_infos_get        = ixgbevf_dev_info_get,
@@ -2855,6 +2859,32 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 	ixgbe_set_rar(hw, 0, hw->mac.addr, 0, IXGBE_RAH_AV);
 }
 
+/*
+ * Reest PF device.
+ */
+static int
+ixgbe_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	/* When a DPDK PMD PF begin to reset PF port, it should notify all
+	 * its VF to make them align with it. The detailed notification
+	 * mechanism is PMD specific. As to ixgbe PF, it is rather complex.
+	 * To avoid unexpected behavior in VF, currently reset of PF with
+	 * SR-IOV activation is not supported. It might be supported later.
+	 */
+	if (dev->data->sriov.active)
+		return -ENOTSUP;
+
+	ret = eth_ixgbe_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_ixgbe_dev_init(dev);
+
+	return ret;
+}
+
 static void
 ixgbe_read_stats_registers(struct ixgbe_hw *hw,
 			   struct ixgbe_hw_stats *hw_stats,
@@ -5051,6 +5081,23 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	ixgbevf_remove_mac_addr(dev, 0);
 }
 
+/*
+ * Reset VF device
+ */
+static int
+ixgbevf_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	ret = eth_ixgbevf_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_ixgbevf_dev_init(dev);
+
+	return ret;
+}
+
 static void ixgbevf_set_vfta_all(struct rte_eth_dev *dev, bool on)
 {
 	struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-- 
2.7.5

[PATCH v8 3/5] net/i40e: add support of reset

[Wei Dai]

Reset a NIC by calling dev_uninit() and then dev_init().
Go through the same way in NIC PCI remove without release
of ethdev resource and then NIC PCI probe function without
ethdev resource allocation.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 drivers/net/i40e/i40e_ethdev.c    | 28 ++++++++++++++++++++++++++++
 drivers/net/i40e/i40e_ethdev_vf.c | 19 +++++++++++++++++++
 2 files changed, 47 insertions(+)

diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 97a73e1..4ed4cf0 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -250,6 +250,7 @@ static int i40e_dev_configure(struct rte_eth_dev *dev);
 static int i40e_dev_start(struct rte_eth_dev *dev);
 static void i40e_dev_stop(struct rte_eth_dev *dev);
 static void i40e_dev_close(struct rte_eth_dev *dev);
+static int  i40e_dev_reset(struct rte_eth_dev *dev);
 static void i40e_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void i40e_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void i40e_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -449,6 +450,7 @@ static const struct eth_dev_ops i40e_eth_dev_ops = {
 	.dev_start                    = i40e_dev_start,
 	.dev_stop                     = i40e_dev_stop,
 	.dev_close                    = i40e_dev_close,
+	.dev_reset		      = i40e_dev_reset,
 	.promiscuous_enable           = i40e_dev_promiscuous_enable,
 	.promiscuous_disable          = i40e_dev_promiscuous_disable,
 	.allmulticast_enable          = i40e_dev_allmulticast_enable,
@@ -2151,6 +2153,32 @@ i40e_dev_close(struct rte_eth_dev *dev)
 	I40E_WRITE_FLUSH(hw);
 }
 
+/*
+ * Reest PF device only to re-initialize resources in PMD layer
+ */
+static int
+i40e_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	/* When a DPDK PMD PF begin to reset PF port, it should notify all
+	 * its VF to make them align with it. The detailed notification
+	 * mechanism is PMD specific. As to i40e PF, it is rather complex.
+	 * To avoid unexpected behavior in VF, currently reset of PF with
+	 * SR-IOV activation is not supported. It might be supported later.
+	 */
+	if (dev->data->sriov.active)
+		return -ENOTSUP;
+
+	ret = eth_i40e_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_i40e_dev_init(dev);
+
+	return ret;
+}
+
 static void
 i40e_dev_promiscuous_enable(struct rte_eth_dev *dev)
 {
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index dc6c794..e0031d1 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -122,6 +122,7 @@ static void i40evf_vlan_offload_set(struct rte_eth_dev *dev, int mask);
 static int i40evf_vlan_pvid_set(struct rte_eth_dev *dev, uint16_t pvid,
 				int on);
 static void i40evf_dev_close(struct rte_eth_dev *dev);
+static int  i40evf_dev_reset(struct rte_eth_dev *dev);
 static void i40evf_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void i40evf_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void i40evf_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -203,6 +204,7 @@ static const struct eth_dev_ops i40evf_eth_dev_ops = {
 	.xstats_get_names     = i40evf_dev_xstats_get_names,
 	.xstats_reset         = i40evf_dev_xstats_reset,
 	.dev_close            = i40evf_dev_close,
+	.dev_reset	      = i40evf_dev_reset,
 	.dev_infos_get        = i40evf_dev_info_get,
 	.dev_supported_ptypes_get = i40e_dev_supported_ptypes_get,
 	.vlan_filter_set      = i40evf_vlan_filter_set,
@@ -2355,6 +2357,23 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	i40evf_disable_irq0(hw);
 }
 
+/*
+ * Reest VF device only to re-initialize resources in PMD layer
+ */
+static int
+i40evf_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	ret = i40evf_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = i40evf_dev_init(dev);
+
+	return ret;
+}
+
 static int
 i40evf_get_rss_lut(struct i40e_vsi *vsi, uint8_t *lut, uint16_t lut_size)
 {
-- 
2.7.5

[PATCH v8 4/5] app/testpmd: enhance command to test NIC reset

[Wei Dai]

When PF is reset, a message will show it and all its
VF need to be reset.
User can run the command "port reset port_id"
to reset the VF port and to keep same port id without
any configuration. Then user can run "port stop port_id"
and "port start port_id" to reconfigure its forwarding
mode and parmaters as previous ones.
To avoid crash, current forwarding should be stopped
before running "port reset port_id".

Signed-off-by: Wei Dai <wei.dai@intel.com>
Tested-by: Yuan Peng <yuan.peng@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>
---
 app/test-pmd/cmdline.c | 12 ++++++++----
 app/test-pmd/testpmd.c | 41 +++++++++++++++++++++++++++++++++++++++++
 app/test-pmd/testpmd.h |  1 +
 3 files changed, 50 insertions(+), 4 deletions(-)

diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
index 7decb96..84a7180 100644
--- a/app/test-pmd/cmdline.c
+++ b/app/test-pmd/cmdline.c
@@ -969,6 +969,8 @@ static void cmd_operate_port_parsed(void *parsed_result,
 		stop_port(RTE_PORT_ALL);
 	else if (!strcmp(res->name, "close"))
 		close_port(RTE_PORT_ALL);
+	else if (!strcmp(res->name, "reset"))
+		reset_port(RTE_PORT_ALL);
 	else
 		printf("Unknown parameter\n");
 }
@@ -978,14 +980,14 @@ cmdline_parse_token_string_t cmd_operate_port_all_cmd =
 								"port");
 cmdline_parse_token_string_t cmd_operate_port_all_port =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_port_result, name,
-						"start#stop#close");
+						"start#stop#close#reset");
 cmdline_parse_token_string_t cmd_operate_port_all_all =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_port_result, value, "all");
 
 cmdline_parse_inst_t cmd_operate_port = {
 	.f = cmd_operate_port_parsed,
 	.data = NULL,
-	.help_str = "port start|stop|close all: Start/Stop/Close all ports",
+	.help_str = "port start|stop|close all: Start/Stop/Close/Reset all ports",
 	.tokens = {
 		(void *)&cmd_operate_port_all_cmd,
 		(void *)&cmd_operate_port_all_port,
@@ -1013,6 +1015,8 @@ static void cmd_operate_specific_port_parsed(void *parsed_result,
 		stop_port(res->value);
 	else if (!strcmp(res->name, "close"))
 		close_port(res->value);
+	else if (!strcmp(res->name, "reset"))
+		reset_port(res->value);
 	else
 		printf("Unknown parameter\n");
 }
@@ -1022,7 +1026,7 @@ cmdline_parse_token_string_t cmd_operate_specific_port_cmd =
 							keyword, "port");
 cmdline_parse_token_string_t cmd_operate_specific_port_port =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_specific_port_result,
-						name, "start#stop#close");
+						name, "start#stop#close#reset");
 cmdline_parse_token_num_t cmd_operate_specific_port_id =
 	TOKEN_NUM_INITIALIZER(struct cmd_operate_specific_port_result,
 							value, UINT8);
@@ -1030,7 +1034,7 @@ cmdline_parse_token_num_t cmd_operate_specific_port_id =
 cmdline_parse_inst_t cmd_operate_specific_port = {
 	.f = cmd_operate_specific_port_parsed,
 	.data = NULL,
-	.help_str = "port start|stop|close <port_id>: Start/Stop/Close port_id",
+	.help_str = "port start|stop|close <port_id>: Start/Stop/Close/Reset port_id",
 	.tokens = {
 		(void *)&cmd_operate_specific_port_cmd,
 		(void *)&cmd_operate_specific_port_port,
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index e754d12..59fc441 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -1683,6 +1683,47 @@ close_port(portid_t pid)
 }
 
 void
+reset_port(portid_t pid)
+{
+	int diag;
+	portid_t pi;
+	struct rte_port *port;
+
+	if (port_id_is_invalid(pid, ENABLED_WARN))
+		return;
+
+	printf("Resetting ports...\n");
+
+	RTE_ETH_FOREACH_DEV(pi) {
+		if (pid != pi && pid != (portid_t)RTE_PORT_ALL)
+			continue;
+
+		if (port_is_forwarding(pi) != 0 && test_done == 0) {
+			printf("Please remove port %d from forwarding "
+			       "configuration.\n", pi);
+			continue;
+		}
+
+		if (port_is_bonding_slave(pi)) {
+			printf("Please remove port %d from bonded device.\n",
+			       pi);
+			continue;
+		}
+
+		diag = rte_eth_dev_reset(pi);
+		if (diag == 0) {
+			port = &ports[pi];
+			port->need_reconfig = 1;
+			port->need_reconfig_queues = 1;
+		} else {
+			printf("Failed to reset port %d. diag=%d\n", pi, diag);
+		}
+	}
+
+	printf("Done\n");
+}
+
+void
 attach_port(char *identifier)
 {
 	portid_t pi = 0;
diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
index c9d7739..1d1ee75 100644
--- a/app/test-pmd/testpmd.h
+++ b/app/test-pmd/testpmd.h
@@ -610,6 +610,7 @@ int init_port_dcb_config(portid_t pid, enum dcb_mode_enable dcb_mode,
 int start_port(portid_t pid);
 void stop_port(portid_t pid);
 void close_port(portid_t pid);
+void reset_port(portid_t pid);
 void attach_port(char *identifier);
 void detach_port(uint8_t port_id);
 int all_ports_stopped(void);
-- 
2.7.5

[PATCH v8 5/5] doc: add description of the NIC reset API

[Wei Dai]

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 doc/guides/prog_guide/poll_mode_drv.rst | 40 +++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 4987f70..60518fc 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -525,3 +525,43 @@ call. As an end result, the application is able to achieve its goal of
 monitoring a single statistic ("rx_errors" in this case), and if that shows
 packets being dropped, it can easily retrieve a "set" of statistics using the
 IDs array parameter to ``rte_eth_xstats_get_by_id`` function.
+
+NIC Reset API
+~~~~~~~~~~~~~
+
+.. code-block:: c
+
+    int rte_eth_dev_reset(uint8_t port_id);
+
+Sometimes a port have to be reset passively. For example a PF is reset, all its
+VFs should also be reset by application itself to align with the PF. A DPDK
+application also can call this function to trigger an initative port reset.
+
+Normally, a DPDK application can invake this function when
+RTE_ETH_EVENT_INTR_RESET event is detected.
+
+It is duty of PMD to trigger RTE_ETH_EVENT_INTR_RESET event and application
+should also register some callback function to handle this event.
+When PMD needs to trigger a reset, it can trigger RTE_ETH_EVENT_INTR_RESET.
+On the received event of RTE_ETH_EVENT_INTR_RESET, application can begin to
+handle it:  stop working queues,  make rx and tx function not be called and
+then call rte_eth_dev_reset( ).For thread safety, all these controlling
+operations had better be made in same thread.
+
+For example, when PF is reset, PF send a message to notify VF this event and
+also trigger an interrupt to VF.  And then in the interrupt service routine
+DPDK VF detect this notification message and calls
+_rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_RESET, NULL, NULL).
+So it means that PF reset trigger RTE_ETH_EVENT_INTR_RESET event in VF.
+The function _rte_eth_dev_callback_process( ) will call the registered
+callback function. The callback function can trigger application to handle
+all operations of VF reset including something like stopping working Rx/Tx
+queues and call this rte_eth_dev_reset().
+
+The rte_eth_dev_reset( ) itself is generic function which only does some HW
+reset operations through calling dev_unint() and dev_init(). And itself doesn't
+handle above synchronization which is handled by application.
+
+PMD itself should not call rte_eth_dev_reset( ). PMD can trigger application to
+handle reset event. It is duty of application to handle all synchronization
+befort it calls rte_eth_dev_reset( ).
-- 
2.7.5

Re: [PATCH v8 5/5] doc: add description of the NIC reset API

[Remy Horton]

Unless the plan is for this patch to be squashed, ought to have a brief 
commit message..

On 17/07/2017 16:15, Wei Dai wrote:
> Signed-off-by: Wei Dai <wei.dai@intel.com>
> ---
>  doc/guides/prog_guide/poll_mode_drv.rst | 40 +++++++++++++++++++++++++++++++++
>  1 file changed, 40 insertions(+)

Reviewed-by: Remy Horton <remy.horton@intel.com>


> +.. code-block:: c
> +
> +    int rte_eth_dev_reset(uint8_t port_id);
> +

Suggest following text:

Sometimes a port has to be reset passively. For example when a PF is 
reset, all its VFs should also be reset by the application to make them 
consistent with the PF. A DPDK application also can call this function 
to trigger a port reset. Normally, a DPDK application would invokes this 
function when an RTE_ETH_EVENT_INTR_RESET event is detected.

It is the duty of the PMD to trigger RTE_ETH_EVENT_INTR_RESET events and 
the application should register a callback function to handle these 
events. When a PMD needs to trigger a reset, it can trigger an 
RTE_ETH_EVENT_INTR_RESET event. On receiving an RTE_ETH_EVENT_INTR_RESET 
event, applications can handle it as follows: Stop working queues, stop 
calling Rx and Tx functions, and then call rte_eth_dev_reset( ). For 
thread safety all these operations should be called from the same thread.

For example when PF is reset, the PF sends a message to notify VFs of 
this event and also trigger an interrupt to VFs. Then in the interrupt 
service routine the VFs detects this notification message and calls 
_rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_RESET, NULL, 
NULL). This means that a PF reset triggers an RTE_ETH_EVENT_INTR_RESET 
event within VFs. The function _rte_eth_dev_callback_process( ) will 
call the registered callback function. The callback function can trigger 
the application to handle all operations the VF reset requires including 
stopping Rx/Tx queues and calling rte_eth_dev_reset().

The rte_eth_dev_reset( ) itself is a generic function which only does 
some hardware reset operations through calling dev_unint() and 
dev_init(), and itself does not handle synchronization, which is handled 
by application.

The PMD itself should not call rte_eth_dev_reset( ). The PMD can trigger 
the application to handle reset event. It is duty of application to 
handle all synchronization before it calls rte_eth_dev_reset( ).


..Remy

[PATCH v9 0/5] Support of NIC reset and keep same port id

[Wei Dai]

This patch set adds a function rte_eth_dev_reset( ) in rte_ethdev layer.
Sometimes a port have to be reset passively. For example a PF is reset,
all its VFs should also be reset by application itself to align with 
the PF.
A DPDK application also can call this function to trigger an initative
port reset.

When processing reset, if the port goes through PCI remove() and then
PCI probe() for restoration, its port id may be changed and this is not
expected by some DPDK application.

Normally, PCI probe() includes two parts: one is in rte_ethdev layer
to allocate resource in rte_ethdev layer and the other is calling PMD
specific dev_init() to allocate and initialize resource in PMD layer.
PCI remove( ) releases all resource allocated from rte_ethdev layer
in PCI probe( ) and calls PMD specific dev_uninit( ) to releaase
resource allocated by dev_init( ) in PMD layer.

To keep same port id and reset the port, only dev_uninit() and
dev_init( ) in PMD can be called and keep all resources allocated
from rte_ethdev layer poart in PCI probe( ). All these are what
rte_eth_dev_reset() does.

The rte_eth_dev_reset( ) calls rte_eth_dev_stop( ), PMD dev_uninit( )
and then PMD dev_init( ) to reset a port and keep same port id.

After calling rte_eth_dev_reset( ), the application can go through
rte_eth_dev_configure( ), rte_eth_rx_queue_setup( ),
rte_eth_tx_queue_setup( ) and rte_eth_dev_start( ) again to restore
its previous settings or to reconfigure itself with different settings.

To test this new feature, a testpmd command "port reset port_id" is added.
The mapping between port number and its PCI address can be monitored to
confirm its port number is kept.
And following test case can also be used to confirm the port can work
again after reset.

A typical test steps are listed as follows:
For example, run "ifconfig PF-name down" will trigger a reset to VF.
1.  run testpmd with 2 ixgbe VF ports belonging to same PF
2.  testpmd > set verbose 1 //to observe VF working
3.  testpmd > show port info all //show port number and MAC addr
4.  testpmd > start
5.  let all ports forwarding work for a while
6.  testpmd > show port stats all
7.  ifconfig name-of-PF down
8.  A message is shown in testmd to indicate PF reset
9.  ifconfig name-of-PF up
10. testpmd > stop // stop forwarding to avoid crash during reset
11. testpmd > port reset all
12. testpmd > port stop all
13. testpmd > port start all //recofnig all ports
14. testpmd > show port info all
    //get mapping of port id and MAC addr for forwarding
15. testpmd > start // restore forwarding
14. let all ports forwarding work for a while
15. testpmd > show port stats all //confirm all port can work again
16. repeat above step 7 - 15

chagnes:
v9:
  rebase to master branch
  modify some text after it is reviewed by an English native speaker

v8:
  modify the comments before the declaration of rte_eth_dev_reset( ) to
  make doc generated from doxygen be concise.
 
v7:
  add description of NIC reset in doc/poll_mode_drv.rst

v6:
  add more comments to explain why the rte_eth_dev_reset( ) is needeed,
  when it is called and what the application should do after calling it.
  add more comments to explain why reset of PF with SR-IOV is not
  supported currently.
 
v5:
  remove PCI address output to align with other modification which
    will output it in other way
  disable PF reset when its VF is ative to avoid unexpected VF behavior
v4:
  add PCI address to confirm its port number keep same
  correct test method in cover letter
v3:
  update testpmd command
v2:
  only reset PMD layer resource and keep same port id, but
  not restore settings

Signed-off-by: Wei Dai <wei.dai@intel.com>
Tested-by: Yuan Peng <yuan.peng@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>
Reviewed-by: Remy Horton <remy.horton@intel.com>

Wei Dai (5):
  ethdev: add support of NIC reset
  net/ixgbe: add support of reset
  net/i40e: add support of reset
  app/testpmd: enhance command to test NIC reset
  doc: add description of the NIC reset API

 app/test-pmd/cmdline.c                  | 12 ++++++---
 app/test-pmd/testpmd.c                  | 41 ++++++++++++++++++++++++++++
 app/test-pmd/testpmd.h                  |  1 +
 doc/guides/prog_guide/poll_mode_drv.rst | 41 ++++++++++++++++++++++++++++
 drivers/net/i40e/i40e_ethdev.c          | 28 ++++++++++++++++++++
 drivers/net/i40e/i40e_ethdev_vf.c       | 19 +++++++++++++
 drivers/net/ixgbe/ixgbe_ethdev.c        | 47 +++++++++++++++++++++++++++++++++
 lib/librte_ether/rte_ethdev.c           | 17 ++++++++++++
 lib/librte_ether/rte_ethdev.h           | 34 ++++++++++++++++++++++++
 lib/librte_ether/rte_ether_version.map  |  1 +
 10 files changed, 237 insertions(+), 4 deletions(-)

-- 
2.7.5

[PATCH v9 1/5] ethdev: add support of NIC reset

[Wei Dai]

This patch adds a new eth_dev layer API function rte_eth_dev_reset(),
which a DPDK application can call to reset a NIC and keep its port id
afterwards. It means that all software resources allocated in the ethdev
layer are kept, and software & hardware resources of the NIC within the
NIC's PMD are reset to a state simular to that obtained by calling the
PCI dev_uninit() and then dev_init(). This effective sequence of
dev_uninit() and dev_init() is packed into a single API function
rte_eth_dev_reset().

Please see the comments before the declaration of rte_eht_dev_reset()
in lib/librte_ether/rte_ethdev.h to get more details on why this
function is needed, what it does, when it should be called
and what an application should do after calling this function.

Signed-off-by: Wei Dai <wei.dai@intel.com>
Reviewed-by: Remy Horton <remy.horton@intel.com>
---
 lib/librte_ether/rte_ethdev.c          | 17 +++++++++++++++++
 lib/librte_ether/rte_ethdev.h          | 34 ++++++++++++++++++++++++++++++++++
 lib/librte_ether/rte_ether_version.map |  1 +
 3 files changed, 52 insertions(+)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index d4ebb1b..68ba64d 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -1004,6 +1004,23 @@ rte_eth_dev_close(uint8_t port_id)
 }
 
 int
+rte_eth_dev_reset(uint8_t port_id)
+{
+	struct rte_eth_dev *dev;
+	int ret;
+
+	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
+	dev = &rte_eth_devices[port_id];
+
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_reset, -ENOTSUP);
+
+	rte_eth_dev_stop(port_id);
+	ret = dev->dev_ops->dev_reset(dev);
+
+	return ret;
+}
+
+int
 rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id,
 		       uint16_t nb_rx_desc, unsigned int socket_id,
 		       const struct rte_eth_rxconf *rx_conf,
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 0e99090..fde92a1 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -1115,6 +1115,9 @@ typedef int  (*eth_dev_set_link_down_t)(struct rte_eth_dev *dev);
 typedef void (*eth_dev_close_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to close a configured Ethernet device. */
 
+typedef int (*eth_dev_reset_t)(struct rte_eth_dev *dev);
+/** <@internal Function used to reset a configured Ethernet device. */
+
 typedef void (*eth_promiscuous_enable_t)(struct rte_eth_dev *dev);
 /**< @internal Function used to enable the RX promiscuous mode of an Ethernet device. */
 
@@ -1435,6 +1438,7 @@ struct eth_dev_ops {
 	eth_dev_set_link_up_t      dev_set_link_up;   /**< Device link up. */
 	eth_dev_set_link_down_t    dev_set_link_down; /**< Device link down. */
 	eth_dev_close_t            dev_close;     /**< Close device. */
+	eth_dev_reset_t		   dev_reset;	  /**< Reset device. */
 	eth_link_update_t          link_update;   /**< Get device link state. */
 
 	eth_promiscuous_enable_t   promiscuous_enable; /**< Promiscuous ON. */
@@ -2140,6 +2144,36 @@ int rte_eth_dev_set_link_down(uint8_t port_id);
 void rte_eth_dev_close(uint8_t port_id);
 
 /**
+ * Reset a Ethernet device and keep its port id.
+ *
+ * When a port has to be reset passively, the DPDK application can invoke
+ * this function. For example when a PF is reset, all its VFs should also
+ * be reset. Normally a DPDK application can invoke this function when
+ * RTE_ETH_EVENT_INTR_RESET event is detected, but can also use it to start
+ * a port reset in other circumstances.
+ *
+ * When this function is called, it first stops the port and then calls the
+ * PMD specific dev_uninit( ) and dev_init( ) to return the port to initial
+ * state, in which no Tx and Rx queues are setup, as if the port has been
+ * reset and not started. The port keeps the port id it had before the
+ * function call.
+ *
+ * After calling rte_eth_dev_reset( ), the application should use
+ * rte_eth_dev_configure( ), rte_eth_rx_queue_setup( ),
+ * rte_eth_tx_queue_setup( ), and rte_eth_dev_start( )
+ * to reconfigure the device as appropriate.
+ *
+ * Note: To avoid unexpected behavior, the application should stop calling
+ * Tx and Rx functions before calling rte_eth_dev_reset( ). For thread
+ * safety, all these controlling functions should be called from the same
+ * thread.
+ *
+ * @param port_id
+ *   The port identifier of the Ethernet device.
+ */
+int rte_eth_dev_reset(uint8_t port_id);
+
+/**
  * Enable receipt in promiscuous mode for an Ethernet device.
  *
  * @param port_id
diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map
index 4283728..e86d51e 100644
--- a/lib/librte_ether/rte_ether_version.map
+++ b/lib/librte_ether/rte_ether_version.map
@@ -155,6 +155,7 @@ DPDK_17.08 {
 	rte_eth_dev_adjust_nb_rx_tx_desc;
 	rte_flow_copy;
 	rte_flow_isolate;
+	rte_eth_dev_reset;
 	rte_tm_capabilities_get;
 	rte_tm_get_leaf_nodes;
 	rte_tm_hierarchy_commit;
-- 
2.7.5

Re: [PATCH v9 1/5] ethdev: add support of NIC reset

[Thomas Monjalon]

23/07/2017 11:15, Wei Dai:
> --- a/lib/librte_ether/rte_ether_version.map
> +++ b/lib/librte_ether/rte_ether_version.map

This file is now lib/librte_ether/rte_ethdev_version.map

> @@ -155,6 +155,7 @@ DPDK_17.08 {
>         rte_eth_dev_adjust_nb_rx_tx_desc;
>         rte_flow_copy;
>         rte_flow_isolate;
> +       rte_eth_dev_reset;
>         rte_tm_capabilities_get;
>         rte_tm_get_leaf_nodes;
>         rte_tm_hierarchy_commit;

The function must be added in DPDK_17.11 now.

I will do the changes.

Re: [PATCH v9 1/5] ethdev: add support of NIC reset

[Luca Boccassi]

On Sun, 2017-07-23 at 17:15 +0800, Wei Dai wrote:
> This patch adds a new eth_dev layer API function rte_eth_dev_reset(),
> which a DPDK application can call to reset a NIC and keep its port id
> afterwards. It means that all software resources allocated in the
> ethdev
> layer are kept, and software & hardware resources of the NIC within
> the
> NIC's PMD are reset to a state simular to that obtained by calling
> the
> PCI dev_uninit() and then dev_init(). This effective sequence of
> dev_uninit() and dev_init() is packed into a single API function
> rte_eth_dev_reset().
> 
> Please see the comments before the declaration of rte_eht_dev_reset()
> in lib/librte_ether/rte_ethdev.h to get more details on why this
> function is needed, what it does, when it should be called
> and what an application should do after calling this function.
> 
> Signed-off-by: Wei Dai <wei.dai@intel.com>
> Reviewed-by: Remy Horton <remy.horton@intel.com>
> ---
>  lib/librte_ether/rte_ethdev.c          | 17 +++++++++++++++++
>  lib/librte_ether/rte_ethdev.h          | 34
> ++++++++++++++++++++++++++++++++++
>  lib/librte_ether/rte_ether_version.map |  1 +
>  3 files changed, 52 insertions(+)
> 
> diff --git a/lib/librte_ether/rte_ethdev.c
> b/lib/librte_ether/rte_ethdev.c
> index d4ebb1b..68ba64d 100644
> --- a/lib/librte_ether/rte_ethdev.c
> +++ b/lib/librte_ether/rte_ethdev.c
> @@ -1004,6 +1004,23 @@ rte_eth_dev_close(uint8_t port_id)
>  }
>  
>  int
> +rte_eth_dev_reset(uint8_t port_id)
> +{
> +	struct rte_eth_dev *dev;
> +	int ret;
> +
> +	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
> +	dev = &rte_eth_devices[port_id];
> +
> +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_reset, -ENOTSUP);
> +
> +	rte_eth_dev_stop(port_id);
> +	ret = dev->dev_ops->dev_reset(dev);
> +
> +	return ret;
> +}
> +
> +int
>  rte_eth_rx_queue_setup(uint8_t port_id, uint16_t rx_queue_id,
>  		       uint16_t nb_rx_desc, unsigned int socket_id,
>  		       const struct rte_eth_rxconf *rx_conf,
> diff --git a/lib/librte_ether/rte_ethdev.h
> b/lib/librte_ether/rte_ethdev.h
> index 0e99090..fde92a1 100644
> --- a/lib/librte_ether/rte_ethdev.h
> +++ b/lib/librte_ether/rte_ethdev.h
> @@ -1115,6 +1115,9 @@ typedef int  (*eth_dev_set_link_down_t)(struct
> rte_eth_dev *dev);
>  typedef void (*eth_dev_close_t)(struct rte_eth_dev *dev);
>  /**< @internal Function used to close a configured Ethernet device.
> */
>  
> +typedef int (*eth_dev_reset_t)(struct rte_eth_dev *dev);
> +/** <@internal Function used to reset a configured Ethernet device.
> */
> +
>  typedef void (*eth_promiscuous_enable_t)(struct rte_eth_dev *dev);
>  /**< @internal Function used to enable the RX promiscuous mode of an
> Ethernet device. */
>  
> @@ -1435,6 +1438,7 @@ struct eth_dev_ops {
>  	eth_dev_set_link_up_t      dev_set_link_up;   /**< Device
> link up. */
>  	eth_dev_set_link_down_t    dev_set_link_down; /**< Device
> link down. */
>  	eth_dev_close_t            dev_close;     /**< Close device.
> */
> +	eth_dev_reset_t		   dev_reset;	  /**<
> Reset device. */
>  	eth_link_update_t          link_update;   /**< Get device
> link state. */
>  
>  	eth_promiscuous_enable_t   promiscuous_enable; /**<
> Promiscuous ON. */
> @@ -2140,6 +2144,36 @@ int rte_eth_dev_set_link_down(uint8_t
> port_id);
>  void rte_eth_dev_close(uint8_t port_id);
>  
>  /**
> + * Reset a Ethernet device and keep its port id.
> + *
> + * When a port has to be reset passively, the DPDK application can
> invoke
> + * this function. For example when a PF is reset, all its VFs should
> also
> + * be reset. Normally a DPDK application can invoke this function
> when
> + * RTE_ETH_EVENT_INTR_RESET event is detected, but can also use it
> to start
> + * a port reset in other circumstances.
> + *
> + * When this function is called, it first stops the port and then
> calls the
> + * PMD specific dev_uninit( ) and dev_init( ) to return the port to
> initial
> + * state, in which no Tx and Rx queues are setup, as if the port has
> been
> + * reset and not started. The port keeps the port id it had before
> the
> + * function call.
> + *
> + * After calling rte_eth_dev_reset( ), the application should use
> + * rte_eth_dev_configure( ), rte_eth_rx_queue_setup( ),
> + * rte_eth_tx_queue_setup( ), and rte_eth_dev_start( )
> + * to reconfigure the device as appropriate.
> + *
> + * Note: To avoid unexpected behavior, the application should stop
> calling
> + * Tx and Rx functions before calling rte_eth_dev_reset( ). For
> thread
> + * safety, all these controlling functions should be called from the
> same
> + * thread.
> + *
> + * @param port_id
> + *   The port identifier of the Ethernet device.
> + */
> +int rte_eth_dev_reset(uint8_t port_id);
> +
> +/**
>   * Enable receipt in promiscuous mode for an Ethernet device.
>   *
>   * @param port_id
> diff --git a/lib/librte_ether/rte_ether_version.map
> b/lib/librte_ether/rte_ether_version.map
> index 4283728..e86d51e 100644
> --- a/lib/librte_ether/rte_ether_version.map
> +++ b/lib/librte_ether/rte_ether_version.map
> @@ -155,6 +155,7 @@ DPDK_17.08 {
>  	rte_eth_dev_adjust_nb_rx_tx_desc;
>  	rte_flow_copy;
>  	rte_flow_isolate;
> +	rte_eth_dev_reset;
>  	rte_tm_capabilities_get;
>  	rte_tm_get_leaf_nodes;
>  	rte_tm_hierarchy_commit;

Hello Wei,

First of all thanks for your work, we have a use case where it's
extremely important to be able to reset the VFs when the PF (Linux)
changes state.

Question: can the rte_eth_dev_reset call block in the current
implementation that was merged?

Last year a patchset was sent by, I believe, a colleague of yours,
Wenzhuo Lu, and I've been using it in production.

One of the things that I immediately noticed is that the _reset call
was blocking - that was a show stopper for our application, so I sent a
couple of patches to take a "blocking" boolean, and return EAGAIN
instead of blocking.

You can find the original patches this link:

http://dpdk.org/ml/archives/dev/2016-June/042018.html

The description of the problem we had with the blocking call:

http://dpdk.org/ml/archives/dev/2016-July/043446.html

The "blocking" and EAGAIN changes:

http://dpdk.org/ml/archives/dev/2016-July/043535.html
http://dpdk.org/ml/archives/dev/2016-July/043818.html

Functionality-wise, from the point of view of an application, are you
aware of big differences between that patchset and what was merged
today?

Thanks!

-- 
Kind regards,
Luca Boccassi

[PATCH v9 2/5] net/ixgbe: add support of reset

[Wei Dai]

Reset a NIC by calling dev_uninit and then dev_init.
Go through same way in NIC PCI remove without release of
ethdev resource and then NIC PCI probe function without
ethdev resource allocation.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 drivers/net/ixgbe/ixgbe_ethdev.c | 47 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)

diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 194058f..3f176fe 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -169,6 +169,7 @@ static void ixgbe_dev_stop(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_up(struct rte_eth_dev *dev);
 static int  ixgbe_dev_set_link_down(struct rte_eth_dev *dev);
 static void ixgbe_dev_close(struct rte_eth_dev *dev);
+static int  ixgbe_dev_reset(struct rte_eth_dev *dev);
 static void ixgbe_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void ixgbe_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void ixgbe_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -265,6 +266,7 @@ static int ixgbevf_dev_link_update(struct rte_eth_dev *dev,
 				   int wait_to_complete);
 static void ixgbevf_dev_stop(struct rte_eth_dev *dev);
 static void ixgbevf_dev_close(struct rte_eth_dev *dev);
+static int  ixgbevf_dev_reset(struct rte_eth_dev *dev);
 static void ixgbevf_intr_disable(struct ixgbe_hw *hw);
 static void ixgbevf_intr_enable(struct ixgbe_hw *hw);
 static void ixgbevf_dev_stats_get(struct rte_eth_dev *dev,
@@ -523,6 +525,7 @@ static const struct eth_dev_ops ixgbe_eth_dev_ops = {
 	.dev_set_link_up    = ixgbe_dev_set_link_up,
 	.dev_set_link_down  = ixgbe_dev_set_link_down,
 	.dev_close            = ixgbe_dev_close,
+	.dev_reset	      = ixgbe_dev_reset,
 	.promiscuous_enable   = ixgbe_dev_promiscuous_enable,
 	.promiscuous_disable  = ixgbe_dev_promiscuous_disable,
 	.allmulticast_enable  = ixgbe_dev_allmulticast_enable,
@@ -613,6 +616,7 @@ static const struct eth_dev_ops ixgbevf_eth_dev_ops = {
 	.xstats_reset         = ixgbevf_dev_stats_reset,
 	.xstats_get_names     = ixgbevf_dev_xstats_get_names,
 	.dev_close            = ixgbevf_dev_close,
+	.dev_reset	      = ixgbevf_dev_reset,
 	.allmulticast_enable  = ixgbevf_dev_allmulticast_enable,
 	.allmulticast_disable = ixgbevf_dev_allmulticast_disable,
 	.dev_infos_get        = ixgbevf_dev_info_get,
@@ -2857,6 +2861,32 @@ ixgbe_dev_close(struct rte_eth_dev *dev)
 	ixgbe_set_rar(hw, 0, hw->mac.addr, 0, IXGBE_RAH_AV);
 }
 
+/*
+ * Reest PF device.
+ */
+static int
+ixgbe_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	/* When a DPDK PMD PF begin to reset PF port, it should notify all
+	 * its VF to make them align with it. The detailed notification
+	 * mechanism is PMD specific. As to ixgbe PF, it is rather complex.
+	 * To avoid unexpected behavior in VF, currently reset of PF with
+	 * SR-IOV activation is not supported. It might be supported later.
+	 */
+	if (dev->data->sriov.active)
+		return -ENOTSUP;
+
+	ret = eth_ixgbe_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_ixgbe_dev_init(dev);
+
+	return ret;
+}
+
 static void
 ixgbe_read_stats_registers(struct ixgbe_hw *hw,
 			   struct ixgbe_hw_stats *hw_stats,
@@ -5058,6 +5088,23 @@ ixgbevf_dev_close(struct rte_eth_dev *dev)
 	ixgbevf_remove_mac_addr(dev, 0);
 }
 
+/*
+ * Reset VF device
+ */
+static int
+ixgbevf_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	ret = eth_ixgbevf_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_ixgbevf_dev_init(dev);
+
+	return ret;
+}
+
 static void ixgbevf_set_vfta_all(struct rte_eth_dev *dev, bool on)
 {
 	struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
-- 
2.7.5

Re: [PATCH v9 2/5] net/ixgbe: add support of reset

[Thomas Monjalon]

23/07/2017 11:15, Wei Dai:
> +/*
> + * Reest PF device.
> + */

Small typo here. I will fix it.

[PATCH v9 3/5] net/i40e: add support of reset

[Wei Dai]

Reset a NIC by calling dev_uninit() and then dev_init().
Go through the same way in NIC PCI remove without release
of ethdev resource and then NIC PCI probe function without
ethdev resource allocation.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 drivers/net/i40e/i40e_ethdev.c    | 28 ++++++++++++++++++++++++++++
 drivers/net/i40e/i40e_ethdev_vf.c | 19 +++++++++++++++++++
 2 files changed, 47 insertions(+)

diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 9fcccda..1641e00 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -250,6 +250,7 @@ static int i40e_dev_configure(struct rte_eth_dev *dev);
 static int i40e_dev_start(struct rte_eth_dev *dev);
 static void i40e_dev_stop(struct rte_eth_dev *dev);
 static void i40e_dev_close(struct rte_eth_dev *dev);
+static int  i40e_dev_reset(struct rte_eth_dev *dev);
 static void i40e_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void i40e_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void i40e_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -449,6 +450,7 @@ static const struct eth_dev_ops i40e_eth_dev_ops = {
 	.dev_start                    = i40e_dev_start,
 	.dev_stop                     = i40e_dev_stop,
 	.dev_close                    = i40e_dev_close,
+	.dev_reset		      = i40e_dev_reset,
 	.promiscuous_enable           = i40e_dev_promiscuous_enable,
 	.promiscuous_disable          = i40e_dev_promiscuous_disable,
 	.allmulticast_enable          = i40e_dev_allmulticast_enable,
@@ -2155,6 +2157,32 @@ i40e_dev_close(struct rte_eth_dev *dev)
 	I40E_WRITE_FLUSH(hw);
 }
 
+/*
+ * Reest PF device only to re-initialize resources in PMD layer
+ */
+static int
+i40e_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	/* When a DPDK PMD PF begin to reset PF port, it should notify all
+	 * its VF to make them align with it. The detailed notification
+	 * mechanism is PMD specific. As to i40e PF, it is rather complex.
+	 * To avoid unexpected behavior in VF, currently reset of PF with
+	 * SR-IOV activation is not supported. It might be supported later.
+	 */
+	if (dev->data->sriov.active)
+		return -ENOTSUP;
+
+	ret = eth_i40e_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = eth_i40e_dev_init(dev);
+
+	return ret;
+}
+
 static void
 i40e_dev_promiscuous_enable(struct rte_eth_dev *dev)
 {
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index f6d8293..f951d4e 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -122,6 +122,7 @@ static void i40evf_vlan_offload_set(struct rte_eth_dev *dev, int mask);
 static int i40evf_vlan_pvid_set(struct rte_eth_dev *dev, uint16_t pvid,
 				int on);
 static void i40evf_dev_close(struct rte_eth_dev *dev);
+static int  i40evf_dev_reset(struct rte_eth_dev *dev);
 static void i40evf_dev_promiscuous_enable(struct rte_eth_dev *dev);
 static void i40evf_dev_promiscuous_disable(struct rte_eth_dev *dev);
 static void i40evf_dev_allmulticast_enable(struct rte_eth_dev *dev);
@@ -203,6 +204,7 @@ static const struct eth_dev_ops i40evf_eth_dev_ops = {
 	.xstats_get_names     = i40evf_dev_xstats_get_names,
 	.xstats_reset         = i40evf_dev_xstats_reset,
 	.dev_close            = i40evf_dev_close,
+	.dev_reset	      = i40evf_dev_reset,
 	.dev_infos_get        = i40evf_dev_info_get,
 	.dev_supported_ptypes_get = i40e_dev_supported_ptypes_get,
 	.vlan_filter_set      = i40evf_vlan_filter_set,
@@ -2373,6 +2375,23 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	i40evf_disable_irq0(hw);
 }
 
+/*
+ * Reest VF device only to re-initialize resources in PMD layer
+ */
+static int
+i40evf_dev_reset(struct rte_eth_dev *dev)
+{
+	int ret;
+
+	ret = i40evf_dev_uninit(dev);
+	if (ret)
+		return ret;
+
+	ret = i40evf_dev_init(dev);
+
+	return ret;
+}
+
 static int
 i40evf_get_rss_lut(struct i40e_vsi *vsi, uint8_t *lut, uint16_t lut_size)
 {
-- 
2.7.5

[PATCH v9 4/5] app/testpmd: enhance command to test NIC reset

[Wei Dai]

When PF is reset, a message will show it and all its
VF need to be reset.
User can run the command "port reset port_id"
to reset the VF port and to keep same port id without
any configuration. Then user can run "port stop port_id"
and "port start port_id" to reconfigure its forwarding
mode and parmaters as previous ones.
To avoid crash, current forwarding should be stopped
before running "port reset port_id".

Signed-off-by: Wei Dai <wei.dai@intel.com>
Tested-by: Yuan Peng <yuan.peng@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>
---
 app/test-pmd/cmdline.c | 12 ++++++++----
 app/test-pmd/testpmd.c | 41 +++++++++++++++++++++++++++++++++++++++++
 app/test-pmd/testpmd.h |  1 +
 3 files changed, 50 insertions(+), 4 deletions(-)

diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
index b1b36c1..e37d6d7 100644
--- a/app/test-pmd/cmdline.c
+++ b/app/test-pmd/cmdline.c
@@ -972,6 +972,8 @@ static void cmd_operate_port_parsed(void *parsed_result,
 		stop_port(RTE_PORT_ALL);
 	else if (!strcmp(res->name, "close"))
 		close_port(RTE_PORT_ALL);
+	else if (!strcmp(res->name, "reset"))
+		reset_port(RTE_PORT_ALL);
 	else
 		printf("Unknown parameter\n");
 }
@@ -981,14 +983,14 @@ cmdline_parse_token_string_t cmd_operate_port_all_cmd =
 								"port");
 cmdline_parse_token_string_t cmd_operate_port_all_port =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_port_result, name,
-						"start#stop#close");
+						"start#stop#close#reset");
 cmdline_parse_token_string_t cmd_operate_port_all_all =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_port_result, value, "all");
 
 cmdline_parse_inst_t cmd_operate_port = {
 	.f = cmd_operate_port_parsed,
 	.data = NULL,
-	.help_str = "port start|stop|close all: Start/Stop/Close all ports",
+	.help_str = "port start|stop|close all: Start/Stop/Close/Reset all ports",
 	.tokens = {
 		(void *)&cmd_operate_port_all_cmd,
 		(void *)&cmd_operate_port_all_port,
@@ -1016,6 +1018,8 @@ static void cmd_operate_specific_port_parsed(void *parsed_result,
 		stop_port(res->value);
 	else if (!strcmp(res->name, "close"))
 		close_port(res->value);
+	else if (!strcmp(res->name, "reset"))
+		reset_port(res->value);
 	else
 		printf("Unknown parameter\n");
 }
@@ -1025,7 +1029,7 @@ cmdline_parse_token_string_t cmd_operate_specific_port_cmd =
 							keyword, "port");
 cmdline_parse_token_string_t cmd_operate_specific_port_port =
 	TOKEN_STRING_INITIALIZER(struct cmd_operate_specific_port_result,
-						name, "start#stop#close");
+						name, "start#stop#close#reset");
 cmdline_parse_token_num_t cmd_operate_specific_port_id =
 	TOKEN_NUM_INITIALIZER(struct cmd_operate_specific_port_result,
 							value, UINT8);
@@ -1033,7 +1037,7 @@ cmdline_parse_token_num_t cmd_operate_specific_port_id =
 cmdline_parse_inst_t cmd_operate_specific_port = {
 	.f = cmd_operate_specific_port_parsed,
 	.data = NULL,
-	.help_str = "port start|stop|close <port_id>: Start/Stop/Close port_id",
+	.help_str = "port start|stop|close <port_id>: Start/Stop/Close/Reset port_id",
 	.tokens = {
 		(void *)&cmd_operate_specific_port_cmd,
 		(void *)&cmd_operate_specific_port_port,
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index e754d12..59fc441 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -1683,6 +1683,47 @@ close_port(portid_t pid)
 }
 
 void
+reset_port(portid_t pid)
+{
+	int diag;
+	portid_t pi;
+	struct rte_port *port;
+
+	if (port_id_is_invalid(pid, ENABLED_WARN))
+		return;
+
+	printf("Resetting ports...\n");
+
+	RTE_ETH_FOREACH_DEV(pi) {
+		if (pid != pi && pid != (portid_t)RTE_PORT_ALL)
+			continue;
+
+		if (port_is_forwarding(pi) != 0 && test_done == 0) {
+			printf("Please remove port %d from forwarding "
+			       "configuration.\n", pi);
+			continue;
+		}
+
+		if (port_is_bonding_slave(pi)) {
+			printf("Please remove port %d from bonded device.\n",
+			       pi);
+			continue;
+		}
+
+		diag = rte_eth_dev_reset(pi);
+		if (diag == 0) {
+			port = &ports[pi];
+			port->need_reconfig = 1;
+			port->need_reconfig_queues = 1;
+		} else {
+			printf("Failed to reset port %d. diag=%d\n", pi, diag);
+		}
+	}
+
+	printf("Done\n");
+}
+
+void
 attach_port(char *identifier)
 {
 	portid_t pi = 0;
diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
index c9d7739..1d1ee75 100644
--- a/app/test-pmd/testpmd.h
+++ b/app/test-pmd/testpmd.h
@@ -610,6 +610,7 @@ int init_port_dcb_config(portid_t pid, enum dcb_mode_enable dcb_mode,
 int start_port(portid_t pid);
 void stop_port(portid_t pid);
 void close_port(portid_t pid);
+void reset_port(portid_t pid);
 void attach_port(char *identifier);
 void detach_port(uint8_t port_id);
 int all_ports_stopped(void);
-- 
2.7.5

[PATCH v9 5/5] doc: add description of the NIC reset API

[Wei Dai]

This patch add the description of NIC reset API in
doc/guides/prog_guide/poll_mode_drv.rst .
It explains why this API is needed, when it should
be called and some noticeable information.

Signed-off-by: Wei Dai <wei.dai@intel.com>
Reviewed-by: Remy Horton <remy.horton@intel.com>
---
 doc/guides/prog_guide/poll_mode_drv.rst | 41 +++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)

diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
index 1ac8f7e..7c95b3e 100644
--- a/doc/guides/prog_guide/poll_mode_drv.rst
+++ b/doc/guides/prog_guide/poll_mode_drv.rst
@@ -536,3 +536,44 @@ call. As an end result, the application is able to achieve its goal of
 monitoring a single statistic ("rx_errors" in this case), and if that shows
 packets being dropped, it can easily retrieve a "set" of statistics using the
 IDs array parameter to ``rte_eth_xstats_get_by_id`` function.
+
+NIC Reset API
+~~~~~~~~~~~~~
+
+.. code-block:: c
+
+    int rte_eth_dev_reset(uint8_t port_id);
+
+Sometimes a port has to be reset passively. For example when a PF is
+reset, all its VFs should also be reset by the application to make them
+consistent with the PF. A DPDK application also can call this function
+to trigger a port reset. Normally, a DPDK application would invokes this
+function when an RTE_ETH_EVENT_INTR_RESET event is detected.
+
+It is the duty of the PMD to trigger RTE_ETH_EVENT_INTR_RESET events and
+the application should register a callback function to handle these
+events. When a PMD needs to trigger a reset, it can trigger an
+RTE_ETH_EVENT_INTR_RESET event. On receiving an RTE_ETH_EVENT_INTR_RESET
+event, applications can handle it as follows: Stop working queues, stop
+calling Rx and Tx functions, and then call rte_eth_dev_reset( ). For
+thread safety all these operations should be called from the same thread.
+
+For example when PF is reset, the PF sends a message to notify VFs of
+this event and also trigger an interrupt to VFs. Then in the interrupt
+service routine the VFs detects this notification message and calls
+_rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_RESET, NULL,
+NULL). This means that a PF reset triggers an RTE_ETH_EVENT_INTR_RESET
+event within VFs. The function _rte_eth_dev_callback_process( ) will
+call the registered callback function. The callback function can trigger
+the application to handle all operations the VF reset requires including
+stopping Rx/Tx queues and calling rte_eth_dev_reset().
+
+The rte_eth_dev_reset( ) itself is a generic function which only does
+some hardware reset operations through calling dev_unint() and
+dev_init(), and itself does not handle synchronization, which is handled
+by application.
+
+The PMD itself should not call rte_eth_dev_reset( ). The PMD can trigger
+the application to handle reset event. It is duty of application to
+handle all synchronization before it calls rte_eth_dev_reset( ).
+
-- 
2.7.5

Re: [PATCH v9 0/5] Support of NIC reset and keep same port id

[Thomas Monjalon]

> Wei Dai (5):
>   ethdev: add support of NIC reset
>   net/ixgbe: add support of reset
>   net/i40e: add support of reset
>   app/testpmd: enhance command to test NIC reset
>   doc: add description of the NIC reset API

Sorry for being so late to get back to it.

Applied, thanks for the improvements done in latest versions.

Re: [PATCH v9 0/5] Support of NIC reset and keep same port id

[Dai, Wei]

Hi, Thomas

Thanks for your great support !

> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas@monjalon.net]
> Sent: Thursday, September 14, 2017 5:02 AM
> To: Dai, Wei <wei.dai@intel.com>
> Cc: dev@dpdk.org; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev,
> Konstantin <konstantin.ananyev@intel.com>; Wu, Jingjing
> <jingjing.wu@intel.com>; Xing, Beilei <beilei.xing@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v9 0/5] Support of NIC reset and keep same
> port id
> 
> > Wei Dai (5):
> >   ethdev: add support of NIC reset
> >   net/ixgbe: add support of reset
> >   net/i40e: add support of reset
> >   app/testpmd: enhance command to test NIC reset
> >   doc: add description of the NIC reset API
> 
> Sorry for being so late to get back to it.
> 
> Applied, thanks for the improvements done in latest versions.