Re: [PATCH v4 0/5] Support NIC reset and keep same port id

["Peng, Yuan" <yuan.peng@intel.com>]

Tested-by: Peng, Yuan <yuan.peng@intel.com>

- Tested commit bbe569daa7e99b36d44b12bb3d23ddfbc26d383c+the 5 patches.
- OS: 4.8.6-300.fc25.x86_64
- GCC: gcc version 6.2.1 20160916 (Red Hat 6.2.1-2) (GCC)
- CPU: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz
- NIC: Ethernet controller [0200]: Intel Corporation Ethernet Controller 10G X550T [8086:1563] (rev 01)
- Default x86_64-native-linuxapp-gcc configuration
- Prerequisites:
- Total 1cases, 1 passed, 0 failed

Steps:
DUT:
1. echo 2 >/sys/bus/pci/devices/0000:83:00.0/sriov_numvfs
./usertools/dpdk-devbind.py -b vfio-pci 83:10.0 83:10.2
./x86_64-native-linuxapp-gcc/app/testpmd -c 0x3 -n 4 -- -i

2.testpmd> set verbose 1
testpmd> set fwd mac
testpmd> start
tester:
scapy
sendp([Ether(dst="02:09:C0:63:DA:4B")/IP()/UDP()], iface="ens6f0", count=1)
dut:
testpmd>show port stats all
port0 can fwd the packet normally
testpmd>show port info all
show port number, PCI addr and MAC addr

3. ifconfig ens801f0 down
Port 0: Interrupt reset event
Port 1: Interrupt reset event
ifconfig ens801f0 up

4.testpmd> stop

5. testpmd> port reset all
Resetting ports...
Finish resetting Port 0 with PCI Address: 0000:83:10.0
Finish resetting Port 1 with PCI Address: 0000:83:10.2
Done

6. testpmd> port stop all
Stopping ports...
Checking link statuses...
Done
testpmd> port start all
Configuring Port 0 (socket 1) with PCI Address: 0000:83:10.0
Port 0: 02:09:C0:63:DA:4B
Configuring Port 1 (socket 1) with PCI Address: 0000:83:10.2
Port 1: 02:09:C0:37:93:6F
Checking link statuses...
Done

7. testpmd> show port info all
confirm same mapping of port id and PCI address.

8.testpmd> start
Tester:
scapy
sendp([Ether(dst="02:09:C0:63:DA:4B")/IP()/UDP()], iface="ens6f0", count=1)
dut:
testpmd>show port stats all
port0 can fwd the packet normally

9.repeat step3 to step8, the same result.



-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wei Dai
Sent: Thursday, June 29, 2017 10:58 PM
To: thomas@monjalon.net; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Zhang, Helin <helin.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; yuan.pntel.com@dpdk.org
Cc: dev@dpdk.org; Dai, Wei <wei.dai@intel.com>
Subject: [dpdk-dev] [PATCH v4 0/5] Support NIC reset and keep same port id

Sometimes a port have to be reset. For example PF is reset, all its VF should also be reset. After reset, if the port goes through PCI remove() and then PCI probe() for restoration, its port id may be changed and this is not expected by some customer DPDK application.
Normally, PCI probe() includes two parts: one is in rte_ethdev layer and the other is calling PMD dev_init(). PCI remove( ) release all resource allocated from rte_ethdev layer in PCI probe( ) and calls PMD dev_unit( ).
To keep same port id and reset the port, only dev_uninit() and dev_init( ) in PMD can be called and keep all resources allocated from rte_ethdev layer poart in PCI probe( ).

New rte_eth_dev_reset( ) calls rte_eth_dev_stop( ), PMD dev_uninit( ) and then PMD dev_init( ) to reset a port and keep same port id.
And then application can go through rte_eth_dev_configure( ), rte_eth_rx_queue_setup( ), rte_eth_tx_queue_setup( ) and rte_eth_dev_start( ) again to restore its previous settings or to reconfigure itself with different settings.

To test this new feature, a testpmd command "port reset port_id" is added.
The mapping between port number and its PCI address can be monitored to confirm its port number is kept.
And following test case can also be used to confirm the port can work again after reset.

A typical test steps are listed as follows:
For example, run "ifconfig PF-name down" will trigger a reset to VF.
1.  run testpmd with 2 ixgbe VF ports belonging to same PF 2.  testpmd > set verbose 1 //to observe VF working 3.  testpmd > show port info all //show port number, PCI addr and MAC addr 4.  testpmd > start 5.  let all ports forwarding work for a while 6.  testpmd > show port stats all 7.  ifconfig name-of-PF down 8.  A message is shown in testmd to indicate PF reset 9.  ifconfig name-of-PF up 10. testpmd > stop // stop forwarding to avoid crash during reset 11. testpmd > port reset all 12. testpmd > port stop all 13. testpmd > port start all //recofnig all ports 14. testpmd > show port info all
    //confirm same mapping of port id and PCI addr 15. testpmd > start // restore forwarding 14. let all ports forwarding work for a while 15. testpmd > show port stats all //confirm all port can work again 16. repeat above step 7 - 15

chagnes:
v4:
  add PCI address to confirm its port number keep same
  correct test method in cover letter
v3:
  update testpmd command
v2:
  only reset PMD layer resource and keep same port id, but
  not restore settings


Wei Dai (5):
  ethdev: add support of NIC reset
  net/ixgbe: add support of reset
  net/i40e: add support of reset
  app/testpmd: display PCI address in port info
  app/testpmd: enhance command to test NIC reset

 app/test-pmd/cmdline.c                 | 10 ++++--
 app/test-pmd/config.c                  |  5 +++
 app/test-pmd/testpmd.c                 | 61 ++++++++++++++++++++++++++++++++--
 app/test-pmd/testpmd.h                 |  1 +
 drivers/net/i40e/i40e_ethdev.c         | 16 +++++++++
 drivers/net/i40e/i40e_ethdev_vf.c      | 16 +++++++++
 drivers/net/ixgbe/ixgbe_ethdev.c       | 38 +++++++++++++++++++++
 lib/librte_ether/rte_ethdev.c          | 16 +++++++++
 lib/librte_ether/rte_ethdev.h          | 12 +++++++
 lib/librte_ether/rte_ether_version.map |  2 +-
 10 files changed, 170 insertions(+), 7 deletions(-)

--
2.7.4