[PATCH 0/7] dpdk-devbind.py refactor

[PATCH 0/7] dpdk-devbind.py refactor

[Jerin Jacob]

This patchset refactor the dpdk-devbind.py script to

1) Optimize the bind and status operation delay(It is noticeable
when the system has fairly large number of PCIe device. For instance,
OCTEONTX system has around 170 PCIe devices and it takes around
19 seconds to bind devices. With this patch it is reduced to less
than one seconds)

Patch 3 and 4 address this optimization

2) There is a lot of common code in NIC and crypto device
to get the device info, display status.

Patch 1 and 2 create a common code to add new devices
without duplicating the code

3) This patch creates the framework to define the DPDK PCI functional
device by specifying the pci attributes like Vendor ID, Device ID,
Sub Vendor ID, Sub Device ID and Class.This enables a flexible way to
add DPDK function devices based on PCI attributes.

Crypto devices can belong to Encryption class(0x10) or Processor
class(0x0b) based on the vendor preference.

Using this framework, The above disparity can be encoded in the following
format

encryption_class = [{'Class': '10', 'Vendor': None,
                     'Device': None, 'SVendor': None, 'SDevice': None}]

intel_processor_class = [{'Class': '0b', 'Vendor': '8086', 'Device': None,
                        'SVendor': None, 'SDevice': None}]

crypto_devices = [encryption_class, intel_processor_class]

4) Add eventdev, mempool PCI devices support

Guduri Prathyusha (7):
  usertools: refactor the get NIC and crypto details
  usertools: refactor the show status function
  usertools: optimize lspci invocation
  usertools: use optimized driver override scheme to bind
  usertools: define DPDK PCI functional device
  usertools: add eventdev PCI functional device
  usertools: add mempool PCI functional device

 usertools/dpdk-devbind.py | 356 +++++++++++++++++++++++++---------------------
 1 file changed, 195 insertions(+), 161 deletions(-)

-- 
2.5.5

[PATCH 1/7] usertools: refactor the get NIC and crypto details

[Jerin Jacob]

From: Guduri Prathyusha <gprathyusha@caviumnetworks.com>

get_nic_details() and get_crypto_details() shares a lot of common code.
Created a new unified get_device_details() function get the device details.

Signed-off-by: Guduri Prathyusha <gprathyusha@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
---
 usertools/dpdk-devbind.py | 115 +++++++++++++++-------------------------------
 1 file changed, 36 insertions(+), 79 deletions(-)

diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py
index 1b9c651..0444b54 100755
--- a/usertools/dpdk-devbind.py
+++ b/usertools/dpdk-devbind.py
@@ -234,23 +234,24 @@ def get_pci_device_details(dev_id):
 
     return device
 
-
-def get_nic_details():
-    '''This function populates the "devices" dictionary. The keys used are
-    the pci addresses (domain:bus:slot.func). The values are themselves
-    dictionaries - one for each NIC.'''
-    global devices
-    global dpdk_drivers
-
-    # clear any old data
+def clear_data():
+    '''This function clears any old data'''
     devices = {}
+
+def get_device_details(devices_type):
+    '''This function populates the "devices" dictionary. The keys used are
+    the pci addresses (domain:bus:slot.func). The values are themselves
+    dictionaries - one for each NIC.'''
+    global devices
+    global dpdk_drivers
+
     # first loop through and read details for all devices
     # request machine readable format, with numeric IDs
     dev = {}
     dev_lines = check_output(["lspci", "-Dvmmn"]).splitlines()
     for dev_line in dev_lines:
         if len(dev_line) == 0:
-            if dev["Class"][0:2] == NETWORK_BASE_CLASS:
+            if dev["Class"][0:2] == devices_type:
                 # convert device and vendor ids to numbers, then add to global
                 dev["Vendor"] = int(dev["Vendor"], 16)
                 dev["Device"] = int(dev["Device"], 16)
@@ -260,81 +261,35 @@ def get_nic_details():
             name, value = dev_line.decode().split("\t", 1)
             dev[name.rstrip(":")] = value
 
-    # check what is the interface if any for an ssh connection if
-    # any to this host, so we can mark it later.
-    ssh_if = []
-    route = check_output(["ip", "-o", "route"])
-    # filter out all lines for 169.254 routes
-    route = "\n".join(filter(lambda ln: not ln.startswith("169.254"),
+    if devices_type == NETWORK_BASE_CLASS:
+        # check what is the interface if any for an ssh connection if
+        # any to this host, so we can mark it later.
+        ssh_if = []
+        route = check_output(["ip", "-o", "route"])
+        # filter out all lines for 169.254 routes
+        route = "\n".join(filter(lambda ln: not ln.startswith("169.254"),
                              route.decode().splitlines()))
-    rt_info = route.split()
-    for i in range(len(rt_info) - 1):
-        if rt_info[i] == "dev":
-            ssh_if.append(rt_info[i+1])
+        rt_info = route.split()
+        for i in range(len(rt_info) - 1):
+            if rt_info[i] == "dev":
+                ssh_if.append(rt_info[i+1])
 
     # based on the basic info, get extended text details
     for d in devices.keys():
-        # get additional info and add it to existing data
-        devices[d] = devices[d].copy()
-        devices[d].update(get_pci_device_details(d).items())
-
-        for _if in ssh_if:
-            if _if in devices[d]["Interface"].split(","):
-                devices[d]["Ssh_if"] = True
-                devices[d]["Active"] = "*Active*"
-                break
-
-        # add igb_uio to list of supporting modules if needed
-        if "Module_str" in devices[d]:
-            for driver in dpdk_drivers:
-                if driver not in devices[d]["Module_str"]:
-                    devices[d]["Module_str"] = \
-                        devices[d]["Module_str"] + ",%s" % driver
-        else:
-            devices[d]["Module_str"] = ",".join(dpdk_drivers)
-
-        # make sure the driver and module strings do not have any duplicates
-        if has_driver(d):
-            modules = devices[d]["Module_str"].split(",")
-            if devices[d]["Driver_str"] in modules:
-                modules.remove(devices[d]["Driver_str"])
-                devices[d]["Module_str"] = ",".join(modules)
-
-
-def get_crypto_details():
-    '''This function populates the "devices" dictionary. The keys used are
-    the pci addresses (domain:bus:slot.func). The values are themselves
-    dictionaries - one for each NIC.'''
-    global devices
-    global dpdk_drivers
-
-    # clear any old data
-    # devices = {}
-    # first loop through and read details for all devices
-    # request machine readable format, with numeric IDs
-    dev = {}
-    dev_lines = check_output(["lspci", "-Dvmmn"]).splitlines()
-    for dev_line in dev_lines:
-        if len(dev_line) == 0:
-            if dev["Class"][0:2] == CRYPTO_BASE_CLASS:
-                # convert device and vendor ids to numbers, then add to global
-                dev["Vendor"] = int(dev["Vendor"], 16)
-                dev["Device"] = int(dev["Device"], 16)
-                # use dict to make copy of dev
-                devices[dev["Slot"]] = dict(dev)
-        else:
-            name, value = dev_line.decode().split("\t", 1)
-            dev[name.rstrip(":")] = value
-
-    # based on the basic info, get extended text details
-    for d in devices.keys():
-        if devices[d]["Class"][0:2] != CRYPTO_BASE_CLASS:
+        if devices[d]["Class"][0:2] != devices_type:
             continue
 
         # get additional info and add it to existing data
         devices[d] = devices[d].copy()
         devices[d].update(get_pci_device_details(d).items())
 
+        if devices_type == NETWORK_BASE_CLASS:
+            for _if in ssh_if:
+                if _if in devices[d]["Interface"].split(","):
+                    devices[d]["Ssh_if"] = True
+                    devices[d]["Active"] = "*Active*"
+                    break
+
         # add igb_uio to list of supporting modules if needed
         if "Module_str" in devices[d]:
             for driver in dpdk_drivers:
@@ -638,8 +593,9 @@ def do_arg_actions():
         bind_all(args, b_flag, force_flag)
     if status_flag:
         if b_flag is not None:
-            get_nic_details()  # refresh if we have changed anything
-            get_crypto_details()  # refresh if we have changed anything
+            clear_data()
+            get_device_details(NETWORK_BASE_CLASS)  # refresh if we have changed anything
+            get_device_details(CRYPTO_BASE_CLASS)  # refresh if we have changed anything
         show_status()
 
 
@@ -647,8 +603,9 @@ def main():
     '''program main function'''
     parse_args()
     check_modules()
-    get_nic_details()
-    get_crypto_details()
+    clear_data()
+    get_device_details(NETWORK_BASE_CLASS)
+    get_device_details(CRYPTO_BASE_CLASS)
     do_arg_actions()
 
 if __name__ == "__main__":
-- 
2.5.5

[PATCH 2/7] usertools: refactor the show status function

[Jerin Jacob]

From: Guduri Prathyusha <gprathyusha@caviumnetworks.com>

Identified and parameterized the common code in show_status() function as
show_device_status().This will enable to avoid code duplication when
additional devices added to the script.

Signed-off-by: Guduri Prathyusha <gprathyusha@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
---
 usertools/dpdk-devbind.py | 41 +++++++++++------------------------------
 1 file changed, 11 insertions(+), 30 deletions(-)

diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py
index 0444b54..83c4c17 100755
--- a/usertools/dpdk-devbind.py
+++ b/usertools/dpdk-devbind.py
@@ -479,11 +479,7 @@ def display_devices(title, dev_list, extra_params=None):
     strings.sort()
     print("\n".join(strings))  # print one per line
 
-
-def show_status():
-    '''Function called when the script is passed the "--status" option.
-    Displays to the user what devices are bound to the igb_uio driver, the
-    kernel driver or to no driver'''
+def show_device_status(devices_type, device_name):
     global dpdk_drivers
     kernel_drv = []
     dpdk_drv = []
@@ -491,7 +487,7 @@ def show_status():
 
     # split our list of network devices into the three categories above
     for d in devices.keys():
-        if NETWORK_BASE_CLASS in devices[d]["Class"]:
+        if devices_type in devices[d]["Class"]:
             if not has_driver(d):
                 no_drv.append(devices[d])
                 continue
@@ -501,35 +497,20 @@ def show_status():
                 kernel_drv.append(devices[d])
 
     # print each category separately, so we can clearly see what's used by DPDK
-    display_devices("Network devices using DPDK-compatible driver", dpdk_drv,
+    display_devices("%s devices using DPDK-compatible driver" % device_name, dpdk_drv,
                     "drv=%(Driver_str)s unused=%(Module_str)s")
-    display_devices("Network devices using kernel driver", kernel_drv,
+    display_devices("%s devices using kernel driver" % device_name, kernel_drv,
                     "if=%(Interface)s drv=%(Driver_str)s "
                     "unused=%(Module_str)s %(Active)s")
-    display_devices("Other network devices", no_drv, "unused=%(Module_str)s")
+    display_devices("Other %s devices" % device_name, no_drv, "unused=%(Module_str)s")
 
-    # split our list of crypto devices into the three categories above
-    kernel_drv = []
-    dpdk_drv = []
-    no_drv = []
-
-    for d in devices.keys():
-        if CRYPTO_BASE_CLASS in devices[d]["Class"]:
-            if not has_driver(d):
-                no_drv.append(devices[d])
-                continue
-            if devices[d]["Driver_str"] in dpdk_drivers:
-                dpdk_drv.append(devices[d])
-            else:
-                kernel_drv.append(devices[d])
-
-    display_devices("Crypto devices using DPDK-compatible driver", dpdk_drv,
-                    "drv=%(Driver_str)s unused=%(Module_str)s")
-    display_devices("Crypto devices using kernel driver", kernel_drv,
-                    "drv=%(Driver_str)s "
-                    "unused=%(Module_str)s")
-    display_devices("Other crypto devices", no_drv, "unused=%(Module_str)s")
+def show_status():
+    '''Function called when the script is passed the "--status" option.
+    Displays to the user what devices are bound to the igb_uio driver, the
+    kernel driver or to no driver'''
 
+    show_device_status(network_devices, "Network")
+    show_device_status(crypto_devices, "Crypto")
 
 def parse_args():
     '''Parses the command-line arguments given by the user and takes the
-- 
2.5.5

[PATCH 3/7] usertools: optimize lspci invocation

[Jerin Jacob]

From: Guduri Prathyusha <gprathyusha@caviumnetworks.com>

lspci invoked twice over all the pci devices in the system.
The first pass is to extract Numeric IDs and second pass to get extended
device details.

As an optimization, Used lspci with -nn option in get_device_details()
to obtain Numeric ID and extended device details in one shot.

In addition to this, After binding the PCI device, lspci needs to be
invoked again to confirm the proper bind operation. Used a boolean
argument to express this case in get_pci_device_details()

Signed-off-by: Guduri Prathyusha <gprathyusha@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
---
 usertools/dpdk-devbind.py | 52 ++++++++++++++++++++++++++++++-----------------
 1 file changed, 33 insertions(+), 19 deletions(-)

diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py
index 83c4c17..9beadde 100755
--- a/usertools/dpdk-devbind.py
+++ b/usertools/dpdk-devbind.py
@@ -208,19 +208,20 @@ def has_driver(dev_id):
     return "Driver_str" in devices[dev_id]
 
 
-def get_pci_device_details(dev_id):
+def get_pci_device_details(dev_id, probe_lspci):
     '''This function gets additional details for a PCI device'''
     device = {}
 
-    extra_info = check_output(["lspci", "-vmmks", dev_id]).splitlines()
+    if probe_lspci:
+        extra_info = check_output(["lspci", "-vmmks", dev_id]).splitlines()
 
-    # parse lspci details
-    for line in extra_info:
-        if len(line) == 0:
-            continue
-        name, value = line.decode().split("\t", 1)
-        name = name.strip(":") + "_str"
-        device[name] = value
+        # parse lspci details
+        for line in extra_info:
+            if len(line) == 0:
+                continue
+            name, value = line.decode().split("\t", 1)
+            name = name.strip(":") + "_str"
+            device[name] = value
     # check for a unix interface name
     device["Interface"] = ""
     for base, dirs, _ in os.walk("/sys/bus/pci/devices/%s/" % dev_id):
@@ -246,20 +247,30 @@ def get_device_details(devices_type):
     global dpdk_drivers
 
     # first loop through and read details for all devices
-    # request machine readable format, with numeric IDs
+    # request machine readable format, with numeric IDs and String
     dev = {}
-    dev_lines = check_output(["lspci", "-Dvmmn"]).splitlines()
+    dev_lines = check_output(["lspci", "-Dvmmnnk"]).splitlines()
     for dev_line in dev_lines:
         if len(dev_line) == 0:
             if dev["Class"][0:2] == devices_type:
                 # convert device and vendor ids to numbers, then add to global
                 dev["Vendor"] = int(dev["Vendor"], 16)
                 dev["Device"] = int(dev["Device"], 16)
+                if "Driver" in dev.keys():
+                    dev["Driver_str"] = dev.pop("Driver")
                 # use dict to make copy of dev
                 devices[dev["Slot"]] = dict(dev)
+            # Clear previous device's data
+            dev = {}
         else:
             name, value = dev_line.decode().split("\t", 1)
-            dev[name.rstrip(":")] = value
+            value_list = value.rsplit(' ', 1)
+            if len(value_list) > 1:
+                # String stored in <name>_str
+                dev[name.rstrip(":") + '_str'] = value_list[0]
+            # Numeric IDs
+            dev[name.rstrip(":")] = value_list[len(value_list) - 1] \
+                .rstrip("]").lstrip("[")
 
     if devices_type == NETWORK_BASE_CLASS:
         # check what is the interface if any for an ssh connection if
@@ -281,7 +292,8 @@ def get_device_details(devices_type):
 
         # get additional info and add it to existing data
         devices[d] = devices[d].copy()
-        devices[d].update(get_pci_device_details(d).items())
+        # No need to probe lspci
+        devices[d].update(get_pci_device_details(d, False).items())
 
         if devices_type == NETWORK_BASE_CLASS:
             for _if in ssh_if:
@@ -412,7 +424,7 @@ def bind_one(dev_id, driver, force):
         # for some reason, closing dev_id after adding a new PCI ID to new_id
         # results in IOError. however, if the device was successfully bound,
         # we don't care for any errors and can safely ignore IOError
-        tmp = get_pci_device_details(dev_id)
+        tmp = get_pci_device_details(dev_id, True)
         if "Driver_str" in tmp and tmp["Driver_str"] == driver:
             return
         print("Error: bind failed for %s - Cannot bind to driver %s"
@@ -450,7 +462,7 @@ def bind_all(dev_list, driver, force=False):
 
         # update information about this device
         devices[d] = dict(devices[d].items() +
-                          get_pci_device_details(d).items())
+                          get_pci_device_details(d, True).items())
 
         # check if updated information indicates that the device was bound
         if "Driver_str" in devices[d]:
@@ -470,8 +482,9 @@ def display_devices(title, dev_list, extra_params=None):
     else:
         for dev in dev_list:
             if extra_params is not None:
-                strings.append("%s '%s' %s" % (dev["Slot"],
+                strings.append("%s '%s %s' %s" % (dev["Slot"],
                                                dev["Device_str"],
+                                               dev["Device"],
                                                extra_params % dev))
             else:
                 strings.append("%s '%s'" % (dev["Slot"], dev["Device_str"]))
@@ -497,12 +510,13 @@ def show_device_status(devices_type, device_name):
                 kernel_drv.append(devices[d])
 
     # print each category separately, so we can clearly see what's used by DPDK
-    display_devices("%s devices using DPDK-compatible driver" % device_name, dpdk_drv,
-                    "drv=%(Driver_str)s unused=%(Module_str)s")
+    display_devices("%s devices using DPDK-compatible driver" % device_name,
+                    dpdk_drv, "drv=%(Driver_str)s unused=%(Module_str)s")
     display_devices("%s devices using kernel driver" % device_name, kernel_drv,
                     "if=%(Interface)s drv=%(Driver_str)s "
                     "unused=%(Module_str)s %(Active)s")
-    display_devices("Other %s devices" % device_name, no_drv, "unused=%(Module_str)s")
+    display_devices("Other %s devices" % device_name, no_drv,
+                    "unused=%(Module_str)s")
 
 def show_status():
     '''Function called when the script is passed the "--status" option.
-- 
2.5.5

[PATCH 4/7] usertools: use optimized driver override scheme to bind

[Jerin Jacob]

From: Guduri Prathyusha <gprathyusha@caviumnetworks.com>

The current device bind model uses /sys/bus/pci/driver/new_id scheme to
bind devices to the driver. This scheme has following operations to bind
a device to the driver.
1) Write device ID and vendor ID to /sys/bus/pci/driver/new_id
2) Write PCI BDF number to /sys/bus/pci/driver/bind
3) On step (1), _All_ the devices that match the device ID and vendor ID
get bound to the driver
4) Except for requested devices, Unbind the remaining devices

In kernels >= 3.15, An alternative scheme driver_override can be used to
bind a device to driver.This scheme has following operations to bind a
device to driver.
1) Write driver to /sys/bus/pci/device/driver_override
2) Write PCI BDF number to /sys/bus/pci/driver/bind

This script detects the presence of /sys/bus/pci/device/driver_override,
if available use optimized bind scheme to bind it

Signed-off-by: Guduri Prathyusha <gprathyusha@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
---
 usertools/dpdk-devbind.py | 109 +++++++++++++++++++++++++++++++---------------
 1 file changed, 75 insertions(+), 34 deletions(-)

diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py
index 9beadde..4dc6a66 100755
--- a/usertools/dpdk-devbind.py
+++ b/usertools/dpdk-devbind.py
@@ -355,6 +355,25 @@ def unbind_one(dev_id, force):
               "Skipping unbind" % (dev_id))
         return
 
+    # For kernels > 3.15 driver_override is used to bind a device to a driver.
+    # Before unbinding it, overwrite driver_override with empty string so that
+    # the device can be bound to any other driver
+    filename = "/sys/bus/pci/devices/%s/driver_override" % dev_id
+    if os.path.exists(filename):
+        try:
+            f = open(filename, "w")
+        except:
+            print("Error: unbind failed for %s - Cannot open %s"
+                  % (dev_id, filename))
+            sys.exit(1)
+        try:
+            f.write("\00")
+            f.close()
+        except:
+            print("Error: unbind failed for %s - Cannot open %s"
+                  % (dev_id, filename))
+            sys.exit(1)
+
     # write to /sys to unbind
     filename = "/sys/bus/pci/drivers/%s/unbind" % dev["Driver_str"]
     try:
@@ -390,22 +409,44 @@ def bind_one(dev_id, driver, force):
             unbind_one(dev_id, force)
             dev["Driver_str"] = ""  # clear driver string
 
-    # if we are binding to one of DPDK drivers, add PCI id's to that driver
+    # For kernels >= 3.15 driver_override can be used to specify the driver
+    # for a device rather than relying on the driver to provide a positive
+    # match of the device.  The existing process of looking up
+    # the vendor and device ID, adding them to the driver new_id,
+    # will erroneously bind other devices too which has the additional burden
+    # of unbinding those devices
     if driver in dpdk_drivers:
-        filename = "/sys/bus/pci/drivers/%s/new_id" % driver
-        try:
-            f = open(filename, "w")
-        except:
-            print("Error: bind failed for %s - Cannot open %s"
-                  % (dev_id, filename))
-            return
-        try:
-            f.write("%04x %04x" % (dev["Vendor"], dev["Device"]))
-            f.close()
-        except:
-            print("Error: bind failed for %s - Cannot write new PCI ID to "
-                  "driver %s" % (dev_id, driver))
-            return
+        filename = "/sys/bus/pci/devices/%s/driver_override" % dev_id
+        if os.path.exists(filename):
+            try:
+                f = open(filename, "w")
+            except:
+                print("Error: bind failed for %s - Cannot open %s"
+                      % (dev_id, filename))
+                return
+            try:
+                f.write("%s" % driver)
+                f.close()
+            except:
+                print("Error: bind failed for %s - Cannot write driver %s to "
+                      "PCI ID " % (dev_id, driver))
+                return
+        # For kernels < 3.15 use new_id to add PCI id's to the driver
+        else:
+            filename = "/sys/bus/pci/drivers/%s/new_id" % driver
+            try:
+                f = open(filename, "w")
+            except:
+                print("Error: bind failed for %s - Cannot open %s"
+                      % (dev_id, filename))
+                return
+            try:
+                f.write("%04x %04x" % (dev["Vendor"], dev["Device"]))
+                f.close()
+            except:
+                print("Error: bind failed for %s - Cannot write new PCI ID to "
+                      "driver %s" % (dev_id, driver))
+                return
 
     # do the bind by writing to /sys
     filename = "/sys/bus/pci/drivers/%s/bind" % driver
@@ -450,23 +491,24 @@ def bind_all(dev_list, driver, force=False):
     for d in dev_list:
         bind_one(d, driver, force)
 
-    # when binding devices to a generic driver (i.e. one that doesn't have a
-    # PCI ID table), some devices that are not bound to any other driver could
-    # be bound even if no one has asked them to. hence, we check the list of
-    # drivers again, and see if some of the previously-unbound devices were
-    # erroneously bound.
-    for d in devices.keys():
-        # skip devices that were already bound or that we know should be bound
-        if "Driver_str" in devices[d] or d in dev_list:
-            continue
+    # For kenels < 3.15 when binding devices to a generic driver
+    # (i.e. one that doesn't have a PCI ID table) using new_id, some devices
+    # that are not bound to any other driver could be bound even if no one has
+    # asked them to. hence, we check the list of drivers again, and see if
+    # some of the previously-unbound devices were erroneously bound.
+    if not os.path.exists("/sys/bus/pci/devices/%s/driver_override" % d):
+        for d in devices.keys():
+            # skip devices that were already bound or that we know should be bound
+            if "Driver_str" in devices[d] or d in dev_list:
+                continue
 
-        # update information about this device
-        devices[d] = dict(devices[d].items() +
-                          get_pci_device_details(d, True).items())
+            # update information about this device
+            devices[d] = dict(devices[d].items() +
+                              get_pci_device_details(d, True).items())
 
-        # check if updated information indicates that the device was bound
-        if "Driver_str" in devices[d]:
-            unbind_one(d, force)
+            # check if updated information indicates that the device was bound
+            if "Driver_str" in devices[d]:
+                unbind_one(d, force)
 
 
 def display_devices(title, dev_list, extra_params=None):
@@ -510,13 +552,12 @@ def show_device_status(devices_type, device_name):
                 kernel_drv.append(devices[d])
 
     # print each category separately, so we can clearly see what's used by DPDK
-    display_devices("%s devices using DPDK-compatible driver" % device_name,
-                    dpdk_drv, "drv=%(Driver_str)s unused=%(Module_str)s")
+    display_devices("%s devices using DPDK-compatible driver" % device_name, dpdk_drv,
+                    "drv=%(Driver_str)s unused=%(Module_str)s")
     display_devices("%s devices using kernel driver" % device_name, kernel_drv,
                     "if=%(Interface)s drv=%(Driver_str)s "
                     "unused=%(Module_str)s %(Active)s")
-    display_devices("Other %s devices" % device_name, no_drv,
-                    "unused=%(Module_str)s")
+    display_devices("Other %s devices" % device_name, no_drv, "unused=%(Module_str)s")
 
 def show_status():
     '''Function called when the script is passed the "--status" option.
-- 
2.5.5

[PATCH 5/7] usertools: define DPDK PCI functional device

[Jerin Jacob]

From: Guduri Prathyusha <gprathyusha@caviumnetworks.com>

This patch creates the framework to define the DPDK PCI functional
device by specifying the pci attributes like Vendor ID, Device ID,
Sub Vendor ID, Sub Device ID and Class.This enables a flexible way to
add DPDK function devices based on PCI attributes.

Crypto devices can belong to Encryption class(0x10) or Processor
class(0x0b) based on the vendor preference.

Using this framework, The above disparity can be encoded in the following
format

encryption_class = [{'Class': '10', 'Vendor': None,
                     'Device': None, 'SVendor': None, 'SDevice': None}]

intel_processor_class = [{'Class': '0b', 'Vendor': '8086', 'Device': None,
                    'SVendor': None, 'SDevice': None}]

crypto_devices = [encryption_class, intel_processor_class]

Signed-off-by: Guduri Prathyusha <gprathyusha@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
---
 usertools/dpdk-devbind.py | 67 +++++++++++++++++++++++++++++++++--------------
 1 file changed, 48 insertions(+), 19 deletions(-)

diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py
index 4dc6a66..4ea0adf 100755
--- a/usertools/dpdk-devbind.py
+++ b/usertools/dpdk-devbind.py
@@ -38,9 +38,16 @@
 import subprocess
 from os.path import exists, abspath, dirname, basename
 
-# The PCI base class for NETWORK devices
-NETWORK_BASE_CLASS = "02"
-CRYPTO_BASE_CLASS = "0b"
+# The PCI base class for all devices
+network_class = {'Class': '02', 'Vendor': None, 'Device': None,
+                    'SVendor': None, 'SDevice': None}
+encryption_class = {'Class': '10', 'Vendor': None, 'Device': None,
+                   'SVendor': None, 'SDevice': None}
+intel_processor_class = {'Class': '0b', 'Vendor': '8086', 'Device': None,
+                   'SVendor': None, 'SDevice': None}
+
+network_devices = [network_class]
+crypto_devices = [encryption_class, intel_processor_class]
 
 # global dict ethernet devices present. Dictionary indexed by PCI address.
 # Each device within this is itself a dictionary of device properties
@@ -252,10 +259,9 @@ def get_device_details(devices_type):
     dev_lines = check_output(["lspci", "-Dvmmnnk"]).splitlines()
     for dev_line in dev_lines:
         if len(dev_line) == 0:
-            if dev["Class"][0:2] == devices_type:
-                # convert device and vendor ids to numbers, then add to global
-                dev["Vendor"] = int(dev["Vendor"], 16)
-                dev["Device"] = int(dev["Device"], 16)
+            if device_type_match(dev, devices_type):
+                # Replace "Driver" with "Driver_str" to have consistency of
+                # of dictionary key names
                 if "Driver" in dev.keys():
                     dev["Driver_str"] = dev.pop("Driver")
                 # use dict to make copy of dev
@@ -272,7 +278,7 @@ def get_device_details(devices_type):
             dev[name.rstrip(":")] = value_list[len(value_list) - 1] \
                 .rstrip("]").lstrip("[")
 
-    if devices_type == NETWORK_BASE_CLASS:
+    if devices_type == network_devices:
         # check what is the interface if any for an ssh connection if
         # any to this host, so we can mark it later.
         ssh_if = []
@@ -287,7 +293,7 @@ def get_device_details(devices_type):
 
     # based on the basic info, get extended text details
     for d in devices.keys():
-        if devices[d]["Class"][0:2] != devices_type:
+        if not device_type_match(devices[d], devices_type):
             continue
 
         # get additional info and add it to existing data
@@ -295,7 +301,7 @@ def get_device_details(devices_type):
         # No need to probe lspci
         devices[d].update(get_pci_device_details(d, False).items())
 
-        if devices_type == NETWORK_BASE_CLASS:
+        if devices_type == network_devices:
             for _if in ssh_if:
                 if _if in devices[d]["Interface"].split(","):
                     devices[d]["Ssh_if"] = True
@@ -319,6 +325,25 @@ def get_device_details(devices_type):
                 devices[d]["Module_str"] = ",".join(modules)
 
 
+def device_type_match(dev, devices_type):
+    for i in range(len(devices_type)):
+        param_count = len(
+            [x for x in devices_type[i].values() if x is not None])
+        match_count = 0
+        if dev["Class"][0:2] == devices_type[i]["Class"]:
+            match_count = match_count + 1
+            for key in devices_type[i].keys():
+                if key != 'Class' and devices_type[i][key]:
+                    value_list = devices_type[i][key].split(',')
+                    for value in value_list:
+                        if value.strip(' ') == dev[key]:
+                            match_count = match_count + 1
+            # count must be the number of non None parameters to match
+            if match_count == param_count:
+                return True
+            else:
+                return False
+
 def dev_id_from_dev_name(dev_name):
     '''Take a device "name" - a string passed in by user to identify a NIC
     device, and determine the device id - i.e. the domain:bus:slot.func - for
@@ -441,7 +466,9 @@ def bind_one(dev_id, driver, force):
                       % (dev_id, filename))
                 return
             try:
-                f.write("%04x %04x" % (dev["Vendor"], dev["Device"]))
+                # Convert Device and Vendor Id to int to write to new_id
+                f.write("%04x %04x" % (int(dev["Vendor"],16),
+                        int(dev["Device"], 16)))
                 f.close()
             except:
                 print("Error: bind failed for %s - Cannot write new PCI ID to "
@@ -542,7 +569,7 @@ def show_device_status(devices_type, device_name):
 
     # split our list of network devices into the three categories above
     for d in devices.keys():
-        if devices_type in devices[d]["Class"]:
+        if device_type_match(devices[d], devices_type):
             if not has_driver(d):
                 no_drv.append(devices[d])
                 continue
@@ -552,12 +579,13 @@ def show_device_status(devices_type, device_name):
                 kernel_drv.append(devices[d])
 
     # print each category separately, so we can clearly see what's used by DPDK
-    display_devices("%s devices using DPDK-compatible driver" % device_name, dpdk_drv,
-                    "drv=%(Driver_str)s unused=%(Module_str)s")
+    display_devices("%s devices using DPDK-compatible driver" % device_name,
+                    dpdk_drv, "drv=%(Driver_str)s unused=%(Module_str)s")
     display_devices("%s devices using kernel driver" % device_name, kernel_drv,
                     "if=%(Interface)s drv=%(Driver_str)s "
                     "unused=%(Module_str)s %(Active)s")
-    display_devices("Other %s devices" % device_name, no_drv, "unused=%(Module_str)s")
+    display_devices("Other %s devices" % device_name, no_drv,
+                    "unused=%(Module_str)s")
 
 def show_status():
     '''Function called when the script is passed the "--status" option.
@@ -630,8 +658,9 @@ def do_arg_actions():
     if status_flag:
         if b_flag is not None:
             clear_data()
-            get_device_details(NETWORK_BASE_CLASS)  # refresh if we have changed anything
-            get_device_details(CRYPTO_BASE_CLASS)  # refresh if we have changed anything
+            # refresh if we have changed anything
+            get_device_details(network_devices)
+            get_device_details(crypto_devices)
         show_status()
 
 
@@ -640,8 +669,8 @@ def main():
     parse_args()
     check_modules()
     clear_data()
-    get_device_details(NETWORK_BASE_CLASS)
-    get_device_details(CRYPTO_BASE_CLASS)
+    get_device_details(network_devices)
+    get_device_details(crypto_devices)
     do_arg_actions()
 
 if __name__ == "__main__":
-- 
2.5.5

[PATCH 6/7] usertools: add eventdev PCI functional device

[Jerin Jacob]

From: Guduri Prathyusha <gprathyusha@caviumnetworks.com>

Signed-off-by: Guduri Prathyusha <gprathyusha@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
---
 usertools/dpdk-devbind.py | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py
index 4ea0adf..87e1d30 100755
--- a/usertools/dpdk-devbind.py
+++ b/usertools/dpdk-devbind.py
@@ -45,9 +45,12 @@
                    'SVendor': None, 'SDevice': None}
 intel_processor_class = {'Class': '0b', 'Vendor': '8086', 'Device': None,
                    'SVendor': None, 'SDevice': None}
+cavium_sso = {'Class': '08', 'Vendor': '177d', 'Device': 'a04b,a04d',
+              'SVendor': None, 'SDevice': None}
 
 network_devices = [network_class]
 crypto_devices = [encryption_class, intel_processor_class]
+eventdev_devices = [cavium_sso]
 
 # global dict ethernet devices present. Dictionary indexed by PCI address.
 # Each device within this is itself a dictionary of device properties
@@ -594,6 +597,7 @@ def show_status():
 
     show_device_status(network_devices, "Network")
     show_device_status(crypto_devices, "Crypto")
+    show_device_status(eventdev_devices, "Eventdev")
 
 def parse_args():
     '''Parses the command-line arguments given by the user and takes the
@@ -661,6 +665,7 @@ def do_arg_actions():
             # refresh if we have changed anything
             get_device_details(network_devices)
             get_device_details(crypto_devices)
+            get_device_details(eventdev_devices)
         show_status()
 
 
@@ -671,6 +676,7 @@ def main():
     clear_data()
     get_device_details(network_devices)
     get_device_details(crypto_devices)
+    get_device_details(eventdev_devices)
     do_arg_actions()
 
 if __name__ == "__main__":
-- 
2.5.5

[PATCH 7/7] usertools: add mempool PCI functional device

[Jerin Jacob]

From: Guduri Prathyusha <gprathyusha@caviumnetworks.com>

Signed-off-by: Guduri Prathyusha <gprathyusha@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
---
 usertools/dpdk-devbind.py | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py
index 87e1d30..bb4d536 100755
--- a/usertools/dpdk-devbind.py
+++ b/usertools/dpdk-devbind.py
@@ -47,10 +47,13 @@
                    'SVendor': None, 'SDevice': None}
 cavium_sso = {'Class': '08', 'Vendor': '177d', 'Device': 'a04b,a04d',
               'SVendor': None, 'SDevice': None}
+cavium_fpa = {'Class': '08', 'Vendor': '177d', 'Device': 'a053',
+              'SVendor': None, 'SDevice': None}
 
 network_devices = [network_class]
 crypto_devices = [encryption_class, intel_processor_class]
 eventdev_devices = [cavium_sso]
+mempool_devices = [cavium_fpa]
 
 # global dict ethernet devices present. Dictionary indexed by PCI address.
 # Each device within this is itself a dictionary of device properties
@@ -598,6 +601,7 @@ def show_status():
     show_device_status(network_devices, "Network")
     show_device_status(crypto_devices, "Crypto")
     show_device_status(eventdev_devices, "Eventdev")
+    show_device_status(mempool_devices, "Mempool")
 
 def parse_args():
     '''Parses the command-line arguments given by the user and takes the
@@ -666,6 +670,7 @@ def do_arg_actions():
             get_device_details(network_devices)
             get_device_details(crypto_devices)
             get_device_details(eventdev_devices)
+            get_device_details(mempool_devices)
         show_status()
 
 
@@ -677,6 +682,7 @@ def main():
     get_device_details(network_devices)
     get_device_details(crypto_devices)
     get_device_details(eventdev_devices)
+    get_device_details(mempool_devices)
     do_arg_actions()
 
 if __name__ == "__main__":
-- 
2.5.5

Re: [PATCH 0/7] dpdk-devbind.py refactor

[Thomas Monjalon]

22/03/2017 15:11, Jerin Jacob:
> This patchset refactor the dpdk-devbind.py script to
> 
> 1) Optimize the bind and status operation delay(It is noticeable
> when the system has fairly large number of PCIe device. For instance,
> OCTEONTX system has around 170 PCIe devices and it takes around
> 19 seconds to bind devices. With this patch it is reduced to less
> than one seconds)
> 
> Patch 3 and 4 address this optimization
> 
> 2) There is a lot of common code in NIC and crypto device
> to get the device info, display status.
> 
> Patch 1 and 2 create a common code to add new devices
> without duplicating the code
> 
> 3) This patch creates the framework to define the DPDK PCI functional
> device by specifying the pci attributes like Vendor ID, Device ID,
> Sub Vendor ID, Sub Device ID and Class.This enables a flexible way to
> add DPDK function devices based on PCI attributes.
> 
> Crypto devices can belong to Encryption class(0x10) or Processor
> class(0x0b) based on the vendor preference.
> 
> Using this framework, The above disparity can be encoded in the following
> format
> 
> encryption_class = [{'Class': '10', 'Vendor': None,
>                      'Device': None, 'SVendor': None, 'SDevice': None}]
> 
> intel_processor_class = [{'Class': '0b', 'Vendor': '8086', 'Device': None,
>                         'SVendor': None, 'SDevice': None}]
> 
> crypto_devices = [encryption_class, intel_processor_class]
> 
> 4) Add eventdev, mempool PCI devices support
> 
> Guduri Prathyusha (7):
>   usertools: refactor the get NIC and crypto details
>   usertools: refactor the show status function
>   usertools: optimize lspci invocation
>   usertools: use optimized driver override scheme to bind
>   usertools: define DPDK PCI functional device
>   usertools: add eventdev PCI functional device
>   usertools: add mempool PCI functional device

Applied, thanks