[PATCH 0/4] Implement pmd hardware support exports

[PATCH 0/4] Implement pmd hardware support exports

[Neil Horman]

Hey all-
	So heres attempt number 2 at a method for exporting PMD hardware support
information.  As we discussed previously, the consensus seems to be that pmd
information should be:

1) Able to be interrogated on any ELF binary (application binary or individual
DSO)
2) Equally functional on statically linked applications or on DSO's
3) Resilient to symbol stripping
4) Script friendly
5) Show kernel dependencies
6) List driver options
7) Show driver name
8) Offer human readable output
9) Show DPDK version
10) Show driver version
11) Allow for expansion
12) Not place additional build environment dependencies on an application

I added item 12 myself, because I don't think its reasonable to require
applications to use a specific linker script to get hardware support information
(which precludes the use of a special .modinfo section like the kernel uses)

However, we still can use some tricks from the kernel to make this work.  In
this approach, what I've done is the following:

A) Modified the driver registration macro to also define two variables:
this_pmd_name<n>
this_pmd_table<n>

The second is optional, and omitted for virtual devices.  These symbols are
static, and marked as used, so they will always be emitted by the compiler, and
with their well known names, easy to search for in an object (.o) file

B) Added a utility called pmdinfo.  This utility is not meant for general use,
but rather is used by the dpdk build environment itself when compiling pmds.
for each object that uses the PMD_REGISTER_DRIVER macro, this utiity is run.  It
searches for the symbols in (A), and using those, extracts the hardware support
info, and module name from the object, using that to produce a new c file
containing a single variable in the following format:

static const char <name>[] __attribute__((used)) = "PMD_DRIVER_INFO=<json>";

The <name> is arbitrary, as its static and not referenced.  The relevant bit is
the string value assigned to it.  The <json> is a json encoded string of the
extracted hardware support information pmdinfo found for the corresponding
object.  This C file is suitable for compilation and relocatable linking back
into the parent object file.  The result of this operation is that the object
string table now contains a string that will not e removed by stripping, whos
leading text (PMD_DRIVER_INFO) can be easily searched for at any time weather
the symbol referring to it is stripped or not.

C) Added a utilty called pmd_hw_info.py.  This python script, searches the
string table of the .rodata section of any provided ELF file looking for the
PMD_DRIVER_INFO prefix on a string.  When found, it will interpret the remainder
of the string as json, and output the hardware support for that ELF file (if
any).


This approach ticks most of the above major boxes:
1) Impervious to stripping
2) Works on static and dynamic binaries
3) Human and script friendly
4) Allows for expansion
5) Doesn't require additional build environment needs for applications

Because of (4) the other items should be pretty easy to implement, as its just a
matter of modifying the macros to export the info, pmdinfo to encode it to json,
and pmd_hw_support.py to read it.  I'm not adding them now, as theres alot of
rote work to do to get some of them in place (e.g. DPDK has no current global
VERSION macro, drivers don't have a consistent version scheme, command line
strings have to be built for each driver, etc).  But once this is accepted,
those items can be done as time allows and should be fairly easy to implement.


Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>

[PATCH 1/4] pmdinfo: Add buildtools and pmdinfo utility

[Neil Horman]

pmdinfo is a tool used to parse object files and build json strings for use in
later determining hardware support in a dso or application binary.  pmdinfo
looks for the non-exported symbol names this_pmd_name<n> and this_pmd_tbl<n>
(where n is a integer counter).  It records the name of each of these tuples,
using the later to find the symbolic name of the pci_table for physical devices
that the object supports.  With this information, it outputs a C file with a
single line of the form:

static char *<pmd_name>_driver_info[] __attribute__((used)) = " \
	PMD_DRIVER_INFO=<json string>";

Where <pmd_name> is the arbitrary name of the pmd, and <json_string> is the json
encoded string that hold relevant pmd information, including the pmd name, type
and optional array of pci device/vendor ids that the driver supports.

This c file is suitable for compiling to object code, then relocatably linking
into the parent file from which the C was generated.  This creates an entry in
the string table of the object that can inform a later tool about hardware
support.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 GNUmakefile                  |   2 +-
 buildtools/Makefile          |  36 ++++
 buildtools/pmdinfo/Makefile  |  48 +++++
 buildtools/pmdinfo/pmdinfo.c | 435 +++++++++++++++++++++++++++++++++++++++++++
 buildtools/pmdinfo/pmdinfo.h | 210 +++++++++++++++++++++
 mk/rte.buildtools.mk         | 148 +++++++++++++++
 mk/rte.sdkbuild.mk           |   3 +-
 7 files changed, 880 insertions(+), 2 deletions(-)
 create mode 100644 buildtools/Makefile
 create mode 100644 buildtools/pmdinfo/Makefile
 create mode 100644 buildtools/pmdinfo/pmdinfo.c
 create mode 100644 buildtools/pmdinfo/pmdinfo.h
 create mode 100644 mk/rte.buildtools.mk

diff --git a/GNUmakefile b/GNUmakefile
index b59e4b6..00fe0db 100644
--- a/GNUmakefile
+++ b/GNUmakefile
@@ -40,6 +40,6 @@ export RTE_SDK
 # directory list
 #
 
-ROOTDIRS-y := lib drivers app
+ROOTDIRS-y := buildtools lib drivers app
 
 include $(RTE_SDK)/mk/rte.sdkroot.mk
diff --git a/buildtools/Makefile b/buildtools/Makefile
new file mode 100644
index 0000000..0f15d58
--- /dev/null
+++ b/buildtools/Makefile
@@ -0,0 +1,36 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+DIRS-y += pmdinfo 
+
+include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/buildtools/pmdinfo/Makefile b/buildtools/pmdinfo/Makefile
new file mode 100644
index 0000000..3dea68b
--- /dev/null
+++ b/buildtools/pmdinfo/Makefile
@@ -0,0 +1,48 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+#
+# library name
+#
+APP = pmdinfo
+
+#
+# all sources are stored in SRCS-y
+#
+SRCS-y += pmdinfo.c
+
+#CFLAGS += $(WERROR_FLAGS) -g
+CFLAGS += -g
+
+include $(RTE_SDK)/mk/rte.buildtools.mk
+
diff --git a/buildtools/pmdinfo/pmdinfo.c b/buildtools/pmdinfo/pmdinfo.c
new file mode 100644
index 0000000..5e705ab
--- /dev/null
+++ b/buildtools/pmdinfo/pmdinfo.c
@@ -0,0 +1,435 @@
+/* Postprocess pmd object files to export hw support 
+ *
+ * Copyright 2016 Neil Horman <nhorman@tuxdriver.com>
+ * Based in part on modpost.c from the linux kernel
+ *
+ * This software may be used and distributed according to the terms
+ * of the GNU General Public License V2, incorporated herein by reference.
+ *
+ */
+
+#define _GNU_SOURCE
+#include <stdio.h>
+#include <ctype.h>
+#include <string.h>
+#include <limits.h>
+#include <stdbool.h>
+#include <errno.h>
+#include "pmdinfo.h"
+
+
+static const char *sym_name(struct elf_info *elf, Elf_Sym *sym)
+{
+	if (sym)
+		return elf->strtab + sym->st_name;
+	else
+		return "(unknown)";
+}
+
+void *grab_file(const char *filename, unsigned long *size)
+{
+	struct stat st;
+	void *map = MAP_FAILED;
+	int fd;
+
+	fd = open(filename, O_RDONLY);
+	if (fd < 0)
+		return NULL;
+	if (fstat(fd, &st))
+		goto failed;
+
+	*size = st.st_size;
+	map = mmap(NULL, *size, PROT_READ|PROT_WRITE, MAP_PRIVATE, fd, 0);
+
+failed:
+	close(fd);
+	if (map == MAP_FAILED)
+		return NULL;
+	return map;
+}
+
+/**
+  * Return a copy of the next line in a mmap'ed file.
+  * spaces in the beginning of the line is trimmed away.
+  * Return a pointer to a static buffer.
+  **/
+char *get_next_line(unsigned long *pos, void *file, unsigned long size)
+{
+	static char line[4096];
+	int skip = 1;
+	size_t len = 0;
+	signed char *p = (signed char *)file + *pos;
+	char *s = line;
+
+	for (; *pos < size ; (*pos)++) {
+		if (skip && isspace(*p)) {
+			p++;
+			continue;
+		}
+		skip = 0;
+		if (*p != '\n' && (*pos < size)) {
+			len++;
+			*s++ = *p++;
+			if (len > 4095)
+				break; /* Too long, stop */
+		} else {
+			/* End of string */
+			*s = '\0';
+			return line;
+		}
+	}
+	/* End of buffer */
+	return NULL;
+}
+
+void release_file(void *file, unsigned long size)
+{
+	munmap(file, size);
+}
+
+
+static void *get_sym_value(struct elf_info *info, const Elf_Sym *sym)
+{
+	void *ptr = (void *)info->hdr + info->sechdrs[sym->st_shndx].sh_offset;
+
+	return (void *)(ptr + sym->st_value);
+}
+
+static Elf_Sym *find_sym_in_symtab(struct elf_info *info, 
+				   const char *name, Elf_Sym *last)
+{
+	Elf_Sym *idx;
+	if (last)
+		idx = last+1;
+	else
+		idx = info->symtab_start;
+
+	for(; idx < info->symtab_stop; idx++) {
+		const char *n = sym_name(info, idx);
+		if (!strncmp(n, name, strlen(name)))
+			return idx;
+	}
+	return NULL;
+}
+
+static int parse_elf(struct elf_info *info, const char *filename)
+{
+	unsigned int i;
+	Elf_Ehdr *hdr;
+	Elf_Shdr *sechdrs;
+	Elf_Sym  *sym;
+	const char *secstrings;
+	unsigned int symtab_idx = ~0U, symtab_shndx_idx = ~0U;
+
+	hdr = grab_file(filename, &info->size);
+	if (!hdr) {
+		perror(filename);
+		exit(1);
+	}
+	info->hdr = hdr;
+	if (info->size < sizeof(*hdr)) {
+		/* file too small, assume this is an empty .o file */
+		return 0;
+	}
+	/* Is this a valid ELF file? */
+	if ((hdr->e_ident[EI_MAG0] != ELFMAG0) ||
+	    (hdr->e_ident[EI_MAG1] != ELFMAG1) ||
+	    (hdr->e_ident[EI_MAG2] != ELFMAG2) ||
+	    (hdr->e_ident[EI_MAG3] != ELFMAG3)) {
+		/* Not an ELF file - silently ignore it */
+		return 0;
+	}
+	/* Fix endianness in ELF header */
+	hdr->e_type      = TO_NATIVE(hdr->e_type);
+	hdr->e_machine   = TO_NATIVE(hdr->e_machine);
+	hdr->e_version   = TO_NATIVE(hdr->e_version);
+	hdr->e_entry     = TO_NATIVE(hdr->e_entry);
+	hdr->e_phoff     = TO_NATIVE(hdr->e_phoff);
+	hdr->e_shoff     = TO_NATIVE(hdr->e_shoff);
+	hdr->e_flags     = TO_NATIVE(hdr->e_flags);
+	hdr->e_ehsize    = TO_NATIVE(hdr->e_ehsize);
+	hdr->e_phentsize = TO_NATIVE(hdr->e_phentsize);
+	hdr->e_phnum     = TO_NATIVE(hdr->e_phnum);
+	hdr->e_shentsize = TO_NATIVE(hdr->e_shentsize);
+	hdr->e_shnum     = TO_NATIVE(hdr->e_shnum);
+	hdr->e_shstrndx  = TO_NATIVE(hdr->e_shstrndx);
+	sechdrs = (void *)hdr + hdr->e_shoff;
+	info->sechdrs = sechdrs;
+
+	/* Check if file offset is correct */
+	if (hdr->e_shoff > info->size) {
+		fprintf(stderr, "section header offset=%lu in file '%s' is bigger than "
+		      "filesize=%lu\n", (unsigned long)hdr->e_shoff,
+		      filename, info->size);
+		return 0;
+	}
+
+	if (hdr->e_shnum == SHN_UNDEF) {
+		/*
+		 * There are more than 64k sections,
+		 * read count from .sh_size.
+		 */
+		info->num_sections = TO_NATIVE(sechdrs[0].sh_size);
+	}
+	else {
+		info->num_sections = hdr->e_shnum;
+	}
+	if (hdr->e_shstrndx == SHN_XINDEX) {
+		info->secindex_strings = TO_NATIVE(sechdrs[0].sh_link);
+	}
+	else {
+		info->secindex_strings = hdr->e_shstrndx;
+	}
+
+	/* Fix endianness in section headers */
+	for (i = 0; i < info->num_sections; i++) {
+		sechdrs[i].sh_name      = TO_NATIVE(sechdrs[i].sh_name);
+		sechdrs[i].sh_type      = TO_NATIVE(sechdrs[i].sh_type);
+		sechdrs[i].sh_flags     = TO_NATIVE(sechdrs[i].sh_flags);
+		sechdrs[i].sh_addr      = TO_NATIVE(sechdrs[i].sh_addr);
+		sechdrs[i].sh_offset    = TO_NATIVE(sechdrs[i].sh_offset);
+		sechdrs[i].sh_size      = TO_NATIVE(sechdrs[i].sh_size);
+		sechdrs[i].sh_link      = TO_NATIVE(sechdrs[i].sh_link);
+		sechdrs[i].sh_info      = TO_NATIVE(sechdrs[i].sh_info);
+		sechdrs[i].sh_addralign = TO_NATIVE(sechdrs[i].sh_addralign);
+		sechdrs[i].sh_entsize   = TO_NATIVE(sechdrs[i].sh_entsize);
+	}
+	/* Find symbol table. */
+	secstrings = (void *)hdr + sechdrs[info->secindex_strings].sh_offset;
+	for (i = 1; i < info->num_sections; i++) {
+		const char *secname;
+		int nobits = sechdrs[i].sh_type == SHT_NOBITS;
+
+		if (!nobits && sechdrs[i].sh_offset > info->size) {
+			fprintf(stderr, "%s is truncated. sechdrs[i].sh_offset=%lu > "
+			      "sizeof(*hrd)=%zu\n", filename,
+			      (unsigned long)sechdrs[i].sh_offset,
+			      sizeof(*hdr));
+			return 0;
+		}
+		secname = secstrings + sechdrs[i].sh_name;
+		if (strcmp(secname, ".modinfo") == 0) {
+			if (nobits)
+				fprintf(stderr, "%s has NOBITS .modinfo\n", filename);
+			info->modinfo = (void *)hdr + sechdrs[i].sh_offset;
+			info->modinfo_len = sechdrs[i].sh_size;
+		} else if (strcmp(secname, "__ksymtab") == 0)
+			info->export_sec = i;
+		else if (strcmp(secname, "__ksymtab_unused") == 0)
+			info->export_unused_sec = i;
+		else if (strcmp(secname, "__ksymtab_gpl") == 0)
+			info->export_gpl_sec = i;
+		else if (strcmp(secname, "__ksymtab_unused_gpl") == 0)
+			info->export_unused_gpl_sec = i;
+		else if (strcmp(secname, "__ksymtab_gpl_future") == 0)
+			info->export_gpl_future_sec = i;
+
+		if (sechdrs[i].sh_type == SHT_SYMTAB) {
+			unsigned int sh_link_idx;
+			symtab_idx = i;
+			info->symtab_start = (void *)hdr +
+			    sechdrs[i].sh_offset;
+			info->symtab_stop  = (void *)hdr +
+			    sechdrs[i].sh_offset + sechdrs[i].sh_size;
+			sh_link_idx = sechdrs[i].sh_link;
+			info->strtab       = (void *)hdr +
+			    sechdrs[sh_link_idx].sh_offset;
+		}
+
+		/* 32bit section no. table? ("more than 64k sections") */
+		if (sechdrs[i].sh_type == SHT_SYMTAB_SHNDX) {
+			symtab_shndx_idx = i;
+			info->symtab_shndx_start = (void *)hdr +
+			    sechdrs[i].sh_offset;
+			info->symtab_shndx_stop  = (void *)hdr +
+			    sechdrs[i].sh_offset + sechdrs[i].sh_size;
+		}
+	}
+	if (!info->symtab_start)
+		fprintf(stderr, "%s has no symtab?\n", filename);
+
+	/* Fix endianness in symbols */
+	for (sym = info->symtab_start; sym < info->symtab_stop; sym++) {
+		sym->st_shndx = TO_NATIVE(sym->st_shndx);
+		sym->st_name  = TO_NATIVE(sym->st_name);
+		sym->st_value = TO_NATIVE(sym->st_value);
+		sym->st_size  = TO_NATIVE(sym->st_size);
+	}
+
+	if (symtab_shndx_idx != ~0U) {
+		Elf32_Word *p;
+		if (symtab_idx != sechdrs[symtab_shndx_idx].sh_link)
+			fprintf(stderr, "%s: SYMTAB_SHNDX has bad sh_link: %u!=%u\n",
+			      filename, sechdrs[symtab_shndx_idx].sh_link,
+			      symtab_idx);
+		/* Fix endianness */
+		for (p = info->symtab_shndx_start; p < info->symtab_shndx_stop;
+		     p++)
+			*p = TO_NATIVE(*p);
+	}
+
+	return 1;
+}
+
+static void parse_elf_finish(struct elf_info *info)
+{
+	struct pmd_driver *tmp, *idx = info->drivers;
+	release_file(info->hdr, info->size);
+	while (idx) {
+		tmp = idx->next;
+		free(idx);
+		idx = tmp;
+	}
+}
+
+static const char *sec_name(struct elf_info *elf, int secindex)
+{
+	Elf_Shdr *sechdrs = elf->sechdrs;
+	return (void *)elf->hdr +
+		elf->sechdrs[elf->secindex_strings].sh_offset +
+		sechdrs[secindex].sh_name;
+}
+
+static int get_symbol_index(struct elf_info *info, Elf64_Sym *sym)
+{
+	const char *name =  sym_name(info, sym);
+	const char *idx;
+
+	idx = name;
+	while (idx) {
+		if (isdigit(*idx))
+			return atoi(idx);
+		idx++;
+	}
+	return -1;
+}
+
+static int complete_pmd_entry(struct elf_info *info, struct pmd_driver *drv)
+{
+	const char *tname;
+	int i = get_symbol_index(info, drv->name_sym);
+	char drvsym[128];
+
+	if (i == -1)
+		return -ENOENT;
+
+	drv->name = get_sym_value(info, drv->name_sym);
+
+	sprintf(drvsym, "this_pmd_driver%d", i);
+
+	drv->driver = find_sym_in_symtab(info, drvsym, NULL);
+
+	/*
+ 	 * If this returns NULL, then this is a PMD_VDEV, because
+ 	 * it has no pci table reference
+ 	 */
+	if (!drv->driver) {
+		drv->pci_tbl = NULL;
+		return 0;
+	}
+
+	tname = get_sym_value(info, drv->driver);
+	drv->pci_tbl_sym = find_sym_in_symtab(info, tname, NULL);
+
+	if (!drv->pci_tbl_sym)
+		return -ENOENT;
+
+	drv->pci_tbl = (struct rte_pci_id *)get_sym_value(info, drv->pci_tbl_sym);
+	if (!drv->pci_tbl)
+		return -ENOENT;
+
+
+	return 0;
+	
+}
+
+static int locate_pmd_entries(struct elf_info *info)
+{
+	Elf_Sym *last = NULL;
+	struct pmd_driver *new;
+
+	info->drivers = NULL;
+
+	do {
+		new = malloc(sizeof(struct pmd_driver));
+		new->name_sym = find_sym_in_symtab(info, "this_pmd_name", last);
+		last = new->name_sym;
+		if (!new->name_sym)
+			free(new);
+		else {
+			if (complete_pmd_entry(info, new)) {
+				fprintf(stderr, "Failed to complete pmd entry\n");
+				free(new);
+			} else {
+				new->next = info->drivers;
+				info->drivers = new;
+			}
+		}
+	} while (last);
+}
+
+static void output_pmd_info_string(struct elf_info *info, char *outfile)
+{
+	FILE *ofd;
+	struct pmd_driver *drv;
+	struct rte_pci_id *pci_ids;
+	int idx = 0;
+
+	ofd = fopen(outfile, "w+");
+	if (!ofd) {
+		fprintf(stderr, "Unable to open output file\n");
+		return;
+	}
+
+	drv = info->drivers;
+
+	while (drv) {
+		fprintf(ofd, "const char %s_pmd_info[] __attribute__((used)) = \"PMD_INFO_STRING= {",
+			drv->name);
+		fprintf(ofd,"\\\"name\\\" : \\\"%s\\\", ", drv->name);
+		fprintf(ofd,"\\\"type\\\" : \\\"%s\\\", ", drv->pci_tbl ? "PMD_PDEV" : "PMD_VDEV");
+ 
+		pci_ids = drv->pci_tbl;
+		fprintf(ofd, "\\\"pci_ids\\\" : [");
+
+		while (pci_ids && pci_ids->device_id) {
+			fprintf(ofd, "[%d, %d, %d, %d]",
+				pci_ids->vendor_id, pci_ids->device_id,
+				pci_ids->subsystem_vendor_id,
+				pci_ids->subsystem_device_id);
+			pci_ids++;
+			if (pci_ids->device_id)
+				fprintf(ofd, ",");
+			else
+				fprintf(ofd, " ");
+		}
+		fprintf(ofd, "]}\";");
+		drv = drv->next;
+	}
+
+	fclose(ofd);
+}
+
+int main(int argc, char **argv)
+{
+	struct elf_info info;
+	int rc = 1;
+
+	if (argc < 3) {
+		fprintf(stderr, "usage: pmdinfo <object file> <c output file>\n");
+		exit(127);
+	}
+	parse_elf(&info, argv[1]);
+
+	locate_pmd_entries(&info);
+
+	if (info.drivers) {
+		output_pmd_info_string(&info, argv[2]);
+		rc = 0;
+	} else {
+		fprintf(stderr, "Hmm, Appears to be a driver but no drivers registered\n");
+	}
+
+	parse_elf_finish(&info);
+	exit(rc);
+}
diff --git a/buildtools/pmdinfo/pmdinfo.h b/buildtools/pmdinfo/pmdinfo.h
new file mode 100644
index 0000000..5dafb67
--- /dev/null
+++ b/buildtools/pmdinfo/pmdinfo.h
@@ -0,0 +1,210 @@
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdarg.h>
+#include <string.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/mman.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <elf.h>
+
+
+/* On BSD-alike OSes elf.h defines these according to host's word size */
+#undef ELF_ST_BIND
+#undef ELF_ST_TYPE
+#undef ELF_R_SYM
+#undef ELF_R_TYPE
+
+#if 0 
+
+#define Elf_Ehdr    Elf32_Ehdr
+#define Elf_Shdr    Elf32_Shdr
+#define Elf_Sym     Elf32_Sym
+#define Elf_Addr    Elf32_Addr
+#define Elf_Sword   Elf64_Sword
+#define Elf_Section Elf32_Half
+#define ELF_ST_BIND ELF32_ST_BIND
+#define ELF_ST_TYPE ELF32_ST_TYPE
+
+#define Elf_Rel     Elf32_Rel
+#define Elf_Rela    Elf32_Rela
+#define ELF_R_SYM   ELF32_R_SYM
+#define ELF_R_TYPE  ELF32_R_TYPE
+#else
+
+#define Elf_Ehdr    Elf64_Ehdr
+#define Elf_Shdr    Elf64_Shdr
+#define Elf_Sym     Elf64_Sym
+#define Elf_Addr    Elf64_Addr
+#define Elf_Sword   Elf64_Sxword
+#define Elf_Section Elf64_Half
+#define ELF_ST_BIND ELF64_ST_BIND
+#define ELF_ST_TYPE ELF64_ST_TYPE
+
+#define Elf_Rel     Elf64_Rel
+#define Elf_Rela    Elf64_Rela
+#define ELF_R_SYM   ELF64_R_SYM
+#define ELF_R_TYPE  ELF64_R_TYPE
+#endif
+
+/* The 64-bit MIPS ELF ABI uses an unusual reloc format. */
+typedef struct
+{
+	Elf32_Word    r_sym;	/* Symbol index */
+	unsigned char r_ssym;	/* Special symbol for 2nd relocation */
+	unsigned char r_type3;	/* 3rd relocation type */
+	unsigned char r_type2;	/* 2nd relocation type */
+	unsigned char r_type1;	/* 1st relocation type */
+} _Elf64_Mips_R_Info;
+
+typedef union
+{
+	Elf64_Xword		r_info_number;
+	_Elf64_Mips_R_Info	r_info_fields;
+} _Elf64_Mips_R_Info_union;
+
+#define ELF64_MIPS_R_SYM(i) \
+  ((__extension__ (_Elf64_Mips_R_Info_union)(i)).r_info_fields.r_sym)
+
+#define ELF64_MIPS_R_TYPE(i) \
+  ((__extension__ (_Elf64_Mips_R_Info_union)(i)).r_info_fields.r_type1)
+
+#if 0 
+
+static inline void __endian(const void *src, void *dest, unsigned int size)
+{
+	unsigned int i;
+	for (i = 0; i < size; i++)
+		((unsigned char*)dest)[i] = ((unsigned char*)src)[size - i-1];
+}
+
+#define TO_NATIVE(x)						\
+({								\
+	typeof(x) __x;						\
+	__endian(&(x), &(__x), sizeof(__x));			\
+	__x;							\
+})
+
+#else /* endianness matches */
+
+#define TO_NATIVE(x) (x)
+
+#endif
+
+#define NOFAIL(ptr)   do_nofail((ptr), #ptr)
+void *do_nofail(void *ptr, const char *expr);
+
+struct buffer {
+	char *p;
+	int pos;
+	int size;
+};
+
+void __attribute__((format(printf, 2, 3)))
+buf_printf(struct buffer *buf, const char *fmt, ...);
+
+void
+buf_write(struct buffer *buf, const char *s, int len);
+
+struct module {
+	struct module *next;
+	const char *name;
+	int gpl_compatible;
+	struct symbol *unres;
+	int seen;
+	int skip;
+	int has_init;
+	int has_cleanup;
+	struct buffer dev_table_buf;
+	char	     srcversion[25];
+	int is_dot_o;
+};
+
+struct rte_pci_id {
+	uint16_t vendor_id;           /**< Vendor ID or PCI_ANY_ID. */
+	uint16_t device_id;           /**< Device ID or PCI_ANY_ID. */
+	uint16_t subsystem_vendor_id; /**< Subsystem vendor ID or PCI_ANY_ID. */
+	uint16_t subsystem_device_id; /**< Subsystem device ID or PCI_ANY_ID. */
+};
+
+struct pmd_driver {
+	Elf_Sym *driver;
+	Elf_Sym *pci_tbl_sym;
+	Elf_Sym *name_sym;
+	struct rte_pci_id *pci_tbl;
+	struct pmd_driver *next;
+	const char *name;
+};
+
+struct elf_info {
+	unsigned long size;
+	Elf_Ehdr     *hdr;
+	Elf_Shdr     *sechdrs;
+	Elf_Sym      *symtab_start;
+	Elf_Sym      *symtab_stop;
+	Elf_Section  export_sec;
+	Elf_Section  export_unused_sec;
+	Elf_Section  export_gpl_sec;
+	Elf_Section  export_unused_gpl_sec;
+	Elf_Section  export_gpl_future_sec;
+	char         *strtab;
+	char	     *modinfo;
+	unsigned int modinfo_len;
+
+	/* support for 32bit section numbers */
+
+	unsigned int num_sections; /* max_secindex + 1 */
+	unsigned int secindex_strings;
+	/* if Nth symbol table entry has .st_shndx = SHN_XINDEX,
+	 * take shndx from symtab_shndx_start[N] instead */
+	Elf32_Word   *symtab_shndx_start;
+	Elf32_Word   *symtab_shndx_stop;
+
+	struct pmd_driver *drivers;
+};
+
+static inline int is_shndx_special(unsigned int i)
+{
+	return i != SHN_XINDEX && i >= SHN_LORESERVE && i <= SHN_HIRESERVE;
+}
+
+/*
+ * Move reserved section indices SHN_LORESERVE..SHN_HIRESERVE out of
+ * the way to -256..-1, to avoid conflicting with real section
+ * indices.
+ */
+#define SPECIAL(i) ((i) - (SHN_HIRESERVE + 1))
+
+/* Accessor for sym->st_shndx, hides ugliness of "64k sections" */
+static inline unsigned int get_secindex(const struct elf_info *info,
+					const Elf_Sym *sym)
+{
+	if (is_shndx_special(sym->st_shndx))
+		return SPECIAL(sym->st_shndx);
+	if (sym->st_shndx != SHN_XINDEX)
+		return sym->st_shndx;
+	return info->symtab_shndx_start[sym - info->symtab_start];
+}
+
+/* file2alias.c */
+extern unsigned int cross_build;
+void handle_moddevtable(struct module *mod, struct elf_info *info,
+			Elf_Sym *sym, const char *symname);
+void add_moddevtable(struct buffer *buf, struct module *mod);
+
+/* sumversion.c */
+void maybe_frob_rcs_version(const char *modfilename,
+			    char *version,
+			    void *modinfo,
+			    unsigned long modinfo_offset);
+void get_src_version(const char *modname, char sum[], unsigned sumlen);
+
+/* from modpost.c */
+void *grab_file(const char *filename, unsigned long *size);
+char* get_next_line(unsigned long *pos, void *file, unsigned long size);
+void release_file(void *file, unsigned long size);
+
+void fatal(const char *fmt, ...);
+void warn(const char *fmt, ...);
+void merror(const char *fmt, ...);
diff --git a/mk/rte.buildtools.mk b/mk/rte.buildtools.mk
new file mode 100644
index 0000000..e8bfcef
--- /dev/null
+++ b/mk/rte.buildtools.mk
@@ -0,0 +1,148 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
+#   Copyright(c) 2014-2015 6WIND S.A.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/internal/rte.compile-pre.mk
+include $(RTE_SDK)/mk/internal/rte.install-pre.mk
+include $(RTE_SDK)/mk/internal/rte.clean-pre.mk
+include $(RTE_SDK)/mk/internal/rte.build-pre.mk
+include $(RTE_SDK)/mk/internal/rte.depdirs-pre.mk
+
+# VPATH contains at least SRCDIR
+VPATH += $(SRCDIR)
+
+_BUILD = $(APP)
+_INSTALL = $(INSTALL-FILES-y) $(SYMLINK-FILES-y)
+_INSTALL += $(RTE_OUTPUT)/buildtools/$(APP) $(RTE_OUTPUT)/buildtools/$(APP).map
+POSTINSTALL += target-appinstall
+_CLEAN = doclean
+POSTCLEAN += target-appclean
+
+.PHONY: all
+all: install
+
+.PHONY: install
+install: build _postinstall
+
+_postinstall: build
+
+.PHONY: build
+build: _postbuild
+
+exe2cmd = $(strip $(call dotfile,$(patsubst %,%.cmd,$(1))))
+
+ifeq ($(LINK_USING_CC),1)
+override EXTRA_LDFLAGS := $(call linkerprefix,$(EXTRA_LDFLAGS))
+O_TO_EXE = $(CC) $(CFLAGS) $(LDFLAGS_$(@)) \
+	-Wl,-Map=$(@).map,--cref -o $@ $(OBJS-y) $(call linkerprefix,$(LDFLAGS)) \
+	$(EXTRA_LDFLAGS) $(call linkerprefix,$(LDLIBS))
+else
+O_TO_EXE = $(LD) $(LDFLAGS) $(LDFLAGS_$(@)) $(EXTRA_LDFLAGS) \
+	-Map=$(@).map --cref -o $@ $(OBJS-y) $(LDLIBS)
+endif
+O_TO_EXE_STR = $(subst ','\'',$(O_TO_EXE)) #'# fix syntax highlight
+O_TO_EXE_DISP = $(if $(V),"$(O_TO_EXE_STR)","  LD $(@)")
+O_TO_EXE_CMD = "cmd_$@ = $(O_TO_EXE_STR)"
+O_TO_EXE_DO = @set -e; \
+	echo $(O_TO_EXE_DISP); \
+	$(O_TO_EXE) && \
+	echo $(O_TO_EXE_CMD) > $(call exe2cmd,$(@))
+
+-include .$(APP).cmd
+
+# path where libraries are retrieved
+LDLIBS_PATH := $(subst -Wl$(comma)-L,,$(filter -Wl$(comma)-L%,$(LDLIBS)))
+LDLIBS_PATH += $(subst -L,,$(filter -L%,$(LDLIBS)))
+
+# list of .a files that are linked to this application
+LDLIBS_NAMES := $(patsubst -l%,lib%.a,$(filter -l%,$(LDLIBS)))
+LDLIBS_NAMES += $(patsubst -Wl$(comma)-l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIBS)))
+
+# list of found libraries files (useful for deps). If not found, the
+# library is silently ignored and dep won't be checked
+LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
+	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
+
+#
+# Compile executable file if needed
+#
+$(APP): $(OBJS-y) $(LDLIBS_FILES) $(DEP_$(APP)) $(LDSCRIPT) FORCE
+	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
+	$(if $(D),\
+		@echo -n "$< -> $@ " ; \
+		echo -n "file_missing=$(call boolean,$(file_missing)) " ; \
+		echo -n "cmdline_changed=$(call boolean,$(call cmdline_changed,$(O_TO_EXE_STR))) " ; \
+		echo -n "depfile_missing=$(call boolean,$(depfile_missing)) " ; \
+		echo "depfile_newer=$(call boolean,$(depfile_newer)) ")
+	$(if $(or \
+		$(file_missing),\
+		$(call cmdline_changed,$(O_TO_EXE_STR)),\
+		$(depfile_missing),\
+		$(depfile_newer)),\
+		$(O_TO_EXE_DO))
+
+#
+# install app in $(RTE_OUTPUT)/app
+#
+$(RTE_OUTPUT)/buildtools/$(APP): $(APP)
+	@echo "  INSTALL-APP $(APP)"
+	@[ -d $(RTE_OUTPUT)/buildtools ] || mkdir -p $(RTE_OUTPUT)/buildtools
+	$(Q)cp -f $(APP) $(RTE_OUTPUT)/buildtools
+
+#
+# install app map file in $(RTE_OUTPUT)/app
+#
+$(RTE_OUTPUT)/buildtools/$(APP).map: $(APP)
+	@echo "  INSTALL-MAP $(APP).map"
+	@[ -d $(RTE_OUTPUT)/buildtools ] || mkdir -p $(RTE_OUTPUT)/buildtools
+	$(Q)cp -f $(APP).map $(RTE_OUTPUT)/buildtools
+
+#
+# Clean all generated files
+#
+.PHONY: clean
+clean: _postclean
+	$(Q)rm -f $(_BUILD_TARGETS) $(_INSTALL_TARGETS) $(_CLEAN_TARGETS)
+
+.PHONY: doclean
+doclean:
+	$(Q)rm -rf $(APP) $(OBJS-all) $(DEPS-all) $(DEPSTMP-all) \
+	  $(CMDS-all) $(INSTALL-FILES-all) .$(APP).cmd
+
+
+include $(RTE_SDK)/mk/internal/rte.compile-post.mk
+include $(RTE_SDK)/mk/internal/rte.install-post.mk
+include $(RTE_SDK)/mk/internal/rte.clean-post.mk
+include $(RTE_SDK)/mk/internal/rte.build-post.mk
+include $(RTE_SDK)/mk/internal/rte.depdirs-post.mk
+
+.PHONY: FORCE
+FORCE:
diff --git a/mk/rte.sdkbuild.mk b/mk/rte.sdkbuild.mk
index eec5241..fb68af2 100644
--- a/mk/rte.sdkbuild.mk
+++ b/mk/rte.sdkbuild.mk
@@ -64,7 +64,8 @@ build: $(ROOTDIRS-y)
 clean: $(CLEANDIRS)
 	@rm -rf $(RTE_OUTPUT)/include $(RTE_OUTPUT)/app \
 		$(RTE_OUTPUT)/hostapp $(RTE_OUTPUT)/lib \
-		$(RTE_OUTPUT)/hostlib $(RTE_OUTPUT)/kmod
+		$(RTE_OUTPUT)/hostlib $(RTE_OUTPUT)/kmod \
+		$(RTE_OUTPUT)/buildtools
 	@[ -d $(RTE_OUTPUT)/include ] || mkdir -p $(RTE_OUTPUT)/include
 	@$(RTE_SDK)/scripts/gen-config-h.sh $(RTE_OUTPUT)/.config \
 		> $(RTE_OUTPUT)/include/rte_config.h
-- 
2.5.5

[PATCH 2/4] drivers: Update driver registration macro usage

[Neil Horman]

Modify the PMD_REGISTER_DRIVER macro, bifurcating it into two
(PMD_REGISTER_DRIVER_PDEV and PMD_REGISTER_DRIVER_VDEV.  Both of these do the
same thing the origional macro did, but both add the definition of a string
variable that informs interested parties of the name of the pmd, and the former
also defines an second string that holds the symbol name of the pci table that
is registered by this pmd.

pmdinfo uses this information to extract hardware support from an object file
and create a json string to make hardware support info discoverable later.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 drivers/Makefile                           |  2 ++
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c   |  2 +-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c |  2 +-
 drivers/crypto/null/null_crypto_pmd.c      |  2 +-
 drivers/crypto/qat/rte_qat_cryptodev.c     |  2 +-
 drivers/crypto/snow3g/rte_snow3g_pmd.c     |  2 +-
 drivers/net/af_packet/rte_eth_af_packet.c  |  2 +-
 drivers/net/bnx2x/bnx2x_ethdev.c           |  4 ++--
 drivers/net/bonding/rte_eth_bond_pmd.c     |  2 +-
 drivers/net/cxgbe/cxgbe_ethdev.c           |  2 +-
 drivers/net/e1000/em_ethdev.c              |  2 +-
 drivers/net/e1000/igb_ethdev.c             |  4 ++--
 drivers/net/ena/ena_ethdev.c               |  2 +-
 drivers/net/enic/enic_ethdev.c             |  2 +-
 drivers/net/fm10k/fm10k_ethdev.c           |  2 +-
 drivers/net/i40e/i40e_ethdev.c             |  2 +-
 drivers/net/i40e/i40e_ethdev_vf.c          |  2 +-
 drivers/net/ixgbe/ixgbe_ethdev.c           |  4 ++--
 drivers/net/mlx4/mlx4.c                    |  2 +-
 drivers/net/mlx5/mlx5.c                    |  2 +-
 drivers/net/mpipe/mpipe_tilegx.c           |  4 ++--
 drivers/net/nfp/nfp_net.c                  |  2 +-
 drivers/net/null/rte_eth_null.c            |  2 +-
 drivers/net/pcap/rte_eth_pcap.c            |  2 +-
 drivers/net/ring/rte_eth_ring.c            |  2 +-
 drivers/net/szedata2/rte_eth_szedata2.c    |  2 +-
 drivers/net/vhost/rte_eth_vhost.c          |  2 +-
 drivers/net/virtio/virtio_ethdev.c         |  2 +-
 drivers/net/vmxnet3/vmxnet3_ethdev.c       |  2 +-
 drivers/net/xenvirt/rte_eth_xenvirt.c      |  2 +-
 lib/librte_eal/common/include/rte_dev.h    | 21 ++++++++++++++++++---
 31 files changed, 53 insertions(+), 36 deletions(-)

diff --git a/drivers/Makefile b/drivers/Makefile
index 81c03a8..75a3168 100644
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -34,4 +34,6 @@ include $(RTE_SDK)/mk/rte.vars.mk
 DIRS-y += net
 DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += crypto
 
+DEPDIRS-y += buildtools/pmdinfo
+
 include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
index 2987ef6..1c1cbf7 100644
--- a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
+++ b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
@@ -521,4 +521,4 @@ static struct rte_driver aesni_gcm_pmd_drv = {
 	.uninit = aesni_gcm_uninit
 };
 
-PMD_REGISTER_DRIVER(aesni_gcm_pmd_drv);
+PMD_REGISTER_DRIVER_VDEV(aesni_gcm_pmd_drv, aesni_gcm);
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
index 3415ac1..bbd44ff 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
@@ -716,4 +716,4 @@ static struct rte_driver cryptodev_aesni_mb_pmd_drv = {
 	.uninit = cryptodev_aesni_mb_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drv);
+PMD_REGISTER_DRIVER_VDEV(cryptodev_aesni_mb_pmd_drvi, aesni_mb);
diff --git a/drivers/crypto/null/null_crypto_pmd.c b/drivers/crypto/null/null_crypto_pmd.c
index bdaf13c..03b79d9 100644
--- a/drivers/crypto/null/null_crypto_pmd.c
+++ b/drivers/crypto/null/null_crypto_pmd.c
@@ -275,4 +275,4 @@ static struct rte_driver cryptodev_null_pmd_drv = {
 	.uninit = cryptodev_null_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_null_pmd_drv);
+PMD_REGISTER_DRIVER_VDEV(cryptodev_null_pmd_drv, cryptodev_null_pmd);
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index a7912f5..14794c2 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -137,4 +137,4 @@ static struct rte_driver pmd_qat_drv = {
 	.init = rte_qat_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(pmd_qat_drv);
+PMD_REGISTER_DRIVER_PDEV(pmd_qat_drv, pci_id_qat_map, qat);
diff --git a/drivers/crypto/snow3g/rte_snow3g_pmd.c b/drivers/crypto/snow3g/rte_snow3g_pmd.c
index f3e0e66..0203670 100644
--- a/drivers/crypto/snow3g/rte_snow3g_pmd.c
+++ b/drivers/crypto/snow3g/rte_snow3g_pmd.c
@@ -548,4 +548,4 @@ static struct rte_driver cryptodev_snow3g_pmd_drv = {
 	.uninit = cryptodev_snow3g_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_snow3g_pmd_drv);
+PMD_REGISTER_DRIVER_VDEV(cryptodev_snow3g_pmd_drv, snow3g);
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index f17bd7e..1f4a87c 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -878,4 +878,4 @@ static struct rte_driver pmd_af_packet_drv = {
 	.uninit = rte_pmd_af_packet_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_af_packet_drv);
+PMD_REGISTER_DRIVER_VDEV(pmd_af_packet_drv, eth_af_packet);
diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c b/drivers/net/bnx2x/bnx2x_ethdev.c
index 071b44f..e0254fa 100644
--- a/drivers/net/bnx2x/bnx2x_ethdev.c
+++ b/drivers/net/bnx2x/bnx2x_ethdev.c
@@ -550,5 +550,5 @@ static struct rte_driver rte_bnx2xvf_driver = {
 	.init = rte_bnx2xvf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_bnx2x_driver);
-PMD_REGISTER_DRIVER(rte_bnx2xvf_driver);
+PMD_REGISTER_DRIVER_PDEV(rte_bnx2x_driver, pci_id_bnx2x_map, bnx2x);
+PMD_REGISTER_DRIVER_PDEV(rte_bnx2xvf_driver, pci_id_bnx2xvf_map, bnx2xvf);
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 129f04b..3ab1c00 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -2511,4 +2511,4 @@ static struct rte_driver bond_drv = {
 	.uninit = bond_uninit,
 };
 
-PMD_REGISTER_DRIVER(bond_drv);
+PMD_REGISTER_DRIVER_VDEV(bond_drv, bonding);
diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index 04eddaf..0d265c2 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -894,4 +894,4 @@ static struct rte_driver rte_cxgbe_driver = {
 	.init = rte_cxgbe_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_cxgbe_driver);
+PMD_REGISTER_DRIVER_PDEV(rte_cxgbe_driver, cxgb4_pci_tbl, cxgb4);
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index 653be09..2d35f36 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -1777,4 +1777,4 @@ struct rte_driver em_pmd_drv = {
 	.init = rte_em_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(em_pmd_drv);
+PMD_REGISTER_DRIVER_PDEV(em_pmd_drv, pci_id_em_map, em);
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index f0921ee..90fb003 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -4973,5 +4973,5 @@ eth_igb_configure_msix_intr(struct rte_eth_dev *dev)
 	E1000_WRITE_FLUSH(hw);
 }
 
-PMD_REGISTER_DRIVER(pmd_igb_drv);
-PMD_REGISTER_DRIVER(pmd_igbvf_drv);
+PMD_REGISTER_DRIVER_PDEV(pmd_igb_drv, pci_id_igb_map, igb);
+PMD_REGISTER_DRIVER_PDEV(pmd_igbvf_drv, pci_id_igbvf_map, igbvf);
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index e157587..ac50623 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -1450,4 +1450,4 @@ struct rte_driver ena_pmd_drv = {
 	.init = rte_ena_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(ena_pmd_drv);
+PMD_REGISTER_DRIVER_PDEV(ena_pmd_drv, pci_id_ena_map, ena);
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index 6bea940..1126efb 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -684,4 +684,4 @@ static struct rte_driver rte_enic_driver = {
 	.init = rte_enic_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_enic_driver);
+PMD_REGISTER_DRIVER_PDEV(rte_enic_driver, pci_id_enic_map, enic);
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index c2d377f..21ef214 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -3055,4 +3055,4 @@ static struct rte_driver rte_fm10k_driver = {
 	.init = rte_pmd_fm10k_init,
 };
 
-PMD_REGISTER_DRIVER(rte_fm10k_driver);
+PMD_REGISTER_DRIVER_PDEV(rte_fm10k_driver, pci_id_fm10k_map, fm10k);
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index d8b6bd7..9ff350e 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -695,7 +695,7 @@ static struct rte_driver rte_i40e_driver = {
 	.init = rte_i40e_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_i40e_driver);
+PMD_REGISTER_DRIVER_PDEV(rte_i40e_driver, pci_id_i40e_map, i40e);
 
 /*
  * Initialize registers for flexible payload, which should be set by NVM.
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 2bce69b..29b1f48 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1560,7 +1560,7 @@ static struct rte_driver rte_i40evf_driver = {
 	.init = rte_i40evf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_i40evf_driver);
+PMD_REGISTER_DRIVER_PDEV(rte_i40evf_driver, pci_id_i40evf_map, i40evf);
 
 static int
 i40evf_dev_configure(struct rte_eth_dev *dev)
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index eec607c..642da5c 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -7141,5 +7141,5 @@ static struct rte_driver rte_ixgbevf_driver = {
 	.init = rte_ixgbevf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_ixgbe_driver);
-PMD_REGISTER_DRIVER(rte_ixgbevf_driver);
+PMD_REGISTER_DRIVER_PDEV(rte_ixgbe_driver, pci_id_ixgbe_map, ixgbe);
+PMD_REGISTER_DRIVER_PDEV(rte_ixgbevf_driver, pci_id_ixgbevf_map, ixgbevf);
diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index 4f21dbe..60f6068 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -5817,4 +5817,4 @@ static struct rte_driver rte_mlx4_driver = {
 	.init = rte_mlx4_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_mlx4_driver)
+PMD_REGISTER_DRIVER_PDEV(rte_mlx4_driveri, mlx4_pci_id_map, mlx4)
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 041cfc3..0efc35d 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -672,4 +672,4 @@ static struct rte_driver rte_mlx5_driver = {
 	.init = rte_mlx5_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_mlx5_driver)
+PMD_REGISTER_DRIVER_PDEV(rte_mlx5_driveri, mlx5_pci_id_map, mlx5)
diff --git a/drivers/net/mpipe/mpipe_tilegx.c b/drivers/net/mpipe/mpipe_tilegx.c
index adcbc19..a539391 100644
--- a/drivers/net/mpipe/mpipe_tilegx.c
+++ b/drivers/net/mpipe/mpipe_tilegx.c
@@ -1635,8 +1635,8 @@ static struct rte_driver pmd_mpipe_gbe_drv = {
 	.init = rte_pmd_mpipe_devinit,
 };
 
-PMD_REGISTER_DRIVER(pmd_mpipe_xgbe_drv);
-PMD_REGISTER_DRIVER(pmd_mpipe_gbe_drv);
+PMD_REGISTER_DRIVER_VDEV(pmd_mpipe_xgbe_drv, mpipe_xgbe);
+PMD_REGISTER_DRIVER_VDEV(pmd_mpipe_gbe_drv, mpipe_gbe);
 
 static void __attribute__((constructor, used))
 mpipe_init_contexts(void)
diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
index bc0a3d8..3cf6d24 100644
--- a/drivers/net/nfp/nfp_net.c
+++ b/drivers/net/nfp/nfp_net.c
@@ -2482,7 +2482,7 @@ static struct rte_driver rte_nfp_net_driver = {
 	.init = nfp_net_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_nfp_net_driver);
+PMD_REGISTER_DRIVER_PDEV(rte_nfp_net_driver, pci_id_nfp_net_map, nfp);
 
 /*
  * Local variables:
diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index 5e8e203..029d2d5 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -689,4 +689,4 @@ static struct rte_driver pmd_null_drv = {
 	.uninit = rte_pmd_null_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_null_drv);
+PMD_REGISTER_DRIVER_VDEV(pmd_null_drv, eth_null);
diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
index c98e234..7e17f19 100644
--- a/drivers/net/pcap/rte_eth_pcap.c
+++ b/drivers/net/pcap/rte_eth_pcap.c
@@ -1090,4 +1090,4 @@ static struct rte_driver pmd_pcap_drv = {
 	.uninit = rte_pmd_pcap_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_pcap_drv);
+PMD_REGISTER_DRIVER_VDEV(pmd_pcap_drv, pcap);
diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index b1783c3..cb2b438 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -630,4 +630,4 @@ static struct rte_driver pmd_ring_drv = {
 	.uninit = rte_pmd_ring_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_ring_drv);
+PMD_REGISTER_DRIVER_VDEV(pmd_ring_drv, eth_ring);
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 78c43b0..31d2940 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1601,4 +1601,4 @@ static struct rte_driver rte_szedata2_driver = {
 	.uninit = rte_szedata2_uninit,
 };
 
-PMD_REGISTER_DRIVER(rte_szedata2_driver);
+PMD_REGISTER_DRIVER_PDEV(rte_szedata2_driver, rte_szedata2_pci_id_table ,rte_szedata2_pmd);
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index 310cbef..148ac82 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -924,4 +924,4 @@ static struct rte_driver pmd_vhost_drv = {
 	.uninit = rte_pmd_vhost_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_vhost_drv);
+PMD_REGISTER_DRIVER_VDEV(pmd_vhost_drv, eth_vhost);
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 63a368a..44f0cbc 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -1458,4 +1458,4 @@ static struct rte_driver rte_virtio_driver = {
 	.init = rte_virtio_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_virtio_driver);
+PMD_REGISTER_DRIVER_PDEV(rte_virtio_driver, pci_id_virtio_map, virtio_net);
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index 29b469c..82a6c39 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -954,4 +954,4 @@ static struct rte_driver rte_vmxnet3_driver = {
 	.init = rte_vmxnet3_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_vmxnet3_driver);
+PMD_REGISTER_DRIVER_PDEV(rte_vmxnet3_driver, pci_id_vmxnet3_map,vmxnet3);
diff --git a/drivers/net/xenvirt/rte_eth_xenvirt.c b/drivers/net/xenvirt/rte_eth_xenvirt.c
index b9638d9..7998cab 100644
--- a/drivers/net/xenvirt/rte_eth_xenvirt.c
+++ b/drivers/net/xenvirt/rte_eth_xenvirt.c
@@ -775,4 +775,4 @@ static struct rte_driver pmd_xenvirt_drv = {
 	.uninit = rte_pmd_xenvirt_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_xenvirt_drv);
+PMD_REGISTER_DRIVER_VDEV(pmd_xenvirt_drv, xenvirt);
diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
index f1b5507..77c1f42 100644
--- a/lib/librte_eal/common/include/rte_dev.h
+++ b/lib/librte_eal/common/include/rte_dev.h
@@ -48,7 +48,7 @@ extern "C" {
 
 #include <stdio.h>
 #include <sys/queue.h>
-
+#include <rte_pci.h>
 #include <rte_log.h>
 
 __attribute__((format(printf, 2, 0)))
@@ -178,12 +178,27 @@ int rte_eal_vdev_init(const char *name, const char *args);
  */
 int rte_eal_vdev_uninit(const char *name);
 
+#define DRIVER_EXPORT_NAME_ARRAY(n, idx) n##idx[] __attribute__((used))
+
+#define DRIVER_EXPORT_PDEV(d, n, idx) static const char DRIVER_EXPORT_NAME_ARRAY(this_pmd_driver, idx) = RTE_STR(d);\
+static const char DRIVER_EXPORT_NAME_ARRAY(this_pmd_name, idx) = RTE_STR(n)
+
+#define DRIVER_EXPORT_VDEV(d, idx) static const char DRIVER_EXPORT_NAME_ARRAY(this_pmd_name, idx) = RTE_STR(d)
+
 #define PMD_REGISTER_DRIVER(d)\
 void devinitfn_ ##d(void);\
 void __attribute__((constructor, used)) devinitfn_ ##d(void)\
 {\
-	rte_eal_driver_register(&d);\
-}
+        rte_eal_driver_register(&d);\
+}\
+
+#define PMD_REGISTER_DRIVER_PDEV(d, t, n)\
+PMD_REGISTER_DRIVER(d) \
+DRIVER_EXPORT_PDEV(t, n, __COUNTER__)
+
+#define PMD_REGISTER_DRIVER_VDEV(d, n)\
+PMD_REGISTER_DRIVER(d) \
+DRIVER_EXPORT_VDEV(n, __COUNTER__)
 
 #ifdef __cplusplus
 }
-- 
2.5.5

[PATCH 3/4] Makefile: Do post processing on objects that register a driver

[Neil Horman]

Modify the compilation makefile to identify C files that export PMD information,
and use that to trigger execution of the pmdinfo binary.  If the execution of
pmdinfo is successful, compile the output C file to an object, and use the
linker to do relocatable linking on the resultant object file into the parent
object that it came from.  This effectively just adds the json string into the
string table of the object that defines the PMD to the outside world.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 mk/internal/rte.compile-pre.mk | 19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/mk/internal/rte.compile-pre.mk b/mk/internal/rte.compile-pre.mk
index b9bff4a..2771887 100644
--- a/mk/internal/rte.compile-pre.mk
+++ b/mk/internal/rte.compile-pre.mk
@@ -80,7 +80,8 @@ C_TO_O_STR = $(subst ','\'',$(C_TO_O)) #'# fix syntax highlight
 C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)","  HOSTCC $(@)")
 else
 C_TO_O = $(CC) -Wp,-MD,$(call obj2dep,$(@)).tmp $(CFLAGS) \
-	$(CFLAGS_$(@)) $(EXTRA_CFLAGS) -o $@ -c $<
+	 $(CFLAGS_$(@)) $(EXTRA_CFLAGS) -o $@ -c $<
+
 C_TO_O_STR = $(subst ','\'',$(C_TO_O)) #'# fix syntax highlight
 C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)","  CC $(@)")
 endif
@@ -88,10 +89,26 @@ C_TO_O_CMD = 'cmd_$@ = $(C_TO_O_STR)'
 C_TO_O_DO = @set -e; \
 	echo $(C_TO_O_DISP); \
 	$(C_TO_O) && \
+	sh -c "grep -q \"PMD_REGISTER_DRIVER_[PV]DEV(.*)\" $<; \
+	if [ \$$? -eq 0 ]; \
+	then \
+		echo MODGEN $@; \
+		OBJF=`readlink -f $@`; \
+		${RTE_OUTPUT}/buildtools/pmdinfo \$$OBJF \$$OBJF.mod.c; \
+		if [ \$$? -eq 0 ]; \
+		then \
+			echo MODBUILD $@; \
+			$(CC) -c -o \$$OBJF.mod.o \$$OBJF.mod.c; \
+			$(CROSS)ld -r -o \$$OBJF.o \$$OBJF.mod.o \$$OBJF; \
+			mv -f \$$OBJF.o \$$OBJF; \
+		fi; \
+	fi; \
+	true" && \
 	echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \
 	sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \
 	rm -f $(call obj2dep,$(@)).tmp
 
+
 # return an empty string if string are equal
 compare = $(strip $(subst $(1),,$(2)) $(subst $(2),,$(1)))
 
-- 
2.5.5

[PATCH 4/4] pmd_hw_support.py: Add tool to query binaries for hw support information

[Neil Horman]

This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
and, if found parses the remainder of the string as a json encoded string,
outputting the results in either a human readable or raw, script parseable
format

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 tools/pmd_hw_support.py | 174 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 174 insertions(+)
 create mode 100755 tools/pmd_hw_support.py

diff --git a/tools/pmd_hw_support.py b/tools/pmd_hw_support.py
new file mode 100755
index 0000000..0669aca
--- /dev/null
+++ b/tools/pmd_hw_support.py
@@ -0,0 +1,174 @@
+#!/usr/bin/python3
+#-------------------------------------------------------------------------------
+# scripts/pmd_hw_support.py
+#
+# Utility to dump PMD_INFO_STRING support from an object file
+#
+#-------------------------------------------------------------------------------
+import os, sys
+from optparse import OptionParser
+import string
+import json
+
+# For running from development directory. It should take precedence over the
+# installed pyelftools.
+sys.path.insert(0, '.')
+
+
+from elftools import __version__
+from elftools.common.exceptions import ELFError
+from elftools.common.py3compat import (
+        ifilter, byte2int, bytes2str, itervalues, str2bytes)
+from elftools.elf.elffile import ELFFile
+from elftools.elf.dynamic import DynamicSection, DynamicSegment
+from elftools.elf.enums import ENUM_D_TAG
+from elftools.elf.segments import InterpSegment
+from elftools.elf.sections import SymbolTableSection
+from elftools.elf.gnuversions import (
+    GNUVerSymSection, GNUVerDefSection,
+    GNUVerNeedSection,
+    )
+from elftools.elf.relocation import RelocationSection
+from elftools.elf.descriptions import (
+    describe_ei_class, describe_ei_data, describe_ei_version,
+    describe_ei_osabi, describe_e_type, describe_e_machine,
+    describe_e_version_numeric, describe_p_type, describe_p_flags,
+    describe_sh_type, describe_sh_flags,
+    describe_symbol_type, describe_symbol_bind, describe_symbol_visibility,
+    describe_symbol_shndx, describe_reloc_type, describe_dyn_tag,
+    describe_ver_flags,
+    )
+from elftools.elf.constants import E_FLAGS
+from elftools.dwarf.dwarfinfo import DWARFInfo
+from elftools.dwarf.descriptions import (
+    describe_reg_name, describe_attr_value, set_global_machine_arch,
+    describe_CFI_instructions, describe_CFI_register_rule,
+    describe_CFI_CFA_rule,
+    )
+from elftools.dwarf.constants import (
+    DW_LNS_copy, DW_LNS_set_file, DW_LNE_define_file)
+from elftools.dwarf.callframe import CIE, FDE
+
+raw_output = False;
+
+class ReadElf(object):
+    """ display_* methods are used to emit output into the output stream
+    """
+    def __init__(self, file, output):
+        """ file:
+                stream object with the ELF file to read
+
+            output:
+                output stream to write to
+        """
+        self.elffile = ELFFile(file)
+        self.output = output
+
+        # Lazily initialized if a debug dump is requested
+        self._dwarfinfo = None
+
+        self._versioninfo = None
+
+    def _section_from_spec(self, spec):
+        """ Retrieve a section given a "spec" (either number or name).
+            Return None if no such section exists in the file.
+        """
+        try:
+            num = int(spec)
+            if num < self.elffile.num_sections():
+                return self.elffile.get_section(num)
+            else:
+                return None
+        except ValueError:
+            # Not a number. Must be a name then
+            return self.elffile.get_section_by_name(str2bytes(spec))
+
+    def parse_pmd_info_string(self, mystring):
+        global raw_output
+        i = mystring.index("=");
+        mystring = mystring[i+2:]
+        pmdinfo = json.loads(mystring)
+
+        if raw_output:
+            print(pmdinfo)
+            return
+
+        print("PMD NAME: " + pmdinfo["name"])
+        print("PMD TYPE: " + pmdinfo["type"])
+        if (pmdinfo["type"] == "PMD_PDEV"):
+            print("PMD HW SUPPORT:")
+            print("VENDOR\t DEVICE\t SUBVENDOR\t SUBDEVICE")
+            for i in pmdinfo["pci_ids"]:
+                print("0x%04x\t 0x%04x\t 0x%04x\t\t 0x%04x" % (i[0], i[1], i[2], i[3]))
+
+        print("")
+
+
+    def display_pmd_info_strings(self, section_spec):
+        """ Display a strings dump of a section. section_spec is either a
+            section number or a name.
+        """
+        section = self._section_from_spec(section_spec)
+        if section is None:
+            return
+
+
+        found = False
+        data = section.data()
+        dataptr = 0
+
+        while dataptr < len(data):
+            while ( dataptr < len(data) and
+                    not (32 <= byte2int(data[dataptr]) <= 127)):
+                dataptr += 1
+
+            if dataptr >= len(data):
+                break
+
+            endptr = dataptr
+            while endptr < len(data) and byte2int(data[endptr]) != 0:
+                endptr += 1
+
+            found = True
+            mystring = bytes2str(data[dataptr:endptr])
+            rc = mystring.find("PMD_INFO_STRING")
+            if (rc != -1):
+                self.parse_pmd_info_string(mystring)
+
+            dataptr = endptr
+
+
+def main(stream=None):
+    global raw_output
+
+    optparser = OptionParser(
+            usage='usage: %prog [-h|-r] <elf-file>',
+            description="Dump pmd hardware support info",
+            add_help_option=True,
+            prog='pmd_hw_support.py')
+    optparser.add_option('-r', '--raw',
+            action='store_true', dest='raw_output',
+            help='Dump raw json strings')
+
+    options, args = optparser.parse_args()
+
+    if options.raw_output:
+        raw_output = True
+
+    with open(args[0], 'rb') as file:
+        try:
+            readelf = ReadElf(file, stream or sys.stdout)
+   
+            readelf.display_pmd_info_strings(".rodata") 
+            sys.exit(0)
+ 
+        except ELFError as ex:
+            sys.stderr.write('ELF error: %s\n' % ex)
+            sys.exit(1)
+
+
+#-------------------------------------------------------------------------------
+if __name__ == '__main__':
+    main()
+
+
-- 
2.5.5

Re: [PATCH 4/4] pmd_hw_support.py: Add tool to query binaries for hw support information

[Panu Matilainen]

On 05/16/2016 11:41 PM, Neil Horman wrote:
> This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
> and, if found parses the remainder of the string as a json encoded string,
> outputting the results in either a human readable or raw, script parseable
> format
>
> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> CC: Bruce Richardson <bruce.richardson@intel.com>
> CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> CC: Stephen Hemminger <stephen@networkplumber.org>
> CC: Panu Matilainen <pmatilai@redhat.com>
> ---
>  tools/pmd_hw_support.py | 174 ++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 174 insertions(+)
>  create mode 100755 tools/pmd_hw_support.py
>
> diff --git a/tools/pmd_hw_support.py b/tools/pmd_hw_support.py
> new file mode 100755
> index 0000000..0669aca
> --- /dev/null
> +++ b/tools/pmd_hw_support.py
> @@ -0,0 +1,174 @@
> +#!/usr/bin/python3

I think this should use /usr/bin/python to be consistent with the other 
python scripts, and like the others work with python 2 and 3. I only 
tested it with python2 after changing this and it seemed to work fine so 
the compatibility side should be fine as-is.

On the whole, AFAICT the patch series does what it promises, and works 
for both static and shared linkage. Using JSON formatted strings in an 
ELF section is a sound working technical solution for the storage of the 
data. But the difference between the two cases makes me wonder about 
this all...

For static library build, you'd query the application executable, eg 
testpmd, to get the data out. For a shared library build, that method 
gives absolutely nothing because the data is scattered around in 
individual libraries which might be just about wherever, and you need to 
somehow discover the location + correct library files to be able to 
query that. For the shared case, perhaps the script could be taught to 
walk files in CONFIG_RTE_EAL_PMD_PATH to give in-the-ballpark 
correct/identical results when querying the executable as with static 
builds. If identical operation between static and shared versions is a 
requirement (without running the app in question) then query through the 
executable itself is practically the only option. Unless some kind of 
(auto-generated) external config file system ala kernel depmod / 
modules.dep etc is brought into the picture.

For shared library configurations, having the data in the individual 
pmds is valuable as one could for example have rpm autogenerate provides 
from the data to ease/automate installation (in case of split packaging 
and/or 3rd party drivers). And no doubt other interesting possibilities. 
With static builds that kind of thing is not possible.

Calling up on the list of requirements from 
http://dpdk.org/ml/archives/dev/2016-May/038324.html, I see a pile of 
technical requirements but perhaps we should stop for a moment to think 
about the use-cases first?

To name some from the top of my head:
- user wants to know whether the hardware on the system is supported
- user wants to know which package(s) need to be installed to support 
the system hardware
- user wants to list all supported hardware before going shopping
- [what else?]

...and then think how these things would look like from the user 
perspective, in the light of the two quite dramatically differing cases 
of static vs shared linkage.

P.S. Sorry for being late to this party, I'm having some health issues 
so my level of participation is a bit on-and-off at the moment.

	- Panu -

Re: [PATCH 4/4] pmd_hw_support.py: Add tool to query binaries for hw support information

[Neil Horman]

On Wed, May 18, 2016 at 02:48:30PM +0300, Panu Matilainen wrote:
> On 05/16/2016 11:41 PM, Neil Horman wrote:
> > This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
> > and, if found parses the remainder of the string as a json encoded string,
> > outputting the results in either a human readable or raw, script parseable
> > format
> > 
> > Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> > CC: Bruce Richardson <bruce.richardson@intel.com>
> > CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> > CC: Stephen Hemminger <stephen@networkplumber.org>
> > CC: Panu Matilainen <pmatilai@redhat.com>
> > ---
> >  tools/pmd_hw_support.py | 174 ++++++++++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 174 insertions(+)
> >  create mode 100755 tools/pmd_hw_support.py
> > 
> > diff --git a/tools/pmd_hw_support.py b/tools/pmd_hw_support.py
> > new file mode 100755
> > index 0000000..0669aca
> > --- /dev/null
> > +++ b/tools/pmd_hw_support.py
> > @@ -0,0 +1,174 @@
> > +#!/usr/bin/python3
> 
> I think this should use /usr/bin/python to be consistent with the other
> python scripts, and like the others work with python 2 and 3. I only tested
> it with python2 after changing this and it seemed to work fine so the
> compatibility side should be fine as-is.
> 
Sure, I can change the python executable, that makes sense.

> On the whole, AFAICT the patch series does what it promises, and works for
> both static and shared linkage. Using JSON formatted strings in an ELF
> section is a sound working technical solution for the storage of the data.
> But the difference between the two cases makes me wonder about this all...
You mean the difference between checking static binaries and dynamic binaries?
yes, there is some functional difference there

> 
> For static library build, you'd query the application executable, eg
Correct.

> testpmd, to get the data out. For a shared library build, that method gives
> absolutely nothing because the data is scattered around in individual
> libraries which might be just about wherever, and you need to somehow
Correct, I figured that users would be smart enough to realize that with
dynamically linked executables, they would need to look at DSO's, but I agree,
its a glaring diffrence.

> discover the location + correct library files to be able to query that. For
> the shared case, perhaps the script could be taught to walk files in
> CONFIG_RTE_EAL_PMD_PATH to give in-the-ballpark correct/identical results
My initial thought would be to run ldd on the executable, and use a heuristic to
determine relevant pmd DSO's, and then feed each of those through the python
script.  I didn't want to go to that trouble unless there was consensus on it
though.


> when querying the executable as with static builds. If identical operation
> between static and shared versions is a requirement (without running the app
> in question) then query through the executable itself is practically the
> only option. Unless some kind of (auto-generated) external config file
> system ala kernel depmod / modules.dep etc is brought into the picture.
Yeah, I'm really trying to avoid that, as I think its really not a typical part
of how user space libraries are interacted with.

> 
> For shared library configurations, having the data in the individual pmds is
> valuable as one could for example have rpm autogenerate provides from the
> data to ease/automate installation (in case of split packaging and/or 3rd
> party drivers). And no doubt other interesting possibilities. With static
> builds that kind of thing is not possible.
Right.

Note, this also leaves out PMD's that are loaded dynamically (i.e. via dlopen).
For those situations I don't think we have any way of 'knowing' that the
application intends to use them.

> 
> Calling up on the list of requirements from
> http://dpdk.org/ml/archives/dev/2016-May/038324.html, I see a pile of
> technical requirements but perhaps we should stop for a moment to think
> about the use-cases first?

To ennumerate the list:

- query all drivers in static binary or shared library (works)
- stripping resiliency (works)
- human friendly (works)
- script friendly (works)
- show driver name (works)
- list supported device id / name (works)
- list driver options (not yet, but possible)
- show driver version if available (nope, but possible)
- show dpdk version (nope, but possible)
- show kernel dependencies (vfio/uio_pci_generic/etc) (nope)
- room for extra information? (works)

Of the items that are missing, I've already got a V2 started that can do driver
options, and is easier to expand.  Adding in the the DPDK and PMD version should
be easy (though I think they can be left out, as theres currently no globaly
defined DPDK release version, its all just implicit, and driver versions aren't
really there either).  I'm also hesitant to include kernel dependencies without
defining exactly what they mean (just module dependencies, or feature
enablement, or something else?).  Once we define it though, adding it can be
easy.

I'll have a v2 posted soon, with the consensus corrections you have above, as
well as some other cleanups

Best
Neil

> 
> To name some from the top of my head:
> - user wants to know whether the hardware on the system is supported
> - user wants to know which package(s) need to be installed to support the
> system hardware
> - user wants to list all supported hardware before going shopping
> - [what else?]
> 
> ...and then think how these things would look like from the user
> perspective, in the light of the two quite dramatically differing cases of
> static vs shared linkage.
> 
> P.S. Sorry for being late to this party, I'm having some health issues so my
> level of participation is a bit on-and-off at the moment.
> 
> 	- Panu -
> 

Re: [PATCH 4/4] pmd_hw_support.py: Add tool to query binaries for hw support information

[Thomas Monjalon]

2016-05-18 14:48, Panu Matilainen:
> Calling up on the list of requirements from 
> http://dpdk.org/ml/archives/dev/2016-May/038324.html, I see a pile of 
> technical requirements but perhaps we should stop for a moment to think 
> about the use-cases first?
> 
> To name some from the top of my head:
> - user wants to know whether the hardware on the system is supported

supported by what?
* by a statically linked app
* by a DPDK he has downloaded and built
* by a DPDK provided as shared library by its Linux vendor
In the first 2 cases he knows where the files are.
In the Linux distribution case, there can be a default directory set
by the Linux vendor for the script looking at the infos. Only the Linux
vendor knows where the PMDs files are.

> - user wants to know which package(s) need to be installed to support 
> the system hardware

You mean "which DPDK packages"?
Are some informations showed when doing "packager search dpdk"?
or "packager show dpdk-driverX"?
Do you want to show the PCI ids in the description of the packages?

> - user wants to list all supported hardware before going shopping

Why doing shopping? For a DPDK usage or for a specific application?
The application should mentions the supported hardware.
For more general DPDK information, there is this this page:
	http://dpdk.org/doc/nics
But it may be not enough accurate for some PCI id exceptions.
For more details, he must use a listing tool.

Re: [PATCH 4/4] pmd_hw_support.py: Add tool to query binaries for hw support information

[Panu Matilainen]

On 05/18/2016 03:03 PM, Neil Horman wrote:
> On Wed, May 18, 2016 at 02:48:30PM +0300, Panu Matilainen wrote:
>> On 05/16/2016 11:41 PM, Neil Horman wrote:
>>> This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
>>> and, if found parses the remainder of the string as a json encoded string,
>>> outputting the results in either a human readable or raw, script parseable
>>> format
>>>
>>> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
>>> CC: Bruce Richardson <bruce.richardson@intel.com>
>>> CC: Thomas Monjalon <thomas.monjalon@6wind.com>
>>> CC: Stephen Hemminger <stephen@networkplumber.org>
>>> CC: Panu Matilainen <pmatilai@redhat.com>
>>> ---
>>>  tools/pmd_hw_support.py | 174 ++++++++++++++++++++++++++++++++++++++++++++++++
>>>  1 file changed, 174 insertions(+)
>>>  create mode 100755 tools/pmd_hw_support.py
>>>
>>> diff --git a/tools/pmd_hw_support.py b/tools/pmd_hw_support.py
>>> new file mode 100755
>>> index 0000000..0669aca
>>> --- /dev/null
>>> +++ b/tools/pmd_hw_support.py
>>> @@ -0,0 +1,174 @@
>>> +#!/usr/bin/python3
>>
>> I think this should use /usr/bin/python to be consistent with the other
>> python scripts, and like the others work with python 2 and 3. I only tested
>> it with python2 after changing this and it seemed to work fine so the
>> compatibility side should be fine as-is.
>>
> Sure, I can change the python executable, that makes sense.
>
>> On the whole, AFAICT the patch series does what it promises, and works for
>> both static and shared linkage. Using JSON formatted strings in an ELF
>> section is a sound working technical solution for the storage of the data.
>> But the difference between the two cases makes me wonder about this all...
> You mean the difference between checking static binaries and dynamic binaries?
> yes, there is some functional difference there
>
>>
>> For static library build, you'd query the application executable, eg
> Correct.
>
>> testpmd, to get the data out. For a shared library build, that method gives
>> absolutely nothing because the data is scattered around in individual
>> libraries which might be just about wherever, and you need to somehow
> Correct, I figured that users would be smart enough to realize that with
> dynamically linked executables, they would need to look at DSO's, but I agree,
> its a glaring diffrence.

Being able to look at DSOs is good, but expecting the user to figure out 
which DSOs might be loaded and not and where to look is going to be well 
above many users. At very least it's not what I would call user-friendly.

>> discover the location + correct library files to be able to query that. For
>> the shared case, perhaps the script could be taught to walk files in
>> CONFIG_RTE_EAL_PMD_PATH to give in-the-ballpark correct/identical results
> My initial thought would be to run ldd on the executable, and use a heuristic to
> determine relevant pmd DSO's, and then feed each of those through the python
> script.  I didn't want to go to that trouble unless there was consensus on it
> though.

Problem is, ldd doesn't know about them either because the pmds are not 
linked to the executables at all anymore. They could be force-linked of 
course, but that means giving up the flexibility of plugins, which IMO 
is a no-go. Except maybe as an option, but then that would be a third 
case to support.


>
>> when querying the executable as with static builds. If identical operation
>> between static and shared versions is a requirement (without running the app
>> in question) then query through the executable itself is practically the
>> only option. Unless some kind of (auto-generated) external config file
>> system ala kernel depmod / modules.dep etc is brought into the picture.
> Yeah, I'm really trying to avoid that, as I think its really not a typical part
> of how user space libraries are interacted with.
>
>>
>> For shared library configurations, having the data in the individual pmds is
>> valuable as one could for example have rpm autogenerate provides from the
>> data to ease/automate installation (in case of split packaging and/or 3rd
>> party drivers). And no doubt other interesting possibilities. With static
>> builds that kind of thing is not possible.
> Right.
>
> Note, this also leaves out PMD's that are loaded dynamically (i.e. via dlopen).
> For those situations I don't think we have any way of 'knowing' that the
> application intends to use them.

Hence my comment about CONFIG_RTE_EAL_PMD_PATH above, it at least 
provides a reasonable heuristic of what would be loaded by the app when 
run. But ultimately the only way to know what hardware is supported at a 
given time is to run an app which calls rte_eal_init() to load all the 
drivers that are present and work from there, because besides 
CONFIG_RTE_EAL_PMD_PATH this can be affected by runtime commandline 
switches and applies to both shared and static builds.

>>
>> Calling up on the list of requirements from
>> http://dpdk.org/ml/archives/dev/2016-May/038324.html, I see a pile of
>> technical requirements but perhaps we should stop for a moment to think
>> about the use-cases first?
>
> To ennumerate the list:
>
> - query all drivers in static binary or shared library (works)
> - stripping resiliency (works)
> - human friendly (works)
> - script friendly (works)
> - show driver name (works)
> - list supported device id / name (works)
> - list driver options (not yet, but possible)
> - show driver version if available (nope, but possible)
> - show dpdk version (nope, but possible)
> - show kernel dependencies (vfio/uio_pci_generic/etc) (nope)
> - room for extra information? (works)
>
> Of the items that are missing, I've already got a V2 started that can do driver
> options, and is easier to expand.  Adding in the the DPDK and PMD version should
> be easy (though I think they can be left out, as theres currently no globaly
> defined DPDK release version, its all just implicit, and driver versions aren't
> really there either).  I'm also hesitant to include kernel dependencies without
> defining exactly what they mean (just module dependencies, or feature
> enablement, or something else?).  Once we define it though, adding it can be
> easy.

Yup. I just think the shared/static difference needs to be sorted out 
somehow, eg requiring user to know about DSOs is not human-friendly at 
all. That's why I called for the higher level use-cases in my previous 
email.

>
> I'll have a v2 posted soon, with the consensus corrections you have above, as
> well as some other cleanups
>
> Best
> Neil
>
>>
>> To name some from the top of my head:
>> - user wants to know whether the hardware on the system is supported
>> - user wants to know which package(s) need to be installed to support the
>> system hardware
>> - user wants to list all supported hardware before going shopping
>> - [what else?]
>>
>> ...and then think how these things would look like from the user
>> perspective, in the light of the two quite dramatically differing cases of
>> static vs shared linkage.


	- Panu -

Re: [PATCH 4/4] pmd_hw_support.py: Add tool to query binaries for hw support information

[Panu Matilainen]

On 05/18/2016 03:38 PM, Thomas Monjalon wrote:
> 2016-05-18 14:48, Panu Matilainen:
>> Calling up on the list of requirements from
>> http://dpdk.org/ml/archives/dev/2016-May/038324.html, I see a pile of
>> technical requirements but perhaps we should stop for a moment to think
>> about the use-cases first?
>>
>> To name some from the top of my head:
>> - user wants to know whether the hardware on the system is supported
>
> supported by what?
> * by a statically linked app
> * by a DPDK he has downloaded and built
> * by a DPDK provided as shared library by its Linux vendor

All three?

> In the first 2 cases he knows where the files are.
> In the Linux distribution case, there can be a default directory set
> by the Linux vendor for the script looking at the infos. Only the Linux
> vendor knows where the PMDs files are.

For case 3), EAL and the DPDK build system know where the PMDs are via 
CONFIG_RTE_EAL_PMD_PATH (if set of course, otherwise there's not much hope)

>
>> - user wants to know which package(s) need to be installed to support
>> the system hardware
>
> You mean "which DPDK packages"?

Yes. This is of course only relevant if PMDs are split across several 
different packages (splitting might not make much sense yet, but as the 
number grows that might well change)

> Are some informations showed when doing "packager search dpdk"?
> or "packager show dpdk-driverX"?
> Do you want to show the PCI ids in the description of the packages?

Something along those lines - such things are being done by distros for 
eg firmware, printer drivers, kernel drivers by modalias etc.

>> - user wants to list all supported hardware before going shopping
>
> Why doing shopping? For a DPDK usage or for a specific application?

To buy hardware which is supported by DPDK, in a general case.

> The application should mentions the supported hardware.
> For more general DPDK information, there is this this page:
> 	http://dpdk.org/doc/nics
> But it may be not enough accurate for some PCI id exceptions.
> For more details, he must use a listing tool.

Yes. The point is, what kind of tool/solution can be made to behave 
identically between shared and static configs, in a user-friendly way. I 
just listed a few obvious (to me at least) use-cases, and was asking for 
others that I didn't think of.

	- Panu -

Re: [PATCH 4/4] pmd_hw_support.py: Add tool to query binaries for hw support information

[Thomas Monjalon]

2016-05-18 16:09, Panu Matilainen:
> On 05/18/2016 03:38 PM, Thomas Monjalon wrote:
> > 2016-05-18 14:48, Panu Matilainen:
> >> Calling up on the list of requirements from
> >> http://dpdk.org/ml/archives/dev/2016-May/038324.html, I see a pile of
> >> technical requirements but perhaps we should stop for a moment to think
> >> about the use-cases first?
> >>
> >> To name some from the top of my head:
> >> - user wants to know whether the hardware on the system is supported
> >
> > supported by what?
> > * by a statically linked app
> > * by a DPDK he has downloaded and built
> > * by a DPDK provided as shared library by its Linux vendor
> 
> All three?

Not at the same time ;)

> > In the first 2 cases he knows where the files are.
> > In the Linux distribution case, there can be a default directory set
> > by the Linux vendor for the script looking at the infos. Only the Linux
> > vendor knows where the PMDs files are.
> 
> For case 3), EAL and the DPDK build system know where the PMDs are via 
> CONFIG_RTE_EAL_PMD_PATH (if set of course, otherwise there's not much hope)

In case 3 (DPDK packaged in distribution), I would rely on the packager (you)
who knows where the libraries are installed.
You can even have a script calling system tools (lspci or other from your
distribution) to get hardware infos and then check if it matches the PCI ids
listed by the DPDK tool.

> >> - user wants to know which package(s) need to be installed to support
> >> the system hardware
> >
> > You mean "which DPDK packages"?
> 
> Yes. This is of course only relevant if PMDs are split across several 
> different packages (splitting might not make much sense yet, but as the 
> number grows that might well change)
> 
> > Are some informations showed when doing "packager search dpdk"?
> > or "packager show dpdk-driverX"?
> > Do you want to show the PCI ids in the description of the packages?
> 
> Something along those lines - such things are being done by distros for 
> eg firmware, printer drivers, kernel drivers by modalias etc.

So the packager would call the DPDK tool listing PCI ids of compiled libs.

> >> - user wants to list all supported hardware before going shopping
> >
> > Why doing shopping? For a DPDK usage or for a specific application?
> 
> To buy hardware which is supported by DPDK, in a general case.
> 
> > The application should mentions the supported hardware.
> > For more general DPDK information, there is this this page:
> > 	http://dpdk.org/doc/nics
> > But it may be not enough accurate for some PCI id exceptions.
> > For more details, he must use a listing tool.
> 
> Yes. The point is, what kind of tool/solution can be made to behave 
> identically between shared and static configs, in a user-friendly way. I 
> just listed a few obvious (to me at least) use-cases, and was asking for 
> others that I didn't think of.

For a user-friendly output, we should not export only PCI ids but also
the commercial names.

About the static/shared case, we can have a script which look at testpmd
plus the shared libs. In a dev space, it is easy to find the files.
In a packaged system, the script can get some configuration variables from
the distribution.

Re: [PATCH 4/4] pmd_hw_support.py: Add tool to query binaries for hw support information

[Neil Horman]

On Wed, May 18, 2016 at 03:48:12PM +0300, Panu Matilainen wrote:
> On 05/18/2016 03:03 PM, Neil Horman wrote:
> > On Wed, May 18, 2016 at 02:48:30PM +0300, Panu Matilainen wrote:
> > > On 05/16/2016 11:41 PM, Neil Horman wrote:
> > > > This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
> > > > and, if found parses the remainder of the string as a json encoded string,
> > > > outputting the results in either a human readable or raw, script parseable
> > > > format
> > > > 
> > > > Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> > > > CC: Bruce Richardson <bruce.richardson@intel.com>
> > > > CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> > > > CC: Stephen Hemminger <stephen@networkplumber.org>
> > > > CC: Panu Matilainen <pmatilai@redhat.com>
> > > > ---
> > > >  tools/pmd_hw_support.py | 174 ++++++++++++++++++++++++++++++++++++++++++++++++
> > > >  1 file changed, 174 insertions(+)
> > > >  create mode 100755 tools/pmd_hw_support.py
> > > > 
> > > > diff --git a/tools/pmd_hw_support.py b/tools/pmd_hw_support.py
> > > > new file mode 100755
> > > > index 0000000..0669aca
> > > > --- /dev/null
> > > > +++ b/tools/pmd_hw_support.py
> > > > @@ -0,0 +1,174 @@
> > > > +#!/usr/bin/python3
> > > 
> > > I think this should use /usr/bin/python to be consistent with the other
> > > python scripts, and like the others work with python 2 and 3. I only tested
> > > it with python2 after changing this and it seemed to work fine so the
> > > compatibility side should be fine as-is.
> > > 
> > Sure, I can change the python executable, that makes sense.
> > 
> > > On the whole, AFAICT the patch series does what it promises, and works for
> > > both static and shared linkage. Using JSON formatted strings in an ELF
> > > section is a sound working technical solution for the storage of the data.
> > > But the difference between the two cases makes me wonder about this all...
> > You mean the difference between checking static binaries and dynamic binaries?
> > yes, there is some functional difference there
> > 
> > > 
> > > For static library build, you'd query the application executable, eg
> > Correct.
> > 
> > > testpmd, to get the data out. For a shared library build, that method gives
> > > absolutely nothing because the data is scattered around in individual
> > > libraries which might be just about wherever, and you need to somehow
> > Correct, I figured that users would be smart enough to realize that with
> > dynamically linked executables, they would need to look at DSO's, but I agree,
> > its a glaring diffrence.
> 
> Being able to look at DSOs is good, but expecting the user to figure out
> which DSOs might be loaded and not and where to look is going to be well
> above many users. At very least it's not what I would call user-friendly.
> 
I disagree, there is no linkage between an application and the dso's it opens
via dlopen that is exportable.  The only way to handle that is to have a
standard search path for the pmd_hw_info python script.  Thats just like modinfo
works (i.e. "modinfo bnx2" finds the bnx2 module for the running kernel).  We
can of course do something simmilar, but we have no existing implicit path
information to draw from to do that (because you can have multiple dpdk installs
side by side).  The only way around that is to explicitly call out the path on
the command line.

> > > discover the location + correct library files to be able to query that. For
> > > the shared case, perhaps the script could be taught to walk files in
> > > CONFIG_RTE_EAL_PMD_PATH to give in-the-ballpark correct/identical results
> > My initial thought would be to run ldd on the executable, and use a heuristic to
> > determine relevant pmd DSO's, and then feed each of those through the python
> > script.  I didn't want to go to that trouble unless there was consensus on it
> > though.
> 
> Problem is, ldd doesn't know about them either because the pmds are not
> linked to the executables at all anymore. They could be force-linked of
> course, but that means giving up the flexibility of plugins, which IMO is a
> no-go. Except maybe as an option, but then that would be a third case to
> support.
> 
Thats not true at all, or at least its a perfectly valid way to link the DSO's
in at link time via -lrte_pmd_<driver>.  Its really just the dlopen case we need
to worry about.  I would argue that, if they're not explicitly linked in like
that, then its correct to indicate that an application supports no hardware,
because it actually doesn't, it only supports the pmds that it chooses to list
on the command line.  And if a user is savy enough to specify a pmd on the
application command line, then they are perfectly capable of specifying the same
path to the hw_info script.

> 
> > 
> > > when querying the executable as with static builds. If identical operation
> > > between static and shared versions is a requirement (without running the app
> > > in question) then query through the executable itself is practically the
> > > only option. Unless some kind of (auto-generated) external config file
> > > system ala kernel depmod / modules.dep etc is brought into the picture.
> > Yeah, I'm really trying to avoid that, as I think its really not a typical part
> > of how user space libraries are interacted with.
> > 
> > > 
> > > For shared library configurations, having the data in the individual pmds is
> > > valuable as one could for example have rpm autogenerate provides from the
> > > data to ease/automate installation (in case of split packaging and/or 3rd
> > > party drivers). And no doubt other interesting possibilities. With static
> > > builds that kind of thing is not possible.
> > Right.
> > 
> > Note, this also leaves out PMD's that are loaded dynamically (i.e. via dlopen).
> > For those situations I don't think we have any way of 'knowing' that the
> > application intends to use them.
> 
> Hence my comment about CONFIG_RTE_EAL_PMD_PATH above, it at least provides a
> reasonable heuristic of what would be loaded by the app when run. But
> ultimately the only way to know what hardware is supported at a given time
> is to run an app which calls rte_eal_init() to load all the drivers that are
> present and work from there, because besides CONFIG_RTE_EAL_PMD_PATH this
> can be affected by runtime commandline switches and applies to both shared
> and static builds.
> 
I'm not sure I agree with that.  Its clearly tempting to use, but its not
at all guaranteed to be accurate (the default is just set to "", and there is no
promise anyone will set it properly). And it also requires that the binary will
be tied to a specific release.  I really think that, given the fact that
distributions generally try to package dpdk in such a way that multiple dpdk
versions might be available, the right solution is to just require a full path
specification if you want to get hw info for a DSO that is dynamically loaded
via dlopen from the command line.  Otherwise you're going to fall into this trap
where you might be looking implicitly at an older version of the PMD while your
application may use a newer version.

> > > 
> > > Calling up on the list of requirements from
> > > http://dpdk.org/ml/archives/dev/2016-May/038324.html, I see a pile of
> > > technical requirements but perhaps we should stop for a moment to think
> > > about the use-cases first?
> > 
> > To ennumerate the list:
> > 
> > - query all drivers in static binary or shared library (works)
> > - stripping resiliency (works)
> > - human friendly (works)
> > - script friendly (works)
> > - show driver name (works)
> > - list supported device id / name (works)
> > - list driver options (not yet, but possible)
> > - show driver version if available (nope, but possible)
> > - show dpdk version (nope, but possible)
> > - show kernel dependencies (vfio/uio_pci_generic/etc) (nope)
> > - room for extra information? (works)
> > 
> > Of the items that are missing, I've already got a V2 started that can do driver
> > options, and is easier to expand.  Adding in the the DPDK and PMD version should
> > be easy (though I think they can be left out, as theres currently no globaly
> > defined DPDK release version, its all just implicit, and driver versions aren't
> > really there either).  I'm also hesitant to include kernel dependencies without
> > defining exactly what they mean (just module dependencies, or feature
> > enablement, or something else?).  Once we define it though, adding it can be
> > easy.
> 
> Yup. I just think the shared/static difference needs to be sorted out
> somehow, eg requiring user to know about DSOs is not human-friendly at all.
> That's why I called for the higher level use-cases in my previous email.
> 

I disagree with that.  While its reasonable to give users the convienience of
scanning the DT_NEEDED entries of a binary and scanning those DSO's.  If a user
has to specify the PMD to load in an application (either on the command line or
via a configuration file), then its reasonable assume that they (a) know where
to find that pmd and (b) are savy enough to pass that same path to a hardware
info tool.  Thats the exact same way that modinfo works (save for the fact that
modinfo can implicitly check the running kernel version to find the appropriate
path for a modular driver).

The only other thing that seems reasonable to me would be to scan
LD_LIBRARY_PATH.  I would assume that, if an application is linked dynamically,
the individual DSO's (librte_sched.so, etc), need to be in LD_LIBRARY_PATH.  If
thats the case, then we can assume that the appropriate PMD DSO's are there too,
and we can search there.  We can also check the standard /usr/lib and /lib paths
with that.  I think that would make fairly good sense.


> > 
> > I'll have a v2 posted soon, with the consensus corrections you have above, as
> > well as some other cleanups
> > 
> > Best
> > Neil
> > 
> > > 
> > > To name some from the top of my head:
> > > - user wants to know whether the hardware on the system is supported
> > > - user wants to know which package(s) need to be installed to support the
> > > system hardware
> > > - user wants to list all supported hardware before going shopping
> > > - [what else?]
> > > 
> > > ...and then think how these things would look like from the user
> > > perspective, in the light of the two quite dramatically differing cases of
> > > static vs shared linkage.
> 
> 
> 	- Panu -
> 

Re: [PATCH 4/4] pmd_hw_support.py: Add tool to query binaries for hw support information

[Neil Horman]

On Wed, May 18, 2016 at 03:26:42PM +0200, Thomas Monjalon wrote:
> 2016-05-18 16:09, Panu Matilainen:
> > On 05/18/2016 03:38 PM, Thomas Monjalon wrote:
> > > 2016-05-18 14:48, Panu Matilainen:
> > >> Calling up on the list of requirements from
> > >> http://dpdk.org/ml/archives/dev/2016-May/038324.html, I see a pile of
> > >> technical requirements but perhaps we should stop for a moment to think
> > >> about the use-cases first?
> > >>
> > >> To name some from the top of my head:
> > >> - user wants to know whether the hardware on the system is supported
> > >
> > > supported by what?
> > > * by a statically linked app
> > > * by a DPDK he has downloaded and built
> > > * by a DPDK provided as shared library by its Linux vendor
> > 
> > All three?
> 
> Not at the same time ;)
> 
> > > In the first 2 cases he knows where the files are.
> > > In the Linux distribution case, there can be a default directory set
> > > by the Linux vendor for the script looking at the infos. Only the Linux
> > > vendor knows where the PMDs files are.
> > 
> > For case 3), EAL and the DPDK build system know where the PMDs are via 
> > CONFIG_RTE_EAL_PMD_PATH (if set of course, otherwise there's not much hope)
> 
> In case 3 (DPDK packaged in distribution), I would rely on the packager (you)
> who knows where the libraries are installed.
> You can even have a script calling system tools (lspci or other from your
> distribution) to get hardware infos and then check if it matches the PCI ids
> listed by the DPDK tool.
> 

I think the only sane solution here is to scan for the file in
/lib:/usr/lib:$LD_LIBRAR_PATH.  Thats the only way that we can come close to
mimicing what the application will do when linking.

Truthfully, the RTE_EAL_PMD_PATH variable should be dropped and set to that
anyway to ensure that the system admin can point to the right libraries when
installing an application.

> > >> - user wants to know which package(s) need to be installed to support
> > >> the system hardware
> > >
> > > You mean "which DPDK packages"?
> > 
> > Yes. This is of course only relevant if PMDs are split across several 
> > different packages (splitting might not make much sense yet, but as the 
> > number grows that might well change)
> > 
> > > Are some informations showed when doing "packager search dpdk"?
> > > or "packager show dpdk-driverX"?
> > > Do you want to show the PCI ids in the description of the packages?
> > 
> > Something along those lines - such things are being done by distros for 
> > eg firmware, printer drivers, kernel drivers by modalias etc.
> 
> So the packager would call the DPDK tool listing PCI ids of compiled libs.
> 
> > >> - user wants to list all supported hardware before going shopping
> > >
> > > Why doing shopping? For a DPDK usage or for a specific application?
> > 
> > To buy hardware which is supported by DPDK, in a general case.
> > 
> > > The application should mentions the supported hardware.
> > > For more general DPDK information, there is this this page:
> > > 	http://dpdk.org/doc/nics
> > > But it may be not enough accurate for some PCI id exceptions.
> > > For more details, he must use a listing tool.
> > 
> > Yes. The point is, what kind of tool/solution can be made to behave 
> > identically between shared and static configs, in a user-friendly way. I 
> > just listed a few obvious (to me at least) use-cases, and was asking for 
> > others that I didn't think of.
> 
> For a user-friendly output, we should not export only PCI ids but also
> the commercial names.
> 
Thats not something that the DSO's should export explicitly. If you want
commercial names, the hw_info tool should incorporate the use of the pci.ids
file from the pci hardware database project (thats how lspci works).  That seems
like a nice bell and whistle to add later though.  Lets get the initial
functionality working first before we start adding features like that

> About the static/shared case, we can have a script which look at testpmd
> plus the shared libs. In a dev space, it is easy to find the files.
> In a packaged system, the script can get some configuration variables from
> the distribution.
> 
> 

[PATCHv2 0/4] Implement pmd hardware support exports

[Neil Horman]

Hey all-
	So heres attempt number 2 at a method for exporting PMD hardware support
information.  As we discussed previously, the consensus seems to be that pmd
information should be:

1) Able to be interrogated on any ELF binary (application binary or individual
DSO)
2) Equally functional on statically linked applications or on DSO's
3) Resilient to symbol stripping
4) Script friendly
5) Show kernel dependencies
6) List driver options
7) Show driver name
8) Offer human readable output
9) Show DPDK version
10) Show driver version
11) Allow for expansion
12) Not place additional build environment dependencies on an application

I added item 12 myself, because I don't think its reasonable to require
applications to use a specific linker script to get hardware support information
(which precludes the use of a special .modinfo section like the kernel uses)

However, we still can use some tricks from the kernel to make this work.  In
this approach, what I've done is the following:

A) Modified the driver registration macro to also define a variable:
this_pmd_name<n>= "name"

Based on the unique name string pointed to by the above variable, we can
query for an arbitrary number of other symbols following the pattern:
<name>_<tag>

Where tag is some well known identifier for information we wish to export

B) Added a utility called pmdinfogen.  This utility is not meant for general use,
but rather is used by the dpdk build environment itself when compiling pmds.
for each object that uses the PMD_REGISTER_DRIVER macro, this utiity is run.  It
searches for the symbols in (A), and using those, extracts the hardware support
info, and module name from the object, using that to produce a new c file
containing a single variable in the following format:

static const char <name>[] __attribute__((used)) = "PMD_DRIVER_INFO=<json>";

The <name> is arbitrary, as its static and not referenced.  The relevant bit is
the string value assigned to it.  The <json> is a json encoded string of the
extracted hardware support information pmdinfo found for the corresponding
object.  This C file is suitable for compilation and relocatable linking back
into the parent object file.  The result of this operation is that the object
string table now contains a string that will not e removed by stripping, whos
leading text (PMD_DRIVER_INFO) can be easily searched for at any time weather
the symbol referring to it is stripped or not.

C) Added a utilty called pmdinfo.py.  This python script, searches the
string table of the .rodata section of any provided ELF file looking for the
PMD_DRIVER_INFO prefix on a string.  When found, it will interpret the remainder
of the string as json, and output the hardware support for that ELF file (if
any).


This approach ticks most of the above boxes:
1) Impervious to stripping
2) Works on static and dynamic binaries
3) Human and script friendly
4) Allows for expansion

Because of (4) the other items should be pretty easy to implement, as its just a
matter of modifying the macros to export the info, pmdinfo to encode it to json,
and pmd_hw_support.py to read it.  I'm not adding them now, as theres alot of
rote work to do to get some of them in place (e.g. DPDK has no current global
VERSION macro, drivers don't have a consistent version scheme, command line
strings have to be built for each driver, etc).  But once this is accepted,
those items can be done as time allows and should be fairly easy to implement.

Change Notes:
v2)
 * Made the export macros a bit easier to expand
 * Added a macro to optionally export command line strings
 * Renamed utilties to be more appropriate
   (pmdinfo -> pmdinfogen, pmd_hw_support.py -> pmdinfo.py)
 * Added search capabilities to pmdinfo.py so that we search for libraries
   linked using DT_NEEDINFO entries.  We search DT_RUNPATH, LD_LIBRARY_PATH,
   /usr/lib and /lib
 * Added serch abilities if the specified binary to pmdinfo isn't found, we
   search LD_LIBRARY_PATH, /usr/lib and /lib
 * Added an option to pmdinfo.py to pretty-print hardware support info using the
   pci.ids database

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>

[PATCHv2 1/4] pmdinfogen: Add buildtools and pmdinfogen utility

[Neil Horman]

pmdinfogen is a tool used to parse object files and build json strings for use in
later determining hardware support in a dso or application binary.  pmdinfo
looks for the non-exported symbol names this_pmd_name<n> and this_pmd_tbl<n>
(where n is a integer counter).  It records the name of each of these tuples,
using the later to find the symbolic name of the pci_table for physical devices
that the object supports.  With this information, it outputs a C file with a
single line of the form:

static char *<pmd_name>_driver_info[] __attribute__((used)) = " \
	PMD_DRIVER_INFO=<json string>";

Where <pmd_name> is the arbitrary name of the pmd, and <json_string> is the json
encoded string that hold relevant pmd information, including the pmd name, type
and optional array of pci device/vendor ids that the driver supports.

This c file is suitable for compiling to object code, then relocatably linking
into the parent file from which the C was generated.  This creates an entry in
the string table of the object that can inform a later tool about hardware
support.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 GNUmakefile                        |   2 +-
 buildtools/Makefile                |  36 ++++
 buildtools/pmdinfogen/Makefile     |  48 +++++
 buildtools/pmdinfogen/pmdinfogen.c | 430 +++++++++++++++++++++++++++++++++++++
 buildtools/pmdinfogen/pmdinfogen.h |  83 +++++++
 mk/rte.buildtools.mk               | 148 +++++++++++++
 mk/rte.sdkbuild.mk                 |   3 +-
 7 files changed, 748 insertions(+), 2 deletions(-)
 create mode 100644 buildtools/Makefile
 create mode 100644 buildtools/pmdinfogen/Makefile
 create mode 100644 buildtools/pmdinfogen/pmdinfogen.c
 create mode 100644 buildtools/pmdinfogen/pmdinfogen.h
 create mode 100644 mk/rte.buildtools.mk

diff --git a/GNUmakefile b/GNUmakefile
index b59e4b6..00fe0db 100644
--- a/GNUmakefile
+++ b/GNUmakefile
@@ -40,6 +40,6 @@ export RTE_SDK
 # directory list
 #
 
-ROOTDIRS-y := lib drivers app
+ROOTDIRS-y := buildtools lib drivers app
 
 include $(RTE_SDK)/mk/rte.sdkroot.mk
diff --git a/buildtools/Makefile b/buildtools/Makefile
new file mode 100644
index 0000000..eb565eb
--- /dev/null
+++ b/buildtools/Makefile
@@ -0,0 +1,36 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+DIRS-y += pmdinfogen
+
+include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/buildtools/pmdinfogen/Makefile b/buildtools/pmdinfogen/Makefile
new file mode 100644
index 0000000..4de9506
--- /dev/null
+++ b/buildtools/pmdinfogen/Makefile
@@ -0,0 +1,48 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+#
+# library name
+#
+APP = pmdinfogen
+
+#
+# all sources are stored in SRCS-y
+#
+SRCS-y += pmdinfogen.c
+
+#CFLAGS += $(WERROR_FLAGS) -g
+CFLAGS += -g
+
+include $(RTE_SDK)/mk/rte.buildtools.mk
+
diff --git a/buildtools/pmdinfogen/pmdinfogen.c b/buildtools/pmdinfogen/pmdinfogen.c
new file mode 100644
index 0000000..9183937
--- /dev/null
+++ b/buildtools/pmdinfogen/pmdinfogen.c
@@ -0,0 +1,430 @@
+/* Postprocess pmd object files to export hw support 
+ *
+ * Copyright 2016 Neil Horman <nhorman@tuxdriver.com>
+ * Based in part on modpost.c from the linux kernel
+ *
+ * This software may be used and distributed according to the terms
+ * of the GNU General Public License V2, incorporated herein by reference.
+ *
+ */
+
+#define _GNU_SOURCE
+#include <stdio.h>
+#include <ctype.h>
+#include <string.h>
+#include <limits.h>
+#include <stdbool.h>
+#include <errno.h>
+#include "pmdinfogen.h"
+
+
+static const char *sym_name(struct elf_info *elf, Elf_Sym *sym)
+{
+	if (sym)
+		return elf->strtab + sym->st_name;
+	else
+		return "(unknown)";
+}
+
+void *grab_file(const char *filename, unsigned long *size)
+{
+	struct stat st;
+	void *map = MAP_FAILED;
+	int fd;
+
+	fd = open(filename, O_RDONLY);
+	if (fd < 0)
+		return NULL;
+	if (fstat(fd, &st))
+		goto failed;
+
+	*size = st.st_size;
+	map = mmap(NULL, *size, PROT_READ|PROT_WRITE, MAP_PRIVATE, fd, 0);
+
+failed:
+	close(fd);
+	if (map == MAP_FAILED)
+		return NULL;
+	return map;
+}
+
+/**
+  * Return a copy of the next line in a mmap'ed file.
+  * spaces in the beginning of the line is trimmed away.
+  * Return a pointer to a static buffer.
+  **/
+void release_file(void *file, unsigned long size)
+{
+	munmap(file, size);
+}
+
+
+static void *get_sym_value(struct elf_info *info, const Elf_Sym *sym)
+{
+	void *ptr = (void *)info->hdr + info->sechdrs[sym->st_shndx].sh_offset;
+
+	return (void *)(ptr + sym->st_value);
+}
+
+static Elf_Sym *find_sym_in_symtab(struct elf_info *info, 
+				   const char *name, Elf_Sym *last)
+{
+	Elf_Sym *idx;
+	if (last)
+		idx = last+1;
+	else
+		idx = info->symtab_start;
+
+	for(; idx < info->symtab_stop; idx++) {
+		const char *n = sym_name(info, idx);
+		if (!strncmp(n, name, strlen(name)))
+			return idx;
+	}
+	return NULL;
+}
+
+static int parse_elf(struct elf_info *info, const char *filename)
+{
+	unsigned int i;
+	Elf_Ehdr *hdr;
+	Elf_Shdr *sechdrs;
+	Elf_Sym  *sym;
+	const char *secstrings;
+	unsigned int symtab_idx = ~0U, symtab_shndx_idx = ~0U;
+
+	hdr = grab_file(filename, &info->size);
+	if (!hdr) {
+		perror(filename);
+		exit(1);
+	}
+	info->hdr = hdr;
+	if (info->size < sizeof(*hdr)) {
+		/* file too small, assume this is an empty .o file */
+		return 0;
+	}
+	/* Is this a valid ELF file? */
+	if ((hdr->e_ident[EI_MAG0] != ELFMAG0) ||
+	    (hdr->e_ident[EI_MAG1] != ELFMAG1) ||
+	    (hdr->e_ident[EI_MAG2] != ELFMAG2) ||
+	    (hdr->e_ident[EI_MAG3] != ELFMAG3)) {
+		/* Not an ELF file - silently ignore it */
+		return 0;
+	}
+	/* Fix endianness in ELF header */
+	hdr->e_type      = TO_NATIVE(hdr->e_type);
+	hdr->e_machine   = TO_NATIVE(hdr->e_machine);
+	hdr->e_version   = TO_NATIVE(hdr->e_version);
+	hdr->e_entry     = TO_NATIVE(hdr->e_entry);
+	hdr->e_phoff     = TO_NATIVE(hdr->e_phoff);
+	hdr->e_shoff     = TO_NATIVE(hdr->e_shoff);
+	hdr->e_flags     = TO_NATIVE(hdr->e_flags);
+	hdr->e_ehsize    = TO_NATIVE(hdr->e_ehsize);
+	hdr->e_phentsize = TO_NATIVE(hdr->e_phentsize);
+	hdr->e_phnum     = TO_NATIVE(hdr->e_phnum);
+	hdr->e_shentsize = TO_NATIVE(hdr->e_shentsize);
+	hdr->e_shnum     = TO_NATIVE(hdr->e_shnum);
+	hdr->e_shstrndx  = TO_NATIVE(hdr->e_shstrndx);
+	sechdrs = (void *)hdr + hdr->e_shoff;
+	info->sechdrs = sechdrs;
+
+	/* Check if file offset is correct */
+	if (hdr->e_shoff > info->size) {
+		fprintf(stderr, "section header offset=%lu in file '%s' is bigger than "
+		      "filesize=%lu\n", (unsigned long)hdr->e_shoff,
+		      filename, info->size);
+		return 0;
+	}
+
+	if (hdr->e_shnum == SHN_UNDEF) {
+		/*
+		 * There are more than 64k sections,
+		 * read count from .sh_size.
+		 */
+		info->num_sections = TO_NATIVE(sechdrs[0].sh_size);
+	}
+	else {
+		info->num_sections = hdr->e_shnum;
+	}
+	if (hdr->e_shstrndx == SHN_XINDEX) {
+		info->secindex_strings = TO_NATIVE(sechdrs[0].sh_link);
+	}
+	else {
+		info->secindex_strings = hdr->e_shstrndx;
+	}
+
+	/* Fix endianness in section headers */
+	for (i = 0; i < info->num_sections; i++) {
+		sechdrs[i].sh_name      = TO_NATIVE(sechdrs[i].sh_name);
+		sechdrs[i].sh_type      = TO_NATIVE(sechdrs[i].sh_type);
+		sechdrs[i].sh_flags     = TO_NATIVE(sechdrs[i].sh_flags);
+		sechdrs[i].sh_addr      = TO_NATIVE(sechdrs[i].sh_addr);
+		sechdrs[i].sh_offset    = TO_NATIVE(sechdrs[i].sh_offset);
+		sechdrs[i].sh_size      = TO_NATIVE(sechdrs[i].sh_size);
+		sechdrs[i].sh_link      = TO_NATIVE(sechdrs[i].sh_link);
+		sechdrs[i].sh_info      = TO_NATIVE(sechdrs[i].sh_info);
+		sechdrs[i].sh_addralign = TO_NATIVE(sechdrs[i].sh_addralign);
+		sechdrs[i].sh_entsize   = TO_NATIVE(sechdrs[i].sh_entsize);
+	}
+	/* Find symbol table. */
+	secstrings = (void *)hdr + sechdrs[info->secindex_strings].sh_offset;
+	for (i = 1; i < info->num_sections; i++) {
+		const char *secname;
+		int nobits = sechdrs[i].sh_type == SHT_NOBITS;
+
+		if (!nobits && sechdrs[i].sh_offset > info->size) {
+			fprintf(stderr, "%s is truncated. sechdrs[i].sh_offset=%lu > "
+			      "sizeof(*hrd)=%zu\n", filename,
+			      (unsigned long)sechdrs[i].sh_offset,
+			      sizeof(*hdr));
+			return 0;
+		}
+		secname = secstrings + sechdrs[i].sh_name;
+		if (strcmp(secname, ".modinfo") == 0) {
+			if (nobits)
+				fprintf(stderr, "%s has NOBITS .modinfo\n", filename);
+			info->modinfo = (void *)hdr + sechdrs[i].sh_offset;
+			info->modinfo_len = sechdrs[i].sh_size;
+		} else if (strcmp(secname, "__ksymtab") == 0)
+			info->export_sec = i;
+		else if (strcmp(secname, "__ksymtab_unused") == 0)
+			info->export_unused_sec = i;
+		else if (strcmp(secname, "__ksymtab_gpl") == 0)
+			info->export_gpl_sec = i;
+		else if (strcmp(secname, "__ksymtab_unused_gpl") == 0)
+			info->export_unused_gpl_sec = i;
+		else if (strcmp(secname, "__ksymtab_gpl_future") == 0)
+			info->export_gpl_future_sec = i;
+
+		if (sechdrs[i].sh_type == SHT_SYMTAB) {
+			unsigned int sh_link_idx;
+			symtab_idx = i;
+			info->symtab_start = (void *)hdr +
+			    sechdrs[i].sh_offset;
+			info->symtab_stop  = (void *)hdr +
+			    sechdrs[i].sh_offset + sechdrs[i].sh_size;
+			sh_link_idx = sechdrs[i].sh_link;
+			info->strtab       = (void *)hdr +
+			    sechdrs[sh_link_idx].sh_offset;
+		}
+
+		/* 32bit section no. table? ("more than 64k sections") */
+		if (sechdrs[i].sh_type == SHT_SYMTAB_SHNDX) {
+			symtab_shndx_idx = i;
+			info->symtab_shndx_start = (void *)hdr +
+			    sechdrs[i].sh_offset;
+			info->symtab_shndx_stop  = (void *)hdr +
+			    sechdrs[i].sh_offset + sechdrs[i].sh_size;
+		}
+	}
+	if (!info->symtab_start)
+		fprintf(stderr, "%s has no symtab?\n", filename);
+
+	/* Fix endianness in symbols */
+	for (sym = info->symtab_start; sym < info->symtab_stop; sym++) {
+		sym->st_shndx = TO_NATIVE(sym->st_shndx);
+		sym->st_name  = TO_NATIVE(sym->st_name);
+		sym->st_value = TO_NATIVE(sym->st_value);
+		sym->st_size  = TO_NATIVE(sym->st_size);
+	}
+
+	if (symtab_shndx_idx != ~0U) {
+		Elf32_Word *p;
+		if (symtab_idx != sechdrs[symtab_shndx_idx].sh_link)
+			fprintf(stderr, "%s: SYMTAB_SHNDX has bad sh_link: %u!=%u\n",
+			      filename, sechdrs[symtab_shndx_idx].sh_link,
+			      symtab_idx);
+		/* Fix endianness */
+		for (p = info->symtab_shndx_start; p < info->symtab_shndx_stop;
+		     p++)
+			*p = TO_NATIVE(*p);
+	}
+
+	return 1;
+}
+
+static void parse_elf_finish(struct elf_info *info)
+{
+	struct pmd_driver *tmp, *idx = info->drivers;
+	release_file(info->hdr, info->size);
+	while (idx) {
+		tmp = idx->next;
+		free(idx);
+		idx = tmp;
+	}
+}
+
+static const char *sec_name(struct elf_info *elf, int secindex)
+{
+	Elf_Shdr *sechdrs = elf->sechdrs;
+	return (void *)elf->hdr +
+		elf->sechdrs[elf->secindex_strings].sh_offset +
+		sechdrs[secindex].sh_name;
+}
+
+static int get_symbol_index(struct elf_info *info, Elf64_Sym *sym)
+{
+	const char *name =  sym_name(info, sym);
+	const char *idx;
+
+	idx = name;
+	while (idx) {
+		if (isdigit(*idx))
+			return atoi(idx);
+		idx++;
+	}
+	return -1;
+}
+
+struct opt_tag {
+	const char* suffix;
+	const char* json_id;
+};
+
+static const struct opt_tag opt_tags[] = {
+	{"_param_string_export", "params"},
+};
+
+static int complete_pmd_entry(struct elf_info *info, struct pmd_driver *drv)
+{
+	const char *tname;
+	int i;
+	char tmpsymname[128];
+	Elf_Sym *tmpsym;
+	
+
+	drv->name = get_sym_value(info, drv->name_sym);
+
+	for (i=0; i<PMD_OPT_MAX; i++) {
+		memset(tmpsymname, 0, 128);
+		sprintf(tmpsymname, "%s%s", drv->name, opt_tags[i].suffix);
+		tmpsym = find_sym_in_symtab(info, tmpsymname, NULL);
+		if (!tmpsym)
+			continue;
+		drv->opt_vals[i] = get_sym_value(info, tmpsym);
+	}
+
+	memset(tmpsymname, 0, 128);
+	sprintf(tmpsymname, "%s_pci_tbl_export", drv->name);
+
+	tmpsym = find_sym_in_symtab(info, tmpsymname, NULL);
+
+
+	/*
+ 	 * If this returns NULL, then this is a PMD_VDEV, because
+ 	 * it has no pci table reference
+ 	 */
+	if (!tmpsym) {
+		drv->pci_tbl = NULL;
+		return 0;
+	}
+
+	tname = get_sym_value(info, tmpsym);
+	tmpsym = find_sym_in_symtab(info, tname, NULL);
+	if (!tmpsym)
+		return -ENOENT;
+
+	drv->pci_tbl = (struct rte_pci_id *)get_sym_value(info, tmpsym);
+	if (!drv->pci_tbl)
+		return -ENOENT;
+
+
+	return 0;
+	
+}
+
+static int locate_pmd_entries(struct elf_info *info)
+{
+	Elf_Sym *last = NULL;
+	struct pmd_driver *new;
+
+	info->drivers = NULL;
+
+	do {
+		new = calloc(sizeof(struct pmd_driver), 1);
+		new->name_sym = find_sym_in_symtab(info, "this_pmd_name", last);
+		last = new->name_sym;
+		if (!new->name_sym)
+			free(new);
+		else {
+			if (complete_pmd_entry(info, new)) {
+				fprintf(stderr, "Failed to complete pmd entry\n");
+				free(new);
+			} else {
+				new->next = info->drivers;
+				info->drivers = new;
+			}
+		}
+	} while (last);
+}
+
+static void output_pmd_info_string(struct elf_info *info, char *outfile)
+{
+	FILE *ofd;
+	struct pmd_driver *drv;
+	struct rte_pci_id *pci_ids;
+	int idx = 0;
+
+	ofd = fopen(outfile, "w+");
+	if (!ofd) {
+		fprintf(stderr, "Unable to open output file\n");
+		return;
+	}
+
+	drv = info->drivers;
+
+	while (drv) {
+		fprintf(ofd, "const char %s_pmd_info[] __attribute__((used)) = \"PMD_INFO_STRING= {",
+			drv->name);
+		fprintf(ofd,"\\\"name\\\" : \\\"%s\\\", ", drv->name);
+		fprintf(ofd,"\\\"type\\\" : \\\"%s\\\", ", drv->pci_tbl ? "PMD_PDEV" : "PMD_VDEV");
+
+		for(idx=0; idx<PMD_OPT_MAX; idx++) {
+			if (drv->opt_vals[idx])
+				fprintf(ofd,"\\\"%s\\\" : \\\"%s\\\", ", opt_tags[idx].json_id,
+					drv->opt_vals[idx]);
+		}
+
+		pci_ids = drv->pci_tbl;
+		fprintf(ofd, "\\\"pci_ids\\\" : [");
+
+		while (pci_ids && pci_ids->device_id) {
+			fprintf(ofd, "[%d, %d, %d, %d]",
+				pci_ids->vendor_id, pci_ids->device_id,
+				pci_ids->subsystem_vendor_id,
+				pci_ids->subsystem_device_id);
+			pci_ids++;
+			if (pci_ids->device_id)
+				fprintf(ofd, ",");
+			else
+				fprintf(ofd, " ");
+		}
+		fprintf(ofd, "]}\";");
+		drv = drv->next;
+	}
+
+	fclose(ofd);
+}
+
+int main(int argc, char **argv)
+{
+	struct elf_info info;
+	int rc = 1;
+
+	if (argc < 3) {
+		fprintf(stderr, "usage: pmdinfo <object file> <c output file>\n");
+		exit(127);
+	}
+	parse_elf(&info, argv[1]);
+
+	locate_pmd_entries(&info);
+
+	if (info.drivers) {
+		output_pmd_info_string(&info, argv[2]);
+		rc = 0;
+	} else {
+		fprintf(stderr, "Hmm, Appears to be a driver but no drivers registered\n");
+	}
+
+	parse_elf_finish(&info);
+	exit(rc);
+}
diff --git a/buildtools/pmdinfogen/pmdinfogen.h b/buildtools/pmdinfogen/pmdinfogen.h
new file mode 100644
index 0000000..580ed9f
--- /dev/null
+++ b/buildtools/pmdinfogen/pmdinfogen.h
@@ -0,0 +1,83 @@
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdarg.h>
+#include <string.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/mman.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <elf.h>
+
+
+/* On BSD-alike OSes elf.h defines these according to host's word size */
+#undef ELF_ST_BIND
+#undef ELF_ST_TYPE
+#undef ELF_R_SYM
+#undef ELF_R_TYPE
+
+#define Elf_Ehdr    Elf64_Ehdr
+#define Elf_Shdr    Elf64_Shdr
+#define Elf_Sym     Elf64_Sym
+#define Elf_Addr    Elf64_Addr
+#define Elf_Sword   Elf64_Sxword
+#define Elf_Section Elf64_Half
+#define ELF_ST_BIND ELF64_ST_BIND
+#define ELF_ST_TYPE ELF64_ST_TYPE
+
+#define Elf_Rel     Elf64_Rel
+#define Elf_Rela    Elf64_Rela
+#define ELF_R_SYM   ELF64_R_SYM
+#define ELF_R_TYPE  ELF64_R_TYPE
+
+#define TO_NATIVE(x) (x)
+
+
+struct rte_pci_id {
+	uint16_t vendor_id;           /**< Vendor ID or PCI_ANY_ID. */
+	uint16_t device_id;           /**< Device ID or PCI_ANY_ID. */
+	uint16_t subsystem_vendor_id; /**< Subsystem vendor ID or PCI_ANY_ID. */
+	uint16_t subsystem_device_id; /**< Subsystem device ID or PCI_ANY_ID. */
+};
+
+enum opt_params {
+	PMD_PARAM_STRING = 0,
+	PMD_OPT_MAX
+};
+
+struct pmd_driver {
+	Elf_Sym *name_sym;
+	const char *name;
+	struct rte_pci_id *pci_tbl;
+	struct pmd_driver *next;
+
+	const char* opt_vals[PMD_OPT_MAX];
+};
+
+struct elf_info {
+	unsigned long size;
+	Elf_Ehdr     *hdr;
+	Elf_Shdr     *sechdrs;
+	Elf_Sym      *symtab_start;
+	Elf_Sym      *symtab_stop;
+	Elf_Section  export_sec;
+	Elf_Section  export_unused_sec;
+	Elf_Section  export_gpl_sec;
+	Elf_Section  export_unused_gpl_sec;
+	Elf_Section  export_gpl_future_sec;
+	char         *strtab;
+	char	     *modinfo;
+	unsigned int modinfo_len;
+
+	/* support for 32bit section numbers */
+
+	unsigned int num_sections; /* max_secindex + 1 */
+	unsigned int secindex_strings;
+	/* if Nth symbol table entry has .st_shndx = SHN_XINDEX,
+	 * take shndx from symtab_shndx_start[N] instead */
+	Elf32_Word   *symtab_shndx_start;
+	Elf32_Word   *symtab_shndx_stop;
+
+	struct pmd_driver *drivers;
+};
+
diff --git a/mk/rte.buildtools.mk b/mk/rte.buildtools.mk
new file mode 100644
index 0000000..e8bfcef
--- /dev/null
+++ b/mk/rte.buildtools.mk
@@ -0,0 +1,148 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
+#   Copyright(c) 2014-2015 6WIND S.A.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/internal/rte.compile-pre.mk
+include $(RTE_SDK)/mk/internal/rte.install-pre.mk
+include $(RTE_SDK)/mk/internal/rte.clean-pre.mk
+include $(RTE_SDK)/mk/internal/rte.build-pre.mk
+include $(RTE_SDK)/mk/internal/rte.depdirs-pre.mk
+
+# VPATH contains at least SRCDIR
+VPATH += $(SRCDIR)
+
+_BUILD = $(APP)
+_INSTALL = $(INSTALL-FILES-y) $(SYMLINK-FILES-y)
+_INSTALL += $(RTE_OUTPUT)/buildtools/$(APP) $(RTE_OUTPUT)/buildtools/$(APP).map
+POSTINSTALL += target-appinstall
+_CLEAN = doclean
+POSTCLEAN += target-appclean
+
+.PHONY: all
+all: install
+
+.PHONY: install
+install: build _postinstall
+
+_postinstall: build
+
+.PHONY: build
+build: _postbuild
+
+exe2cmd = $(strip $(call dotfile,$(patsubst %,%.cmd,$(1))))
+
+ifeq ($(LINK_USING_CC),1)
+override EXTRA_LDFLAGS := $(call linkerprefix,$(EXTRA_LDFLAGS))
+O_TO_EXE = $(CC) $(CFLAGS) $(LDFLAGS_$(@)) \
+	-Wl,-Map=$(@).map,--cref -o $@ $(OBJS-y) $(call linkerprefix,$(LDFLAGS)) \
+	$(EXTRA_LDFLAGS) $(call linkerprefix,$(LDLIBS))
+else
+O_TO_EXE = $(LD) $(LDFLAGS) $(LDFLAGS_$(@)) $(EXTRA_LDFLAGS) \
+	-Map=$(@).map --cref -o $@ $(OBJS-y) $(LDLIBS)
+endif
+O_TO_EXE_STR = $(subst ','\'',$(O_TO_EXE)) #'# fix syntax highlight
+O_TO_EXE_DISP = $(if $(V),"$(O_TO_EXE_STR)","  LD $(@)")
+O_TO_EXE_CMD = "cmd_$@ = $(O_TO_EXE_STR)"
+O_TO_EXE_DO = @set -e; \
+	echo $(O_TO_EXE_DISP); \
+	$(O_TO_EXE) && \
+	echo $(O_TO_EXE_CMD) > $(call exe2cmd,$(@))
+
+-include .$(APP).cmd
+
+# path where libraries are retrieved
+LDLIBS_PATH := $(subst -Wl$(comma)-L,,$(filter -Wl$(comma)-L%,$(LDLIBS)))
+LDLIBS_PATH += $(subst -L,,$(filter -L%,$(LDLIBS)))
+
+# list of .a files that are linked to this application
+LDLIBS_NAMES := $(patsubst -l%,lib%.a,$(filter -l%,$(LDLIBS)))
+LDLIBS_NAMES += $(patsubst -Wl$(comma)-l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIBS)))
+
+# list of found libraries files (useful for deps). If not found, the
+# library is silently ignored and dep won't be checked
+LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
+	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
+
+#
+# Compile executable file if needed
+#
+$(APP): $(OBJS-y) $(LDLIBS_FILES) $(DEP_$(APP)) $(LDSCRIPT) FORCE
+	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
+	$(if $(D),\
+		@echo -n "$< -> $@ " ; \
+		echo -n "file_missing=$(call boolean,$(file_missing)) " ; \
+		echo -n "cmdline_changed=$(call boolean,$(call cmdline_changed,$(O_TO_EXE_STR))) " ; \
+		echo -n "depfile_missing=$(call boolean,$(depfile_missing)) " ; \
+		echo "depfile_newer=$(call boolean,$(depfile_newer)) ")
+	$(if $(or \
+		$(file_missing),\
+		$(call cmdline_changed,$(O_TO_EXE_STR)),\
+		$(depfile_missing),\
+		$(depfile_newer)),\
+		$(O_TO_EXE_DO))
+
+#
+# install app in $(RTE_OUTPUT)/app
+#
+$(RTE_OUTPUT)/buildtools/$(APP): $(APP)
+	@echo "  INSTALL-APP $(APP)"
+	@[ -d $(RTE_OUTPUT)/buildtools ] || mkdir -p $(RTE_OUTPUT)/buildtools
+	$(Q)cp -f $(APP) $(RTE_OUTPUT)/buildtools
+
+#
+# install app map file in $(RTE_OUTPUT)/app
+#
+$(RTE_OUTPUT)/buildtools/$(APP).map: $(APP)
+	@echo "  INSTALL-MAP $(APP).map"
+	@[ -d $(RTE_OUTPUT)/buildtools ] || mkdir -p $(RTE_OUTPUT)/buildtools
+	$(Q)cp -f $(APP).map $(RTE_OUTPUT)/buildtools
+
+#
+# Clean all generated files
+#
+.PHONY: clean
+clean: _postclean
+	$(Q)rm -f $(_BUILD_TARGETS) $(_INSTALL_TARGETS) $(_CLEAN_TARGETS)
+
+.PHONY: doclean
+doclean:
+	$(Q)rm -rf $(APP) $(OBJS-all) $(DEPS-all) $(DEPSTMP-all) \
+	  $(CMDS-all) $(INSTALL-FILES-all) .$(APP).cmd
+
+
+include $(RTE_SDK)/mk/internal/rte.compile-post.mk
+include $(RTE_SDK)/mk/internal/rte.install-post.mk
+include $(RTE_SDK)/mk/internal/rte.clean-post.mk
+include $(RTE_SDK)/mk/internal/rte.build-post.mk
+include $(RTE_SDK)/mk/internal/rte.depdirs-post.mk
+
+.PHONY: FORCE
+FORCE:
diff --git a/mk/rte.sdkbuild.mk b/mk/rte.sdkbuild.mk
index eec5241..fb68af2 100644
--- a/mk/rte.sdkbuild.mk
+++ b/mk/rte.sdkbuild.mk
@@ -64,7 +64,8 @@ build: $(ROOTDIRS-y)
 clean: $(CLEANDIRS)
 	@rm -rf $(RTE_OUTPUT)/include $(RTE_OUTPUT)/app \
 		$(RTE_OUTPUT)/hostapp $(RTE_OUTPUT)/lib \
-		$(RTE_OUTPUT)/hostlib $(RTE_OUTPUT)/kmod
+		$(RTE_OUTPUT)/hostlib $(RTE_OUTPUT)/kmod \
+		$(RTE_OUTPUT)/buildtools
 	@[ -d $(RTE_OUTPUT)/include ] || mkdir -p $(RTE_OUTPUT)/include
 	@$(RTE_SDK)/scripts/gen-config-h.sh $(RTE_OUTPUT)/.config \
 		> $(RTE_OUTPUT)/include/rte_config.h
-- 
2.5.5

[PATCHv2 2/4] drivers: Update driver registration macro usage

[Neil Horman]

Modify the PMD_REGISTER_DRIVER macro, bifurcating it into two
(PMD_REGISTER_DRIVER_PDEV and PMD_REGISTER_DRIVER_VDEV.  Both of these do the
same thing the origional macro did, but both add the definition of a string
variable that informs interested parties of the name of the pmd, and the former
also defines an second string that holds the symbol name of the pci table that
is registered by this pmd.

pmdinfo uses this information to extract hardware support from an object file
and create a json string to make hardware support info discoverable later.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 drivers/Makefile                           |  2 ++
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c   |  4 +++-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c |  4 +++-
 drivers/crypto/null/null_crypto_pmd.c      |  4 +++-
 drivers/crypto/qat/rte_qat_cryptodev.c     |  4 +++-
 drivers/crypto/snow3g/rte_snow3g_pmd.c     |  4 +++-
 drivers/net/af_packet/rte_eth_af_packet.c  |  4 +++-
 drivers/net/bnx2x/bnx2x_ethdev.c           |  6 ++++--
 drivers/net/bonding/rte_eth_bond_pmd.c     |  7 ++++++-
 drivers/net/cxgbe/cxgbe_ethdev.c           |  4 +++-
 drivers/net/e1000/em_ethdev.c              |  3 ++-
 drivers/net/e1000/igb_ethdev.c             |  6 ++++--
 drivers/net/ena/ena_ethdev.c               |  3 ++-
 drivers/net/enic/enic_ethdev.c             |  3 ++-
 drivers/net/fm10k/fm10k_ethdev.c           |  3 ++-
 drivers/net/i40e/i40e_ethdev.c             |  3 ++-
 drivers/net/i40e/i40e_ethdev_vf.c          |  3 ++-
 drivers/net/ixgbe/ixgbe_ethdev.c           |  6 ++++--
 drivers/net/mlx4/mlx4.c                    |  3 ++-
 drivers/net/mlx5/mlx5.c                    |  3 ++-
 drivers/net/mpipe/mpipe_tilegx.c           |  4 ++--
 drivers/net/nfp/nfp_net.c                  |  3 ++-
 drivers/net/null/rte_eth_null.c            |  3 ++-
 drivers/net/pcap/rte_eth_pcap.c            |  4 +++-
 drivers/net/ring/rte_eth_ring.c            |  3 ++-
 drivers/net/szedata2/rte_eth_szedata2.c    |  3 ++-
 drivers/net/vhost/rte_eth_vhost.c          |  3 ++-
 drivers/net/virtio/virtio_ethdev.c         |  3 ++-
 drivers/net/vmxnet3/vmxnet3_ethdev.c       |  3 ++-
 drivers/net/xenvirt/rte_eth_xenvirt.c      |  2 +-
 lib/librte_eal/common/include/rte_dev.h    | 20 ++++++++++++++++----
 31 files changed, 93 insertions(+), 37 deletions(-)

diff --git a/drivers/Makefile b/drivers/Makefile
index 81c03a8..75a3168 100644
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -34,4 +34,6 @@ include $(RTE_SDK)/mk/rte.vars.mk
 DIRS-y += net
 DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += crypto
 
+DEPDIRS-y += buildtools/pmdinfo
+
 include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
index 2987ef6..593769f 100644
--- a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
+++ b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
@@ -521,4 +521,6 @@ static struct rte_driver aesni_gcm_pmd_drv = {
 	.uninit = aesni_gcm_uninit
 };
 
-PMD_REGISTER_DRIVER(aesni_gcm_pmd_drv);
+PMD_REGISTER_DRIVER(aesni_gcm_pmd_drv, aesni_gcm);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> \
+max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
index 3415ac1..2e30939 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
@@ -716,4 +716,6 @@ static struct rte_driver cryptodev_aesni_mb_pmd_drv = {
 	.uninit = cryptodev_aesni_mb_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drv);
+PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drvi, aesni_mb);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> \
+max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/null/null_crypto_pmd.c b/drivers/crypto/null/null_crypto_pmd.c
index bdaf13c..44c4fba 100644
--- a/drivers/crypto/null/null_crypto_pmd.c
+++ b/drivers/crypto/null/null_crypto_pmd.c
@@ -275,4 +275,6 @@ static struct rte_driver cryptodev_null_pmd_drv = {
 	.uninit = cryptodev_null_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_null_pmd_drv);
+PMD_REGISTER_DRIVER(cryptodev_null_pmd_drv, cryptodev_null_pmd);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> \
+max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index a7912f5..92f0d40 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -137,4 +137,6 @@ static struct rte_driver pmd_qat_drv = {
 	.init = rte_qat_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(pmd_qat_drv);
+PMD_REGISTER_DRIVER(pmd_qat_drv, qat);
+DRIVER_EXPORT_PCI_TABLE(qat, pci_id_qat_map);
+
diff --git a/drivers/crypto/snow3g/rte_snow3g_pmd.c b/drivers/crypto/snow3g/rte_snow3g_pmd.c
index f3e0e66..eee2544 100644
--- a/drivers/crypto/snow3g/rte_snow3g_pmd.c
+++ b/drivers/crypto/snow3g/rte_snow3g_pmd.c
@@ -548,4 +548,6 @@ static struct rte_driver cryptodev_snow3g_pmd_drv = {
 	.uninit = cryptodev_snow3g_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_snow3g_pmd_drv);
+PMD_REGISTER_DRIVER(cryptodev_snow3g_pmd_drv, snow3g);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> \
+max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index f17bd7e..e960512 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -878,4 +878,6 @@ static struct rte_driver pmd_af_packet_drv = {
 	.uninit = rte_pmd_af_packet_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_af_packet_drv);
+PMD_REGISTER_DRIVER(pmd_af_packet_drv, eth_af_packet);
+DRIVER_REGISTER_PARAM_STRING(eth_af_packet, "iface=<string> \
+qpairs=<int> blocksz=<int> framesz=<int> framecnt=<int>");
diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c b/drivers/net/bnx2x/bnx2x_ethdev.c
index 071b44f..ee9581b 100644
--- a/drivers/net/bnx2x/bnx2x_ethdev.c
+++ b/drivers/net/bnx2x/bnx2x_ethdev.c
@@ -550,5 +550,7 @@ static struct rte_driver rte_bnx2xvf_driver = {
 	.init = rte_bnx2xvf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_bnx2x_driver);
-PMD_REGISTER_DRIVER(rte_bnx2xvf_driver);
+PMD_REGISTER_DRIVER(rte_bnx2x_driver, pci_id_bnx2x_map, bnx2x);
+DRIVER_EXPORT_PCI_TABLE(bnx2x, pci_id_bnx2x_map);
+PMD_REGISTER_DRIVER(rte_bnx2xvf_driver, pci_id_bnx2xvf_map, bnx2xvf);
+DRIVER_EXPORT_PCI_TABLE(bnx2xvf, pci_id_bnx2xvf_map);
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 129f04b..b4556be 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -2511,4 +2511,9 @@ static struct rte_driver bond_drv = {
 	.uninit = bond_uninit,
 };
 
-PMD_REGISTER_DRIVER(bond_drv);
+PMD_REGISTER_DRIVER(bond_drv, bonding);
+
+DRIVER_REGISTER_PARAM_STRING(bonding, "slave=<ifc> primary=<ifc> mode=[0-4] \
+xmit_policy=[l2 | l23 | l34] socket_id=<int> mac=<mac addr> \
+lsc_poll_period_ms=<int> up_delay=<int> down_delay=<int>");
+
diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index 04eddaf..e84e48b 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -894,4 +894,6 @@ static struct rte_driver rte_cxgbe_driver = {
 	.init = rte_cxgbe_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_cxgbe_driver);
+PMD_REGISTER_DRIVER(rte_cxgbe_driver, cxgb4);
+DRIVER_REGISTER_PCI_TABLE(cxgb4, cxgb4_pci_tbl);
+
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index 653be09..4de5eb2 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -1777,4 +1777,5 @@ struct rte_driver em_pmd_drv = {
 	.init = rte_em_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(em_pmd_drv);
+PMD_REGISTER_DRIVER(em_pmd_drv, em);
+DRIVER_REGISTER_PCI_TABLE(em, pci_id_em_map);
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index f0921ee..f49aae7 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -4973,5 +4973,7 @@ eth_igb_configure_msix_intr(struct rte_eth_dev *dev)
 	E1000_WRITE_FLUSH(hw);
 }
 
-PMD_REGISTER_DRIVER(pmd_igb_drv);
-PMD_REGISTER_DRIVER(pmd_igbvf_drv);
+PMD_REGISTER_DRIVER(pmd_igb_drv, igb);
+DRIVER_REGISTER_PCI_TABLE(igb, pci_id_igb_map);
+PMD_REGISTER_DRIVER(pmd_igbvf_drv, igbvf);
+DRIVER_REGISTER_PCI_TABLE(igbvf, pci_id_igbvf_map);
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index e157587..702289b 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -1450,4 +1450,5 @@ struct rte_driver ena_pmd_drv = {
 	.init = rte_ena_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(ena_pmd_drv);
+PMD_REGISTER_DRIVER(ena_pmd_drv, ena);
+DRIVER_REGISTER_PCI_TABLE(ena, pci_id_ena_map);
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index 6bea940..36ee7ca 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -684,4 +684,5 @@ static struct rte_driver rte_enic_driver = {
 	.init = rte_enic_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_enic_driver);
+PMD_REGISTER_DRIVER(rte_enic_driver, enic);
+DRIVER_REGISTER_PCI_TABLE(enic, pci_id_enic_map);
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index c2d377f..9b2a7ee 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -3055,4 +3055,5 @@ static struct rte_driver rte_fm10k_driver = {
 	.init = rte_pmd_fm10k_init,
 };
 
-PMD_REGISTER_DRIVER(rte_fm10k_driver);
+PMD_REGISTER_DRIVER(rte_fm10k_driver, fm10k);
+DRIVER_REGISTER_PCI_TABLE(fm10k, pci_id_fm10k_map);
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index d8b6bd7..57f89d9 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -695,7 +695,8 @@ static struct rte_driver rte_i40e_driver = {
 	.init = rte_i40e_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_i40e_driver);
+PMD_REGISTER_DRIVER(rte_i40e_driver, i40e);
+DRIVER_REGISTER_PCI_TABLE(i40e, pci_id_i40e_map);
 
 /*
  * Initialize registers for flexible payload, which should be set by NVM.
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 2bce69b..348d9a8 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1560,7 +1560,8 @@ static struct rte_driver rte_i40evf_driver = {
 	.init = rte_i40evf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_i40evf_driver);
+PMD_REGISTER_DRIVER(rte_i40evf_driver, i40evf);
+DRIVER_REGISTER_PCI_TABLE(i40evf, pci_id_i40evf_map);
 
 static int
 i40evf_dev_configure(struct rte_eth_dev *dev)
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index eec607c..933f013 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -7141,5 +7141,7 @@ static struct rte_driver rte_ixgbevf_driver = {
 	.init = rte_ixgbevf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_ixgbe_driver);
-PMD_REGISTER_DRIVER(rte_ixgbevf_driver);
+PMD_REGISTER_DRIVER(rte_ixgbe_driver, ixgbe);
+DRIVER_REGISTER_PCI_TABLE(ixgbe, pci_id_ixgbe_map);
+PMD_REGISTER_DRIVER(rte_ixgbevf_driver, ixgbevf);
+DRIVER_REGISTER_PCI_TABLE(ixgbevf, pci_id_ixgbevf_map);
diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index 4f21dbe..b0d2f3c 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -5817,4 +5817,5 @@ static struct rte_driver rte_mlx4_driver = {
 	.init = rte_mlx4_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_mlx4_driver)
+PMD_REGISTER_DRIVER(rte_mlx4_driveri, mlx4)
+DRIVER_REGISTER_PCI_TABLE(mlx4, mlx4_pci_id_map);
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 041cfc3..5f2d6c8 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -672,4 +672,5 @@ static struct rte_driver rte_mlx5_driver = {
 	.init = rte_mlx5_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_mlx5_driver)
+PMD_REGISTER_DRIVER(rte_mlx5_driveri, mlx5)
+DRIVER_REGISTER_PCI_TABLE(mlx5, mlx5_pci_id_map);
diff --git a/drivers/net/mpipe/mpipe_tilegx.c b/drivers/net/mpipe/mpipe_tilegx.c
index adcbc19..4717a43 100644
--- a/drivers/net/mpipe/mpipe_tilegx.c
+++ b/drivers/net/mpipe/mpipe_tilegx.c
@@ -1635,8 +1635,8 @@ static struct rte_driver pmd_mpipe_gbe_drv = {
 	.init = rte_pmd_mpipe_devinit,
 };
 
-PMD_REGISTER_DRIVER(pmd_mpipe_xgbe_drv);
-PMD_REGISTER_DRIVER(pmd_mpipe_gbe_drv);
+PMD_REGISTER_DRIVER(pmd_mpipe_xgbe_drv, mpipe_xgbe);
+PMD_REGISTER_DRIVER(pmd_mpipe_gbe_drv, mpipe_gbe);
 
 static void __attribute__((constructor, used))
 mpipe_init_contexts(void)
diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
index bc0a3d8..75e780f 100644
--- a/drivers/net/nfp/nfp_net.c
+++ b/drivers/net/nfp/nfp_net.c
@@ -2482,7 +2482,8 @@ static struct rte_driver rte_nfp_net_driver = {
 	.init = nfp_net_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_nfp_net_driver);
+PMD_REGISTER_DRIVER(rte_nfp_net_driver, nfp);
+DRIVER_REGISTER_PCI_TABLE(nfp, pci_id_nfp_net_map);
 
 /*
  * Local variables:
diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index 5e8e203..88eb582 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -689,4 +689,5 @@ static struct rte_driver pmd_null_drv = {
 	.uninit = rte_pmd_null_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_null_drv);
+PMD_REGISTER_DRIVER(pmd_null_drv, eth_null);
+DRIVER_REGISTER_PARAM_STRING(eth_null, "size=<int> copy=<int>");
diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
index c98e234..397f01c 100644
--- a/drivers/net/pcap/rte_eth_pcap.c
+++ b/drivers/net/pcap/rte_eth_pcap.c
@@ -1090,4 +1090,6 @@ static struct rte_driver pmd_pcap_drv = {
 	.uninit = rte_pmd_pcap_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_pcap_drv);
+PMD_REGISTER_DRIVER(pmd_pcap_drv, pcap);
+DRIVER_REGISTER_PARAM_STRING(pcap, "rx_pcap=<string> tx_pcap=<string> \
+rx_iface=<ifc> tx_iface=<ifc> iface=<ifc>");
diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index b1783c3..f2f5217 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -630,4 +630,5 @@ static struct rte_driver pmd_ring_drv = {
 	.uninit = rte_pmd_ring_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_ring_drv);
+PMD_REGISTER_DRIVER(pmd_ring_drv, eth_ring);
+DRIVER_REGISTER_PARAM_STRING(eth_ring, "nodeaction=[attach|detach]");
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 78c43b0..abbc907 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1601,4 +1601,5 @@ static struct rte_driver rte_szedata2_driver = {
 	.uninit = rte_szedata2_uninit,
 };
 
-PMD_REGISTER_DRIVER(rte_szedata2_driver);
+PMD_REGISTER_DRIVER(rte_szedata2_driver, rte_szedata2_pmd);
+DRIVER_REGISTER_PCI_TABLE(rte_szedata2_pmd, rte_szedata2_pci_id_table);
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index 310cbef..d321c3e 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -924,4 +924,5 @@ static struct rte_driver pmd_vhost_drv = {
 	.uninit = rte_pmd_vhost_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_vhost_drv);
+PMD_REGISTER_DRIVER(pmd_vhost_drv, eth_vhost);
+DRIVER_REGISTER_PARAM_STRING(eth_vhost, "iface=<ifc> queues=<int>");
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 63a368a..030a206 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -1458,4 +1458,5 @@ static struct rte_driver rte_virtio_driver = {
 	.init = rte_virtio_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_virtio_driver);
+PMD_REGISTER_DRIVER(rte_virtio_driver, virtio_net);
+DRIVER_REGISTER_PCI_TABLE(virtio_net, pci_id_virtio_map);
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index 29b469c..8da4449 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -954,4 +954,5 @@ static struct rte_driver rte_vmxnet3_driver = {
 	.init = rte_vmxnet3_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_vmxnet3_driver);
+PMD_REGISTER_DRIVER(rte_vmxnet3_driver, vmxnet3);
+DRIVER_REGISTER_PCI_TABLE(vmxnet3, pci_id_vmxnet3_map);
diff --git a/drivers/net/xenvirt/rte_eth_xenvirt.c b/drivers/net/xenvirt/rte_eth_xenvirt.c
index b9638d9..bcc1847 100644
--- a/drivers/net/xenvirt/rte_eth_xenvirt.c
+++ b/drivers/net/xenvirt/rte_eth_xenvirt.c
@@ -775,4 +775,4 @@ static struct rte_driver pmd_xenvirt_drv = {
 	.uninit = rte_pmd_xenvirt_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_xenvirt_drv);
+PMD_REGISTER_DRIVER(pmd_xenvirt_drv, xenvirt);
diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
index f1b5507..871089a 100644
--- a/lib/librte_eal/common/include/rte_dev.h
+++ b/lib/librte_eal/common/include/rte_dev.h
@@ -48,7 +48,7 @@ extern "C" {
 
 #include <stdio.h>
 #include <sys/queue.h>
-
+#include <rte_pci.h>
 #include <rte_log.h>
 
 __attribute__((format(printf, 2, 0)))
@@ -178,12 +178,24 @@ int rte_eal_vdev_init(const char *name, const char *args);
  */
 int rte_eal_vdev_uninit(const char *name);
 
-#define PMD_REGISTER_DRIVER(d)\
+#define DRIVER_EXPORT_NAME_ARRAY(n, idx) n##idx[] __attribute__((used))
+
+#define DRIVER_EXPORT_NAME(d, idx) \
+static const char DRIVER_EXPORT_NAME_ARRAY(this_pmd_name, idx) = RTE_STR(d);\
+ 
+#define PMD_REGISTER_DRIVER(d, n)\
 void devinitfn_ ##d(void);\
 void __attribute__((constructor, used)) devinitfn_ ##d(void)\
 {\
-	rte_eal_driver_register(&d);\
-}
+        rte_eal_driver_register(&d);\
+}\
+DRIVER_EXPORT_NAME(n, __COUNTER__)
+
+#define DRIVER_REGISTER_PCI_TABLE(n, t) \
+static const char n##_pci_tbl_export[] __attribute__((used)) = RTE_STR(t)
+
+#define DRIVER_REGISTER_PARAM_STRING(n, s) \
+static const char n##_param_string_export[] __attribute__((used)) = s
 
 #ifdef __cplusplus
 }
-- 
2.5.5

[PATCHv2 3/4] Makefile: Do post processing on objects that register a driver

[Neil Horman]

Modify the compilation makefile to identify C files that export PMD information,
and use that to trigger execution of the pmdinfo binary.  If the execution of
pmdinfo is successful, compile the output C file to an object, and use the
linker to do relocatable linking on the resultant object file into the parent
object that it came from.  This effectively just adds the json string into the
string table of the object that defines the PMD to the outside world.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 mk/internal/rte.compile-pre.mk | 19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/mk/internal/rte.compile-pre.mk b/mk/internal/rte.compile-pre.mk
index b9bff4a..6fa4042 100644
--- a/mk/internal/rte.compile-pre.mk
+++ b/mk/internal/rte.compile-pre.mk
@@ -80,7 +80,8 @@ C_TO_O_STR = $(subst ','\'',$(C_TO_O)) #'# fix syntax highlight
 C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)","  HOSTCC $(@)")
 else
 C_TO_O = $(CC) -Wp,-MD,$(call obj2dep,$(@)).tmp $(CFLAGS) \
-	$(CFLAGS_$(@)) $(EXTRA_CFLAGS) -o $@ -c $<
+	 $(CFLAGS_$(@)) $(EXTRA_CFLAGS) -o $@ -c $<
+
 C_TO_O_STR = $(subst ','\'',$(C_TO_O)) #'# fix syntax highlight
 C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)","  CC $(@)")
 endif
@@ -88,10 +89,26 @@ C_TO_O_CMD = 'cmd_$@ = $(C_TO_O_STR)'
 C_TO_O_DO = @set -e; \
 	echo $(C_TO_O_DISP); \
 	$(C_TO_O) && \
+	sh -c "grep -q \"PMD_REGISTER_DRIVER(.*)\" $<; \
+	if [ \$$? -eq 0 ]; \
+	then \
+		echo MODGEN $@; \
+		OBJF=`readlink -f $@`; \
+		${RTE_OUTPUT}/buildtools/pmdinfogen \$$OBJF \$$OBJF.mod.c; \
+		if [ \$$? -eq 0 ]; \
+		then \
+			echo MODBUILD $@; \
+			$(CC) -c -o \$$OBJF.mod.o \$$OBJF.mod.c; \
+			$(CROSS)ld -r -o \$$OBJF.o \$$OBJF.mod.o \$$OBJF; \
+			mv -f \$$OBJF.o \$$OBJF; \
+		fi; \
+	fi; \
+	true" && \
 	echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \
 	sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \
 	rm -f $(call obj2dep,$(@)).tmp
 
+
 # return an empty string if string are equal
 compare = $(strip $(subst $(1),,$(2)) $(subst $(2),,$(1)))
 
-- 
2.5.5

[PATCHv2 4/4] pmdinfo.py: Add tool to query binaries for hw and other support information

[Neil Horman]

This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
and, if found parses the remainder of the string as a json encoded string,
outputting the results in either a human readable or raw, script parseable
format

Note that, in the case of dynamically linked applications, pmdinfo.py will scan
for implicitly linked PMDs by searching the specified binaries .dynamic section
for DT_NEEDED entries that contain the substring librte_pmd.  The DT_RUNPATH,
LD_LIBRARY_PATH, /usr/lib and /lib are searched for these libraries, in that
order

If a file is specified with no path, it is assumed to be a PMD DSO, and the
LD_LIBRARY_PATH, /usr/lib/ and /lib is searched for it

Currently the tool can output data in 3 formats:

a) raw, suitable for scripting, where the raw JSON strings are dumped out
b) table format (default) where hex pci ids are dumped in a table format
c) pretty, where a user supplied pci.ids file is used to print out vendor and
device strings

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 tools/pmdinfo.py | 457 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 457 insertions(+)
 create mode 100755 tools/pmdinfo.py

diff --git a/tools/pmdinfo.py b/tools/pmdinfo.py
new file mode 100755
index 0000000..e1cc4ae
--- /dev/null
+++ b/tools/pmdinfo.py
@@ -0,0 +1,457 @@
+#!/usr/bin/python
+#-------------------------------------------------------------------------------
+# scripts/pmd_hw_support.py
+#
+# Utility to dump PMD_INFO_STRING support from an object file
+#
+#-------------------------------------------------------------------------------
+import os, sys
+from optparse import OptionParser
+import string
+import json
+
+# For running from development directory. It should take precedence over the
+# installed pyelftools.
+sys.path.insert(0, '.')
+
+
+from elftools import __version__
+from elftools.common.exceptions import ELFError
+from elftools.common.py3compat import (
+        ifilter, byte2int, bytes2str, itervalues, str2bytes)
+from elftools.elf.elffile import ELFFile
+from elftools.elf.dynamic import DynamicSection, DynamicSegment
+from elftools.elf.enums import ENUM_D_TAG
+from elftools.elf.segments import InterpSegment
+from elftools.elf.sections import SymbolTableSection
+from elftools.elf.gnuversions import (
+    GNUVerSymSection, GNUVerDefSection,
+    GNUVerNeedSection,
+    )
+from elftools.elf.relocation import RelocationSection
+from elftools.elf.descriptions import (
+    describe_ei_class, describe_ei_data, describe_ei_version,
+    describe_ei_osabi, describe_e_type, describe_e_machine,
+    describe_e_version_numeric, describe_p_type, describe_p_flags,
+    describe_sh_type, describe_sh_flags,
+    describe_symbol_type, describe_symbol_bind, describe_symbol_visibility,
+    describe_symbol_shndx, describe_reloc_type, describe_dyn_tag,
+    describe_ver_flags,
+    )
+from elftools.elf.constants import E_FLAGS
+from elftools.dwarf.dwarfinfo import DWARFInfo
+from elftools.dwarf.descriptions import (
+    describe_reg_name, describe_attr_value, set_global_machine_arch,
+    describe_CFI_instructions, describe_CFI_register_rule,
+    describe_CFI_CFA_rule,
+    )
+from elftools.dwarf.constants import (
+    DW_LNS_copy, DW_LNS_set_file, DW_LNE_define_file)
+from elftools.dwarf.callframe import CIE, FDE
+
+raw_output = False
+pcidb = None
+
+#===========================================
+
+class Vendor:
+    """
+    Class for vendors. This is the top level class
+    for the devices belong to a specific vendor.
+    self.devices is the device dictionary
+    subdevices are in each device.
+    """
+    def __init__(self, vendorStr):
+        """
+        Class initializes with the raw line from pci.ids
+        Parsing takes place inside __init__
+        """
+        self.ID = vendorStr.split()[0]
+        self.name = vendorStr.replace("%s " % self.ID,"").rstrip()
+        self.devices = {}
+
+    def addDevice(self, deviceStr):
+        """
+        Adds a device to self.devices
+        takes the raw line from pci.ids
+        """
+        s = deviceStr.strip()
+        devID = s.split()[0]
+        if devID in self.devices:
+            pass
+        else:
+            self.devices[devID] = Device(deviceStr)
+
+    def report(self):
+        print self.ID, self.name
+        for id, dev in self.devices.items():
+            dev.report()
+
+    def find_device(self, devid):
+        # convert to a hex string and remove 0x
+        devid = hex(devid)[2:]
+        try:
+            return self.devices[devid]
+        except:
+            return Device("%s (Unknown Device %s)" % (devid, devid)) 
+
+class Device:
+    def __init__(self, deviceStr):
+        """
+        Class for each device.
+        Each vendor has its own devices dictionary.
+        """
+        s = deviceStr.strip()
+        self.ID = s.split()[0]
+        self.name = s.replace("%s  " % self.ID,"")
+        self.subdevices = {}
+
+    def report(self):
+        print "\t%s\t%s" % (self.ID, self.name)
+        for subID, subdev in self.subdevices.items():
+            subdev.report()
+
+    def addSubDevice(self, subDeviceStr):
+        """
+        Adds a subvendor, subdevice to device.
+        Uses raw line from pci.ids
+        """
+        s = subDeviceStr.strip()
+        spl = s.split()
+        subVendorID  = spl[0]
+        subDeviceID  = spl[1]
+        subDeviceName = s.split("  ")[-1]
+        devID = "%s:%s" % (subVendorID,subDeviceID)
+        self.subdevices[devID] = SubDevice(subVendorID,subDeviceID,subDeviceName)
+
+    def find_subid(self, subven, subdev):
+        subven = hex(subven)[2:]
+        subdev = hex(subdev)[2:]
+        devid ="%s:%s" % (subven, subdev)
+
+        try:
+            return self.subdevices[devid]
+        except:
+            if (subven == "ffff" and subdev == "ffff"):
+                return SubDevice("ffff", "ffff", "(All Subdevices)");
+            else:
+                return SubDevice(subven, subdev, "(Unknown Subdevice)")
+        
+
+class SubDevice:
+    """
+    Class for subdevices.
+    """
+    def __init__(self, vendor, device, name):
+        """
+        Class initializes with vendorid, deviceid and name
+        """
+        self.vendorID = vendor
+        self.deviceID = device
+        self.name = name
+
+    def report(self):
+        print "\t\t%s\t%s\t%s" % (self.vendorID, self.deviceID,self.name)
+
+class PCIIds:
+    """
+    Top class for all pci.ids entries.
+    All queries will be asked to this class.
+    PCIIds.vendors["0e11"].devices["0046"].subdevices["0e11:4091"].name  =  "Smart Array 6i"
+    """
+    def __init__(self, filename):
+        """
+        Prepares the directories.
+        Checks local data file.
+        Tries to load from local, if not found, downloads from web
+        """
+        self.version = ""
+        self.date = ""
+        self.vendors = {}
+        self.contents = None
+        self.readLocal(filename)
+        self.parse()
+
+    def reportVendors(self):
+        """Reports the vendors
+        """
+        for vid, v in self.vendors.items():
+            print v.ID, v.name
+
+    def report(self, vendor = None):
+        """
+        Reports everything for all vendors or a specific vendor
+        PCIIds.report()  reports everything
+        PCIIDs.report("0e11") reports only "Compaq Computer Corporation"
+        """
+        if vendor != None:
+            self.vendors[vendor].report()
+        else:
+            for vID, v in self.vendors.items():
+                v.report()
+
+    def find_vendor(self, vid):
+        # convert vid to a hex string and remove the 0x
+        vid = hex(vid)[2:]
+
+        try:
+            return self.vendors[vid]
+        except: 
+            return Vendor("%s (Unknown Vendor %s)" % (vid, vid))
+
+    def findDate(self, content):
+        for l in content:
+            if l.find("Date:") > -1:
+                return l.split()[-2].replace("-", "")
+        return None
+
+    def parse(self):
+        if len(self.contents) < 1:
+            print "data/%s-pci.ids not found" % self.date
+        else:
+            vendorID = ""
+            deviceID = ""
+            for l in self.contents:
+                if l[0] == "#":
+                    continue
+                elif len(l.strip()) == 0:
+                    continue
+                else:
+                    if l.find("\t\t") == 0:
+                        self.vendors[vendorID].devices[deviceID].addSubDevice(l)
+                    elif l.find("\t") == 0:
+                        deviceID = l.strip().split()[0]
+                        self.vendors[vendorID].addDevice(l)
+                    else:
+                        vendorID = l.split()[0]
+                        self.vendors[vendorID] = Vendor(l)
+
+    def readLocal(self, filename):
+        """
+        Reads the local file
+        """
+        self.contents = open(filename).readlines()
+        self.date = self.findDate(self.contents)
+
+    def loadLocal(self):
+        """
+        Loads database from local. If there is no file,
+        it creates a new one from web
+        """
+        self.date = idsfile[0].split("/")[1].split("-")[0]
+        self.readLocal()
+
+
+
+#=======================================
+
+def search_file(filename, search_path):
+    """ Given a search path, find file with requested name """
+    for path in string.split(search_path, ":"):
+        candidate = os.path.join(path, filename)
+        if os.path.exists(candidate): return os.path.abspath(candidate)
+    return None
+
+class ReadElf(object):
+    """ display_* methods are used to emit output into the output stream
+    """
+    def __init__(self, file, output):
+        """ file:
+                stream object with the ELF file to read
+
+            output:
+                output stream to write to
+        """
+        self.elffile = ELFFile(file)
+        self.output = output
+
+        # Lazily initialized if a debug dump is requested
+        self._dwarfinfo = None
+
+        self._versioninfo = None
+
+    def _section_from_spec(self, spec):
+        """ Retrieve a section given a "spec" (either number or name).
+            Return None if no such section exists in the file.
+        """
+        try:
+            num = int(spec)
+            if num < self.elffile.num_sections():
+                return self.elffile.get_section(num)
+            else:
+                return None
+        except ValueError:
+            # Not a number. Must be a name then
+            return self.elffile.get_section_by_name(str2bytes(spec))
+
+    def pretty_print_pmdinfo(self, pmdinfo):
+        global pcidb
+
+        for i in pmdinfo["pci_ids"]:
+            vendor = pcidb.find_vendor(i[0])
+            device = vendor.find_device(i[1])
+            subdev = device.find_subid(i[2], i[3])
+            print("%s (%s) : %s (%s) %s" % (vendor.name, vendor.ID, device.name, device.ID, subdev.name))
+   
+    def parse_pmd_info_string(self, mystring):
+        global raw_output
+        global pcidb
+
+        optional_pmd_info = [{'id': 'params', 'tag' : 'PMD PARAMETERS'}]
+
+        i = mystring.index("=");
+        mystring = mystring[i+2:]
+        pmdinfo = json.loads(mystring)
+
+        if raw_output:
+            print(pmdinfo)
+            return
+
+        print("PMD NAME: " + pmdinfo["name"])
+        print("PMD TYPE: " + pmdinfo["type"])
+        for i in optional_pmd_info:
+            try:
+                print("%s: %s" % (i['tag'], pmdinfo[i['id']]))
+            except KeyError as e:
+                continue
+
+        if (pmdinfo["type"] == "PMD_PDEV"):
+            print("PMD HW SUPPORT:")
+            if pcidb != None:
+                self.pretty_print_pmdinfo(pmdinfo)
+            else:
+                print("VENDOR\t DEVICE\t SUBVENDOR\t SUBDEVICE")
+                for i in pmdinfo["pci_ids"]:
+                    print("0x%04x\t 0x%04x\t 0x%04x\t\t 0x%04x" % (i[0], i[1], i[2], i[3]))
+
+        print("")
+
+
+    def display_pmd_info_strings(self, section_spec):
+        """ Display a strings dump of a section. section_spec is either a
+            section number or a name.
+        """
+        section = self._section_from_spec(section_spec)
+        if section is None:
+            return
+
+
+        data = section.data()
+        dataptr = 0
+
+        while dataptr < len(data):
+            while ( dataptr < len(data) and
+                    not (32 <= byte2int(data[dataptr]) <= 127)):
+                dataptr += 1
+
+            if dataptr >= len(data):
+                break
+
+            endptr = dataptr
+            while endptr < len(data) and byte2int(data[endptr]) != 0:
+                endptr += 1
+
+            mystring = bytes2str(data[dataptr:endptr])
+            rc = mystring.find("PMD_INFO_STRING")
+            if (rc != -1):
+                self.parse_pmd_info_string(mystring)
+
+            dataptr = endptr
+
+    def get_dt_runpath(self, dynsec):
+        for tag in dynsec.iter_tags():
+            if tag.entry.d_tag == 'DT_RUNPATH':
+                return tag.runpath
+        return ""
+
+
+    def process_dt_needed_entries(self):
+        """ Look to see if there are any DT_NEEDED entries in the binary
+            And process those if there are
+        """
+        global raw_output
+        runpath = ""
+        ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+        if (ldlibpath == None):
+            ldlibpath = ""
+
+        dynsec = self._section_from_spec(".dynamic")
+        try:
+            runpath = self.get_dt_runpath(dynsec)
+        except AttributeError:
+            # dynsec is None, just return
+            return
+
+        for tag in dynsec.iter_tags():
+            if tag.entry.d_tag == 'DT_NEEDED':
+                rc = tag.needed.find("librte_pmd")
+                if (rc != -1):
+                    library = search_file(tag.needed,
+                            runpath + ":" + ldlibpath +
+                            ":/usr/lib:/lib")
+                    if (library != None):
+                        if (raw_output == False):
+                            print("Scanning %s for pmd information" % library)
+                        with open(library, 'rb') as file:
+                            libelf = ReadElf(file, sys.stdout)
+                            libelf.process_dt_needed_entries()
+                            libelf.display_pmd_info_strings(".rodata")
+                            file.close()
+
+
+def main(stream=None):
+    global raw_output
+    global pcidb
+
+    optparser = OptionParser(
+            usage='usage: %prog [-h|-r] <elf-file>',
+            description="Dump pmd hardware support info",
+            add_help_option=True,
+            prog='pmdinfo.py')
+    optparser.add_option('-r', '--raw',
+            action='store_true', dest='raw_output',
+            help='Dump raw json strings')
+    optparser.add_option("-d", "--pcidb", dest="pcifile",
+            help="", metavar="FILE")
+
+    options, args = optparser.parse_args()
+
+    if options.raw_output:
+        raw_output = True
+
+    if options.pcifile:
+        pcidb = PCIIds(options.pcifile)
+        if pcidb == None:
+            print("Pci DB file not found")
+            exit(1)
+   
+    ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+    if (ldlibpath == None):
+        ldlibpath = ""
+
+    if (os.path.exists(args[0]) == True):
+        myelffile = args[0]
+    else:
+        myelffile = search_file(args[0], ldlibpath + ":/usr/lib:/lib")
+
+    if (myelffile == None):
+        print("File not found")
+        sys.exit(1)
+
+    with open(myelffile, 'rb') as file:
+        try:
+            readelf = ReadElf(file, sys.stdout)
+            readelf.process_dt_needed_entries()
+            readelf.display_pmd_info_strings(".rodata") 
+            sys.exit(0)
+ 
+        except ELFError as ex:
+            sys.stderr.write('ELF error: %s\n' % ex)
+            sys.exit(1)
+
+
+#-------------------------------------------------------------------------------
+if __name__ == '__main__':
+    main()
+
+
-- 
2.5.5

Re: [PATCH 4/4] pmd_hw_support.py: Add tool to query binaries for hw support information

[Panu Matilainen]

On 05/18/2016 04:48 PM, Neil Horman wrote:
> On Wed, May 18, 2016 at 03:48:12PM +0300, Panu Matilainen wrote:
>> On 05/18/2016 03:03 PM, Neil Horman wrote:
>>> On Wed, May 18, 2016 at 02:48:30PM +0300, Panu Matilainen wrote:
>>>> On 05/16/2016 11:41 PM, Neil Horman wrote:
>>>>> This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
>>>>> and, if found parses the remainder of the string as a json encoded string,
>>>>> outputting the results in either a human readable or raw, script parseable
>>>>> format
>>>>>
>>>>> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
>>>>> CC: Bruce Richardson <bruce.richardson@intel.com>
>>>>> CC: Thomas Monjalon <thomas.monjalon@6wind.com>
>>>>> CC: Stephen Hemminger <stephen@networkplumber.org>
>>>>> CC: Panu Matilainen <pmatilai@redhat.com>
>>>>> ---
>>>>>  tools/pmd_hw_support.py | 174 ++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>  1 file changed, 174 insertions(+)
>>>>>  create mode 100755 tools/pmd_hw_support.py
>>>>>
>>>>> diff --git a/tools/pmd_hw_support.py b/tools/pmd_hw_support.py
>>>>> new file mode 100755
>>>>> index 0000000..0669aca
>>>>> --- /dev/null
>>>>> +++ b/tools/pmd_hw_support.py
>>>>> @@ -0,0 +1,174 @@
>>>>> +#!/usr/bin/python3
>>>>
>>>> I think this should use /usr/bin/python to be consistent with the other
>>>> python scripts, and like the others work with python 2 and 3. I only tested
>>>> it with python2 after changing this and it seemed to work fine so the
>>>> compatibility side should be fine as-is.
>>>>
>>> Sure, I can change the python executable, that makes sense.
>>>
>>>> On the whole, AFAICT the patch series does what it promises, and works for
>>>> both static and shared linkage. Using JSON formatted strings in an ELF
>>>> section is a sound working technical solution for the storage of the data.
>>>> But the difference between the two cases makes me wonder about this all...
>>> You mean the difference between checking static binaries and dynamic binaries?
>>> yes, there is some functional difference there
>>>
>>>>
>>>> For static library build, you'd query the application executable, eg
>>> Correct.
>>>
>>>> testpmd, to get the data out. For a shared library build, that method gives
>>>> absolutely nothing because the data is scattered around in individual
>>>> libraries which might be just about wherever, and you need to somehow
>>> Correct, I figured that users would be smart enough to realize that with
>>> dynamically linked executables, they would need to look at DSO's, but I agree,
>>> its a glaring diffrence.
>>
>> Being able to look at DSOs is good, but expecting the user to figure out
>> which DSOs might be loaded and not and where to look is going to be well
>> above many users. At very least it's not what I would call user-friendly.
>>
> I disagree, there is no linkage between an application and the dso's it opens
> via dlopen that is exportable.  The only way to handle that is to have a
> standard search path for the pmd_hw_info python script.  Thats just like modinfo
> works (i.e. "modinfo bnx2" finds the bnx2 module for the running kernel).  We
> can of course do something simmilar, but we have no existing implicit path
> information to draw from to do that (because you can have multiple dpdk installs
> side by side).  The only way around that is to explicitly call out the path on
> the command line.

There's no telling what libraries user might load at runtime with -D, 
that is true for both static and shared libraries.

When CONFIG_RTE_EAL_PMD_PATH is set, as it is likely to be on distro 
builds, you *know* that everything in that path will be loaded on 
runtime regardless of what commandline options there might be so the 
situation is actually on par with static builds. Of course you still 
dont know about ones added with -D but that's a limitation of any 
solution that works without actually running the app.

>
>>>> discover the location + correct library files to be able to query that. For
>>>> the shared case, perhaps the script could be taught to walk files in
>>>> CONFIG_RTE_EAL_PMD_PATH to give in-the-ballpark correct/identical results
>>> My initial thought would be to run ldd on the executable, and use a heuristic to
>>> determine relevant pmd DSO's, and then feed each of those through the python
>>> script.  I didn't want to go to that trouble unless there was consensus on it
>>> though.
>>
>> Problem is, ldd doesn't know about them either because the pmds are not
>> linked to the executables at all anymore. They could be force-linked of
>> course, but that means giving up the flexibility of plugins, which IMO is a
>> no-go. Except maybe as an option, but then that would be a third case to
>> support.
>>
> Thats not true at all, or at least its a perfectly valid way to link the DSO's
> in at link time via -lrte_pmd_<driver>.  Its really just the dlopen case we need
> to worry about.  I would argue that, if they're not explicitly linked in like
> that, then its correct to indicate that an application supports no hardware,
> because it actually doesn't, it only supports the pmds that it chooses to list
> on the command line.  And if a user is savy enough to specify a pmd on the
> application command line, then they are perfectly capable of specifying the same
> path to the hw_info script.

Yes you can force-link apps to every driver on existence, but it 
requires not just linking but using --whole-archive. The apps in DPDK 
itself dont in shared link setup (take a look at testpmd) and I think 
its for a damn good reason - the drivers are plugins and that's how 
plugins are expected to work: they are not linked to, they reside in a 
specific path which is scanned at runtime and plugins loaded to provide 
extra functionality.

>>
>>>
>>>> when querying the executable as with static builds. If identical operation
>>>> between static and shared versions is a requirement (without running the app
>>>> in question) then query through the executable itself is practically the
>>>> only option. Unless some kind of (auto-generated) external config file
>>>> system ala kernel depmod / modules.dep etc is brought into the picture.
>>> Yeah, I'm really trying to avoid that, as I think its really not a typical part
>>> of how user space libraries are interacted with.
>>>
>>>>
>>>> For shared library configurations, having the data in the individual pmds is
>>>> valuable as one could for example have rpm autogenerate provides from the
>>>> data to ease/automate installation (in case of split packaging and/or 3rd
>>>> party drivers). And no doubt other interesting possibilities. With static
>>>> builds that kind of thing is not possible.
>>> Right.
>>>
>>> Note, this also leaves out PMD's that are loaded dynamically (i.e. via dlopen).
>>> For those situations I don't think we have any way of 'knowing' that the
>>> application intends to use them.
>>
>> Hence my comment about CONFIG_RTE_EAL_PMD_PATH above, it at least provides a
>> reasonable heuristic of what would be loaded by the app when run. But
>> ultimately the only way to know what hardware is supported at a given time
>> is to run an app which calls rte_eal_init() to load all the drivers that are
>> present and work from there, because besides CONFIG_RTE_EAL_PMD_PATH this
>> can be affected by runtime commandline switches and applies to both shared
>> and static builds.
>>
> I'm not sure I agree with that.  Its clearly tempting to use, but its not
> at all guaranteed to be accurate (the default is just set to "", and there is no
> promise anyone will set it properly).

The promise is that shared builds are barely functional unless its set 
correctly, because zero drivers are linked to testpmd in shared config. 
So you're kinda likely to notice if its not set.

It defaults to empty because at the time there was no standard 
installation available at that time. Setting a reasonable default is 
tricky still because it needs to be set before build whereas install 
path is set at install time.

> And it also requires that the binary will
> be tied to a specific release.  I really think that, given the fact that
> distributions generally try to package dpdk in such a way that multiple dpdk
> versions might be available, the right solution is to just require a full path
> specification if you want to get hw info for a DSO that is dynamically loaded
> via dlopen from the command line.  Otherwise you're going to fall into this trap
> where you might be looking implicitly at an older version of the PMD while your
> application may use a newer version.

If there are multiple dpdk versions available then they just need to 
have separate PMD paths, but that's not a problem.

>>>>
>>>> Calling up on the list of requirements from
>>>> http://dpdk.org/ml/archives/dev/2016-May/038324.html, I see a pile of
>>>> technical requirements but perhaps we should stop for a moment to think
>>>> about the use-cases first?
>>>
>>> To ennumerate the list:
>>>
>>> - query all drivers in static binary or shared library (works)
>>> - stripping resiliency (works)
>>> - human friendly (works)
>>> - script friendly (works)
>>> - show driver name (works)
>>> - list supported device id / name (works)
>>> - list driver options (not yet, but possible)
>>> - show driver version if available (nope, but possible)
>>> - show dpdk version (nope, but possible)
>>> - show kernel dependencies (vfio/uio_pci_generic/etc) (nope)
>>> - room for extra information? (works)
>>>
>>> Of the items that are missing, I've already got a V2 started that can do driver
>>> options, and is easier to expand.  Adding in the the DPDK and PMD version should
>>> be easy (though I think they can be left out, as theres currently no globaly
>>> defined DPDK release version, its all just implicit, and driver versions aren't
>>> really there either).  I'm also hesitant to include kernel dependencies without
>>> defining exactly what they mean (just module dependencies, or feature
>>> enablement, or something else?).  Once we define it though, adding it can be
>>> easy.
>>
>> Yup. I just think the shared/static difference needs to be sorted out
>> somehow, eg requiring user to know about DSOs is not human-friendly at all.
>> That's why I called for the higher level use-cases in my previous email.
>>
>
> I disagree with that.  While its reasonable to give users the convienience of
> scanning the DT_NEEDED entries of a binary and scanning those DSO's.  If a user

Scanning DT_NEEDED is of course ok sane and right thing to do, its just 
not sufficient.

> has to specify the PMD to load in an application (either on the command line or
> via a configuration file), then its reasonable assume that they (a) know where

But when the PMD path is set (as it should be on a distro build), this 
is all automatic with zero action or extra config required from the user.

> to find that pmd and (b) are savy enough to pass that same path to a hardware
> info tool.  Thats the exact same way that modinfo works (save for the fact that
> modinfo can implicitly check the running kernel version to find the appropriate
> path for a modular driver).
>
> The only other thing that seems reasonable to me would be to scan
> LD_LIBRARY_PATH.  I would assume that, if an application is linked dynamically,
> the individual DSO's (librte_sched.so, etc), need to be in LD_LIBRARY_PATH.  If
> thats the case, then we can assume that the appropriate PMD DSO's are there too,
> and we can search there.  We can also check the standard /usr/lib and /lib paths
> with that.  I think that would make fairly good sense.

You really don't want go crashing through the potentially thousands of 
libraries in the standard library path going "is it a pmd, no, is it a 
pmd, no..."

	- Panu -

>
>>>
>>> I'll have a v2 posted soon, with the consensus corrections you have above, as
>>> well as some other cleanups
>>>
>>> Best
>>> Neil
>>>
>>>>
>>>> To name some from the top of my head:
>>>> - user wants to know whether the hardware on the system is supported
>>>> - user wants to know which package(s) need to be installed to support the
>>>> system hardware
>>>> - user wants to list all supported hardware before going shopping
>>>> - [what else?]
>>>>
>>>> ...and then think how these things would look like from the user
>>>> perspective, in the light of the two quite dramatically differing cases of
>>>> static vs shared linkage.
>>
>>
>> 	- Panu -
>>

Re: [PATCHv2 1/4] pmdinfogen: Add buildtools and pmdinfogen utility

[Panu Matilainen]

On 05/19/2016 12:08 AM, Neil Horman wrote:
[...]
> +		if (strcmp(secname, ".modinfo") == 0) {
> +			if (nobits)
> +				fprintf(stderr, "%s has NOBITS .modinfo\n", filename);
> +			info->modinfo = (void *)hdr + sechdrs[i].sh_offset;
> +			info->modinfo_len = sechdrs[i].sh_size;
> +		} else if (strcmp(secname, "__ksymtab") == 0)
> +			info->export_sec = i;
> +		else if (strcmp(secname, "__ksymtab_unused") == 0)
> +			info->export_unused_sec = i;
> +		else if (strcmp(secname, "__ksymtab_gpl") == 0)
> +			info->export_gpl_sec = i;
> +		else if (strcmp(secname, "__ksymtab_unused_gpl") == 0)
> +			info->export_unused_gpl_sec = i;
> +		else if (strcmp(secname, "__ksymtab_gpl_future") == 0)
> +			info->export_gpl_future_sec = i;
> +

Looks like a leftover from kernel modpost.c, not needed in DPDK.

	- Panu -

Re: [PATCHv2 2/4] drivers: Update driver registration macro usage

[Panu Matilainen]

On 05/19/2016 12:08 AM, Neil Horman wrote:
> Modify the PMD_REGISTER_DRIVER macro, bifurcating it into two
> (PMD_REGISTER_DRIVER_PDEV and PMD_REGISTER_DRIVER_VDEV.  Both of these do the
> same thing the origional macro did, but both add the definition of a string
> variable that informs interested parties of the name of the pmd, and the former
> also defines an second string that holds the symbol name of the pci table that
> is registered by this pmd.
>
> pmdinfo uses this information to extract hardware support from an object file
> and create a json string to make hardware support info discoverable later.
>
> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> CC: Bruce Richardson <bruce.richardson@intel.com>
> CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> CC: Stephen Hemminger <stephen@networkplumber.org>
> CC: Panu Matilainen <pmatilai@redhat.com>
> ---
>  drivers/Makefile                           |  2 ++
>  drivers/crypto/aesni_gcm/aesni_gcm_pmd.c   |  4 +++-
>  drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c |  4 +++-
>  drivers/crypto/null/null_crypto_pmd.c      |  4 +++-
>  drivers/crypto/qat/rte_qat_cryptodev.c     |  4 +++-
>  drivers/crypto/snow3g/rte_snow3g_pmd.c     |  4 +++-
>  drivers/net/af_packet/rte_eth_af_packet.c  |  4 +++-
>  drivers/net/bnx2x/bnx2x_ethdev.c           |  6 ++++--
>  drivers/net/bonding/rte_eth_bond_pmd.c     |  7 ++++++-
>  drivers/net/cxgbe/cxgbe_ethdev.c           |  4 +++-
>  drivers/net/e1000/em_ethdev.c              |  3 ++-
>  drivers/net/e1000/igb_ethdev.c             |  6 ++++--
>  drivers/net/ena/ena_ethdev.c               |  3 ++-
>  drivers/net/enic/enic_ethdev.c             |  3 ++-
>  drivers/net/fm10k/fm10k_ethdev.c           |  3 ++-
>  drivers/net/i40e/i40e_ethdev.c             |  3 ++-
>  drivers/net/i40e/i40e_ethdev_vf.c          |  3 ++-
>  drivers/net/ixgbe/ixgbe_ethdev.c           |  6 ++++--
>  drivers/net/mlx4/mlx4.c                    |  3 ++-
>  drivers/net/mlx5/mlx5.c                    |  3 ++-
>  drivers/net/mpipe/mpipe_tilegx.c           |  4 ++--
>  drivers/net/nfp/nfp_net.c                  |  3 ++-
>  drivers/net/null/rte_eth_null.c            |  3 ++-
>  drivers/net/pcap/rte_eth_pcap.c            |  4 +++-
>  drivers/net/ring/rte_eth_ring.c            |  3 ++-
>  drivers/net/szedata2/rte_eth_szedata2.c    |  3 ++-
>  drivers/net/vhost/rte_eth_vhost.c          |  3 ++-
>  drivers/net/virtio/virtio_ethdev.c         |  3 ++-
>  drivers/net/vmxnet3/vmxnet3_ethdev.c       |  3 ++-
>  drivers/net/xenvirt/rte_eth_xenvirt.c      |  2 +-
>  lib/librte_eal/common/include/rte_dev.h    | 20 ++++++++++++++++----
>  31 files changed, 93 insertions(+), 37 deletions(-)
>

drivers/net/qede is missing and causes a build failure with a fresh config.

It seems to be missing in v1 but I managed to test it, guess it must've 
been an old .config generated before QEDE got merged.

	- Panu -

Re: [PATCHv2 4/4] pmdinfo.py: Add tool to query binaries for hw and other support information

[Panu Matilainen]

On 05/19/2016 12:08 AM, Neil Horman wrote:
> This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
> and, if found parses the remainder of the string as a json encoded string,
> outputting the results in either a human readable or raw, script parseable
> format
>
> Note that, in the case of dynamically linked applications, pmdinfo.py will scan
> for implicitly linked PMDs by searching the specified binaries .dynamic section
> for DT_NEEDED entries that contain the substring librte_pmd.  The DT_RUNPATH,
> LD_LIBRARY_PATH, /usr/lib and /lib are searched for these libraries, in that
> order

Scanning /usr/lib and /lib does little good on systems where /usr/lib64 
and /lib64 are the standard path, such as x86_64 Fedora / RHEL and 
derivates.

With the path changed (or LD_LIBRARY_PATH set manually), I can confirm 
it works for a shared binary which is --whole-archive linked to all of 
DPDK such as ovs-vswitchd currently is (because it needs to for static 
DPDK linkage and is not aware of plugin autoloading).

It doesn't help testpmd though because its not linked with 
--whole-archive in the shared case, so its not working for the main DPDK 
executable...

In any case, using --whole-archive is a sledgehammer solution at best, 
and against the spirit of shared libs and plugins in particular.

I think the shared linkage case can be solved by exporting the PMD path 
from librte_eal (either through an elf section or c-level symbol) and 
teach the script to detect the case of an executable dynamically linked 
to librte_eal, fish the path from there and then process everything in 
that path.

>
> If a file is specified with no path, it is assumed to be a PMD DSO, and the
> LD_LIBRARY_PATH, /usr/lib/ and /lib is searched for it

Same as above, /usr/lib/ and /lib is incorrect for a large number of 
systems.

>
> Currently the tool can output data in 3 formats:
>
> a) raw, suitable for scripting, where the raw JSON strings are dumped out
> b) table format (default) where hex pci ids are dumped in a table format
> c) pretty, where a user supplied pci.ids file is used to print out vendor and
> device strings

c) is a nice addition. Would be even nicer if it knew the most common 
pci.ids locations so it doesn't need extra arguments in those cases, ie 
see if /usr/share/hwdata/pci.ids or /usr/share/misc/pci.ids exists and 
use that unless overridden on the cli.

	- Panu -

Re: [PATCHv2 2/4] drivers: Update driver registration macro usage

[Neil Horman]

On Thu, May 19, 2016 at 10:58:23AM +0300, Panu Matilainen wrote:
> On 05/19/2016 12:08 AM, Neil Horman wrote:
> > Modify the PMD_REGISTER_DRIVER macro, bifurcating it into two
> > (PMD_REGISTER_DRIVER_PDEV and PMD_REGISTER_DRIVER_VDEV.  Both of these do the
> > same thing the origional macro did, but both add the definition of a string
> > variable that informs interested parties of the name of the pmd, and the former
> > also defines an second string that holds the symbol name of the pci table that
> > is registered by this pmd.
> > 
> > pmdinfo uses this information to extract hardware support from an object file
> > and create a json string to make hardware support info discoverable later.
> > 
> > Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> > CC: Bruce Richardson <bruce.richardson@intel.com>
> > CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> > CC: Stephen Hemminger <stephen@networkplumber.org>
> > CC: Panu Matilainen <pmatilai@redhat.com>
> > ---
> >  drivers/Makefile                           |  2 ++
> >  drivers/crypto/aesni_gcm/aesni_gcm_pmd.c   |  4 +++-
> >  drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c |  4 +++-
> >  drivers/crypto/null/null_crypto_pmd.c      |  4 +++-
> >  drivers/crypto/qat/rte_qat_cryptodev.c     |  4 +++-
> >  drivers/crypto/snow3g/rte_snow3g_pmd.c     |  4 +++-
> >  drivers/net/af_packet/rte_eth_af_packet.c  |  4 +++-
> >  drivers/net/bnx2x/bnx2x_ethdev.c           |  6 ++++--
> >  drivers/net/bonding/rte_eth_bond_pmd.c     |  7 ++++++-
> >  drivers/net/cxgbe/cxgbe_ethdev.c           |  4 +++-
> >  drivers/net/e1000/em_ethdev.c              |  3 ++-
> >  drivers/net/e1000/igb_ethdev.c             |  6 ++++--
> >  drivers/net/ena/ena_ethdev.c               |  3 ++-
> >  drivers/net/enic/enic_ethdev.c             |  3 ++-
> >  drivers/net/fm10k/fm10k_ethdev.c           |  3 ++-
> >  drivers/net/i40e/i40e_ethdev.c             |  3 ++-
> >  drivers/net/i40e/i40e_ethdev_vf.c          |  3 ++-
> >  drivers/net/ixgbe/ixgbe_ethdev.c           |  6 ++++--
> >  drivers/net/mlx4/mlx4.c                    |  3 ++-
> >  drivers/net/mlx5/mlx5.c                    |  3 ++-
> >  drivers/net/mpipe/mpipe_tilegx.c           |  4 ++--
> >  drivers/net/nfp/nfp_net.c                  |  3 ++-
> >  drivers/net/null/rte_eth_null.c            |  3 ++-
> >  drivers/net/pcap/rte_eth_pcap.c            |  4 +++-
> >  drivers/net/ring/rte_eth_ring.c            |  3 ++-
> >  drivers/net/szedata2/rte_eth_szedata2.c    |  3 ++-
> >  drivers/net/vhost/rte_eth_vhost.c          |  3 ++-
> >  drivers/net/virtio/virtio_ethdev.c         |  3 ++-
> >  drivers/net/vmxnet3/vmxnet3_ethdev.c       |  3 ++-
> >  drivers/net/xenvirt/rte_eth_xenvirt.c      |  2 +-
> >  lib/librte_eal/common/include/rte_dev.h    | 20 ++++++++++++++++----
> >  31 files changed, 93 insertions(+), 37 deletions(-)
> > 
> 
> drivers/net/qede is missing and causes a build failure with a fresh config.
> 
> It seems to be missing in v1 but I managed to test it, guess it must've been
> an old .config generated before QEDE got merged.
> 
> 	- Panu -
> 
No, It only got added recently.  I pulled when I started writing this (about two
weeks ago), and it got added during its development.  I'll rebase
Neil

Re: [dpdk-dev, PATCHv2, 2/4] drivers: Update driver registration macro usage

[Jan Viktorin]

Hello Neil,

just few notes...

(sorry if you've recevied this twice, importing mbox files from patchwork
always changes my default From: field)

On Wed, 18 May 2016 17:08:05 -0400
Neil Horman <nhorman@tuxdriver.com> wrote:

> Modify the PMD_REGISTER_DRIVER macro, bifurcating it into two
> (PMD_REGISTER_DRIVER_PDEV and PMD_REGISTER_DRIVER_VDEV.  Both of these do the
> same thing the origional macro did, but both add the definition of a string

I could not find any of those: PMD_REGISTER_DRIVER_PDEV, PMD_REGISTER_DRIVER_VDEV
in this patch. I think the message is misleading...

I am interested as this may lead to merge conflicts when generalizing
rte_pci_device/driver and stuff around it.

> variable that informs interested parties of the name of the pmd, and the former
> also defines an second string that holds the symbol name of the pci table that
> is registered by this pmd.

> 
> pmdinfo uses this information to extract hardware support from an object file
> and create a json string to make hardware support info discoverable later.
> 
> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> CC: Bruce Richardson <bruce.richardson@intel.com>
> CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> CC: Stephen Hemminger <stephen@networkplumber.org>
> CC: Panu Matilainen <pmatilai@redhat.com>
> 
> ---
> drivers/Makefile                           |  2 ++
>  drivers/crypto/aesni_gcm/aesni_gcm_pmd.c   |  4 +++-
>  drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c |  4 +++-
>  drivers/crypto/null/null_crypto_pmd.c      |  4 +++-
>  drivers/crypto/qat/rte_qat_cryptodev.c     |  4 +++-
>  drivers/crypto/snow3g/rte_snow3g_pmd.c     |  4 +++-
>  drivers/net/af_packet/rte_eth_af_packet.c  |  4 +++-
>  drivers/net/bnx2x/bnx2x_ethdev.c           |  6 ++++--
>  drivers/net/bonding/rte_eth_bond_pmd.c     |  7 ++++++-
>  drivers/net/cxgbe/cxgbe_ethdev.c           |  4 +++-
>  drivers/net/e1000/em_ethdev.c              |  3 ++-
>  drivers/net/e1000/igb_ethdev.c             |  6 ++++--
>  drivers/net/ena/ena_ethdev.c               |  3 ++-
>  drivers/net/enic/enic_ethdev.c             |  3 ++-
>  drivers/net/fm10k/fm10k_ethdev.c           |  3 ++-
>  drivers/net/i40e/i40e_ethdev.c             |  3 ++-
>  drivers/net/i40e/i40e_ethdev_vf.c          |  3 ++-
>  drivers/net/ixgbe/ixgbe_ethdev.c           |  6 ++++--
>  drivers/net/mlx4/mlx4.c                    |  3 ++-
>  drivers/net/mlx5/mlx5.c                    |  3 ++-
>  drivers/net/mpipe/mpipe_tilegx.c           |  4 ++--
>  drivers/net/nfp/nfp_net.c                  |  3 ++-
>  drivers/net/null/rte_eth_null.c            |  3 ++-
>  drivers/net/pcap/rte_eth_pcap.c            |  4 +++-
>  drivers/net/ring/rte_eth_ring.c            |  3 ++-
>  drivers/net/szedata2/rte_eth_szedata2.c    |  3 ++-
>  drivers/net/vhost/rte_eth_vhost.c          |  3 ++-
>  drivers/net/virtio/virtio_ethdev.c         |  3 ++-
>  drivers/net/vmxnet3/vmxnet3_ethdev.c       |  3 ++-
>  drivers/net/xenvirt/rte_eth_xenvirt.c      |  2 +-
>  lib/librte_eal/common/include/rte_dev.h    | 20 ++++++++++++++++----
>  31 files changed, 93 insertions(+), 37 deletions(-)
> 

[...]

>  
> -PMD_REGISTER_DRIVER(pmd_xenvirt_drv);
> +PMD_REGISTER_DRIVER(pmd_xenvirt_drv, xenvirt);
> diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
> index f1b5507..871089a 100644
> --- a/lib/librte_eal/common/include/rte_dev.h
> +++ b/lib/librte_eal/common/include/rte_dev.h
> @@ -48,7 +48,7 @@ extern "C" {
>  
>  #include <stdio.h>
>  #include <sys/queue.h>
> -
> +#include <rte_pci.h>

This should be done in an opposite way. The rte_pci.h should include rte_dev.h
(more specific includes a generic one). I've introduced this in the following patch:

 [PATCH v1 01/28] eal: make enum rte_kernel_driver non-PCI specific
 http://dpdk.org/dev/patchwork/patch/12488/

Regards
Jan

>  #include <rte_log.h>
>  
>  __attribute__((format(printf, 2, 0)))
> @@ -178,12 +178,24 @@ int rte_eal_vdev_init(const char *name, const char *args);
>   */
>  int rte_eal_vdev_uninit(const char *name);
>  
> -#define PMD_REGISTER_DRIVER(d)\
> +#define DRIVER_EXPORT_NAME_ARRAY(n, idx) n##idx[] __attribute__((used))
> +
> +#define DRIVER_EXPORT_NAME(d, idx) \
> +static const char DRIVER_EXPORT_NAME_ARRAY(this_pmd_name, idx) = RTE_STR(d);\
> + 
> +#define PMD_REGISTER_DRIVER(d, n)\
>  void devinitfn_ ##d(void);\
>  void __attribute__((constructor, used)) devinitfn_ ##d(void)\
>  {\
> -	rte_eal_driver_register(&d);\
> -}
> +        rte_eal_driver_register(&d);\
> +}\
> +DRIVER_EXPORT_NAME(n, __COUNTER__)
> +
> +#define DRIVER_REGISTER_PCI_TABLE(n, t) \
> +static const char n##_pci_tbl_export[] __attribute__((used)) = RTE_STR(t)
> +
> +#define DRIVER_REGISTER_PARAM_STRING(n, s) \
> +static const char n##_param_string_export[] __attribute__((used)) = s
>  
>  #ifdef __cplusplus
>  }

-- 
   Jan Viktorin                  E-mail: Viktorin@RehiveTech.com
   System Architect              Web:    www.RehiveTech.com
   RehiveTech
   Brno, Czech Republic

Re: [dpdk-dev, PATCHv2, 2/4] drivers: Update driver registration macro usage

[Neil Horman]

On Thu, May 19, 2016 at 12:46:50PM +0200, Jan Viktorin wrote:
> Hello Neil,
> 
> just few notes...
> 
> On Wed, 18 May 2016 17:08:05 -0400
> Neil Horman <nhorman@tuxdriver.com> wrote:
> 
> > Modify the PMD_REGISTER_DRIVER macro, bifurcating it into two
> > (PMD_REGISTER_DRIVER_PDEV and PMD_REGISTER_DRIVER_VDEV.  Both of these do the
> > same thing the origional macro did, but both add the definition of a string
> 

> I could not find any of those: PMD_REGISTER_DRIVER_PDEV, PMD_REGISTER_DRIVER_VDEV
> in this patch. I think the message is misleading...
> 
Forgot to fix up the changelog entry, when I merged these two macros back to the
single PMD_REGISTER_DRIVER

> I am interested as this may lead to merge conflicts when generalizing
> rte_pci_device/driver and stuff around it.
> 
Not sure what you mean by that

> > variable that informs interested parties of the name of the pmd, and the former
> > also defines an second string that holds the symbol name of the pci table that
> > is registered by this pmd.
> 
> > 
> > pmdinfo uses this information to extract hardware support from an object file
> > and create a json string to make hardware support info discoverable later.
> > 
> > Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> > CC: Bruce Richardson <bruce.richardson@intel.com>
> > CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> > CC: Stephen Hemminger <stephen@networkplumber.org>
> > CC: Panu Matilainen <pmatilai@redhat.com>
> > 
> > ---
> > drivers/Makefile                           |  2 ++
> >  drivers/crypto/aesni_gcm/aesni_gcm_pmd.c   |  4 +++-
> >  drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c |  4 +++-
> >  drivers/crypto/null/null_crypto_pmd.c      |  4 +++-
> >  drivers/crypto/qat/rte_qat_cryptodev.c     |  4 +++-
> >  drivers/crypto/snow3g/rte_snow3g_pmd.c     |  4 +++-
> >  drivers/net/af_packet/rte_eth_af_packet.c  |  4 +++-
> >  drivers/net/bnx2x/bnx2x_ethdev.c           |  6 ++++--
> >  drivers/net/bonding/rte_eth_bond_pmd.c     |  7 ++++++-
> >  drivers/net/cxgbe/cxgbe_ethdev.c           |  4 +++-
> >  drivers/net/e1000/em_ethdev.c              |  3 ++-
> >  drivers/net/e1000/igb_ethdev.c             |  6 ++++--
> >  drivers/net/ena/ena_ethdev.c               |  3 ++-
> >  drivers/net/enic/enic_ethdev.c             |  3 ++-
> >  drivers/net/fm10k/fm10k_ethdev.c           |  3 ++-
> >  drivers/net/i40e/i40e_ethdev.c             |  3 ++-
> >  drivers/net/i40e/i40e_ethdev_vf.c          |  3 ++-
> >  drivers/net/ixgbe/ixgbe_ethdev.c           |  6 ++++--
> >  drivers/net/mlx4/mlx4.c                    |  3 ++-
> >  drivers/net/mlx5/mlx5.c                    |  3 ++-
> >  drivers/net/mpipe/mpipe_tilegx.c           |  4 ++--
> >  drivers/net/nfp/nfp_net.c                  |  3 ++-
> >  drivers/net/null/rte_eth_null.c            |  3 ++-
> >  drivers/net/pcap/rte_eth_pcap.c            |  4 +++-
> >  drivers/net/ring/rte_eth_ring.c            |  3 ++-
> >  drivers/net/szedata2/rte_eth_szedata2.c    |  3 ++-
> >  drivers/net/vhost/rte_eth_vhost.c          |  3 ++-
> >  drivers/net/virtio/virtio_ethdev.c         |  3 ++-
> >  drivers/net/vmxnet3/vmxnet3_ethdev.c       |  3 ++-
> >  drivers/net/xenvirt/rte_eth_xenvirt.c      |  2 +-
> >  lib/librte_eal/common/include/rte_dev.h    | 20 ++++++++++++++++----
> >  31 files changed, 93 insertions(+), 37 deletions(-)
> > 
> 
> [...]
> 
> >  
> > -PMD_REGISTER_DRIVER(pmd_xenvirt_drv);
> > +PMD_REGISTER_DRIVER(pmd_xenvirt_drv, xenvirt);
> > diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
> > index f1b5507..871089a 100644
> > --- a/lib/librte_eal/common/include/rte_dev.h
> > +++ b/lib/librte_eal/common/include/rte_dev.h
> > @@ -48,7 +48,7 @@ extern "C" {
> >  
> >  #include <stdio.h>
> >  #include <sys/queue.h>
> > -
> > +#include <rte_pci.h>
> 
> This should be done in an opposite way. The rte_pci.h should include rte_dev.h
> (more specific includes a generic one). I've introduced this in the following patch:
> 
>  [PATCH v1 01/28] eal: make enum rte_kernel_driver non-PCI specific
>  http://dpdk.org/dev/patchwork/patch/12488/
> 
> Regards
> Jan
> 
That seems to be still in flight.  If your change lands by the time this gets
merged I'll change it accordingly.

Re: [PATCHv2 4/4] pmdinfo.py: Add tool to query binaries for hw and other support information

[Neil Horman]

On Thu, May 19, 2016 at 12:02:27PM +0300, Panu Matilainen wrote:
> On 05/19/2016 12:08 AM, Neil Horman wrote:
> > This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
> > and, if found parses the remainder of the string as a json encoded string,
> > outputting the results in either a human readable or raw, script parseable
> > format
> > 
> > Note that, in the case of dynamically linked applications, pmdinfo.py will scan
> > for implicitly linked PMDs by searching the specified binaries .dynamic section
> > for DT_NEEDED entries that contain the substring librte_pmd.  The DT_RUNPATH,
> > LD_LIBRARY_PATH, /usr/lib and /lib are searched for these libraries, in that
> > order
> 
> Scanning /usr/lib and /lib does little good on systems where /usr/lib64 and
> /lib64 are the standard path, such as x86_64 Fedora / RHEL and derivates.
> 
Ah, sorry, forgot the 64 bit variants, I can add those in.

> With the path changed (or LD_LIBRARY_PATH set manually), I can confirm it
> works for a shared binary which is --whole-archive linked to all of DPDK
> such as ovs-vswitchd currently is (because it needs to for static DPDK
> linkage and is not aware of plugin autoloading).
> 
Right, thats why it works, because DPDK always requires --whole-archive for
static linking, and likely always will (see commit
20afd76a504155e947c770783ef5023e87136ad8)

> It doesn't help testpmd though because its not linked with --whole-archive
> in the shared case, so its not working for the main DPDK executable...
> 
This sentence doesn't make sense --whole-archive is only applicable in the
static binary case, and only when linking archive files.

> In any case, using --whole-archive is a sledgehammer solution at best, and
> against the spirit of shared libs and plugins in particular.
> 
It may be a sledgehammer solution, but its the one dpdk uses, and will likely
use in perpituity.

> I think the shared linkage case can be solved by exporting the PMD path from
> librte_eal (either through an elf section or c-level symbol) and teach the
> script to detect the case of an executable dynamically linked to librte_eal,
> fish the path from there and then process everything in that path.
> 
I really disagree with this, because its a half-measure at best.  Yes, if its
set, you will definately get all the shared objects in that directory loaded,
but that is in no way a guarantee that those are the only libraries that get
loaded (the application may load its own independently).  So you're left in this
situation in which you get maybe some of the hardware support an application
offers.  Its also transient.  That is to say, if you configure a plugin
directory and search it when you scan an application, its contents may change
post scan, leading to erroneous results.

The way I see it, we have 3 cases that we need to handle:

1) Statically linked application - in this case, all pmds that are statically
linked in to the application will be reported, so we're good here

2) Dynamically loaded via DT_NEEDED entries - This is effectively the same as a
static linking case, in that we have a list of libraries that must be resolved
at run time, so we are safe to search for and scan the DSO's that the
application ennumerates

3) Dynamically loaded via dlopen - In this case, we don't actually know until
runtime what DSO's are going to get loaded, even if RTE_EAL_PMD_PATH is set,
because the contents of that path can change at arbitrary times.  In this case,
its correct to indicate that the application itself _doesn't_ actually support
the hardware of the PMD's in that path, because until the application is
executed, it has none of the support embodied in any DSO that it loads via
dlopen.  The hardware support travels with the DSO itself, and so its correct to
only display hardware support when the PMD shared library itself is scanned.

Handling case 3 the way I'm proposing is exactly the way the OS does it (that is
to say, it only details hardware support for the module being queried, and you
have to specify the module name to get that).  I don't see there being any
problem with that.

> > 
> > If a file is specified with no path, it is assumed to be a PMD DSO, and the
> > LD_LIBRARY_PATH, /usr/lib/ and /lib is searched for it
> 
> Same as above, /usr/lib/ and /lib is incorrect for a large number of
> systems.
> 
Yeah, I'll add 64 bit detection and correct that path accordingly

> > 
> > Currently the tool can output data in 3 formats:
> > 
> > a) raw, suitable for scripting, where the raw JSON strings are dumped out
> > b) table format (default) where hex pci ids are dumped in a table format
> > c) pretty, where a user supplied pci.ids file is used to print out vendor and
> > device strings
> 
> c) is a nice addition. Would be even nicer if it knew the most common
> pci.ids locations so it doesn't need extra arguments in those cases, ie see
> if /usr/share/hwdata/pci.ids or /usr/share/misc/pci.ids exists and use that
> unless overridden on the cli.
> 
Yeah, I can do that.

> 	- Panu -
> 
> 
> 

Re: [PATCHv2 1/4] pmdinfogen: Add buildtools and pmdinfogen utility

[Neil Horman]

On Thu, May 19, 2016 at 10:51:19AM +0300, Panu Matilainen wrote:
> On 05/19/2016 12:08 AM, Neil Horman wrote:
> [...]
> > +		if (strcmp(secname, ".modinfo") == 0) {
> > +			if (nobits)
> > +				fprintf(stderr, "%s has NOBITS .modinfo\n", filename);
> > +			info->modinfo = (void *)hdr + sechdrs[i].sh_offset;
> > +			info->modinfo_len = sechdrs[i].sh_size;
> > +		} else if (strcmp(secname, "__ksymtab") == 0)
> > +			info->export_sec = i;
> > +		else if (strcmp(secname, "__ksymtab_unused") == 0)
> > +			info->export_unused_sec = i;
> > +		else if (strcmp(secname, "__ksymtab_gpl") == 0)
> > +			info->export_gpl_sec = i;
> > +		else if (strcmp(secname, "__ksymtab_unused_gpl") == 0)
> > +			info->export_unused_gpl_sec = i;
> > +		else if (strcmp(secname, "__ksymtab_gpl_future") == 0)
> > +			info->export_gpl_future_sec = i;
> > +
> 
> Looks like a leftover from kernel modpost.c, not needed in DPDK.
> 
> 	- Panu -
> 
Yup, I'll remove it
Neil

Re: [PATCH 4/4] pmd_hw_support.py: Add tool to query binaries for hw support information

[Neil Horman]

On Thu, May 19, 2016 at 09:08:52AM +0300, Panu Matilainen wrote:
> On 05/18/2016 04:48 PM, Neil Horman wrote:
> > On Wed, May 18, 2016 at 03:48:12PM +0300, Panu Matilainen wrote:
> > > On 05/18/2016 03:03 PM, Neil Horman wrote:
> > > > On Wed, May 18, 2016 at 02:48:30PM +0300, Panu Matilainen wrote:
> > > > > On 05/16/2016 11:41 PM, Neil Horman wrote:
> > > > > > This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
> > > > > > and, if found parses the remainder of the string as a json encoded string,
> > > > > > outputting the results in either a human readable or raw, script parseable
> > > > > > format
> > > > > > 
> > > > > > Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> > > > > > CC: Bruce Richardson <bruce.richardson@intel.com>
> > > > > > CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> > > > > > CC: Stephen Hemminger <stephen@networkplumber.org>
> > > > > > CC: Panu Matilainen <pmatilai@redhat.com>
> > > > > > ---
> > > > > >  tools/pmd_hw_support.py | 174 ++++++++++++++++++++++++++++++++++++++++++++++++
> > > > > >  1 file changed, 174 insertions(+)
> > > > > >  create mode 100755 tools/pmd_hw_support.py
> > > > > > 
> > > > > > diff --git a/tools/pmd_hw_support.py b/tools/pmd_hw_support.py
> > > > > > new file mode 100755
> > > > > > index 0000000..0669aca
> > > > > > --- /dev/null
> > > > > > +++ b/tools/pmd_hw_support.py
> > > > > > @@ -0,0 +1,174 @@
> > > > > > +#!/usr/bin/python3
> > > > > 
> > > > > I think this should use /usr/bin/python to be consistent with the other
> > > > > python scripts, and like the others work with python 2 and 3. I only tested
> > > > > it with python2 after changing this and it seemed to work fine so the
> > > > > compatibility side should be fine as-is.
> > > > > 
> > > > Sure, I can change the python executable, that makes sense.
> > > > 
> > > > > On the whole, AFAICT the patch series does what it promises, and works for
> > > > > both static and shared linkage. Using JSON formatted strings in an ELF
> > > > > section is a sound working technical solution for the storage of the data.
> > > > > But the difference between the two cases makes me wonder about this all...
> > > > You mean the difference between checking static binaries and dynamic binaries?
> > > > yes, there is some functional difference there
> > > > 
> > > > > 
> > > > > For static library build, you'd query the application executable, eg
> > > > Correct.
> > > > 
> > > > > testpmd, to get the data out. For a shared library build, that method gives
> > > > > absolutely nothing because the data is scattered around in individual
> > > > > libraries which might be just about wherever, and you need to somehow
> > > > Correct, I figured that users would be smart enough to realize that with
> > > > dynamically linked executables, they would need to look at DSO's, but I agree,
> > > > its a glaring diffrence.
> > > 
> > > Being able to look at DSOs is good, but expecting the user to figure out
> > > which DSOs might be loaded and not and where to look is going to be well
> > > above many users. At very least it's not what I would call user-friendly.
> > > 
> > I disagree, there is no linkage between an application and the dso's it opens
> > via dlopen that is exportable.  The only way to handle that is to have a
> > standard search path for the pmd_hw_info python script.  Thats just like modinfo
> > works (i.e. "modinfo bnx2" finds the bnx2 module for the running kernel).  We
> > can of course do something simmilar, but we have no existing implicit path
> > information to draw from to do that (because you can have multiple dpdk installs
> > side by side).  The only way around that is to explicitly call out the path on
> > the command line.
> 
> There's no telling what libraries user might load at runtime with -D, that
> is true for both static and shared libraries.
> 
I agree.

> When CONFIG_RTE_EAL_PMD_PATH is set, as it is likely to be on distro builds,
> you *know* that everything in that path will be loaded on runtime regardless
> of what commandline options there might be so the situation is actually on
> par with static builds. Of course you still dont know about ones added with
> -D but that's a limitation of any solution that works without actually
> running the app.
> 
Its not on ours, as the pmd libraries get placed in the same directory as every
other dpdk library, and no one wants to try (and fail to load
rte_sched/rte_acl/etc twice, or deal with the fallout of trying to do so, or
adjust the packaging so that pmds are placed in their own subdirectory, or
handle the need for multiuser support.

Using CONFIG_RTE_EAL_PMD_PATH also doesn't account for directory changes.  This
use case:
1) run pmdinfo <app>
2) remove DSOs from RTE_EAL_PMD_PATH
3) execute <app>

leads to erroneous results, as hardware support that was reported in (1) is no
longer available at (3)

It also completely misses any libraries that we load via the -d option on the
command line, which won't be included in RTE_EAL_PMD_PATH, so following that
path is a half measure at best, and I think that leads to erroneous results.

> > 
> > > > > discover the location + correct library files to be able to query that. For
> > > > > the shared case, perhaps the script could be taught to walk files in
> > > > > CONFIG_RTE_EAL_PMD_PATH to give in-the-ballpark correct/identical results
> > > > My initial thought would be to run ldd on the executable, and use a heuristic to
> > > > determine relevant pmd DSO's, and then feed each of those through the python
> > > > script.  I didn't want to go to that trouble unless there was consensus on it
> > > > though.
> > > 
> > > Problem is, ldd doesn't know about them either because the pmds are not
> > > linked to the executables at all anymore. They could be force-linked of
> > > course, but that means giving up the flexibility of plugins, which IMO is a
> > > no-go. Except maybe as an option, but then that would be a third case to
> > > support.
> > > 
> > Thats not true at all, or at least its a perfectly valid way to link the DSO's
> > in at link time via -lrte_pmd_<driver>.  Its really just the dlopen case we need
> > to worry about.  I would argue that, if they're not explicitly linked in like
> > that, then its correct to indicate that an application supports no hardware,
> > because it actually doesn't, it only supports the pmds that it chooses to list
> > on the command line.  And if a user is savy enough to specify a pmd on the
> > application command line, then they are perfectly capable of specifying the same
> > path to the hw_info script.
> 
> Yes you can force-link apps to every driver on existence, but it requires
> not just linking but using --whole-archive.
For the static case, yes, and thats what DPDK does, and likely will in
perpituity, unless there is a major architectural change in the project (see
commit 20afd76a504155e947c770783ef5023e87136ad8).

> The apps in DPDK itself dont in
> shared link setup (take a look at testpmd) and I think its for a damn good
> reason - the drivers are plugins and that's how plugins are expected to
> work: they are not linked to, they reside in a specific path which is

I know, I'm the one that made that change when we introduced the
PMD_REGISTER_DRIVER macro :).  That doesn't mean its not a valid case when
building apps, and one that we can take advantage of opportunistically.  There
are three cases we have to handle:

1) Static linking - This is taken care of
2) Dynamic linking via DT_NEEDED entries - this is taken care of
3) Dynamic linking via dlopen - This is what we're discussing here

> scanned at runtime and plugins loaded to provide extra functionality.
Its the runtime part that makes this non-functional.  Just because you scan
all the DSO's in the RTE_EAL_PATH, doesn't mean they will be there when the app
is run, nor does it mean you will get a comprehensive list of hardware support,
because it doesn't include additional paths/DSO's added via -d.  I would much
rather have users understand that an app has _no_ hardware support if it uses
DSO's, because the hardware support is included with the DSO's themself, not the
application (saving for the DT_NEEDED case above, where application execution is
predicated on the availability of those shared objects)

> 
> > > 
> > > > 
> > > > > when querying the executable as with static builds. If identical operation
> > > > > between static and shared versions is a requirement (without running the app
> > > > > in question) then query through the executable itself is practically the
> > > > > only option. Unless some kind of (auto-generated) external config file
> > > > > system ala kernel depmod / modules.dep etc is brought into the picture.
> > > > Yeah, I'm really trying to avoid that, as I think its really not a typical part
> > > > of how user space libraries are interacted with.
> > > > 
> > > > > 
> > > > > For shared library configurations, having the data in the individual pmds is
> > > > > valuable as one could for example have rpm autogenerate provides from the
> > > > > data to ease/automate installation (in case of split packaging and/or 3rd
> > > > > party drivers). And no doubt other interesting possibilities. With static
> > > > > builds that kind of thing is not possible.
> > > > Right.
> > > > 
> > > > Note, this also leaves out PMD's that are loaded dynamically (i.e. via dlopen).
> > > > For those situations I don't think we have any way of 'knowing' that the
> > > > application intends to use them.
> > > 
> > > Hence my comment about CONFIG_RTE_EAL_PMD_PATH above, it at least provides a
> > > reasonable heuristic of what would be loaded by the app when run. But
> > > ultimately the only way to know what hardware is supported at a given time
> > > is to run an app which calls rte_eal_init() to load all the drivers that are
> > > present and work from there, because besides CONFIG_RTE_EAL_PMD_PATH this
> > > can be affected by runtime commandline switches and applies to both shared
> > > and static builds.
> > > 
> > I'm not sure I agree with that.  Its clearly tempting to use, but its not
> > at all guaranteed to be accurate (the default is just set to "", and there is no
> > promise anyone will set it properly).
> 
> The promise is that shared builds are barely functional unless its set
> correctly, because zero drivers are linked to testpmd in shared config. So
> you're kinda likely to notice if its not set.
> 
You're twisting the meaning of 'barely functional' here.  I agree that shared
builds are barely functional, because they have no self-contained hardware
support, and as such, running pmdinfo.py on such an application should report
exactly that.

That said, in order to run, and DPDK application built to use shared libraries
has to use one of two methods to obtain hardware support

A) direct shared linking (the DT_NEEDED case) - This case is handled, and we
report hardware support when found, as the application won't run unless those
libraries are resolved

b) dynamic loading via dlopen - This case shouldn't be handled, because the
application in reality doesn't support  any hardware.  Hardware support is
garnered at run time when the EAL_PMD_PATH (and any other paths added via the -d
option) are scanned.  In this case, pmdinfo shouldn't report any hardware
support, it should only do so if the pmd DSO is queried directly.

> It defaults to empty because at the time there was no standard installation
> available at that time. Setting a reasonable default is tricky still because
> it needs to be set before build whereas install path is set at install time.
> 
Exactly, which is why distros don't use it.  It also neglects the multiuser case
(in which different users may want to load different hardware support).

> > And it also requires that the binary will
> > be tied to a specific release.  I really think that, given the fact that
> > distributions generally try to package dpdk in such a way that multiple dpdk
> > versions might be available, the right solution is to just require a full path
> > specification if you want to get hw info for a DSO that is dynamically loaded
> > via dlopen from the command line.  Otherwise you're going to fall into this trap
> > where you might be looking implicitly at an older version of the PMD while your
> > application may use a newer version.
> 
> If there are multiple dpdk versions available then they just need to have
> separate PMD paths, but that's not a problem.
> 
> > > > > 
> > > > > Calling up on the list of requirements from
> > > > > http://dpdk.org/ml/archives/dev/2016-May/038324.html, I see a pile of
> > > > > technical requirements but perhaps we should stop for a moment to think
> > > > > about the use-cases first?
> > > > 
> > > > To ennumerate the list:
> > > > 
> > > > - query all drivers in static binary or shared library (works)
> > > > - stripping resiliency (works)
> > > > - human friendly (works)
> > > > - script friendly (works)
> > > > - show driver name (works)
> > > > - list supported device id / name (works)
> > > > - list driver options (not yet, but possible)
> > > > - show driver version if available (nope, but possible)
> > > > - show dpdk version (nope, but possible)
> > > > - show kernel dependencies (vfio/uio_pci_generic/etc) (nope)
> > > > - room for extra information? (works)
> > > > 
> > > > Of the items that are missing, I've already got a V2 started that can do driver
> > > > options, and is easier to expand.  Adding in the the DPDK and PMD version should
> > > > be easy (though I think they can be left out, as theres currently no globaly
> > > > defined DPDK release version, its all just implicit, and driver versions aren't
> > > > really there either).  I'm also hesitant to include kernel dependencies without
> > > > defining exactly what they mean (just module dependencies, or feature
> > > > enablement, or something else?).  Once we define it though, adding it can be
> > > > easy.
> > > 
> > > Yup. I just think the shared/static difference needs to be sorted out
> > > somehow, eg requiring user to know about DSOs is not human-friendly at all.
> > > That's why I called for the higher level use-cases in my previous email.
> > > 
> > 
> > I disagree with that.  While its reasonable to give users the convienience of
> > scanning the DT_NEEDED entries of a binary and scanning those DSO's.  If a user
> 
> Scanning DT_NEEDED is of course ok sane and right thing to do, its just not
> sufficient.
> 
But its the only sane thing we can do implicitly in the shared case, because we
know those drivers have to be resolved for the app to run.  In the
RTE_EAL_PMD_PATH or -d cases, we dont' know until runtime what drivers that will
include, and so reporting on hardware support prior to run time via scanning of
the application is erroneous.  The sane thing to do is scan the pmd DSO, which
is where the hardware support resides, and make it clear that, for an
application to get that hardware support, they need to either link dynamically
(via a DT_NEEDED entry), or specify it on the command line, or make sure its in
RTE_EAL_PMD_PATH (if the distribution set it, which so far, no one does).

> > has to specify the PMD to load in an application (either on the command line or
> > via a configuration file), then its reasonable assume that they (a) know where
> 
> But when the PMD path is set (as it should be on a distro build), this is
> all automatic with zero action or extra config required from the user.
> 
Its not,  RHEL doesn't do it, Fedora Doesn't do it, Ubuntu doesn't do it.  Even
if it is, it doesn't imply an application will get that support, as the
directory contents may change between scanning and application run time.

> > to find that pmd and (b) are savy enough to pass that same path to a hardware
> > info tool.  Thats the exact same way that modinfo works (save for the fact that
> > modinfo can implicitly check the running kernel version to find the appropriate
> > path for a modular driver).
> > 
> > The only other thing that seems reasonable to me would be to scan
> > LD_LIBRARY_PATH.  I would assume that, if an application is linked dynamically,
> > the individual DSO's (librte_sched.so, etc), need to be in LD_LIBRARY_PATH.  If
> > thats the case, then we can assume that the appropriate PMD DSO's are there too,
> > and we can search there.  We can also check the standard /usr/lib and /lib paths
> > with that.  I think that would make fairly good sense.
> 
> You really don't want go crashing through the potentially thousands of
> libraries in the standard library path going "is it a pmd, no, is it a pmd,
> no..."
What?  No.  You misunderstand.  In the above, all I'm saying is that if you
specify an application, you can scan LD_LIBRARY_PATH for libraries in the
DT_NEEDED list. I don't mean to say that we shoudl check _every_ library in
LD_LIBRARY_PATH, that would be crazy.  Of course, the same problem exists with
RTE_EAL_PMD_PATH.  In the current installation, all pmd libraries are co-located
with the rest of the dpdk library set, so RTE_EAL_PMD_PATH would have to be set
to whatever that installation path was.  And once thats done, we would have to
go crashing through every DPDK library to see if it was a pmd.  Thats just as
insane.

Neil

Re: [PATCHv2 4/4] pmdinfo.py: Add tool to query binaries for hw and other support information

[Panu Matilainen]

On 05/19/2016 03:00 PM, Neil Horman wrote:
> On Thu, May 19, 2016 at 12:02:27PM +0300, Panu Matilainen wrote:
>> On 05/19/2016 12:08 AM, Neil Horman wrote:
>>> This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
>>> and, if found parses the remainder of the string as a json encoded string,
>>> outputting the results in either a human readable or raw, script parseable
>>> format
>>>
>>> Note that, in the case of dynamically linked applications, pmdinfo.py will scan
>>> for implicitly linked PMDs by searching the specified binaries .dynamic section
>>> for DT_NEEDED entries that contain the substring librte_pmd.  The DT_RUNPATH,
>>> LD_LIBRARY_PATH, /usr/lib and /lib are searched for these libraries, in that
>>> order
>>
>> Scanning /usr/lib and /lib does little good on systems where /usr/lib64 and
>> /lib64 are the standard path, such as x86_64 Fedora / RHEL and derivates.
>>
> Ah, sorry, forgot the 64 bit variants, I can add those in.
>
>> With the path changed (or LD_LIBRARY_PATH set manually), I can confirm it
>> works for a shared binary which is --whole-archive linked to all of DPDK
>> such as ovs-vswitchd currently is (because it needs to for static DPDK
>> linkage and is not aware of plugin autoloading).
>>
> Right, thats why it works, because DPDK always requires --whole-archive for
> static linking, and likely always will (see commit
> 20afd76a504155e947c770783ef5023e87136ad8)
>
>> It doesn't help testpmd though because its not linked with --whole-archive
>> in the shared case, so its not working for the main DPDK executable...
>>
> This sentence doesn't make sense --whole-archive is only applicable in the
> static binary case, and only when linking archive files.

Okay sorry I was indeed mixing up things here.

1) Testpmd doesn't get linked to those pmds at all, because of this in 
mk/rte.app.mk:

ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
# plugins (link only if static libraries)
...
endif

2) I'm could swear I've seen --whole-archive it make a difference on 
newer toolchains with the linker script, but I can't reproduce that now 
no matter what. Must've been hallucinating ;)

>
>> In any case, using --whole-archive is a sledgehammer solution at best, and
>> against the spirit of shared libs and plugins in particular.
>>
> It may be a sledgehammer solution, but its the one dpdk uses, and will likely
> use in perpituity.
>
>> I think the shared linkage case can be solved by exporting the PMD path from
>> librte_eal (either through an elf section or c-level symbol) and teach the
>> script to detect the case of an executable dynamically linked to librte_eal,
>> fish the path from there and then process everything in that path.
>>
> I really disagree with this, because its a half-measure at best.  Yes, if its
> set, you will definately get all the shared objects in that directory loaded,
> but that is in no way a guarantee that those are the only libraries that get
> loaded (the application may load its own independently).

That is in no way different from statically linked apps loading 
additional drivers with the EAL -d switch.

> So you're left in this
> situation in which you get maybe some of the hardware support an application
> offers.  Its also transient.  That is to say, if you configure a plugin
> directory and search it when you scan an application, its contents may change
> post scan, leading to erroneous results.

Yes, changing system state such as installing or removing packages 
between scanning and running can "do stuff" such as change results. 
People are quite aware of this because that's how a huge number of 
things in the system works, you install plugins for multimedia formats 
you need, maybe remove others you dont to clean up system etc. I fail to 
see how that is a problem when it's the expected behavior with plugins.

> The way I see it, we have 3 cases that we need to handle:
>
> 1) Statically linked application - in this case, all pmds that are statically
> linked in to the application will be reported, so we're good here
>
> 2) Dynamically loaded via DT_NEEDED entries - This is effectively the same as a
> static linking case, in that we have a list of libraries that must be resolved
> at run time, so we are safe to search for and scan the DSO's that the
> application ennumerates
>
> 3) Dynamically loaded via dlopen - In this case, we don't actually know until
> runtime what DSO's are going to get loaded, even if RTE_EAL_PMD_PATH is set,
> because the contents of that path can change at arbitrary times.  In this case,
> its correct to indicate that the application itself _doesn't_ actually support
> the hardware of the PMD's in that path, because until the application is
> executed, it has none of the support embodied in any DSO that it loads via
> dlopen.  The hardware support travels with the DSO itself, and so its correct to
> only display hardware support when the PMD shared library itself is scanned.
>
> Handling case 3 the way I'm proposing is exactly the way the OS does it (that is
> to say, it only details hardware support for the module being queried, and you
> have to specify the module name to get that).  I don't see there being any
> problem with that.

Ability to query individual DSOs is a building block for other things 
like automation (you dont expect normal users to go manually loading hw 
support modules for the OS either), but its not an end-user solution.

Thomas said in http://dpdk.org/ml/archives/dev/2016-May/038324.html:

"This tool should not behave differently depending of how DPDK was 
compiled (static or shared)."

Its entirely possible to handle all the three above cases virtually 
identically (point the tool to the app executable), so I have a hard 
time understanding the level of resistance to handling the plugin case.

	- Panu -
	
>
>>>
>>> If a file is specified with no path, it is assumed to be a PMD DSO, and the
>>> LD_LIBRARY_PATH, /usr/lib/ and /lib is searched for it
>>
>> Same as above, /usr/lib/ and /lib is incorrect for a large number of
>> systems.
>>
> Yeah, I'll add 64 bit detection and correct that path accordingly
>
>>>
>>> Currently the tool can output data in 3 formats:
>>>
>>> a) raw, suitable for scripting, where the raw JSON strings are dumped out
>>> b) table format (default) where hex pci ids are dumped in a table format
>>> c) pretty, where a user supplied pci.ids file is used to print out vendor and
>>> device strings
>>
>> c) is a nice addition. Would be even nicer if it knew the most common
>> pci.ids locations so it doesn't need extra arguments in those cases, ie see
>> if /usr/share/hwdata/pci.ids or /usr/share/misc/pci.ids exists and use that
>> unless overridden on the cli.
>>
> Yeah, I can do that.
>
>> 	- Panu -
>>
>>
>>

Re: [PATCH 4/4] pmd_hw_support.py: Add tool to query binaries for hw support information

[Panu Matilainen]

On 05/19/2016 04:26 PM, Neil Horman wrote:
> On Thu, May 19, 2016 at 09:08:52AM +0300, Panu Matilainen wrote:
>> On 05/18/2016 04:48 PM, Neil Horman wrote:
>>> On Wed, May 18, 2016 at 03:48:12PM +0300, Panu Matilainen wrote:
>>>> On 05/18/2016 03:03 PM, Neil Horman wrote:
>>>>> On Wed, May 18, 2016 at 02:48:30PM +0300, Panu Matilainen wrote:
>>>>>> On 05/16/2016 11:41 PM, Neil Horman wrote:
>>>>>>> This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
>>>>>>> and, if found parses the remainder of the string as a json encoded string,
>>>>>>> outputting the results in either a human readable or raw, script parseable
>>>>>>> format
>>>>>>>
>>>>>>> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
>>>>>>> CC: Bruce Richardson <bruce.richardson@intel.com>
>>>>>>> CC: Thomas Monjalon <thomas.monjalon@6wind.com>
>>>>>>> CC: Stephen Hemminger <stephen@networkplumber.org>
>>>>>>> CC: Panu Matilainen <pmatilai@redhat.com>
>>>>>>> ---
>>>>>>>  tools/pmd_hw_support.py | 174 ++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>>  1 file changed, 174 insertions(+)
>>>>>>>  create mode 100755 tools/pmd_hw_support.py
>>>>>>>
>>>>>>> diff --git a/tools/pmd_hw_support.py b/tools/pmd_hw_support.py
>>>>>>> new file mode 100755
>>>>>>> index 0000000..0669aca
>>>>>>> --- /dev/null
>>>>>>> +++ b/tools/pmd_hw_support.py
>>>>>>> @@ -0,0 +1,174 @@
>>>>>>> +#!/usr/bin/python3
>>>>>>
>>>>>> I think this should use /usr/bin/python to be consistent with the other
>>>>>> python scripts, and like the others work with python 2 and 3. I only tested
>>>>>> it with python2 after changing this and it seemed to work fine so the
>>>>>> compatibility side should be fine as-is.
>>>>>>
>>>>> Sure, I can change the python executable, that makes sense.
>>>>>
>>>>>> On the whole, AFAICT the patch series does what it promises, and works for
>>>>>> both static and shared linkage. Using JSON formatted strings in an ELF
>>>>>> section is a sound working technical solution for the storage of the data.
>>>>>> But the difference between the two cases makes me wonder about this all...
>>>>> You mean the difference between checking static binaries and dynamic binaries?
>>>>> yes, there is some functional difference there
>>>>>
>>>>>>
>>>>>> For static library build, you'd query the application executable, eg
>>>>> Correct.
>>>>>
>>>>>> testpmd, to get the data out. For a shared library build, that method gives
>>>>>> absolutely nothing because the data is scattered around in individual
>>>>>> libraries which might be just about wherever, and you need to somehow
>>>>> Correct, I figured that users would be smart enough to realize that with
>>>>> dynamically linked executables, they would need to look at DSO's, but I agree,
>>>>> its a glaring diffrence.
>>>>
>>>> Being able to look at DSOs is good, but expecting the user to figure out
>>>> which DSOs might be loaded and not and where to look is going to be well
>>>> above many users. At very least it's not what I would call user-friendly.
>>>>
>>> I disagree, there is no linkage between an application and the dso's it opens
>>> via dlopen that is exportable.  The only way to handle that is to have a
>>> standard search path for the pmd_hw_info python script.  Thats just like modinfo
>>> works (i.e. "modinfo bnx2" finds the bnx2 module for the running kernel).  We
>>> can of course do something simmilar, but we have no existing implicit path
>>> information to draw from to do that (because you can have multiple dpdk installs
>>> side by side).  The only way around that is to explicitly call out the path on
>>> the command line.
>>
>> There's no telling what libraries user might load at runtime with -D, that
>> is true for both static and shared libraries.
>>
> I agree.
>
>> When CONFIG_RTE_EAL_PMD_PATH is set, as it is likely to be on distro builds,
>> you *know* that everything in that path will be loaded on runtime regardless
>> of what commandline options there might be so the situation is actually on
>> par with static builds. Of course you still dont know about ones added with
>> -D but that's a limitation of any solution that works without actually
>> running the app.
>>
> Its not on ours, as the pmd libraries get placed in the same directory as every
> other dpdk library, and no one wants to try (and fail to load
> rte_sched/rte_acl/etc twice, or deal with the fallout of trying to do so, or
> adjust the packaging so that pmds are placed in their own subdirectory, or
> handle the need for multiuser support.

Err. I suggest you actually look at the package.

>
> Using CONFIG_RTE_EAL_PMD_PATH also doesn't account for directory changes.  This
> use case:
> 1) run pmdinfo <app>
> 2) remove DSOs from RTE_EAL_PMD_PATH
> 3) execute <app>
>
> leads to erroneous results, as hardware support that was reported in (1) is no
> longer available at (3)

Yes, and in place of 2) you could also add DSOs there since you found 
something missing. Just like updating statically <app> at 2) could 
change it. RTE_EAL_PMD_PATH is not expected to point to /tmp like 
location where stuff randomly appears and disappears.

>
> It also completely misses any libraries that we load via the -d option on the
> command line, which won't be included in RTE_EAL_PMD_PATH, so following that
> path is a half measure at best, and I think that leads to erroneous results.

This same problem with -d exists for statically linked apps, as you 
actually agreed earlier in your email. So that's hardly an argument here.



>>>
>>>>>> discover the location + correct library files to be able to query that. For
>>>>>> the shared case, perhaps the script could be taught to walk files in
>>>>>> CONFIG_RTE_EAL_PMD_PATH to give in-the-ballpark correct/identical results
>>>>> My initial thought would be to run ldd on the executable, and use a heuristic to
>>>>> determine relevant pmd DSO's, and then feed each of those through the python
>>>>> script.  I didn't want to go to that trouble unless there was consensus on it
>>>>> though.
>>>>
>>>> Problem is, ldd doesn't know about them either because the pmds are not
>>>> linked to the executables at all anymore. They could be force-linked of
>>>> course, but that means giving up the flexibility of plugins, which IMO is a
>>>> no-go. Except maybe as an option, but then that would be a third case to
>>>> support.
>>>>
>>> Thats not true at all, or at least its a perfectly valid way to link the DSO's
>>> in at link time via -lrte_pmd_<driver>.  Its really just the dlopen case we need
>>> to worry about.  I would argue that, if they're not explicitly linked in like
>>> that, then its correct to indicate that an application supports no hardware,
>>> because it actually doesn't, it only supports the pmds that it chooses to list
>>> on the command line.  And if a user is savy enough to specify a pmd on the
>>> application command line, then they are perfectly capable of specifying the same
>>> path to the hw_info script.
>>
>> Yes you can force-link apps to every driver on existence, but it requires
>> not just linking but using --whole-archive.
> For the static case, yes, and thats what DPDK does, and likely will in
> perpituity, unless there is a major architectural change in the project (see
> commit 20afd76a504155e947c770783ef5023e87136ad8).
>
>> The apps in DPDK itself dont in
>> shared link setup (take a look at testpmd) and I think its for a damn good
>> reason - the drivers are plugins and that's how plugins are expected to
>> work: they are not linked to, they reside in a specific path which is
>
> I know, I'm the one that made that change when we introduced the
> PMD_REGISTER_DRIVER macro :).  That doesn't mean its not a valid case when
> building apps, and one that we can take advantage of opportunistically.  There
> are three cases we have to handle:
>
> 1) Static linking - This is taken care of
> 2) Dynamic linking via DT_NEEDED entries - this is taken care of
> 3) Dynamic linking via dlopen - This is what we're discussing here
>
>> scanned at runtime and plugins loaded to provide extra functionality.
> Its the runtime part that makes this non-functional.  Just because you scan
> all the DSO's in the RTE_EAL_PATH, doesn't mean they will be there when the app
> is run, nor does it mean you will get a comprehensive list of hardware support,
> because it doesn't include additional paths/DSO's added via -d.  I would much
> rather have users understand that an app has _no_ hardware support if it uses
> DSO's, because the hardware support is included with the DSO's themself, not the
> application (saving for the DT_NEEDED case above, where application execution is
> predicated on the availability of those shared objects)

I'm not going to repeat all the earlier arguments from above, but -d is 
different because its specified by the user at runtime.

RTE_EAL_PMD_PATH is built into the EAL library and you can't disable it 
at runtime. So an app linked to EAL with RTE_EAL_PMD_PATH configured is 
guaranteed to load everything from that path, regardless of what the 
user specifies at the runtime. I agree it is somewhat different from the 
static case because its, well, dynamic, by design. Note that I fully 
agree there is value in being able to query *just* the binary and no 
magic lookups, because for some uses you want just that so it'd need to 
be possible to disable such lookup in the tool.

Anyway, unless something really new turns up in this discussion I'm 
going to shut up now since I would hope I've made my opinion and point 
clear by now.

>>
>>>>
>>>>>
>>>>>> when querying the executable as with static builds. If identical operation
>>>>>> between static and shared versions is a requirement (without running the app
>>>>>> in question) then query through the executable itself is practically the
>>>>>> only option. Unless some kind of (auto-generated) external config file
>>>>>> system ala kernel depmod / modules.dep etc is brought into the picture.
>>>>> Yeah, I'm really trying to avoid that, as I think its really not a typical part
>>>>> of how user space libraries are interacted with.
>>>>>
>>>>>>
>>>>>> For shared library configurations, having the data in the individual pmds is
>>>>>> valuable as one could for example have rpm autogenerate provides from the
>>>>>> data to ease/automate installation (in case of split packaging and/or 3rd
>>>>>> party drivers). And no doubt other interesting possibilities. With static
>>>>>> builds that kind of thing is not possible.
>>>>> Right.
>>>>>
>>>>> Note, this also leaves out PMD's that are loaded dynamically (i.e. via dlopen).
>>>>> For those situations I don't think we have any way of 'knowing' that the
>>>>> application intends to use them.
>>>>
>>>> Hence my comment about CONFIG_RTE_EAL_PMD_PATH above, it at least provides a
>>>> reasonable heuristic of what would be loaded by the app when run. But
>>>> ultimately the only way to know what hardware is supported at a given time
>>>> is to run an app which calls rte_eal_init() to load all the drivers that are
>>>> present and work from there, because besides CONFIG_RTE_EAL_PMD_PATH this
>>>> can be affected by runtime commandline switches and applies to both shared
>>>> and static builds.
>>>>
>>> I'm not sure I agree with that.  Its clearly tempting to use, but its not
>>> at all guaranteed to be accurate (the default is just set to "", and there is no
>>> promise anyone will set it properly).
>>
>> The promise is that shared builds are barely functional unless its set
>> correctly, because zero drivers are linked to testpmd in shared config. So
>> you're kinda likely to notice if its not set.
>>
> You're twisting the meaning of 'barely functional' here.  I agree that shared
> builds are barely functional, because they have no self-contained hardware
> support, and as such, running pmdinfo.py on such an application should report
> exactly that.
>
> That said, in order to run, and DPDK application built to use shared libraries
> has to use one of two methods to obtain hardware support
>
> A) direct shared linking (the DT_NEEDED case) - This case is handled, and we
> report hardware support when found, as the application won't run unless those
> libraries are resolved
>
> b) dynamic loading via dlopen - This case shouldn't be handled, because the
> application in reality doesn't support  any hardware.  Hardware support is
> garnered at run time when the EAL_PMD_PATH (and any other paths added via the -d
> option) are scanned.  In this case, pmdinfo shouldn't report any hardware
> support, it should only do so if the pmd DSO is queried directly.
>
>> It defaults to empty because at the time there was no standard installation
>> available at that time. Setting a reasonable default is tricky still because
>> it needs to be set before build whereas install path is set at install time.
>>
> Exactly, which is why distros don't use it.  It also neglects the multiuser case
> (in which different users may want to load different hardware support).

Ehh? It exists *primarily* for distro needs. I suggest you take a look 
how it all works in current Fedora and RHEL packages. The packaging is 
monolithic at the moment but thanks to the plugin autoloading, it would 
be possible to split drivers into different subpackages to eg provide 
minimal driver package for use in virtual guests where all space counts 
etc, and to allow 3rd party drivers to be dropped in and so on.

>>> And it also requires that the binary will
>>> be tied to a specific release.  I really think that, given the fact that
>>> distributions generally try to package dpdk in such a way that multiple dpdk
>>> versions might be available, the right solution is to just require a full path
>>> specification if you want to get hw info for a DSO that is dynamically loaded
>>> via dlopen from the command line.  Otherwise you're going to fall into this trap
>>> where you might be looking implicitly at an older version of the PMD while your
>>> application may use a newer version.
>>
>> If there are multiple dpdk versions available then they just need to have
>> separate PMD paths, but that's not a problem.
>>
>>>>>>
>>>>>> Calling up on the list of requirements from
>>>>>> http://dpdk.org/ml/archives/dev/2016-May/038324.html, I see a pile of
>>>>>> technical requirements but perhaps we should stop for a moment to think
>>>>>> about the use-cases first?
>>>>>
>>>>> To ennumerate the list:
>>>>>
>>>>> - query all drivers in static binary or shared library (works)
>>>>> - stripping resiliency (works)
>>>>> - human friendly (works)
>>>>> - script friendly (works)
>>>>> - show driver name (works)
>>>>> - list supported device id / name (works)
>>>>> - list driver options (not yet, but possible)
>>>>> - show driver version if available (nope, but possible)
>>>>> - show dpdk version (nope, but possible)
>>>>> - show kernel dependencies (vfio/uio_pci_generic/etc) (nope)
>>>>> - room for extra information? (works)
>>>>>
>>>>> Of the items that are missing, I've already got a V2 started that can do driver
>>>>> options, and is easier to expand.  Adding in the the DPDK and PMD version should
>>>>> be easy (though I think they can be left out, as theres currently no globaly
>>>>> defined DPDK release version, its all just implicit, and driver versions aren't
>>>>> really there either).  I'm also hesitant to include kernel dependencies without
>>>>> defining exactly what they mean (just module dependencies, or feature
>>>>> enablement, or something else?).  Once we define it though, adding it can be
>>>>> easy.
>>>>
>>>> Yup. I just think the shared/static difference needs to be sorted out
>>>> somehow, eg requiring user to know about DSOs is not human-friendly at all.
>>>> That's why I called for the higher level use-cases in my previous email.
>>>>
>>>
>>> I disagree with that.  While its reasonable to give users the convienience of
>>> scanning the DT_NEEDED entries of a binary and scanning those DSO's.  If a user
>>
>> Scanning DT_NEEDED is of course ok sane and right thing to do, its just not
>> sufficient.
>>
> But its the only sane thing we can do implicitly in the shared case, because we
> know those drivers have to be resolved for the app to run.  In the
> RTE_EAL_PMD_PATH or -d cases, we dont' know until runtime what drivers that will
> include, and so reporting on hardware support prior to run time via scanning of
> the application is erroneous.  The sane thing to do is scan the pmd DSO, which
> is where the hardware support resides, and make it clear that, for an
> application to get that hardware support, they need to either link dynamically
> (via a DT_NEEDED entry), or specify it on the command line, or make sure its in
> RTE_EAL_PMD_PATH (if the distribution set it, which so far, no one does).
>
>>> has to specify the PMD to load in an application (either on the command line or
>>> via a configuration file), then its reasonable assume that they (a) know where
>>
>> But when the PMD path is set (as it should be on a distro build), this is
>> all automatic with zero action or extra config required from the user.
>>
> Its not,  RHEL doesn't do it, Fedora Doesn't do it, Ubuntu doesn't do it.  Even

Again, check your facts please. I dont know about Ubuntu but Fedora and 
RHEL do set it to make the damn thing actually work out of the box 
without requiring the user to figure out magic -d arguments.

> if it is, it doesn't imply an application will get that support, as the
> directory contents may change between scanning and application run time.



>>> to find that pmd and (b) are savy enough to pass that same path to a hardware
>>> info tool.  Thats the exact same way that modinfo works (save for the fact that
>>> modinfo can implicitly check the running kernel version to find the appropriate
>>> path for a modular driver).
>>>
>>> The only other thing that seems reasonable to me would be to scan
>>> LD_LIBRARY_PATH.  I would assume that, if an application is linked dynamically,
>>> the individual DSO's (librte_sched.so, etc), need to be in LD_LIBRARY_PATH.  If
>>> thats the case, then we can assume that the appropriate PMD DSO's are there too,
>>> and we can search there.  We can also check the standard /usr/lib and /lib paths
>>> with that.  I think that would make fairly good sense.
>>
>> You really don't want go crashing through the potentially thousands of
>> libraries in the standard library path going "is it a pmd, no, is it a pmd,
>> no..."
> What?  No.  You misunderstand.  In the above, all I'm saying is that if you
> specify an application, you can scan LD_LIBRARY_PATH for libraries in the
> DT_NEEDED list. I don't mean to say that we shoudl check _every_ library in
> LD_LIBRARY_PATH, that would be crazy.  Of course, the same problem exists with
> RTE_EAL_PMD_PATH.  In the current installation, all pmd libraries are co-located
> with the rest of the dpdk library set, so RTE_EAL_PMD_PATH would have to be set
> to whatever that installation path was.  And once thats done, we would have to
> go crashing through every DPDK library to see if it was a pmd.  Thats just as
> insane.

Yes I misunderstood what you meant by looking through LD_LIBRARY_PATH, 
and you misunderstood what I meant by it. I guess we can agree on having 
misunderstood each other :) RTE_EAL_PMD_PATH needs to point to a 
directory where nothing but PMDs exist. That is the standard practise 
with plugins on all userland software.

	- Panu -

>
> Neil
>

Re: [PATCHv2 4/4] pmdinfo.py: Add tool to query binaries for hw and other support information

[Thomas Monjalon]

2016-05-20 08:22, Panu Matilainen:
> Ability to query individual DSOs is a building block for other things 
> like automation (you dont expect normal users to go manually loading hw 
> support modules for the OS either), but its not an end-user solution.
> 
> Thomas said in http://dpdk.org/ml/archives/dev/2016-May/038324.html:
> 
> "This tool should not behave differently depending of how DPDK was 
> compiled (static or shared)."

I meant the basic tool must be usable on static binary and shared library.

> Its entirely possible to handle all the three above cases virtually 
> identically (point the tool to the app executable), so I have a hard 
> time understanding the level of resistance to handling the plugin case.

We need first a basic tool. Then we'll check how to build on it for
end user needs. I think you have some good ideas.
We could also try (later) to inspect a running DPDK app.

Re: [PATCHv2 4/4] pmdinfo.py: Add tool to query binaries for hw and other support information

[Panu Matilainen]

On 05/20/2016 11:55 AM, Thomas Monjalon wrote:
> 2016-05-20 08:22, Panu Matilainen:
>> Ability to query individual DSOs is a building block for other things
>> like automation (you dont expect normal users to go manually loading hw
>> support modules for the OS either), but its not an end-user solution.
>>
>> Thomas said in http://dpdk.org/ml/archives/dev/2016-May/038324.html:
>>
>> "This tool should not behave differently depending of how DPDK was
>> compiled (static or shared)."
>
> I meant the basic tool must be usable on static binary and shared library.

Well, I'm not sure I would interpret that any differently in terms of 
what to expect. But since its all up to interpretation and different 
points of view...

>> Its entirely possible to handle all the three above cases virtually
>> identically (point the tool to the app executable), so I have a hard
>> time understanding the level of resistance to handling the plugin case.
>
> We need first a basic tool. Then we'll check how to build on it for
> end user needs. I think you have some good ideas.
> We could also try (later) to inspect a running DPDK app.
>

...I'll just say fair enough, I've made my points and I'll go shut up 
for now.

	- Panu -

Re: [PATCH 4/4] pmd_hw_support.py: Add tool to query binaries for hw support information

[Neil Horman]

On Fri, May 20, 2016 at 10:30:27AM +0300, Panu Matilainen wrote:
> On 05/19/2016 04:26 PM, Neil Horman wrote:
> > On Thu, May 19, 2016 at 09:08:52AM +0300, Panu Matilainen wrote:
> > > On 05/18/2016 04:48 PM, Neil Horman wrote:
> > > > On Wed, May 18, 2016 at 03:48:12PM +0300, Panu Matilainen wrote:
> > > > > On 05/18/2016 03:03 PM, Neil Horman wrote:
> > > > > > On Wed, May 18, 2016 at 02:48:30PM +0300, Panu Matilainen wrote:
> > > > > > > On 05/16/2016 11:41 PM, Neil Horman wrote:
> > > > > > > > This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
> > > > > > > > and, if found parses the remainder of the string as a json encoded string,
> > > > > > > > outputting the results in either a human readable or raw, script parseable
> > > > > > > > format
> > > > > > > > 
> > > > > > > > Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> > > > > > > > CC: Bruce Richardson <bruce.richardson@intel.com>
> > > > > > > > CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> > > > > > > > CC: Stephen Hemminger <stephen@networkplumber.org>
> > > > > > > > CC: Panu Matilainen <pmatilai@redhat.com>
> > > > > > > > ---
> > > > > > > >  tools/pmd_hw_support.py | 174 ++++++++++++++++++++++++++++++++++++++++++++++++
> > > > > > > >  1 file changed, 174 insertions(+)
> > > > > > > >  create mode 100755 tools/pmd_hw_support.py
> > > > > > > > 
> > > > > > > > diff --git a/tools/pmd_hw_support.py b/tools/pmd_hw_support.py
> > > > > > > > new file mode 100755
> > > > > > > > index 0000000..0669aca
> > > > > > > > --- /dev/null
> > > > > > > > +++ b/tools/pmd_hw_support.py
> > > > > > > > @@ -0,0 +1,174 @@
> > > > > > > > +#!/usr/bin/python3
> > > > > > > 
> > > > > > > I think this should use /usr/bin/python to be consistent with the other
> > > > > > > python scripts, and like the others work with python 2 and 3. I only tested
> > > > > > > it with python2 after changing this and it seemed to work fine so the
> > > > > > > compatibility side should be fine as-is.
> > > > > > > 
> > > > > > Sure, I can change the python executable, that makes sense.
> > > > > > 
> > > > > > > On the whole, AFAICT the patch series does what it promises, and works for
> > > > > > > both static and shared linkage. Using JSON formatted strings in an ELF
> > > > > > > section is a sound working technical solution for the storage of the data.
> > > > > > > But the difference between the two cases makes me wonder about this all...
> > > > > > You mean the difference between checking static binaries and dynamic binaries?
> > > > > > yes, there is some functional difference there
> > > > > > 
> > > > > > > 
> > > > > > > For static library build, you'd query the application executable, eg
> > > > > > Correct.
> > > > > > 
> > > > > > > testpmd, to get the data out. For a shared library build, that method gives
> > > > > > > absolutely nothing because the data is scattered around in individual
> > > > > > > libraries which might be just about wherever, and you need to somehow
> > > > > > Correct, I figured that users would be smart enough to realize that with
> > > > > > dynamically linked executables, they would need to look at DSO's, but I agree,
> > > > > > its a glaring diffrence.
> > > > > 
> > > > > Being able to look at DSOs is good, but expecting the user to figure out
> > > > > which DSOs might be loaded and not and where to look is going to be well
> > > > > above many users. At very least it's not what I would call user-friendly.
> > > > > 
> > > > I disagree, there is no linkage between an application and the dso's it opens
> > > > via dlopen that is exportable.  The only way to handle that is to have a
> > > > standard search path for the pmd_hw_info python script.  Thats just like modinfo
> > > > works (i.e. "modinfo bnx2" finds the bnx2 module for the running kernel).  We
> > > > can of course do something simmilar, but we have no existing implicit path
> > > > information to draw from to do that (because you can have multiple dpdk installs
> > > > side by side).  The only way around that is to explicitly call out the path on
> > > > the command line.
> > > 
> > > There's no telling what libraries user might load at runtime with -D, that
> > > is true for both static and shared libraries.
> > > 
> > I agree.
> > 
> > > When CONFIG_RTE_EAL_PMD_PATH is set, as it is likely to be on distro builds,
> > > you *know* that everything in that path will be loaded on runtime regardless
> > > of what commandline options there might be so the situation is actually on
> > > par with static builds. Of course you still dont know about ones added with
> > > -D but that's a limitation of any solution that works without actually
> > > running the app.
> > > 
> > Its not on ours, as the pmd libraries get placed in the same directory as every
> > other dpdk library, and no one wants to try (and fail to load
> > rte_sched/rte_acl/etc twice, or deal with the fallout of trying to do so, or
> > adjust the packaging so that pmds are placed in their own subdirectory, or
> > handle the need for multiuser support.
> 
> Err. I suggest you actually look at the package.
> 
Oh, you're right, you did make a change to set it.  I never saw a bugzilla for
that.  Sorry.

> > 
> > Using CONFIG_RTE_EAL_PMD_PATH also doesn't account for directory changes.  This
> > use case:
> > 1) run pmdinfo <app>
> > 2) remove DSOs from RTE_EAL_PMD_PATH
> > 3) execute <app>
> > 
> > leads to erroneous results, as hardware support that was reported in (1) is no
> > longer available at (3)
> 
> Yes, and in place of 2) you could also add DSOs there since you found
> something missing. Just like updating statically <app> at 2) could change
> it. RTE_EAL_PMD_PATH is not expected to point to /tmp like location where
> stuff randomly appears and disappears.
> 
So....for this hypothetical admin that we're speaking of, you expect them to be
smart enough to understand that adding additional hardware support requires them
to build an extra DSO, and place it (or a symlink to it) in a distro determined
directory, but then not have the wherewithal to understand that querying
hardware support require that you query that specific binary, rather than
assuming the tool will automagically attempt to glean that path from the core
set of libraries?

> > 
> > It also completely misses any libraries that we load via the -d option on the
> > command line, which won't be included in RTE_EAL_PMD_PATH, so following that
> > path is a half measure at best, and I think that leads to erroneous results.
> 
> This same problem with -d exists for statically linked apps, as you actually
> agreed earlier in your email. So that's hardly an argument here.
> 
Yes, I did agree, but you're twisting my argument.  I'm not arguing that I have
some other solution for this problem, I'm saying that, as a matter of
consistency, for libraries that offer hardware support which are loaded at run
time (which applies equally to those found in RTE_EAL_PMD_PATH, and those loaded
via -d), that we should not automatically report on those libraries, because we
don't know until run time if they will be available.  For both of these cases we
should only report on hardware support if the user queries those binary DSO's
directly.  I don't care if the application automatically loads from that
directory or not, until run time, we don't know what the contents of that
directory will be, and so we aren't guaranteed that the output of pmdinfo, if it
automatically searches that path, will be accurate.

> 
> 
> > > > 
> > > > > > > discover the location + correct library files to be able to query that. For
> > > > > > > the shared case, perhaps the script could be taught to walk files in
> > > > > > > CONFIG_RTE_EAL_PMD_PATH to give in-the-ballpark correct/identical results
> > > > > > My initial thought would be to run ldd on the executable, and use a heuristic to
> > > > > > determine relevant pmd DSO's, and then feed each of those through the python
> > > > > > script.  I didn't want to go to that trouble unless there was consensus on it
> > > > > > though.
> > > > > 
> > > > > Problem is, ldd doesn't know about them either because the pmds are not
> > > > > linked to the executables at all anymore. They could be force-linked of
> > > > > course, but that means giving up the flexibility of plugins, which IMO is a
> > > > > no-go. Except maybe as an option, but then that would be a third case to
> > > > > support.
> > > > > 
> > > > Thats not true at all, or at least its a perfectly valid way to link the DSO's
> > > > in at link time via -lrte_pmd_<driver>.  Its really just the dlopen case we need
> > > > to worry about.  I would argue that, if they're not explicitly linked in like
> > > > that, then its correct to indicate that an application supports no hardware,
> > > > because it actually doesn't, it only supports the pmds that it chooses to list
> > > > on the command line.  And if a user is savy enough to specify a pmd on the
> > > > application command line, then they are perfectly capable of specifying the same
> > > > path to the hw_info script.
> > > 
> > > Yes you can force-link apps to every driver on existence, but it requires
> > > not just linking but using --whole-archive.
> > For the static case, yes, and thats what DPDK does, and likely will in
> > perpituity, unless there is a major architectural change in the project (see
> > commit 20afd76a504155e947c770783ef5023e87136ad8).
> > 
> > > The apps in DPDK itself dont in
> > > shared link setup (take a look at testpmd) and I think its for a damn good
> > > reason - the drivers are plugins and that's how plugins are expected to
> > > work: they are not linked to, they reside in a specific path which is
> > 
> > I know, I'm the one that made that change when we introduced the
> > PMD_REGISTER_DRIVER macro :).  That doesn't mean its not a valid case when
> > building apps, and one that we can take advantage of opportunistically.  There
> > are three cases we have to handle:
> > 
> > 1) Static linking - This is taken care of
> > 2) Dynamic linking via DT_NEEDED entries - this is taken care of
> > 3) Dynamic linking via dlopen - This is what we're discussing here
> > 
> > > scanned at runtime and plugins loaded to provide extra functionality.
> > Its the runtime part that makes this non-functional.  Just because you scan
> > all the DSO's in the RTE_EAL_PATH, doesn't mean they will be there when the app
> > is run, nor does it mean you will get a comprehensive list of hardware support,
> > because it doesn't include additional paths/DSO's added via -d.  I would much
> > rather have users understand that an app has _no_ hardware support if it uses
> > DSO's, because the hardware support is included with the DSO's themself, not the
> > application (saving for the DT_NEEDED case above, where application execution is
> > predicated on the availability of those shared objects)
> 
> I'm not going to repeat all the earlier arguments from above, but -d is
> different because its specified by the user at runtime.
> 
So are the contents of the directory pointed to by RTE_EAL_PMD_PATH.

> RTE_EAL_PMD_PATH is built into the EAL library and you can't disable it at
> runtime. So an app linked to EAL with RTE_EAL_PMD_PATH configured is
> guaranteed to load everything from that path, regardless of what the user
> specifies at the runtime. I agree it is somewhat different from the static
> case because its, well, dynamic, by design. Note that I fully agree there is
> value in being able to query *just* the binary and no magic lookups, because
> for some uses you want just that so it'd need to be possible to disable such
> lookup in the tool.
> 
> Anyway, unless something really new turns up in this discussion I'm going to
> shut up now since I would hope I've made my opinion and point clear by now.
> 
Yes, you've made yourself very clear, and I hope I've done the same.  We're just
not going to agree on this.

> > > 
> > > > > 
> > > > > > 
> > > > > > > when querying the executable as with static builds. If identical operation
> > > > > > > between static and shared versions is a requirement (without running the app
> > > > > > > in question) then query through the executable itself is practically the
> > > > > > > only option. Unless some kind of (auto-generated) external config file
> > > > > > > system ala kernel depmod / modules.dep etc is brought into the picture.
> > > > > > Yeah, I'm really trying to avoid that, as I think its really not a typical part
> > > > > > of how user space libraries are interacted with.
> > > > > > 
> > > > > > > 
> > > > > > > For shared library configurations, having the data in the individual pmds is
> > > > > > > valuable as one could for example have rpm autogenerate provides from the
> > > > > > > data to ease/automate installation (in case of split packaging and/or 3rd
> > > > > > > party drivers). And no doubt other interesting possibilities. With static
> > > > > > > builds that kind of thing is not possible.
> > > > > > Right.
> > > > > > 
> > > > > > Note, this also leaves out PMD's that are loaded dynamically (i.e. via dlopen).
> > > > > > For those situations I don't think we have any way of 'knowing' that the
> > > > > > application intends to use them.
> > > > > 
> > > > > Hence my comment about CONFIG_RTE_EAL_PMD_PATH above, it at least provides a
> > > > > reasonable heuristic of what would be loaded by the app when run. But
> > > > > ultimately the only way to know what hardware is supported at a given time
> > > > > is to run an app which calls rte_eal_init() to load all the drivers that are
> > > > > present and work from there, because besides CONFIG_RTE_EAL_PMD_PATH this
> > > > > can be affected by runtime commandline switches and applies to both shared
> > > > > and static builds.
> > > > > 
> > > > I'm not sure I agree with that.  Its clearly tempting to use, but its not
> > > > at all guaranteed to be accurate (the default is just set to "", and there is no
> > > > promise anyone will set it properly).
> > > 
> > > The promise is that shared builds are barely functional unless its set
> > > correctly, because zero drivers are linked to testpmd in shared config. So
> > > you're kinda likely to notice if its not set.
> > > 
> > You're twisting the meaning of 'barely functional' here.  I agree that shared
> > builds are barely functional, because they have no self-contained hardware
> > support, and as such, running pmdinfo.py on such an application should report
> > exactly that.
> > 
> > That said, in order to run, and DPDK application built to use shared libraries
> > has to use one of two methods to obtain hardware support
> > 
> > A) direct shared linking (the DT_NEEDED case) - This case is handled, and we
> > report hardware support when found, as the application won't run unless those
> > libraries are resolved
> > 
> > b) dynamic loading via dlopen - This case shouldn't be handled, because the
> > application in reality doesn't support  any hardware.  Hardware support is
> > garnered at run time when the EAL_PMD_PATH (and any other paths added via the -d
> > option) are scanned.  In this case, pmdinfo shouldn't report any hardware
> > support, it should only do so if the pmd DSO is queried directly.
> > 
> > > It defaults to empty because at the time there was no standard installation
> > > available at that time. Setting a reasonable default is tricky still because
> > > it needs to be set before build whereas install path is set at install time.
> > > 
> > Exactly, which is why distros don't use it.  It also neglects the multiuser case
> > (in which different users may want to load different hardware support).
> 
> Ehh? It exists *primarily* for distro needs. I suggest you take a look how
> it all works in current Fedora and RHEL packages. The packaging is
> monolithic at the moment but thanks to the plugin autoloading, it would be
> possible to split drivers into different subpackages to eg provide minimal
> driver package for use in virtual guests where all space counts etc, and to
> allow 3rd party drivers to be dropped in and so on.
> 
And if you run pmdinfo, then remove a package that provides a 3rd party driver
which it previously reported support for?

The bottom line is, the application you scan with pmdinfo doesn't acutally
support any hardware (save for whats statically linked or dynamically linked via
DT_NEEDED entries).  It has no support for any hardware provided by the plugin
interface until run time when those directories are queried and loaded.  As such
the only sane consistent thing to do is not report on that hardware support.

> > > > And it also requires that the binary will
> > > > be tied to a specific release.  I really think that, given the fact that
> > > > distributions generally try to package dpdk in such a way that multiple dpdk
> > > > versions might be available, the right solution is to just require a full path
> > > > specification if you want to get hw info for a DSO that is dynamically loaded
> > > > via dlopen from the command line.  Otherwise you're going to fall into this trap
> > > > where you might be looking implicitly at an older version of the PMD while your
> > > > application may use a newer version.
> > > 
> > > If there are multiple dpdk versions available then they just need to have
> > > separate PMD paths, but that's not a problem.
> > > 
> > > > > > > 
> > > > > > > Calling up on the list of requirements from
> > > > > > > http://dpdk.org/ml/archives/dev/2016-May/038324.html, I see a pile of
> > > > > > > technical requirements but perhaps we should stop for a moment to think
> > > > > > > about the use-cases first?
> > > > > > 
> > > > > > To ennumerate the list:
> > > > > > 
> > > > > > - query all drivers in static binary or shared library (works)
> > > > > > - stripping resiliency (works)
> > > > > > - human friendly (works)
> > > > > > - script friendly (works)
> > > > > > - show driver name (works)
> > > > > > - list supported device id / name (works)
> > > > > > - list driver options (not yet, but possible)
> > > > > > - show driver version if available (nope, but possible)
> > > > > > - show dpdk version (nope, but possible)
> > > > > > - show kernel dependencies (vfio/uio_pci_generic/etc) (nope)
> > > > > > - room for extra information? (works)
> > > > > > 
> > > > > > Of the items that are missing, I've already got a V2 started that can do driver
> > > > > > options, and is easier to expand.  Adding in the the DPDK and PMD version should
> > > > > > be easy (though I think they can be left out, as theres currently no globaly
> > > > > > defined DPDK release version, its all just implicit, and driver versions aren't
> > > > > > really there either).  I'm also hesitant to include kernel dependencies without
> > > > > > defining exactly what they mean (just module dependencies, or feature
> > > > > > enablement, or something else?).  Once we define it though, adding it can be
> > > > > > easy.
> > > > > 
> > > > > Yup. I just think the shared/static difference needs to be sorted out
> > > > > somehow, eg requiring user to know about DSOs is not human-friendly at all.
> > > > > That's why I called for the higher level use-cases in my previous email.
> > > > > 
> > > > 
> > > > I disagree with that.  While its reasonable to give users the convienience of
> > > > scanning the DT_NEEDED entries of a binary and scanning those DSO's.  If a user
> > > 
> > > Scanning DT_NEEDED is of course ok sane and right thing to do, its just not
> > > sufficient.
> > > 
> > But its the only sane thing we can do implicitly in the shared case, because we
> > know those drivers have to be resolved for the app to run.  In the
> > RTE_EAL_PMD_PATH or -d cases, we dont' know until runtime what drivers that will
> > include, and so reporting on hardware support prior to run time via scanning of
> > the application is erroneous.  The sane thing to do is scan the pmd DSO, which
> > is where the hardware support resides, and make it clear that, for an
> > application to get that hardware support, they need to either link dynamically
> > (via a DT_NEEDED entry), or specify it on the command line, or make sure its in
> > RTE_EAL_PMD_PATH (if the distribution set it, which so far, no one does).
> > 
> > > > has to specify the PMD to load in an application (either on the command line or
> > > > via a configuration file), then its reasonable assume that they (a) know where
> > > 
> > > But when the PMD path is set (as it should be on a distro build), this is
> > > all automatic with zero action or extra config required from the user.
> > > 
> > Its not,  RHEL doesn't do it, Fedora Doesn't do it, Ubuntu doesn't do it.  Even
> 
> Again, check your facts please. I dont know about Ubuntu but Fedora and RHEL
> do set it to make the damn thing actually work out of the box without
> requiring the user to figure out magic -d arguments.
> 
Yes, I stipulated to your quiet change above, though I take issue to your
referring to a command line argument as being 'magic' while being perfectly fine
with plugin loading happening automatically due to some internal library
configuration that is invisible to the end user.

> > if it is, it doesn't imply an application will get that support, as the
> > directory contents may change between scanning and application run time.
> 
> 
> 
> > > > to find that pmd and (b) are savy enough to pass that same path to a hardware
> > > > info tool.  Thats the exact same way that modinfo works (save for the fact that
> > > > modinfo can implicitly check the running kernel version to find the appropriate
> > > > path for a modular driver).
> > > > 
> > > > The only other thing that seems reasonable to me would be to scan
> > > > LD_LIBRARY_PATH.  I would assume that, if an application is linked dynamically,
> > > > the individual DSO's (librte_sched.so, etc), need to be in LD_LIBRARY_PATH.  If
> > > > thats the case, then we can assume that the appropriate PMD DSO's are there too,
> > > > and we can search there.  We can also check the standard /usr/lib and /lib paths
> > > > with that.  I think that would make fairly good sense.
> > > 
> > > You really don't want go crashing through the potentially thousands of
> > > libraries in the standard library path going "is it a pmd, no, is it a pmd,
> > > no..."
> > What?  No.  You misunderstand.  In the above, all I'm saying is that if you
> > specify an application, you can scan LD_LIBRARY_PATH for libraries in the
> > DT_NEEDED list. I don't mean to say that we shoudl check _every_ library in
> > LD_LIBRARY_PATH, that would be crazy.  Of course, the same problem exists with
> > RTE_EAL_PMD_PATH.  In the current installation, all pmd libraries are co-located
> > with the rest of the dpdk library set, so RTE_EAL_PMD_PATH would have to be set
> > to whatever that installation path was.  And once thats done, we would have to
> > go crashing through every DPDK library to see if it was a pmd.  Thats just as
> > insane.
> 
> Yes I misunderstood what you meant by looking through LD_LIBRARY_PATH, and
> you misunderstood what I meant by it. I guess we can agree on having
> misunderstood each other :) RTE_EAL_PMD_PATH needs to point to a directory
> where nothing but PMDs exist. That is the standard practise with plugins on
> all userland software.
> 
I suppose we did misunderstand each other.

Look, I think we're simply not going to agree on this issue at all.  What about
this in the way of compromise.  I simply am not comfortable with automatically
trying to guess what hardware support will exist in an application based on the
transient contents of a plugin directory, because of all the reasons we've
already gone over, but I do understand the desire to get information about what
_might_ be automatically loaded for an application.  what if we added a 'plugin
mode' to pmdinfo. In this mode you would dpecify a dpdk installation directory
and an appropriate mode option.  When specified pmdinfo would scan librte_eal in
the specified directory, looking for an exported json string that informs us of
the configured plugin directory.  If found, we iterate through all the libraries
there displaying hw support.  That allows you to query the plugin directory for
available hardware support, while not implying that the application is
guaranteed to get it (because you didn't specifically state on the command line
that you wanted to know about the applications hardware support).

Neil

> 	- Panu -
> 
> > 
> > Neil
> > 
> 
> 

Re: [PATCHv2 4/4] pmdinfo.py: Add tool to query binaries for hw and other support information

[Neil Horman]

On Fri, May 20, 2016 at 08:22:34AM +0300, Panu Matilainen wrote:
> On 05/19/2016 03:00 PM, Neil Horman wrote:
> > On Thu, May 19, 2016 at 12:02:27PM +0300, Panu Matilainen wrote:
> > > On 05/19/2016 12:08 AM, Neil Horman wrote:
> > > > This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
> > > > and, if found parses the remainder of the string as a json encoded string,
> > > > outputting the results in either a human readable or raw, script parseable
> > > > format
> > > > 
> > > > Note that, in the case of dynamically linked applications, pmdinfo.py will scan
> > > > for implicitly linked PMDs by searching the specified binaries .dynamic section
> > > > for DT_NEEDED entries that contain the substring librte_pmd.  The DT_RUNPATH,
> > > > LD_LIBRARY_PATH, /usr/lib and /lib are searched for these libraries, in that
> > > > order
> > > 
> > > Scanning /usr/lib and /lib does little good on systems where /usr/lib64 and
> > > /lib64 are the standard path, such as x86_64 Fedora / RHEL and derivates.
> > > 
> > Ah, sorry, forgot the 64 bit variants, I can add those in.
> > 
> > > With the path changed (or LD_LIBRARY_PATH set manually), I can confirm it
> > > works for a shared binary which is --whole-archive linked to all of DPDK
> > > such as ovs-vswitchd currently is (because it needs to for static DPDK
> > > linkage and is not aware of plugin autoloading).
> > > 
> > Right, thats why it works, because DPDK always requires --whole-archive for
> > static linking, and likely always will (see commit
> > 20afd76a504155e947c770783ef5023e87136ad8)
> > 
> > > It doesn't help testpmd though because its not linked with --whole-archive
> > > in the shared case, so its not working for the main DPDK executable...
> > > 
> > This sentence doesn't make sense --whole-archive is only applicable in the
> > static binary case, and only when linking archive files.
> 
> Okay sorry I was indeed mixing up things here.
> 
> 1) Testpmd doesn't get linked to those pmds at all, because of this in
> mk/rte.app.mk:
> 
> ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
> # plugins (link only if static libraries)
> ...
> endif
> 
> 2) I'm could swear I've seen --whole-archive it make a difference on newer
> toolchains with the linker script, but I can't reproduce that now no matter
> what. Must've been hallucinating ;)
> 
I think so.  DSO's don't really mesh with the concept of --whole-archive because
they automatically keep their entire contents (since you won't know what symbols
a given application will use at run time)

> > 
> > > In any case, using --whole-archive is a sledgehammer solution at best, and
> > > against the spirit of shared libs and plugins in particular.
> > > 
> > It may be a sledgehammer solution, but its the one dpdk uses, and will likely
> > use in perpituity.
> > 
> > > I think the shared linkage case can be solved by exporting the PMD path from
> > > librte_eal (either through an elf section or c-level symbol) and teach the
> > > script to detect the case of an executable dynamically linked to librte_eal,
> > > fish the path from there and then process everything in that path.
> > > 
> > I really disagree with this, because its a half-measure at best.  Yes, if its
> > set, you will definately get all the shared objects in that directory loaded,
> > but that is in no way a guarantee that those are the only libraries that get
> > loaded (the application may load its own independently).
> 
> That is in no way different from statically linked apps loading additional
> drivers with the EAL -d switch.
> 


Sooo....we agree?

> > So you're left in this
> > situation in which you get maybe some of the hardware support an application
> > offers.  Its also transient.  That is to say, if you configure a plugin
> > directory and search it when you scan an application, its contents may change
> > post scan, leading to erroneous results.
> 
> Yes, changing system state such as installing or removing packages between
> scanning and running can "do stuff" such as change results. People are quite
> aware of this because that's how a huge number of things in the system
> works, you install plugins for multimedia formats you need, maybe remove
> others you dont to clean up system etc. I fail to see how that is a problem
> when it's the expected behavior with plugins.
> 
> > The way I see it, we have 3 cases that we need to handle:
> > 
> > 1) Statically linked application - in this case, all pmds that are statically
> > linked in to the application will be reported, so we're good here
> > 
> > 2) Dynamically loaded via DT_NEEDED entries - This is effectively the same as a
> > static linking case, in that we have a list of libraries that must be resolved
> > at run time, so we are safe to search for and scan the DSO's that the
> > application ennumerates
> > 
> > 3) Dynamically loaded via dlopen - In this case, we don't actually know until
> > runtime what DSO's are going to get loaded, even if RTE_EAL_PMD_PATH is set,
> > because the contents of that path can change at arbitrary times.  In this case,
> > its correct to indicate that the application itself _doesn't_ actually support
> > the hardware of the PMD's in that path, because until the application is
> > executed, it has none of the support embodied in any DSO that it loads via
> > dlopen.  The hardware support travels with the DSO itself, and so its correct to
> > only display hardware support when the PMD shared library itself is scanned.
> > 
> > Handling case 3 the way I'm proposing is exactly the way the OS does it (that is
> > to say, it only details hardware support for the module being queried, and you
> > have to specify the module name to get that).  I don't see there being any
> > problem with that.
> 
> Ability to query individual DSOs is a building block for other things like
> automation (you dont expect normal users to go manually loading hw support
> modules for the OS either), but its not an end-user solution.
> 
> Thomas said in http://dpdk.org/ml/archives/dev/2016-May/038324.html:
> 
> "This tool should not behave differently depending of how DPDK was compiled
> (static or shared)."
> 
> Its entirely possible to handle all the three above cases virtually
> identically (point the tool to the app executable), so I have a hard time
> understanding the level of resistance to handling the plugin case.
> 
Because you're applying some meaning to the idea that Thomas listed, and
it simply doesn't work like that.  Look, try it by using this statement:

"pmdinfo.py lists hardware support that an application has at run time"

Using that statement:

1) Any pmd that is statically linked into the application is guaranteed to be
present at run time, and so is reported

2) Any pmd that is dynamically linked at link time is guarateed to be available
at run time (because the DT_NEEDED entry implies that said pmd has to be loaded
for the application to run), and so the hw support is reported

3) Any pmd that is dynamically loaded via the the RTE_EAL_PMD_PATH or the -d
command line option is _not_ guaranteed to be there at run time, and so is not
reported.

The only reason that your argument about automatically scanning stuff in the
plugin directory makes sense is because you're enforcing the assumption that the
contents of that directory remain unchanged between scan time and run time.  If
thats true great, if not, we get incorrect resulsts.  And we don't address the
-d option at all.

If you want to make the consistency argument, I assert that what we have now is
in fact the most consistent solution.

Please see my elarier email for a compromise solution I'm willing to implement

Neil

Re: [PATCHv2 4/4] pmdinfo.py: Add tool to query binaries for hw and other support information

[Neil Horman]

On Fri, May 20, 2016 at 10:55:26AM +0200, Thomas Monjalon wrote:
> 2016-05-20 08:22, Panu Matilainen:
> > Ability to query individual DSOs is a building block for other things 
> > like automation (you dont expect normal users to go manually loading hw 
> > support modules for the OS either), but its not an end-user solution.
> > 
> > Thomas said in http://dpdk.org/ml/archives/dev/2016-May/038324.html:
> > 
> > "This tool should not behave differently depending of how DPDK was 
> > compiled (static or shared)."
> 
> I meant the basic tool must be usable on static binary and shared library.
> 
> > Its entirely possible to handle all the three above cases virtually 
> > identically (point the tool to the app executable), so I have a hard 
> > time understanding the level of resistance to handling the plugin case.
> 
> We need first a basic tool. Then we'll check how to build on it for
> end user needs. I think you have some good ideas.
> We could also try (later) to inspect a running DPDK app.
> 

I've thought about that, and its actually pretty easily doable. If we move the
driver registration list to a shared memory segment, we could just attach to it
and read it.


Neil

[PATCHv3 0/5] Implement pmd hardware support exports

[Neil Horman]

Hey all-
	So heres attempt number 2 at a method for exporting PMD hardware support
information.  As we discussed previously, the consensus seems to be that pmd
information should be:

1) Able to be interrogated on any ELF binary (application binary or individual
DSO)
2) Equally functional on statically linked applications or on DSO's
3) Resilient to symbol stripping
4) Script friendly
5) Show kernel dependencies
6) List driver options
7) Show driver name
8) Offer human readable output
9) Show DPDK version
10) Show driver version
11) Allow for expansion
12) Not place additional build environment dependencies on an application

I added item 12 myself, because I don't think its reasonable to require
applications to use a specific linker script to get hardware support information
(which precludes the use of a special .modinfo section like the kernel uses)

However, we still can use some tricks from the kernel to make this work.  In
this approach, what I've done is the following:

A) Modified the driver registration macro to also define a variable:
this_pmd_name<n>= "name"

Based on the unique name string pointed to by the above variable, we can
query for an arbitrary number of other symbols following the pattern:
<name>_<tag>

Where tag is some well known identifier for information we wish to export

B) Added a utility called pmdinfogen.  This utility is not meant for general use,
but rather is used by the dpdk build environment itself when compiling pmds.
for each object that uses the PMD_REGISTER_DRIVER macro, this utiity is run.  It
searches for the symbols in (A), and using those, extracts the hardware support
info, and module name from the object, using that to produce a new c file
containing a single variable in the following format:

static const char <name>[] __attribute__((used)) = "PMD_DRIVER_INFO=<json>";

The <name> is arbitrary, as its static and not referenced.  The relevant bit is
the string value assigned to it.  The <json> is a json encoded string of the
extracted hardware support information pmdinfo found for the corresponding
object.  This C file is suitable for compilation and relocatable linking back
into the parent object file.  The result of this operation is that the object
string table now contains a string that will not e removed by stripping, whos
leading text (PMD_DRIVER_INFO) can be easily searched for at any time weather
the symbol referring to it is stripped or not.

C) Added a utilty called pmdinfo.py.  This python script, searches the
string table of the .rodata section of any provided ELF file looking for the
PMD_DRIVER_INFO prefix on a string.  When found, it will interpret the remainder
of the string as json, and output the hardware support for that ELF file (if
any).


This approach ticks most of the above boxes:
1) Impervious to stripping
2) Works on static and dynamic binaries
3) Human and script friendly
4) Allows for expansion

Because of (4) the other items should be pretty easy to implement, as its just a
matter of modifying the macros to export the info, pmdinfo to encode it to json,
and pmd_hw_support.py to read it.  I'm not adding them now, as theres alot of
rote work to do to get some of them in place (e.g. DPDK has no current global
VERSION macro, drivers don't have a consistent version scheme, command line
strings have to be built for each driver, etc).  But once this is accepted,
those items can be done as time allows and should be fairly easy to implement.

Change Notes:
v2)
 * Made the export macros a bit easier to expand
 * Added a macro to optionally export command line strings
 * Renamed utilties to be more appropriate
   (pmdinfo -> pmdinfogen, pmd_hw_support.py -> pmdinfo.py)
 * Added search capabilities to pmdinfo.py so that we search for libraries
   linked using DT_NEEDINFO entries.  We search DT_RUNPATH, LD_LIBRARY_PATH,
   /usr/lib and /lib
 * Added serch abilities if the specified binary to pmdinfo isn't found, we
   search LD_LIBRARY_PATH, /usr/lib and /lib
 * Added an option to pmdinfo.py to pretty-print hardware support info using the
   pci.ids database

v3)
 * Made table mode printing the default mode
 * Added a default path to the pci.ids file
 * Modifed pmdinfo to use python rather than python3
 * Cleaned up some changelog entries
 * Added an export for RTE_EAL_PMD_PATH
 * Added a plugin option to pmdinfo to scan for autoloaded DSO's

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>

[PATCHv3 1/5] pmdinfogen: Add buildtools and pmdinfogen utility

[Neil Horman]

pmdinfogen is a tool used to parse object files and build json strings for use in
later determining hardware support in a dso or application binary.  pmdinfo
looks for the non-exported symbol names this_pmd_name<n> and this_pmd_tbl<n>
(where n is a integer counter).  It records the name of each of these tuples,
using the later to find the symbolic name of the pci_table for physical devices
that the object supports.  With this information, it outputs a C file with a
single line of the form:

static char *<pmd_name>_driver_info[] __attribute__((used)) = " \
	PMD_DRIVER_INFO=<json string>";

Where <pmd_name> is the arbitrary name of the pmd, and <json_string> is the json
encoded string that hold relevant pmd information, including the pmd name, type
and optional array of pci device/vendor ids that the driver supports.

This c file is suitable for compiling to object code, then relocatably linking
into the parent file from which the C was generated.  This creates an entry in
the string table of the object that can inform a later tool about hardware
support.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 GNUmakefile                        |   2 +-
 buildtools/Makefile                |  36 ++++
 buildtools/pmdinfogen/Makefile     |  48 +++++
 buildtools/pmdinfogen/pmdinfogen.c | 413 +++++++++++++++++++++++++++++++++++++
 buildtools/pmdinfogen/pmdinfogen.h |  83 ++++++++
 mk/rte.buildtools.mk               | 148 +++++++++++++
 mk/rte.sdkbuild.mk                 |   3 +-
 7 files changed, 731 insertions(+), 2 deletions(-)
 create mode 100644 buildtools/Makefile
 create mode 100644 buildtools/pmdinfogen/Makefile
 create mode 100644 buildtools/pmdinfogen/pmdinfogen.c
 create mode 100644 buildtools/pmdinfogen/pmdinfogen.h
 create mode 100644 mk/rte.buildtools.mk

diff --git a/GNUmakefile b/GNUmakefile
index b59e4b6..00fe0db 100644
--- a/GNUmakefile
+++ b/GNUmakefile
@@ -40,6 +40,6 @@ export RTE_SDK
 # directory list
 #
 
-ROOTDIRS-y := lib drivers app
+ROOTDIRS-y := buildtools lib drivers app
 
 include $(RTE_SDK)/mk/rte.sdkroot.mk
diff --git a/buildtools/Makefile b/buildtools/Makefile
new file mode 100644
index 0000000..eb565eb
--- /dev/null
+++ b/buildtools/Makefile
@@ -0,0 +1,36 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+DIRS-y += pmdinfogen
+
+include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/buildtools/pmdinfogen/Makefile b/buildtools/pmdinfogen/Makefile
new file mode 100644
index 0000000..4de9506
--- /dev/null
+++ b/buildtools/pmdinfogen/Makefile
@@ -0,0 +1,48 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+#
+# library name
+#
+APP = pmdinfogen
+
+#
+# all sources are stored in SRCS-y
+#
+SRCS-y += pmdinfogen.c
+
+#CFLAGS += $(WERROR_FLAGS) -g
+CFLAGS += -g
+
+include $(RTE_SDK)/mk/rte.buildtools.mk
+
diff --git a/buildtools/pmdinfogen/pmdinfogen.c b/buildtools/pmdinfogen/pmdinfogen.c
new file mode 100644
index 0000000..af0b2df
--- /dev/null
+++ b/buildtools/pmdinfogen/pmdinfogen.c
@@ -0,0 +1,413 @@
+/* Postprocess pmd object files to export hw support 
+ *
+ * Copyright 2016 Neil Horman <nhorman@tuxdriver.com>
+ * Based in part on modpost.c from the linux kernel
+ *
+ * This software may be used and distributed according to the terms
+ * of the GNU General Public License V2, incorporated herein by reference.
+ *
+ */
+
+#define _GNU_SOURCE
+#include <stdio.h>
+#include <ctype.h>
+#include <string.h>
+#include <limits.h>
+#include <stdbool.h>
+#include <errno.h>
+#include "pmdinfogen.h"
+
+
+static const char *sym_name(struct elf_info *elf, Elf_Sym *sym)
+{
+	if (sym)
+		return elf->strtab + sym->st_name;
+	else
+		return "(unknown)";
+}
+
+void *grab_file(const char *filename, unsigned long *size)
+{
+	struct stat st;
+	void *map = MAP_FAILED;
+	int fd;
+
+	fd = open(filename, O_RDONLY);
+	if (fd < 0)
+		return NULL;
+	if (fstat(fd, &st))
+		goto failed;
+
+	*size = st.st_size;
+	map = mmap(NULL, *size, PROT_READ|PROT_WRITE, MAP_PRIVATE, fd, 0);
+
+failed:
+	close(fd);
+	if (map == MAP_FAILED)
+		return NULL;
+	return map;
+}
+
+/**
+  * Return a copy of the next line in a mmap'ed file.
+  * spaces in the beginning of the line is trimmed away.
+  * Return a pointer to a static buffer.
+  **/
+void release_file(void *file, unsigned long size)
+{
+	munmap(file, size);
+}
+
+
+static void *get_sym_value(struct elf_info *info, const Elf_Sym *sym)
+{
+	void *ptr = (void *)info->hdr + info->sechdrs[sym->st_shndx].sh_offset;
+
+	return (void *)(ptr + sym->st_value);
+}
+
+static Elf_Sym *find_sym_in_symtab(struct elf_info *info, 
+				   const char *name, Elf_Sym *last)
+{
+	Elf_Sym *idx;
+	if (last)
+		idx = last+1;
+	else
+		idx = info->symtab_start;
+
+	for(; idx < info->symtab_stop; idx++) {
+		const char *n = sym_name(info, idx);
+		if (!strncmp(n, name, strlen(name)))
+			return idx;
+	}
+	return NULL;
+}
+
+static int parse_elf(struct elf_info *info, const char *filename)
+{
+	unsigned int i;
+	Elf_Ehdr *hdr;
+	Elf_Shdr *sechdrs;
+	Elf_Sym  *sym;
+	const char *secstrings;
+	unsigned int symtab_idx = ~0U, symtab_shndx_idx = ~0U;
+
+	hdr = grab_file(filename, &info->size);
+	if (!hdr) {
+		perror(filename);
+		exit(1);
+	}
+	info->hdr = hdr;
+	if (info->size < sizeof(*hdr)) {
+		/* file too small, assume this is an empty .o file */
+		return 0;
+	}
+	/* Is this a valid ELF file? */
+	if ((hdr->e_ident[EI_MAG0] != ELFMAG0) ||
+	    (hdr->e_ident[EI_MAG1] != ELFMAG1) ||
+	    (hdr->e_ident[EI_MAG2] != ELFMAG2) ||
+	    (hdr->e_ident[EI_MAG3] != ELFMAG3)) {
+		/* Not an ELF file - silently ignore it */
+		return 0;
+	}
+	/* Fix endianness in ELF header */
+	hdr->e_type      = TO_NATIVE(hdr->e_type);
+	hdr->e_machine   = TO_NATIVE(hdr->e_machine);
+	hdr->e_version   = TO_NATIVE(hdr->e_version);
+	hdr->e_entry     = TO_NATIVE(hdr->e_entry);
+	hdr->e_phoff     = TO_NATIVE(hdr->e_phoff);
+	hdr->e_shoff     = TO_NATIVE(hdr->e_shoff);
+	hdr->e_flags     = TO_NATIVE(hdr->e_flags);
+	hdr->e_ehsize    = TO_NATIVE(hdr->e_ehsize);
+	hdr->e_phentsize = TO_NATIVE(hdr->e_phentsize);
+	hdr->e_phnum     = TO_NATIVE(hdr->e_phnum);
+	hdr->e_shentsize = TO_NATIVE(hdr->e_shentsize);
+	hdr->e_shnum     = TO_NATIVE(hdr->e_shnum);
+	hdr->e_shstrndx  = TO_NATIVE(hdr->e_shstrndx);
+	sechdrs = (void *)hdr + hdr->e_shoff;
+	info->sechdrs = sechdrs;
+
+	/* Check if file offset is correct */
+	if (hdr->e_shoff > info->size) {
+		fprintf(stderr, "section header offset=%lu in file '%s' is bigger than "
+		      "filesize=%lu\n", (unsigned long)hdr->e_shoff,
+		      filename, info->size);
+		return 0;
+	}
+
+	if (hdr->e_shnum == SHN_UNDEF) {
+		/*
+		 * There are more than 64k sections,
+		 * read count from .sh_size.
+		 */
+		info->num_sections = TO_NATIVE(sechdrs[0].sh_size);
+	}
+	else {
+		info->num_sections = hdr->e_shnum;
+	}
+	if (hdr->e_shstrndx == SHN_XINDEX) {
+		info->secindex_strings = TO_NATIVE(sechdrs[0].sh_link);
+	}
+	else {
+		info->secindex_strings = hdr->e_shstrndx;
+	}
+
+	/* Fix endianness in section headers */
+	for (i = 0; i < info->num_sections; i++) {
+		sechdrs[i].sh_name      = TO_NATIVE(sechdrs[i].sh_name);
+		sechdrs[i].sh_type      = TO_NATIVE(sechdrs[i].sh_type);
+		sechdrs[i].sh_flags     = TO_NATIVE(sechdrs[i].sh_flags);
+		sechdrs[i].sh_addr      = TO_NATIVE(sechdrs[i].sh_addr);
+		sechdrs[i].sh_offset    = TO_NATIVE(sechdrs[i].sh_offset);
+		sechdrs[i].sh_size      = TO_NATIVE(sechdrs[i].sh_size);
+		sechdrs[i].sh_link      = TO_NATIVE(sechdrs[i].sh_link);
+		sechdrs[i].sh_info      = TO_NATIVE(sechdrs[i].sh_info);
+		sechdrs[i].sh_addralign = TO_NATIVE(sechdrs[i].sh_addralign);
+		sechdrs[i].sh_entsize   = TO_NATIVE(sechdrs[i].sh_entsize);
+	}
+	/* Find symbol table. */
+	secstrings = (void *)hdr + sechdrs[info->secindex_strings].sh_offset;
+	for (i = 1; i < info->num_sections; i++) {
+		int nobits = sechdrs[i].sh_type == SHT_NOBITS;
+
+		if (!nobits && sechdrs[i].sh_offset > info->size) {
+			fprintf(stderr, "%s is truncated. sechdrs[i].sh_offset=%lu > "
+			      "sizeof(*hrd)=%zu\n", filename,
+			      (unsigned long)sechdrs[i].sh_offset,
+			      sizeof(*hdr));
+			return 0;
+		}
+
+		if (sechdrs[i].sh_type == SHT_SYMTAB) {
+			unsigned int sh_link_idx;
+			symtab_idx = i;
+			info->symtab_start = (void *)hdr +
+			    sechdrs[i].sh_offset;
+			info->symtab_stop  = (void *)hdr +
+			    sechdrs[i].sh_offset + sechdrs[i].sh_size;
+			sh_link_idx = sechdrs[i].sh_link;
+			info->strtab       = (void *)hdr +
+			    sechdrs[sh_link_idx].sh_offset;
+		}
+
+		/* 32bit section no. table? ("more than 64k sections") */
+		if (sechdrs[i].sh_type == SHT_SYMTAB_SHNDX) {
+			symtab_shndx_idx = i;
+			info->symtab_shndx_start = (void *)hdr +
+			    sechdrs[i].sh_offset;
+			info->symtab_shndx_stop  = (void *)hdr +
+			    sechdrs[i].sh_offset + sechdrs[i].sh_size;
+		}
+	}
+	if (!info->symtab_start)
+		fprintf(stderr, "%s has no symtab?\n", filename);
+
+	/* Fix endianness in symbols */
+	for (sym = info->symtab_start; sym < info->symtab_stop; sym++) {
+		sym->st_shndx = TO_NATIVE(sym->st_shndx);
+		sym->st_name  = TO_NATIVE(sym->st_name);
+		sym->st_value = TO_NATIVE(sym->st_value);
+		sym->st_size  = TO_NATIVE(sym->st_size);
+	}
+
+	if (symtab_shndx_idx != ~0U) {
+		Elf32_Word *p;
+		if (symtab_idx != sechdrs[symtab_shndx_idx].sh_link)
+			fprintf(stderr, "%s: SYMTAB_SHNDX has bad sh_link: %u!=%u\n",
+			      filename, sechdrs[symtab_shndx_idx].sh_link,
+			      symtab_idx);
+		/* Fix endianness */
+		for (p = info->symtab_shndx_start; p < info->symtab_shndx_stop;
+		     p++)
+			*p = TO_NATIVE(*p);
+	}
+
+	return 1;
+}
+
+static void parse_elf_finish(struct elf_info *info)
+{
+	struct pmd_driver *tmp, *idx = info->drivers;
+	release_file(info->hdr, info->size);
+	while (idx) {
+		tmp = idx->next;
+		free(idx);
+		idx = tmp;
+	}
+}
+
+static const char *sec_name(struct elf_info *elf, int secindex)
+{
+	Elf_Shdr *sechdrs = elf->sechdrs;
+	return (void *)elf->hdr +
+		elf->sechdrs[elf->secindex_strings].sh_offset +
+		sechdrs[secindex].sh_name;
+}
+
+static int get_symbol_index(struct elf_info *info, Elf64_Sym *sym)
+{
+	const char *name =  sym_name(info, sym);
+	const char *idx;
+
+	idx = name;
+	while (idx) {
+		if (isdigit(*idx))
+			return atoi(idx);
+		idx++;
+	}
+	return -1;
+}
+
+struct opt_tag {
+	const char* suffix;
+	const char* json_id;
+};
+
+static const struct opt_tag opt_tags[] = {
+	{"_param_string_export", "params"},
+};
+
+static int complete_pmd_entry(struct elf_info *info, struct pmd_driver *drv)
+{
+	const char *tname;
+	int i;
+	char tmpsymname[128];
+	Elf_Sym *tmpsym;
+	
+
+	drv->name = get_sym_value(info, drv->name_sym);
+
+	for (i=0; i<PMD_OPT_MAX; i++) {
+		memset(tmpsymname, 0, 128);
+		sprintf(tmpsymname, "__%s%s", drv->name, opt_tags[i].suffix);
+		tmpsym = find_sym_in_symtab(info, tmpsymname, NULL);
+		if (!tmpsym)
+			continue;
+		drv->opt_vals[i] = get_sym_value(info, tmpsym);
+	}
+
+	memset(tmpsymname, 0, 128);
+	sprintf(tmpsymname, "__%s_pci_tbl_export", drv->name);
+
+	tmpsym = find_sym_in_symtab(info, tmpsymname, NULL);
+
+
+	/*
+ 	 * If this returns NULL, then this is a PMD_VDEV, because
+ 	 * it has no pci table reference
+ 	 */
+	if (!tmpsym) {
+		drv->pci_tbl = NULL;
+		return 0;
+	}
+
+	tname = get_sym_value(info, tmpsym);
+	tmpsym = find_sym_in_symtab(info, tname, NULL);
+	if (!tmpsym)
+		return -ENOENT;
+
+	drv->pci_tbl = (struct rte_pci_id *)get_sym_value(info, tmpsym);
+	if (!drv->pci_tbl)
+		return -ENOENT;
+
+
+	return 0;
+	
+}
+
+static int locate_pmd_entries(struct elf_info *info)
+{
+	Elf_Sym *last = NULL;
+	struct pmd_driver *new;
+
+	info->drivers = NULL;
+
+	do {
+		new = calloc(sizeof(struct pmd_driver), 1);
+		new->name_sym = find_sym_in_symtab(info, "this_pmd_name", last);
+		last = new->name_sym;
+		if (!new->name_sym)
+			free(new);
+		else {
+			if (complete_pmd_entry(info, new)) {
+				fprintf(stderr, "Failed to complete pmd entry\n");
+				free(new);
+			} else {
+				new->next = info->drivers;
+				info->drivers = new;
+			}
+		}
+	} while (last);
+}
+
+static void output_pmd_info_string(struct elf_info *info, char *outfile)
+{
+	FILE *ofd;
+	struct pmd_driver *drv;
+	struct rte_pci_id *pci_ids;
+	int idx = 0;
+
+	ofd = fopen(outfile, "w+");
+	if (!ofd) {
+		fprintf(stderr, "Unable to open output file\n");
+		return;
+	}
+
+	drv = info->drivers;
+
+	while (drv) {
+		fprintf(ofd, "const char %s_pmd_info[] __attribute__((used)) = \"PMD_INFO_STRING= {",
+			drv->name);
+		fprintf(ofd,"\\\"name\\\" : \\\"%s\\\", ", drv->name);
+		fprintf(ofd,"\\\"type\\\" : \\\"%s\\\", ", drv->pci_tbl ? "PMD_PDEV" : "PMD_VDEV");
+
+		for(idx=0; idx<PMD_OPT_MAX; idx++) {
+			if (drv->opt_vals[idx])
+				fprintf(ofd,"\\\"%s\\\" : \\\"%s\\\", ", opt_tags[idx].json_id,
+					drv->opt_vals[idx]);
+		}
+
+		pci_ids = drv->pci_tbl;
+		fprintf(ofd, "\\\"pci_ids\\\" : [");
+
+		while (pci_ids && pci_ids->device_id) {
+			fprintf(ofd, "[%d, %d, %d, %d]",
+				pci_ids->vendor_id, pci_ids->device_id,
+				pci_ids->subsystem_vendor_id,
+				pci_ids->subsystem_device_id);
+			pci_ids++;
+			if (pci_ids->device_id)
+				fprintf(ofd, ",");
+			else
+				fprintf(ofd, " ");
+		}
+		fprintf(ofd, "]}\";");
+		drv = drv->next;
+	}
+
+	fclose(ofd);
+}
+
+int main(int argc, char **argv)
+{
+	struct elf_info info;
+	int rc = 1;
+
+	if (argc < 3) {
+		fprintf(stderr, "usage: pmdinfo <object file> <c output file>\n");
+		exit(127);
+	}
+	parse_elf(&info, argv[1]);
+
+	locate_pmd_entries(&info);
+
+	if (info.drivers) {
+		output_pmd_info_string(&info, argv[2]);
+		rc = 0;
+	} else {
+		fprintf(stderr, "Hmm, Appears to be a driver but no drivers registered\n");
+	}
+
+	parse_elf_finish(&info);
+	exit(rc);
+}
diff --git a/buildtools/pmdinfogen/pmdinfogen.h b/buildtools/pmdinfogen/pmdinfogen.h
new file mode 100644
index 0000000..580ed9f
--- /dev/null
+++ b/buildtools/pmdinfogen/pmdinfogen.h
@@ -0,0 +1,83 @@
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdarg.h>
+#include <string.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/mman.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <elf.h>
+
+
+/* On BSD-alike OSes elf.h defines these according to host's word size */
+#undef ELF_ST_BIND
+#undef ELF_ST_TYPE
+#undef ELF_R_SYM
+#undef ELF_R_TYPE
+
+#define Elf_Ehdr    Elf64_Ehdr
+#define Elf_Shdr    Elf64_Shdr
+#define Elf_Sym     Elf64_Sym
+#define Elf_Addr    Elf64_Addr
+#define Elf_Sword   Elf64_Sxword
+#define Elf_Section Elf64_Half
+#define ELF_ST_BIND ELF64_ST_BIND
+#define ELF_ST_TYPE ELF64_ST_TYPE
+
+#define Elf_Rel     Elf64_Rel
+#define Elf_Rela    Elf64_Rela
+#define ELF_R_SYM   ELF64_R_SYM
+#define ELF_R_TYPE  ELF64_R_TYPE
+
+#define TO_NATIVE(x) (x)
+
+
+struct rte_pci_id {
+	uint16_t vendor_id;           /**< Vendor ID or PCI_ANY_ID. */
+	uint16_t device_id;           /**< Device ID or PCI_ANY_ID. */
+	uint16_t subsystem_vendor_id; /**< Subsystem vendor ID or PCI_ANY_ID. */
+	uint16_t subsystem_device_id; /**< Subsystem device ID or PCI_ANY_ID. */
+};
+
+enum opt_params {
+	PMD_PARAM_STRING = 0,
+	PMD_OPT_MAX
+};
+
+struct pmd_driver {
+	Elf_Sym *name_sym;
+	const char *name;
+	struct rte_pci_id *pci_tbl;
+	struct pmd_driver *next;
+
+	const char* opt_vals[PMD_OPT_MAX];
+};
+
+struct elf_info {
+	unsigned long size;
+	Elf_Ehdr     *hdr;
+	Elf_Shdr     *sechdrs;
+	Elf_Sym      *symtab_start;
+	Elf_Sym      *symtab_stop;
+	Elf_Section  export_sec;
+	Elf_Section  export_unused_sec;
+	Elf_Section  export_gpl_sec;
+	Elf_Section  export_unused_gpl_sec;
+	Elf_Section  export_gpl_future_sec;
+	char         *strtab;
+	char	     *modinfo;
+	unsigned int modinfo_len;
+
+	/* support for 32bit section numbers */
+
+	unsigned int num_sections; /* max_secindex + 1 */
+	unsigned int secindex_strings;
+	/* if Nth symbol table entry has .st_shndx = SHN_XINDEX,
+	 * take shndx from symtab_shndx_start[N] instead */
+	Elf32_Word   *symtab_shndx_start;
+	Elf32_Word   *symtab_shndx_stop;
+
+	struct pmd_driver *drivers;
+};
+
diff --git a/mk/rte.buildtools.mk b/mk/rte.buildtools.mk
new file mode 100644
index 0000000..e8bfcef
--- /dev/null
+++ b/mk/rte.buildtools.mk
@@ -0,0 +1,148 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
+#   Copyright(c) 2014-2015 6WIND S.A.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/internal/rte.compile-pre.mk
+include $(RTE_SDK)/mk/internal/rte.install-pre.mk
+include $(RTE_SDK)/mk/internal/rte.clean-pre.mk
+include $(RTE_SDK)/mk/internal/rte.build-pre.mk
+include $(RTE_SDK)/mk/internal/rte.depdirs-pre.mk
+
+# VPATH contains at least SRCDIR
+VPATH += $(SRCDIR)
+
+_BUILD = $(APP)
+_INSTALL = $(INSTALL-FILES-y) $(SYMLINK-FILES-y)
+_INSTALL += $(RTE_OUTPUT)/buildtools/$(APP) $(RTE_OUTPUT)/buildtools/$(APP).map
+POSTINSTALL += target-appinstall
+_CLEAN = doclean
+POSTCLEAN += target-appclean
+
+.PHONY: all
+all: install
+
+.PHONY: install
+install: build _postinstall
+
+_postinstall: build
+
+.PHONY: build
+build: _postbuild
+
+exe2cmd = $(strip $(call dotfile,$(patsubst %,%.cmd,$(1))))
+
+ifeq ($(LINK_USING_CC),1)
+override EXTRA_LDFLAGS := $(call linkerprefix,$(EXTRA_LDFLAGS))
+O_TO_EXE = $(CC) $(CFLAGS) $(LDFLAGS_$(@)) \
+	-Wl,-Map=$(@).map,--cref -o $@ $(OBJS-y) $(call linkerprefix,$(LDFLAGS)) \
+	$(EXTRA_LDFLAGS) $(call linkerprefix,$(LDLIBS))
+else
+O_TO_EXE = $(LD) $(LDFLAGS) $(LDFLAGS_$(@)) $(EXTRA_LDFLAGS) \
+	-Map=$(@).map --cref -o $@ $(OBJS-y) $(LDLIBS)
+endif
+O_TO_EXE_STR = $(subst ','\'',$(O_TO_EXE)) #'# fix syntax highlight
+O_TO_EXE_DISP = $(if $(V),"$(O_TO_EXE_STR)","  LD $(@)")
+O_TO_EXE_CMD = "cmd_$@ = $(O_TO_EXE_STR)"
+O_TO_EXE_DO = @set -e; \
+	echo $(O_TO_EXE_DISP); \
+	$(O_TO_EXE) && \
+	echo $(O_TO_EXE_CMD) > $(call exe2cmd,$(@))
+
+-include .$(APP).cmd
+
+# path where libraries are retrieved
+LDLIBS_PATH := $(subst -Wl$(comma)-L,,$(filter -Wl$(comma)-L%,$(LDLIBS)))
+LDLIBS_PATH += $(subst -L,,$(filter -L%,$(LDLIBS)))
+
+# list of .a files that are linked to this application
+LDLIBS_NAMES := $(patsubst -l%,lib%.a,$(filter -l%,$(LDLIBS)))
+LDLIBS_NAMES += $(patsubst -Wl$(comma)-l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIBS)))
+
+# list of found libraries files (useful for deps). If not found, the
+# library is silently ignored and dep won't be checked
+LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
+	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
+
+#
+# Compile executable file if needed
+#
+$(APP): $(OBJS-y) $(LDLIBS_FILES) $(DEP_$(APP)) $(LDSCRIPT) FORCE
+	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
+	$(if $(D),\
+		@echo -n "$< -> $@ " ; \
+		echo -n "file_missing=$(call boolean,$(file_missing)) " ; \
+		echo -n "cmdline_changed=$(call boolean,$(call cmdline_changed,$(O_TO_EXE_STR))) " ; \
+		echo -n "depfile_missing=$(call boolean,$(depfile_missing)) " ; \
+		echo "depfile_newer=$(call boolean,$(depfile_newer)) ")
+	$(if $(or \
+		$(file_missing),\
+		$(call cmdline_changed,$(O_TO_EXE_STR)),\
+		$(depfile_missing),\
+		$(depfile_newer)),\
+		$(O_TO_EXE_DO))
+
+#
+# install app in $(RTE_OUTPUT)/app
+#
+$(RTE_OUTPUT)/buildtools/$(APP): $(APP)
+	@echo "  INSTALL-APP $(APP)"
+	@[ -d $(RTE_OUTPUT)/buildtools ] || mkdir -p $(RTE_OUTPUT)/buildtools
+	$(Q)cp -f $(APP) $(RTE_OUTPUT)/buildtools
+
+#
+# install app map file in $(RTE_OUTPUT)/app
+#
+$(RTE_OUTPUT)/buildtools/$(APP).map: $(APP)
+	@echo "  INSTALL-MAP $(APP).map"
+	@[ -d $(RTE_OUTPUT)/buildtools ] || mkdir -p $(RTE_OUTPUT)/buildtools
+	$(Q)cp -f $(APP).map $(RTE_OUTPUT)/buildtools
+
+#
+# Clean all generated files
+#
+.PHONY: clean
+clean: _postclean
+	$(Q)rm -f $(_BUILD_TARGETS) $(_INSTALL_TARGETS) $(_CLEAN_TARGETS)
+
+.PHONY: doclean
+doclean:
+	$(Q)rm -rf $(APP) $(OBJS-all) $(DEPS-all) $(DEPSTMP-all) \
+	  $(CMDS-all) $(INSTALL-FILES-all) .$(APP).cmd
+
+
+include $(RTE_SDK)/mk/internal/rte.compile-post.mk
+include $(RTE_SDK)/mk/internal/rte.install-post.mk
+include $(RTE_SDK)/mk/internal/rte.clean-post.mk
+include $(RTE_SDK)/mk/internal/rte.build-post.mk
+include $(RTE_SDK)/mk/internal/rte.depdirs-post.mk
+
+.PHONY: FORCE
+FORCE:
diff --git a/mk/rte.sdkbuild.mk b/mk/rte.sdkbuild.mk
index eec5241..fb68af2 100644
--- a/mk/rte.sdkbuild.mk
+++ b/mk/rte.sdkbuild.mk
@@ -64,7 +64,8 @@ build: $(ROOTDIRS-y)
 clean: $(CLEANDIRS)
 	@rm -rf $(RTE_OUTPUT)/include $(RTE_OUTPUT)/app \
 		$(RTE_OUTPUT)/hostapp $(RTE_OUTPUT)/lib \
-		$(RTE_OUTPUT)/hostlib $(RTE_OUTPUT)/kmod
+		$(RTE_OUTPUT)/hostlib $(RTE_OUTPUT)/kmod \
+		$(RTE_OUTPUT)/buildtools
 	@[ -d $(RTE_OUTPUT)/include ] || mkdir -p $(RTE_OUTPUT)/include
 	@$(RTE_SDK)/scripts/gen-config-h.sh $(RTE_OUTPUT)/.config \
 		> $(RTE_OUTPUT)/include/rte_config.h
-- 
2.5.5

[PATCHv3 2/5] drivers: Update driver registration macro usage

[Neil Horman]

Modify the PMD_REGISTER_DRIVER macro, adding a name argument to it.  The
addition of a name argument creates a token that can be used for subsequent
macros in the creation of unique symbol names to export additional bits of
information for use by the pmdinfogen tool.  For example:

PMD_REGISTER_DRIVER(ena_driver, ena);

registers the ena_driver struct as it always did, and creates a symbol
const char this_pmd_name0[] __attribute__((used)) = "ena";

which pmdinfogen can search for and extract.  The subsequent macro

DRIVER_REGISTER_PCI_TABLE(ena, ena_pci_id_map);

creates a symbol
const char ena_pci_tbl_export[] __attribute__((used)) = "ena_pci_id_map";

Which allows pmdinfogen to find the pci table of this driver

Using this pattern, we can export arbitrary bits of information.

pmdinfo uses this information to extract hardware support from an object file
and create a json string to make hardware support info discoverable later.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 drivers/Makefile                           |  2 ++
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c   |  4 +++-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c |  4 +++-
 drivers/crypto/null/null_crypto_pmd.c      |  4 +++-
 drivers/crypto/qat/rte_qat_cryptodev.c     |  4 +++-
 drivers/crypto/snow3g/rte_snow3g_pmd.c     |  4 +++-
 drivers/net/af_packet/rte_eth_af_packet.c  |  4 +++-
 drivers/net/bnx2x/bnx2x_ethdev.c           |  6 ++++--
 drivers/net/bonding/rte_eth_bond_pmd.c     |  7 ++++++-
 drivers/net/cxgbe/cxgbe_ethdev.c           |  4 +++-
 drivers/net/e1000/em_ethdev.c              |  3 ++-
 drivers/net/e1000/igb_ethdev.c             |  6 ++++--
 drivers/net/ena/ena_ethdev.c               |  3 ++-
 drivers/net/enic/enic_ethdev.c             |  3 ++-
 drivers/net/fm10k/fm10k_ethdev.c           |  3 ++-
 drivers/net/i40e/i40e_ethdev.c             |  3 ++-
 drivers/net/i40e/i40e_ethdev_vf.c          |  3 ++-
 drivers/net/ixgbe/ixgbe_ethdev.c           |  6 ++++--
 drivers/net/mlx4/mlx4.c                    |  3 ++-
 drivers/net/mlx5/mlx5.c                    |  3 ++-
 drivers/net/mpipe/mpipe_tilegx.c           |  4 ++--
 drivers/net/nfp/nfp_net.c                  |  3 ++-
 drivers/net/null/rte_eth_null.c            |  3 ++-
 drivers/net/pcap/rte_eth_pcap.c            |  4 +++-
 drivers/net/qede/qede_ethdev.c             |  6 ++++--
 drivers/net/ring/rte_eth_ring.c            |  3 ++-
 drivers/net/szedata2/rte_eth_szedata2.c    |  3 ++-
 drivers/net/vhost/rte_eth_vhost.c          |  3 ++-
 drivers/net/virtio/virtio_ethdev.c         |  3 ++-
 drivers/net/vmxnet3/vmxnet3_ethdev.c       |  3 ++-
 drivers/net/xenvirt/rte_eth_xenvirt.c      |  2 +-
 lib/librte_eal/common/include/rte_dev.h    | 23 +++++++++++++++++++----
 32 files changed, 100 insertions(+), 39 deletions(-)

diff --git a/drivers/Makefile b/drivers/Makefile
index 81c03a8..75a3168 100644
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -34,4 +34,6 @@ include $(RTE_SDK)/mk/rte.vars.mk
 DIRS-y += net
 DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += crypto
 
+DEPDIRS-y += buildtools/pmdinfo
+
 include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
index 2987ef6..593769f 100644
--- a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
+++ b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
@@ -521,4 +521,6 @@ static struct rte_driver aesni_gcm_pmd_drv = {
 	.uninit = aesni_gcm_uninit
 };
 
-PMD_REGISTER_DRIVER(aesni_gcm_pmd_drv);
+PMD_REGISTER_DRIVER(aesni_gcm_pmd_drv, aesni_gcm);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> \
+max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
index 3415ac1..2e30939 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
@@ -716,4 +716,6 @@ static struct rte_driver cryptodev_aesni_mb_pmd_drv = {
 	.uninit = cryptodev_aesni_mb_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drv);
+PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drvi, aesni_mb);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> \
+max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/null/null_crypto_pmd.c b/drivers/crypto/null/null_crypto_pmd.c
index bdaf13c..44c4fba 100644
--- a/drivers/crypto/null/null_crypto_pmd.c
+++ b/drivers/crypto/null/null_crypto_pmd.c
@@ -275,4 +275,6 @@ static struct rte_driver cryptodev_null_pmd_drv = {
 	.uninit = cryptodev_null_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_null_pmd_drv);
+PMD_REGISTER_DRIVER(cryptodev_null_pmd_drv, cryptodev_null_pmd);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> \
+max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index a7912f5..92f0d40 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -137,4 +137,6 @@ static struct rte_driver pmd_qat_drv = {
 	.init = rte_qat_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(pmd_qat_drv);
+PMD_REGISTER_DRIVER(pmd_qat_drv, qat);
+DRIVER_EXPORT_PCI_TABLE(qat, pci_id_qat_map);
+
diff --git a/drivers/crypto/snow3g/rte_snow3g_pmd.c b/drivers/crypto/snow3g/rte_snow3g_pmd.c
index f3e0e66..eee2544 100644
--- a/drivers/crypto/snow3g/rte_snow3g_pmd.c
+++ b/drivers/crypto/snow3g/rte_snow3g_pmd.c
@@ -548,4 +548,6 @@ static struct rte_driver cryptodev_snow3g_pmd_drv = {
 	.uninit = cryptodev_snow3g_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_snow3g_pmd_drv);
+PMD_REGISTER_DRIVER(cryptodev_snow3g_pmd_drv, snow3g);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> \
+max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index f17bd7e..e960512 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -878,4 +878,6 @@ static struct rte_driver pmd_af_packet_drv = {
 	.uninit = rte_pmd_af_packet_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_af_packet_drv);
+PMD_REGISTER_DRIVER(pmd_af_packet_drv, eth_af_packet);
+DRIVER_REGISTER_PARAM_STRING(eth_af_packet, "iface=<string> \
+qpairs=<int> blocksz=<int> framesz=<int> framecnt=<int>");
diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c b/drivers/net/bnx2x/bnx2x_ethdev.c
index 071b44f..ee9581b 100644
--- a/drivers/net/bnx2x/bnx2x_ethdev.c
+++ b/drivers/net/bnx2x/bnx2x_ethdev.c
@@ -550,5 +550,7 @@ static struct rte_driver rte_bnx2xvf_driver = {
 	.init = rte_bnx2xvf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_bnx2x_driver);
-PMD_REGISTER_DRIVER(rte_bnx2xvf_driver);
+PMD_REGISTER_DRIVER(rte_bnx2x_driver, pci_id_bnx2x_map, bnx2x);
+DRIVER_EXPORT_PCI_TABLE(bnx2x, pci_id_bnx2x_map);
+PMD_REGISTER_DRIVER(rte_bnx2xvf_driver, pci_id_bnx2xvf_map, bnx2xvf);
+DRIVER_EXPORT_PCI_TABLE(bnx2xvf, pci_id_bnx2xvf_map);
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 129f04b..b4556be 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -2511,4 +2511,9 @@ static struct rte_driver bond_drv = {
 	.uninit = bond_uninit,
 };
 
-PMD_REGISTER_DRIVER(bond_drv);
+PMD_REGISTER_DRIVER(bond_drv, bonding);
+
+DRIVER_REGISTER_PARAM_STRING(bonding, "slave=<ifc> primary=<ifc> mode=[0-4] \
+xmit_policy=[l2 | l23 | l34] socket_id=<int> mac=<mac addr> \
+lsc_poll_period_ms=<int> up_delay=<int> down_delay=<int>");
+
diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index 04eddaf..e84e48b 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -894,4 +894,6 @@ static struct rte_driver rte_cxgbe_driver = {
 	.init = rte_cxgbe_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_cxgbe_driver);
+PMD_REGISTER_DRIVER(rte_cxgbe_driver, cxgb4);
+DRIVER_REGISTER_PCI_TABLE(cxgb4, cxgb4_pci_tbl);
+
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index 653be09..4de5eb2 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -1777,4 +1777,5 @@ struct rte_driver em_pmd_drv = {
 	.init = rte_em_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(em_pmd_drv);
+PMD_REGISTER_DRIVER(em_pmd_drv, em);
+DRIVER_REGISTER_PCI_TABLE(em, pci_id_em_map);
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index f0921ee..f49aae7 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -4973,5 +4973,7 @@ eth_igb_configure_msix_intr(struct rte_eth_dev *dev)
 	E1000_WRITE_FLUSH(hw);
 }
 
-PMD_REGISTER_DRIVER(pmd_igb_drv);
-PMD_REGISTER_DRIVER(pmd_igbvf_drv);
+PMD_REGISTER_DRIVER(pmd_igb_drv, igb);
+DRIVER_REGISTER_PCI_TABLE(igb, pci_id_igb_map);
+PMD_REGISTER_DRIVER(pmd_igbvf_drv, igbvf);
+DRIVER_REGISTER_PCI_TABLE(igbvf, pci_id_igbvf_map);
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index e157587..702289b 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -1450,4 +1450,5 @@ struct rte_driver ena_pmd_drv = {
 	.init = rte_ena_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(ena_pmd_drv);
+PMD_REGISTER_DRIVER(ena_pmd_drv, ena);
+DRIVER_REGISTER_PCI_TABLE(ena, pci_id_ena_map);
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index 6bea940..36ee7ca 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -684,4 +684,5 @@ static struct rte_driver rte_enic_driver = {
 	.init = rte_enic_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_enic_driver);
+PMD_REGISTER_DRIVER(rte_enic_driver, enic);
+DRIVER_REGISTER_PCI_TABLE(enic, pci_id_enic_map);
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index c2d377f..9b2a7ee 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -3055,4 +3055,5 @@ static struct rte_driver rte_fm10k_driver = {
 	.init = rte_pmd_fm10k_init,
 };
 
-PMD_REGISTER_DRIVER(rte_fm10k_driver);
+PMD_REGISTER_DRIVER(rte_fm10k_driver, fm10k);
+DRIVER_REGISTER_PCI_TABLE(fm10k, pci_id_fm10k_map);
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 24777d5..3b6c661 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -695,7 +695,8 @@ static struct rte_driver rte_i40e_driver = {
 	.init = rte_i40e_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_i40e_driver);
+PMD_REGISTER_DRIVER(rte_i40e_driver, i40e);
+DRIVER_REGISTER_PCI_TABLE(i40e, pci_id_i40e_map);
 
 /*
  * Initialize registers for flexible payload, which should be set by NVM.
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 90682ac..e94a459 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1563,7 +1563,8 @@ static struct rte_driver rte_i40evf_driver = {
 	.init = rte_i40evf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_i40evf_driver);
+PMD_REGISTER_DRIVER(rte_i40evf_driver, i40evf);
+DRIVER_REGISTER_PCI_TABLE(i40evf, pci_id_i40evf_map);
 
 static int
 i40evf_dev_configure(struct rte_eth_dev *dev)
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index a2b170b..af1ca56 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -7164,5 +7164,7 @@ static struct rte_driver rte_ixgbevf_driver = {
 	.init = rte_ixgbevf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_ixgbe_driver);
-PMD_REGISTER_DRIVER(rte_ixgbevf_driver);
+PMD_REGISTER_DRIVER(rte_ixgbe_driver, ixgbe);
+DRIVER_REGISTER_PCI_TABLE(ixgbe, pci_id_ixgbe_map);
+PMD_REGISTER_DRIVER(rte_ixgbevf_driver, ixgbevf);
+DRIVER_REGISTER_PCI_TABLE(ixgbevf, pci_id_ixgbevf_map);
diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index c5d8535..87add1f 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -5817,4 +5817,5 @@ static struct rte_driver rte_mlx4_driver = {
 	.init = rte_mlx4_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_mlx4_driver)
+PMD_REGISTER_DRIVER(rte_mlx4_driveri, mlx4)
+DRIVER_REGISTER_PCI_TABLE(mlx4, mlx4_pci_id_map);
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 041cfc3..5f2d6c8 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -672,4 +672,5 @@ static struct rte_driver rte_mlx5_driver = {
 	.init = rte_mlx5_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_mlx5_driver)
+PMD_REGISTER_DRIVER(rte_mlx5_driveri, mlx5)
+DRIVER_REGISTER_PCI_TABLE(mlx5, mlx5_pci_id_map);
diff --git a/drivers/net/mpipe/mpipe_tilegx.c b/drivers/net/mpipe/mpipe_tilegx.c
index 26e1424..4a2a905 100644
--- a/drivers/net/mpipe/mpipe_tilegx.c
+++ b/drivers/net/mpipe/mpipe_tilegx.c
@@ -1635,8 +1635,8 @@ static struct rte_driver pmd_mpipe_gbe_drv = {
 	.init = rte_pmd_mpipe_devinit,
 };
 
-PMD_REGISTER_DRIVER(pmd_mpipe_xgbe_drv);
-PMD_REGISTER_DRIVER(pmd_mpipe_gbe_drv);
+PMD_REGISTER_DRIVER(pmd_mpipe_xgbe_drv, mpipe_xgbe);
+PMD_REGISTER_DRIVER(pmd_mpipe_gbe_drv, mpipe_gbe);
 
 static void __attribute__((constructor, used))
 mpipe_init_contexts(void)
diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
index ea5a2a3..78394a1 100644
--- a/drivers/net/nfp/nfp_net.c
+++ b/drivers/net/nfp/nfp_net.c
@@ -2490,7 +2490,8 @@ static struct rte_driver rte_nfp_net_driver = {
 	.init = nfp_net_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_nfp_net_driver);
+PMD_REGISTER_DRIVER(rte_nfp_net_driver, nfp);
+DRIVER_REGISTER_PCI_TABLE(nfp, pci_id_nfp_net_map);
 
 /*
  * Local variables:
diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index 5e8e203..88eb582 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -689,4 +689,5 @@ static struct rte_driver pmd_null_drv = {
 	.uninit = rte_pmd_null_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_null_drv);
+PMD_REGISTER_DRIVER(pmd_null_drv, eth_null);
+DRIVER_REGISTER_PARAM_STRING(eth_null, "size=<int> copy=<int>");
diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
index c98e234..397f01c 100644
--- a/drivers/net/pcap/rte_eth_pcap.c
+++ b/drivers/net/pcap/rte_eth_pcap.c
@@ -1090,4 +1090,6 @@ static struct rte_driver pmd_pcap_drv = {
 	.uninit = rte_pmd_pcap_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_pcap_drv);
+PMD_REGISTER_DRIVER(pmd_pcap_drv, pcap);
+DRIVER_REGISTER_PARAM_STRING(pcap, "rx_pcap=<string> tx_pcap=<string> \
+rx_iface=<ifc> tx_iface=<ifc> iface=<ifc>");
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index 1273fd3..187ccf1 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -1102,5 +1102,7 @@ static struct rte_driver rte_qede_driver = {
 	.init = rte_qedevf_pmd_init
 };
 
-PMD_REGISTER_DRIVER(rte_qede_driver);
-PMD_REGISTER_DRIVER(rte_qedevf_driver);
+PMD_REGISTER_DRIVER(rte_qede_driver, qede);
+DRIVER_REGISTER_PCI_TABLE(qede, pci_id_qede_map);
+PMD_REGISTER_DRIVER(rte_qedevf_driver, qedevf);
+DRIVER_REGISTER_PCI_TABLE(qedevf, pci_id_qedevf_map);
diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index b1783c3..f2f5217 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -630,4 +630,5 @@ static struct rte_driver pmd_ring_drv = {
 	.uninit = rte_pmd_ring_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_ring_drv);
+PMD_REGISTER_DRIVER(pmd_ring_drv, eth_ring);
+DRIVER_REGISTER_PARAM_STRING(eth_ring, "nodeaction=[attach|detach]");
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 78c43b0..abbc907 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1601,4 +1601,5 @@ static struct rte_driver rte_szedata2_driver = {
 	.uninit = rte_szedata2_uninit,
 };
 
-PMD_REGISTER_DRIVER(rte_szedata2_driver);
+PMD_REGISTER_DRIVER(rte_szedata2_driver, rte_szedata2_pmd);
+DRIVER_REGISTER_PCI_TABLE(rte_szedata2_pmd, rte_szedata2_pci_id_table);
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index 310cbef..d321c3e 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -924,4 +924,5 @@ static struct rte_driver pmd_vhost_drv = {
 	.uninit = rte_pmd_vhost_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_vhost_drv);
+PMD_REGISTER_DRIVER(pmd_vhost_drv, eth_vhost);
+DRIVER_REGISTER_PARAM_STRING(eth_vhost, "iface=<ifc> queues=<int>");
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index c3fb628..24d387f 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -1459,4 +1459,5 @@ static struct rte_driver rte_virtio_driver = {
 	.init = rte_virtio_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_virtio_driver);
+PMD_REGISTER_DRIVER(rte_virtio_driver, virtio_net);
+DRIVER_REGISTER_PCI_TABLE(virtio_net, pci_id_virtio_map);
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index 29b469c..8da4449 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -954,4 +954,5 @@ static struct rte_driver rte_vmxnet3_driver = {
 	.init = rte_vmxnet3_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_vmxnet3_driver);
+PMD_REGISTER_DRIVER(rte_vmxnet3_driver, vmxnet3);
+DRIVER_REGISTER_PCI_TABLE(vmxnet3, pci_id_vmxnet3_map);
diff --git a/drivers/net/xenvirt/rte_eth_xenvirt.c b/drivers/net/xenvirt/rte_eth_xenvirt.c
index 1adeb5b..81f8af9 100644
--- a/drivers/net/xenvirt/rte_eth_xenvirt.c
+++ b/drivers/net/xenvirt/rte_eth_xenvirt.c
@@ -763,4 +763,4 @@ static struct rte_driver pmd_xenvirt_drv = {
 	.uninit = rte_pmd_xenvirt_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_xenvirt_drv);
+PMD_REGISTER_DRIVER(pmd_xenvirt_drv, xenvirt);
diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
index f1b5507..e608ff7 100644
--- a/lib/librte_eal/common/include/rte_dev.h
+++ b/lib/librte_eal/common/include/rte_dev.h
@@ -48,7 +48,7 @@ extern "C" {
 
 #include <stdio.h>
 #include <sys/queue.h>
-
+#include <rte_pci.h>
 #include <rte_log.h>
 
 __attribute__((format(printf, 2, 0)))
@@ -178,12 +178,27 @@ int rte_eal_vdev_init(const char *name, const char *args);
  */
 int rte_eal_vdev_uninit(const char *name);
 
-#define PMD_REGISTER_DRIVER(d)\
+#define DRIVER_EXPORT_NAME_ARRAY(n, idx) n##idx[] __attribute__((used))
+
+#define DRIVER_EXPORT_NAME(d, idx) \
+static const char DRIVER_EXPORT_NAME_ARRAY(this_pmd_name, idx) = RTE_STR(d);\
+ 
+#define PMD_REGISTER_DRIVER(d, n)\
 void devinitfn_ ##d(void);\
 void __attribute__((constructor, used)) devinitfn_ ##d(void)\
 {\
-	rte_eal_driver_register(&d);\
-}
+	(d).name = RTE_STR(n);\
+        rte_eal_driver_register(&d);\
+}\
+DRIVER_EXPORT_NAME(n, __COUNTER__)
+
+#define DRV_EXP_TAG(n, t) __##n##_##t
+
+#define DRIVER_REGISTER_PCI_TABLE(n, t) \
+static const char DRV_EXP_TAG(n, pci_tbl_export)[] __attribute__((used)) = RTE_STR(t)
+
+#define DRIVER_REGISTER_PARAM_STRING(n, s) \
+static const char DRV_EXP_TAG(n, param_string_export)[] __attribute__((used)) = s
 
 #ifdef __cplusplus
 }
-- 
2.5.5

[PATCHv3 3/5] eal: Add an export symbol to expose the autoload path to external tools

[Neil Horman]

Export a symbol containing the string:
DPDK_PLUGIN_PATH="$(CONFIG_RTE_EAL_PMD_PATH)"

Where the latter half of the string is set at build time to a location from
which autoloaded DSO's will be found.  This string is used by pmdinfo in
'plugin' mode, whereby a user can specify a dpdk installation directory (or
static binary), and scan the associated path (if found) for pmd DSO's and report
on their hardware support.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 lib/librte_eal/common/eal_common_options.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c
index 3efc90f..1af0fb3 100644
--- a/lib/librte_eal/common/eal_common_options.c
+++ b/lib/librte_eal/common/eal_common_options.c
@@ -115,6 +115,15 @@ TAILQ_HEAD_INITIALIZER(solib_list);
 /* Default path of external loadable drivers */
 static const char *default_solib_dir = RTE_EAL_PMD_PATH;
 
+/*
+ * Stringified version of solib path used by pmdinfo.py 
+ * Note: PLEASE DO NOT ALTER THIS without making a corresponding
+ * change to tools/pmdinfo.py
+ */
+static const char dpdk_solib_path[] __attribute__((used)) = \
+"DPDK_PLUGIN_PATH=" RTE_EAL_PMD_PATH;
+
+
 static int master_lcore_parsed;
 static int mem_parsed;
 
-- 
2.5.5

[PATCHv3 4/5] Makefile: Do post processing on objects that register a driver

[Neil Horman]

Modify the compilation makefile to identify C files that export PMD information,
and use that to trigger execution of the pmdinfo binary.  If the execution of
pmdinfo is successful, compile the output C file to an object, and use the
linker to do relocatable linking on the resultant object file into the parent
object that it came from.  This effectively just adds the json string into the
string table of the object that defines the PMD to the outside world.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 mk/internal/rte.compile-pre.mk | 19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/mk/internal/rte.compile-pre.mk b/mk/internal/rte.compile-pre.mk
index b9bff4a..6fa4042 100644
--- a/mk/internal/rte.compile-pre.mk
+++ b/mk/internal/rte.compile-pre.mk
@@ -80,7 +80,8 @@ C_TO_O_STR = $(subst ','\'',$(C_TO_O)) #'# fix syntax highlight
 C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)","  HOSTCC $(@)")
 else
 C_TO_O = $(CC) -Wp,-MD,$(call obj2dep,$(@)).tmp $(CFLAGS) \
-	$(CFLAGS_$(@)) $(EXTRA_CFLAGS) -o $@ -c $<
+	 $(CFLAGS_$(@)) $(EXTRA_CFLAGS) -o $@ -c $<
+
 C_TO_O_STR = $(subst ','\'',$(C_TO_O)) #'# fix syntax highlight
 C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)","  CC $(@)")
 endif
@@ -88,10 +89,26 @@ C_TO_O_CMD = 'cmd_$@ = $(C_TO_O_STR)'
 C_TO_O_DO = @set -e; \
 	echo $(C_TO_O_DISP); \
 	$(C_TO_O) && \
+	sh -c "grep -q \"PMD_REGISTER_DRIVER(.*)\" $<; \
+	if [ \$$? -eq 0 ]; \
+	then \
+		echo MODGEN $@; \
+		OBJF=`readlink -f $@`; \
+		${RTE_OUTPUT}/buildtools/pmdinfogen \$$OBJF \$$OBJF.mod.c; \
+		if [ \$$? -eq 0 ]; \
+		then \
+			echo MODBUILD $@; \
+			$(CC) -c -o \$$OBJF.mod.o \$$OBJF.mod.c; \
+			$(CROSS)ld -r -o \$$OBJF.o \$$OBJF.mod.o \$$OBJF; \
+			mv -f \$$OBJF.o \$$OBJF; \
+		fi; \
+	fi; \
+	true" && \
 	echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \
 	sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \
 	rm -f $(call obj2dep,$(@)).tmp
 
+
 # return an empty string if string are equal
 compare = $(strip $(subst $(1),,$(2)) $(subst $(2),,$(1)))
 
-- 
2.5.5

[PATCHv3 5/5] pmdinfo.py: Add tool to query binaries for hw and other support information

[Neil Horman]

This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
and, if found parses the remainder of the string as a json encoded string,
outputting the results in either a human readable or raw, script parseable
format

Note that, in the case of dynamically linked applications, pmdinfo.py will scan
for implicitly linked PMDs by searching the specified binaries .dynamic section
for DT_NEEDED entries that contain the substring librte_pmd.  The DT_RUNPATH,
LD_LIBRARY_PATH, /usr/lib and /lib are searched for these libraries, in that
order

If a file is specified with no path, it is assumed to be a PMD DSO, and the
LD_LIBRARY_PATH, /usr/lib[64]/ and /lib[64] is searched for it

Currently the tool can output data in 3 formats:

a) raw, suitable for scripting, where the raw JSON strings are dumped out
b) table format (default) where hex pci ids are dumped in a table format
c) pretty, where a user supplied pci.ids file is used to print out vendor and
device strings

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 tools/pmdinfo.py | 545 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 545 insertions(+)
 create mode 100755 tools/pmdinfo.py

diff --git a/tools/pmdinfo.py b/tools/pmdinfo.py
new file mode 100755
index 0000000..9b4b4a4
--- /dev/null
+++ b/tools/pmdinfo.py
@@ -0,0 +1,545 @@
+#!/usr/bin/python
+#-------------------------------------------------------------------------------
+# scripts/pmd_hw_support.py
+#
+# Utility to dump PMD_INFO_STRING support from an object file
+#
+#-------------------------------------------------------------------------------
+import os, sys
+from optparse import OptionParser
+import string
+import json
+
+# For running from development directory. It should take precedence over the
+# installed pyelftools.
+sys.path.insert(0, '.')
+
+
+from elftools import __version__
+from elftools.common.exceptions import ELFError
+from elftools.common.py3compat import (
+        ifilter, byte2int, bytes2str, itervalues, str2bytes)
+from elftools.elf.elffile import ELFFile
+from elftools.elf.dynamic import DynamicSection, DynamicSegment
+from elftools.elf.enums import ENUM_D_TAG
+from elftools.elf.segments import InterpSegment
+from elftools.elf.sections import SymbolTableSection
+from elftools.elf.gnuversions import (
+    GNUVerSymSection, GNUVerDefSection,
+    GNUVerNeedSection,
+    )
+from elftools.elf.relocation import RelocationSection
+from elftools.elf.descriptions import (
+    describe_ei_class, describe_ei_data, describe_ei_version,
+    describe_ei_osabi, describe_e_type, describe_e_machine,
+    describe_e_version_numeric, describe_p_type, describe_p_flags,
+    describe_sh_type, describe_sh_flags,
+    describe_symbol_type, describe_symbol_bind, describe_symbol_visibility,
+    describe_symbol_shndx, describe_reloc_type, describe_dyn_tag,
+    describe_ver_flags,
+    )
+from elftools.elf.constants import E_FLAGS
+from elftools.dwarf.dwarfinfo import DWARFInfo
+from elftools.dwarf.descriptions import (
+    describe_reg_name, describe_attr_value, set_global_machine_arch,
+    describe_CFI_instructions, describe_CFI_register_rule,
+    describe_CFI_CFA_rule,
+    )
+from elftools.dwarf.constants import (
+    DW_LNS_copy, DW_LNS_set_file, DW_LNE_define_file)
+from elftools.dwarf.callframe import CIE, FDE
+
+raw_output = False
+pcidb = None
+
+#===========================================
+
+class Vendor:
+    """
+    Class for vendors. This is the top level class
+    for the devices belong to a specific vendor.
+    self.devices is the device dictionary
+    subdevices are in each device.
+    """
+    def __init__(self, vendorStr):
+        """
+        Class initializes with the raw line from pci.ids
+        Parsing takes place inside __init__
+        """
+        self.ID = vendorStr.split()[0]
+        self.name = vendorStr.replace("%s " % self.ID,"").rstrip()
+        self.devices = {}
+
+    def addDevice(self, deviceStr):
+        """
+        Adds a device to self.devices
+        takes the raw line from pci.ids
+        """
+        s = deviceStr.strip()
+        devID = s.split()[0]
+        if devID in self.devices:
+            pass
+        else:
+            self.devices[devID] = Device(deviceStr)
+
+    def report(self):
+        print self.ID, self.name
+        for id, dev in self.devices.items():
+            dev.report()
+
+    def find_device(self, devid):
+        # convert to a hex string and remove 0x
+        devid = hex(devid)[2:]
+        try:
+            return self.devices[devid]
+        except:
+            return Device("%s  Unknown Device" % devid) 
+
+class Device:
+    def __init__(self, deviceStr):
+        """
+        Class for each device.
+        Each vendor has its own devices dictionary.
+        """
+        s = deviceStr.strip()
+        self.ID = s.split()[0]
+        self.name = s.replace("%s  " % self.ID,"")
+        self.subdevices = {}
+
+    def report(self):
+        print "\t%s\t%s" % (self.ID, self.name)
+        for subID, subdev in self.subdevices.items():
+            subdev.report()
+
+    def addSubDevice(self, subDeviceStr):
+        """
+        Adds a subvendor, subdevice to device.
+        Uses raw line from pci.ids
+        """
+        s = subDeviceStr.strip()
+        spl = s.split()
+        subVendorID  = spl[0]
+        subDeviceID  = spl[1]
+        subDeviceName = s.split("  ")[-1]
+        devID = "%s:%s" % (subVendorID,subDeviceID)
+        self.subdevices[devID] = SubDevice(subVendorID,subDeviceID,subDeviceName)
+
+    def find_subid(self, subven, subdev):
+        subven = hex(subven)[2:]
+        subdev = hex(subdev)[2:]
+        devid ="%s:%s" % (subven, subdev)
+
+        try:
+            return self.subdevices[devid]
+        except:
+            if (subven == "ffff" and subdev == "ffff"):
+                return SubDevice("ffff", "ffff", "(All Subdevices)");
+            else:
+                return SubDevice(subven, subdev, "(Unknown Subdevice)")
+        
+
+class SubDevice:
+    """
+    Class for subdevices.
+    """
+    def __init__(self, vendor, device, name):
+        """
+        Class initializes with vendorid, deviceid and name
+        """
+        self.vendorID = vendor
+        self.deviceID = device
+        self.name = name
+
+    def report(self):
+        print "\t\t%s\t%s\t%s" % (self.vendorID, self.deviceID,self.name)
+
+class PCIIds:
+    """
+    Top class for all pci.ids entries.
+    All queries will be asked to this class.
+    PCIIds.vendors["0e11"].devices["0046"].subdevices["0e11:4091"].name  =  "Smart Array 6i"
+    """
+    def __init__(self, filename):
+        """
+        Prepares the directories.
+        Checks local data file.
+        Tries to load from local, if not found, downloads from web
+        """
+        self.version = ""
+        self.date = ""
+        self.vendors = {}
+        self.contents = None
+        self.readLocal(filename)
+        self.parse()
+
+    def reportVendors(self):
+        """Reports the vendors
+        """
+        for vid, v in self.vendors.items():
+            print v.ID, v.name
+
+    def report(self, vendor = None):
+        """
+        Reports everything for all vendors or a specific vendor
+        PCIIds.report()  reports everything
+        PCIIDs.report("0e11") reports only "Compaq Computer Corporation"
+        """
+        if vendor != None:
+            self.vendors[vendor].report()
+        else:
+            for vID, v in self.vendors.items():
+                v.report()
+
+    def find_vendor(self, vid):
+        # convert vid to a hex string and remove the 0x
+        vid = hex(vid)[2:]
+
+        try:
+            return self.vendors[vid]
+        except: 
+            return Vendor("%s Unknown Vendor" % (vid))
+
+    def findDate(self, content):
+        for l in content:
+            if l.find("Date:") > -1:
+                return l.split()[-2].replace("-", "")
+        return None
+
+    def parse(self):
+        if len(self.contents) < 1:
+            print "data/%s-pci.ids not found" % self.date
+        else:
+            vendorID = ""
+            deviceID = ""
+            for l in self.contents:
+                if l[0] == "#":
+                    continue
+                elif len(l.strip()) == 0:
+                    continue
+                else:
+                    if l.find("\t\t") == 0:
+                        self.vendors[vendorID].devices[deviceID].addSubDevice(l)
+                    elif l.find("\t") == 0:
+                        deviceID = l.strip().split()[0]
+                        self.vendors[vendorID].addDevice(l)
+                    else:
+                        vendorID = l.split()[0]
+                        self.vendors[vendorID] = Vendor(l)
+
+    def readLocal(self, filename):
+        """
+        Reads the local file
+        """
+        self.contents = open(filename).readlines()
+        self.date = self.findDate(self.contents)
+
+    def loadLocal(self):
+        """
+        Loads database from local. If there is no file,
+        it creates a new one from web
+        """
+        self.date = idsfile[0].split("/")[1].split("-")[0]
+        self.readLocal()
+
+
+
+#=======================================
+
+def search_file(filename, search_path):
+    """ Given a search path, find file with requested name """
+    for path in string.split(search_path, ":"):
+        candidate = os.path.join(path, filename)
+        if os.path.exists(candidate): return os.path.abspath(candidate)
+    return None
+
+class ReadElf(object):
+    """ display_* methods are used to emit output into the output stream
+    """
+    def __init__(self, file, output):
+        """ file:
+                stream object with the ELF file to read
+
+            output:
+                output stream to write to
+        """
+        self.elffile = ELFFile(file)
+        self.output = output
+
+        # Lazily initialized if a debug dump is requested
+        self._dwarfinfo = None
+
+        self._versioninfo = None
+
+    def _section_from_spec(self, spec):
+        """ Retrieve a section given a "spec" (either number or name).
+            Return None if no such section exists in the file.
+        """
+        try:
+            num = int(spec)
+            if num < self.elffile.num_sections():
+                return self.elffile.get_section(num)
+            else:
+                return None
+        except ValueError:
+            # Not a number. Must be a name then
+            return self.elffile.get_section_by_name(str2bytes(spec))
+
+    def pretty_print_pmdinfo(self, pmdinfo):
+        global pcidb
+
+        for i in pmdinfo["pci_ids"]:
+            vendor = pcidb.find_vendor(i[0])
+            device = vendor.find_device(i[1])
+            subdev = device.find_subid(i[2], i[3])
+            print("%s (%s) : %s (%s) %s" % (vendor.name, vendor.ID, device.name, device.ID, subdev.name))
+   
+    def parse_pmd_info_string(self, mystring):
+        global raw_output
+        global pcidb
+
+        optional_pmd_info = [{'id': 'params', 'tag' : 'PMD PARAMETERS'}]
+
+        i = mystring.index("=");
+        mystring = mystring[i+2:]
+        pmdinfo = json.loads(mystring)
+
+        if raw_output:
+            print(pmdinfo)
+            return
+
+        print("PMD NAME: " + pmdinfo["name"])
+        print("PMD TYPE: " + pmdinfo["type"])
+        for i in optional_pmd_info:
+            try:
+                print("%s: %s" % (i['tag'], pmdinfo[i['id']]))
+            except KeyError as e:
+                continue
+
+        if (pmdinfo["type"] == "PMD_PDEV"):
+            print("PMD HW SUPPORT:")
+            if pcidb != None:
+                self.pretty_print_pmdinfo(pmdinfo)
+            else:
+                print("VENDOR\t DEVICE\t SUBVENDOR\t SUBDEVICE")
+                for i in pmdinfo["pci_ids"]:
+                    print("0x%04x\t 0x%04x\t 0x%04x\t\t 0x%04x" % (i[0], i[1], i[2], i[3]))
+
+        print("")
+
+
+    def display_pmd_info_strings(self, section_spec):
+        """ Display a strings dump of a section. section_spec is either a
+            section number or a name.
+        """
+        section = self._section_from_spec(section_spec)
+        if section is None:
+            return
+
+
+        data = section.data()
+        dataptr = 0
+
+        while dataptr < len(data):
+            while ( dataptr < len(data) and
+                    not (32 <= byte2int(data[dataptr]) <= 127)):
+                dataptr += 1
+
+            if dataptr >= len(data):
+                break
+
+            endptr = dataptr
+            while endptr < len(data) and byte2int(data[endptr]) != 0:
+                endptr += 1
+
+            mystring = bytes2str(data[dataptr:endptr])
+            rc = mystring.find("PMD_INFO_STRING")
+            if (rc != -1):
+                self.parse_pmd_info_string(mystring)
+
+            dataptr = endptr
+
+    def search_for_autoload_path(self):
+        section = self._section_from_spec(".rodata")
+        if section is None:
+            return None
+
+        data = section.data()
+        dataptr = 0
+
+        while dataptr < len(data):
+            while ( dataptr < len(data) and
+                    not (32 <= byte2int(data[dataptr]) <= 127)):
+                dataptr += 1
+
+            if dataptr >= len(data):
+                break
+
+            endptr = dataptr
+            while endptr < len(data) and byte2int(data[endptr]) != 0:
+                endptr += 1
+
+            mystring = bytes2str(data[dataptr:endptr])
+            rc = mystring.find("DPDK_PLUGIN_PATH")
+            if (rc != -1):
+                rc = mystring.find("=")
+                return mystring[rc+1:]
+
+            dataptr = endptr
+        return None
+
+    def get_dt_runpath(self, dynsec):
+        for tag in dynsec.iter_tags():
+            if tag.entry.d_tag == 'DT_RUNPATH':
+                return tag.runpath
+        return ""
+
+
+    def process_dt_needed_entries(self):
+        """ Look to see if there are any DT_NEEDED entries in the binary
+            And process those if there are
+        """
+        global raw_output
+        runpath = ""
+        ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+        if (ldlibpath == None):
+            ldlibpath = ""
+
+        dynsec = self._section_from_spec(".dynamic")
+        try:
+            runpath = self.get_dt_runpath(dynsec)
+        except AttributeError:
+            # dynsec is None, just return
+            return
+
+        for tag in dynsec.iter_tags():
+            if tag.entry.d_tag == 'DT_NEEDED':
+                rc = tag.needed.find("librte_pmd")
+                if (rc != -1):
+                    library = search_file(tag.needed,
+                            runpath + ":" + ldlibpath +
+                            ":/usr/lib64:/lib64:/usr/lib:/lib")
+                    if (library != None):
+                        if (raw_output == False):
+                            print("Scanning %s for pmd information" % library)
+                        with open(library, 'rb') as file:
+                            libelf = ReadElf(file, sys.stdout)
+                            libelf.process_dt_needed_entries()
+                            libelf.display_pmd_info_strings(".rodata")
+                            file.close()
+
+def scan_autoload_path(autoload_path):
+    global raw_output
+
+    if os.path.exists(autoload_path) == False:
+        return
+
+    for d in os.listdir(autoload_path):
+        dpath = os.path.join(autoload_path, d)
+        if os.path.isdir(dpath):
+            scan_autoload_path(dpath)
+        if os.path.isfile(dpath):
+            if (raw_output == False):
+                print("Hw Support for library %s" % d)
+            file = open(dpath, 'rb')
+            readelf = ReadElf(file, sys.stdout)
+            readelf.display_pmd_info_strings(".rodata")
+            file.close()
+
+
+def scan_for_autoload_pmds(dpdk_path):
+    """
+    search the specified application or path for a pmd autoload path
+    then scan said path for pmds and report hw support
+    """
+    global raw_output
+
+    if (os.path.isfile(dpdk_path) == False):
+        # Assume this is a directory and we need to scan librte_eal.so
+        dpdk_path=os.path.join(dpdk_path, "librte_eal.so")
+
+    file = open(dpdk_path, 'rb')
+    readelf = ReadElf(file, sys.stdout)
+    autoload_path = readelf.search_for_autoload_path()
+    if (autoload_path == None or autoload_path == ""):
+        if (raw_output == False):
+            print("No autoload path configured in %s" % dpdk_path)
+        return
+    if (raw_output == False):
+        print("Found autoload path %s in %s" % (autoload_path, dpdk_path))
+
+    file.close()
+    if (raw_output == False):
+        print("Discovered Autoload HW Support:") 
+    scan_autoload_path(autoload_path) 
+    return
+    
+    
+def main(stream=None):
+    global raw_output
+    global pcidb
+
+    optparser = OptionParser(
+            usage='usage: %prog [-hrt] [-p <dpdk dir>] [-d <pci id file] <elf-file>',
+            description="Dump pmd hardware support info",
+            add_help_option=True,
+            prog='pmdinfo.py')
+    optparser.add_option('-r', '--raw',
+            action='store_true', dest='raw_output',
+            help='Dump raw json strings')
+    optparser.add_option("-d", "--pcidb", dest="pcifile",
+            help="specify a pci database to get vendor names from",
+            default="/usr/share/hwdata/pci.ids", metavar="FILE")
+    optparser.add_option("-t", "--table", dest="tblout",
+            help="output infomation on hw support as a hex table",
+            action='store_true')
+    optparser.add_option("-p", "--plugindir", dest="pdir",
+            help="scan dpdk for autoload plugins", metavar="PATH|APP")
+
+    options, args = optparser.parse_args()
+
+    if options.raw_output:
+        raw_output = True
+
+    if options.pcifile:
+        pcidb = PCIIds(options.pcifile)
+        if pcidb == None:
+            print("Pci DB file not found")
+            exit(1)
+
+    if options.tblout:   
+        options.pcifile = None
+        pcidb = None
+
+    if options.pdir:
+        exit (scan_for_autoload_pmds(options.pdir))
+
+    ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+    if (ldlibpath == None):
+        ldlibpath = ""
+
+    if (os.path.exists(args[0]) == True):
+        myelffile = args[0]
+    else:
+        myelffile = search_file(args[0], ldlibpath + ":/usr/lib64:/lib64:/usr/lib:/lib")
+
+    if (myelffile == None):
+        print("File not found")
+        sys.exit(1)
+
+    with open(myelffile, 'rb') as file:
+        try:
+            readelf = ReadElf(file, sys.stdout)
+            readelf.process_dt_needed_entries()
+            readelf.display_pmd_info_strings(".rodata") 
+            sys.exit(0)
+ 
+        except ELFError as ex:
+            sys.stderr.write('ELF error: %s\n' % ex)
+            sys.exit(1)
+
+
+#-------------------------------------------------------------------------------
+if __name__ == '__main__':
+    main()
+
+
-- 
2.5.5

Re: [PATCH 4/4] pmd_hw_support.py: Add tool to query binaries for hw support information

[Panu Matilainen]

On 05/20/2016 05:06 PM, Neil Horman wrote:
[...]
> Look, I think we're simply not going to agree on this issue at all.  What about
> this in the way of compromise.  I simply am not comfortable with automatically
> trying to guess what hardware support will exist in an application based on the
> transient contents of a plugin directory, because of all the reasons we've
> already gone over, but I do understand the desire to get information about what
> _might_ be automatically loaded for an application.  what if we added a 'plugin
> mode' to pmdinfo. In this mode you would dpecify a dpdk installation directory
> and an appropriate mode option.  When specified pmdinfo would scan librte_eal in
> the specified directory, looking for an exported json string that informs us of
> the configured plugin directory.  If found, we iterate through all the libraries
> there displaying hw support.  That allows you to query the plugin directory for
> available hardware support, while not implying that the application is
> guaranteed to get it (because you didn't specifically state on the command line
> that you wanted to know about the applications hardware support).

That brings it all to one tiny step away from what I've been asking: 
have the plugin mode automatically locate librte_eal from an executable. 
So I'm not quite sure where the compromise is supposed to be here :)

I do appreciate wanting to differentiate between "physically" linked-in 
and runtime-loaded hardware support, they obviously *are* different from 
a technical POV. But its also a difference an average user might not 
even know about or understand, let alone care about, they just want to 
know "will it work?"

	- Panu -

Re: [PATCH 4/4] pmd_hw_support.py: Add tool to query binaries for hw support information

[Neil Horman]

On Mon, May 23, 2016 at 02:56:18PM +0300, Panu Matilainen wrote:
> On 05/20/2016 05:06 PM, Neil Horman wrote:
> [...]
> > Look, I think we're simply not going to agree on this issue at all.  What about
> > this in the way of compromise.  I simply am not comfortable with automatically
> > trying to guess what hardware support will exist in an application based on the
> > transient contents of a plugin directory, because of all the reasons we've
> > already gone over, but I do understand the desire to get information about what
> > _might_ be automatically loaded for an application.  what if we added a 'plugin
> > mode' to pmdinfo. In this mode you would dpecify a dpdk installation directory
> > and an appropriate mode option.  When specified pmdinfo would scan librte_eal in
> > the specified directory, looking for an exported json string that informs us of
> > the configured plugin directory.  If found, we iterate through all the libraries
> > there displaying hw support.  That allows you to query the plugin directory for
> > available hardware support, while not implying that the application is
> > guaranteed to get it (because you didn't specifically state on the command line
> > that you wanted to know about the applications hardware support).
> 
> That brings it all to one tiny step away from what I've been asking: have
> the plugin mode automatically locate librte_eal from an executable. So I'm
> not quite sure where the compromise is supposed to be here :)
The compromise is that I'm not willing to quietly assume that a given
application linked to the dpdk library in /usr/lib64/dpdk-<version>, will get
hardware support for the cxgb4, mlx5 and ixgbe pmds, because those DSO's are in
the exported RTE_EAL_PMD_PATH.  With this method, you at least have to tell the
pmdinfo application that I wish to scan that path for pmds and report on
hardware support for whatever is found there.  Thats a different operation from
getting a report on what hardware an application supports.  i.e. its the
difference between asking the questions:

"What hardware does the application /usr/bin/foo support"
and
"What hardware support is available via the plugin DSO's pointed to by the dpdk
version in /usr/lib64/dpdk-2.2"

I feel its important for users to understand that autoloading doesn't not
guarantee support for the hardware that is autoloaded to any application.  My
compromise is to provide what your asking for, but doing so in a way that
attempts to make that differentiation clear.

> 
> I do appreciate wanting to differentiate between "physically" linked-in and
> runtime-loaded hardware support, they obviously *are* different from a
> technical POV. But its also a difference an average user might not even know
> about or understand, let alone care about, they just want to know "will it
> work?"
> 

Which average user are we talking about here?  Keep in mind the DPDK is
anything but mainstream.  Its a toolkit to implement high speed network
communications for niche or custom purpose applications.  The users of tools
like this are going to be people who understand the nuance of how
applications are built and want to tune them to work at their most efficient
point, not some person just trying to get firefox to connect to digg.  I think
its reasonable to assume that people who are running the tool have sufficient
knoweldge to understand that DSO's and static binaries may embody hardware
support differently, and that things which are loaded at run time may not be
reported at scan time (by way of corressponding example, though its not perfect,
people seem to understand that iscsid supports lots of different HBA's, but the
specific hardware support for those HBA's isn't codified in /usr/sbin/iscsid,
but rather in the modules under /lib/modules/<kversion/...).  I understand thats
a lousy comparison, as the notion of static linkage of the kernel doesn't really
compare to static application linkage, but still, people can figure out whats
going on there pretty readily, and I think they can here too.

As to your other comment, yes, the end users just want to know "will it work".
But from a philisophical standpont, I don't agree with your assertion that
we can answer this question by doing what you're asking me to do.  The answer to
"Will my application support the hardware on this system with the plugins found
in RTE_EAL_PMD_PATH?" is "I don't know".  Thats because you don't know what the
contents of that directory will be when the application is run later.  The only
things we can tell at the time we run pmdinfo are:

1) What the hardware support of a static binary is for its linked in libraries

2) What the hardware support of a dynamic binary is via its DT_NEEDED entries

3) What the hardware support of a specific PMD DSO is

I am fundamentally opposed to trying to guess what hardware support will be
loaded dynamically via dlopen methods when an application is run at a later
time, and firmly believe that it is more consistent to simply not report that
information in both the static and dynamic case, and educate the user about how
to determine hardware support for dynamically loaded PMD's (perhaps a man page
would be worthwhile here)

Neil

> 	- Panu -
> 
> 

Re: [PATCH 4/4] pmd_hw_support.py: Add tool to query binaries for hw support information

[Panu Matilainen]

On 05/23/2016 04:55 PM, Neil Horman wrote:
> On Mon, May 23, 2016 at 02:56:18PM +0300, Panu Matilainen wrote:
>> On 05/20/2016 05:06 PM, Neil Horman wrote:
>> [...]
>>> Look, I think we're simply not going to agree on this issue at all.  What about
>>> this in the way of compromise.  I simply am not comfortable with automatically
>>> trying to guess what hardware support will exist in an application based on the
>>> transient contents of a plugin directory, because of all the reasons we've
>>> already gone over, but I do understand the desire to get information about what
>>> _might_ be automatically loaded for an application.  what if we added a 'plugin
>>> mode' to pmdinfo. In this mode you would dpecify a dpdk installation directory
>>> and an appropriate mode option.  When specified pmdinfo would scan librte_eal in
>>> the specified directory, looking for an exported json string that informs us of
>>> the configured plugin directory.  If found, we iterate through all the libraries
>>> there displaying hw support.  That allows you to query the plugin directory for
>>> available hardware support, while not implying that the application is
>>> guaranteed to get it (because you didn't specifically state on the command line
>>> that you wanted to know about the applications hardware support).
>>
>> That brings it all to one tiny step away from what I've been asking: have
>> the plugin mode automatically locate librte_eal from an executable. So I'm
>> not quite sure where the compromise is supposed to be here :)
> The compromise is that I'm not willing to quietly assume that a given
> application linked to the dpdk library in /usr/lib64/dpdk-<version>, will get
> hardware support for the cxgb4, mlx5 and ixgbe pmds, because those DSO's are in
> the exported RTE_EAL_PMD_PATH.

Why not? I dont get it.

> With this method, you at least have to tell the
> pmdinfo application that I wish to scan that path for pmds and report on
> hardware support for whatever is found there.  Thats a different operation from
> getting a report on what hardware an application supports.  i.e. its the
> difference between asking the questions:
>
> "What hardware does the application /usr/bin/foo support"
> and
> "What hardware support is available via the plugin DSO's pointed to by the dpdk
> version in /usr/lib64/dpdk-2.2"

Well, for the application to be able to load any PMDs, it will have to 
be linked to some version of librte_eal, which will have to be somewhere 
in the library search path (or use rpath).


> I feel its important for users to understand that autoloading doesn't not
> guarantee support for the hardware that is autoloaded to any application.  My
> compromise is to provide what your asking for, but doing so in a way that
> attempts to make that differentiation clear.

I would think requiring a specific option to enable the plugin scan 
should be quite enough to make that point clear.

>>
>> I do appreciate wanting to differentiate between "physically" linked-in and
>> runtime-loaded hardware support, they obviously *are* different from a
>> technical POV. But its also a difference an average user might not even know
>> about or understand, let alone care about, they just want to know "will it
>> work?"
>>
>
> Which average user are we talking about here?  Keep in mind the DPDK is
> anything but mainstream.  Its a toolkit to implement high speed network
> communications for niche or custom purpose applications.  The users of tools
> like this are going to be people who understand the nuance of how
> applications are built and want to tune them to work at their most efficient
> point, not some person just trying to get firefox to connect to digg.  I think
> its reasonable to assume that people who are running the tool have sufficient
> knoweldge to understand that DSO's and static binaries may embody hardware
> support differently, and that things which are loaded at run time may not be
> reported at scan time (by way of corressponding example, though its not perfect,
> people seem to understand that iscsid supports lots of different HBA's, but the
> specific hardware support for those HBA's isn't codified in /usr/sbin/iscsid,
> but rather in the modules under /lib/modules/<kversion/...).  I understand thats
> a lousy comparison, as the notion of static linkage of the kernel doesn't really
> compare to static application linkage, but still, people can figure out whats
> going on there pretty readily, and I think they can here too.
>
> As to your other comment, yes, the end users just want to know "will it work".
> But from a philisophical standpont, I don't agree with your assertion that
> we can answer this question by doing what you're asking me to do.  The answer to
> "Will my application support the hardware on this system with the plugins found
> in RTE_EAL_PMD_PATH?" is "I don't know".  Thats because you don't know what the
> contents of that directory will be when the application is run later.

Come on. RTE_EAL_PMD_PATH is expected to point to a system directory 
owned by root or such, stuff just doesn't randomly come and go. 
Everything is subject to root changing system configuration between now 
and some later time.

> The only things we can tell at the time we run pmdinfo are:
>
> 1) What the hardware support of a static binary is for its linked in libraries
>
> 2) What the hardware support of a dynamic binary is via its DT_NEEDED entries
>
> 3) What the hardware support of a specific PMD DSO is
>
> I am fundamentally opposed to trying to guess what hardware support will be
> loaded dynamically via dlopen methods when an application is run at a later
> time, and firmly believe that it is more consistent to simply not report that
> information in both the static and dynamic case, and educate the user about how
> to determine hardware support for dynamically loaded PMD's (perhaps a man page
> would be worthwhile here)

Well, with the plugin mode and pmd path export in your v3 patches 
(thanks for that!) all the necessary pieces are there so fishing out the 
information is a rather trivial one-liner of a script now:

---
#!/bin/sh

/usr/share/dpdk/tools/pmdinfo.py -p $(ldd $1 |awk '/librte_eal/{print $3}')
---

I fail to understand how having a separate script to handle this is 
better than having pmdinfo do it directly when run in the plugin mode, 
but as you wish.

	- Panu -

Re: [PATCHv3 2/5] drivers: Update driver registration macro usage

[Panu Matilainen]

On 05/20/2016 08:24 PM, Neil Horman wrote:
> Modify the PMD_REGISTER_DRIVER macro, adding a name argument to it.  The
> addition of a name argument creates a token that can be used for subsequent
> macros in the creation of unique symbol names to export additional bits of
> information for use by the pmdinfogen tool.  For example:
>
> PMD_REGISTER_DRIVER(ena_driver, ena);
>
[..]
> diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c b/drivers/net/bnx2x/bnx2x_ethdev.c
> index 071b44f..ee9581b 100644
> --- a/drivers/net/bnx2x/bnx2x_ethdev.c
> +++ b/drivers/net/bnx2x/bnx2x_ethdev.c
> @@ -550,5 +550,7 @@ static struct rte_driver rte_bnx2xvf_driver = {
>  	.init = rte_bnx2xvf_pmd_init,
>  };
>
> -PMD_REGISTER_DRIVER(rte_bnx2x_driver);
> -PMD_REGISTER_DRIVER(rte_bnx2xvf_driver);
> +PMD_REGISTER_DRIVER(rte_bnx2x_driver, pci_id_bnx2x_map, bnx2x);
> +DRIVER_EXPORT_PCI_TABLE(bnx2x, pci_id_bnx2x_map);
> +PMD_REGISTER_DRIVER(rte_bnx2xvf_driver, pci_id_bnx2xvf_map, bnx2xvf);
> +DRIVER_EXPORT_PCI_TABLE(bnx2xvf, pci_id_bnx2xvf_map);

There are extra arguments to PMD_REGISTER_DRIVER() here, causing build 
failure. Its easy to miss since bnx2x doesn't get built by default, 
dunno if it was in earlier versions already (if it was, sorry for 
missing it).

	- Panu -

Re: [PATCHv3 5/5] pmdinfo.py: Add tool to query binaries for hw and other support information

[Panu Matilainen]

On 05/20/2016 08:24 PM, Neil Horman wrote:
> This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
> and, if found parses the remainder of the string as a json encoded string,
> outputting the results in either a human readable or raw, script parseable
> format
>
> Note that, in the case of dynamically linked applications, pmdinfo.py will scan
> for implicitly linked PMDs by searching the specified binaries .dynamic section
> for DT_NEEDED entries that contain the substring librte_pmd.  The DT_RUNPATH,
> LD_LIBRARY_PATH, /usr/lib and /lib are searched for these libraries, in that
> order
>
> If a file is specified with no path, it is assumed to be a PMD DSO, and the
> LD_LIBRARY_PATH, /usr/lib[64]/ and /lib[64] is searched for it
>
> Currently the tool can output data in 3 formats:
>
> a) raw, suitable for scripting, where the raw JSON strings are dumped out
> b) table format (default) where hex pci ids are dumped in a table format
> c) pretty, where a user supplied pci.ids file is used to print out vendor and
> device strings
>
> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> CC: Bruce Richardson <bruce.richardson@intel.com>
> CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> CC: Stephen Hemminger <stephen@networkplumber.org>
> CC: Panu Matilainen <pmatilai@redhat.com>
> ---
>  tools/pmdinfo.py | 545 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 545 insertions(+)
>  create mode 100755 tools/pmdinfo.py
>
> diff --git a/tools/pmdinfo.py b/tools/pmdinfo.py
> new file mode 100755
> index 0000000..9b4b4a4
> --- /dev/null
> +++ b/tools/pmdinfo.py
> @@ -0,0 +1,545 @@
> +#!/usr/bin/python
> +#-------------------------------------------------------------------------------
> +# scripts/pmd_hw_support.py
> +#
> +# Utility to dump PMD_INFO_STRING support from an object file
> +#
> +#-------------------------------------------------------------------------------
> +import os, sys
> +from optparse import OptionParser
> +import string
> +import json
> +
> +# For running from development directory. It should take precedence over the
> +# installed pyelftools.
> +sys.path.insert(0, '.')
> +
> +
> +from elftools import __version__
> +from elftools.common.exceptions import ELFError
> +from elftools.common.py3compat import (
> +        ifilter, byte2int, bytes2str, itervalues, str2bytes)
> +from elftools.elf.elffile import ELFFile
> +from elftools.elf.dynamic import DynamicSection, DynamicSegment
> +from elftools.elf.enums import ENUM_D_TAG
> +from elftools.elf.segments import InterpSegment
> +from elftools.elf.sections import SymbolTableSection
> +from elftools.elf.gnuversions import (
> +    GNUVerSymSection, GNUVerDefSection,
> +    GNUVerNeedSection,
> +    )
> +from elftools.elf.relocation import RelocationSection
> +from elftools.elf.descriptions import (
> +    describe_ei_class, describe_ei_data, describe_ei_version,
> +    describe_ei_osabi, describe_e_type, describe_e_machine,
> +    describe_e_version_numeric, describe_p_type, describe_p_flags,
> +    describe_sh_type, describe_sh_flags,
> +    describe_symbol_type, describe_symbol_bind, describe_symbol_visibility,
> +    describe_symbol_shndx, describe_reloc_type, describe_dyn_tag,
> +    describe_ver_flags,
> +    )
> +from elftools.elf.constants import E_FLAGS
> +from elftools.dwarf.dwarfinfo import DWARFInfo
> +from elftools.dwarf.descriptions import (
> +    describe_reg_name, describe_attr_value, set_global_machine_arch,
> +    describe_CFI_instructions, describe_CFI_register_rule,
> +    describe_CFI_CFA_rule,
> +    )
> +from elftools.dwarf.constants import (
> +    DW_LNS_copy, DW_LNS_set_file, DW_LNE_define_file)
> +from elftools.dwarf.callframe import CIE, FDE
> +
> +raw_output = False
> +pcidb = None
> +
> +#===========================================
> +
> +class Vendor:
> +    """
> +    Class for vendors. This is the top level class
> +    for the devices belong to a specific vendor.
> +    self.devices is the device dictionary
> +    subdevices are in each device.
> +    """
> +    def __init__(self, vendorStr):
> +        """
> +        Class initializes with the raw line from pci.ids
> +        Parsing takes place inside __init__
> +        """
> +        self.ID = vendorStr.split()[0]
> +        self.name = vendorStr.replace("%s " % self.ID,"").rstrip()
> +        self.devices = {}
> +
> +    def addDevice(self, deviceStr):
> +        """
> +        Adds a device to self.devices
> +        takes the raw line from pci.ids
> +        """
> +        s = deviceStr.strip()
> +        devID = s.split()[0]
> +        if devID in self.devices:
> +            pass
> +        else:
> +            self.devices[devID] = Device(deviceStr)
> +
> +    def report(self):
> +        print self.ID, self.name
> +        for id, dev in self.devices.items():
> +            dev.report()
> +
> +    def find_device(self, devid):
> +        # convert to a hex string and remove 0x
> +        devid = hex(devid)[2:]
> +        try:
> +            return self.devices[devid]
> +        except:
> +            return Device("%s  Unknown Device" % devid)
> +
> +class Device:
> +    def __init__(self, deviceStr):
> +        """
> +        Class for each device.
> +        Each vendor has its own devices dictionary.
> +        """
> +        s = deviceStr.strip()
> +        self.ID = s.split()[0]
> +        self.name = s.replace("%s  " % self.ID,"")
> +        self.subdevices = {}
> +
> +    def report(self):
> +        print "\t%s\t%s" % (self.ID, self.name)
> +        for subID, subdev in self.subdevices.items():
> +            subdev.report()
> +
> +    def addSubDevice(self, subDeviceStr):
> +        """
> +        Adds a subvendor, subdevice to device.
> +        Uses raw line from pci.ids
> +        """
> +        s = subDeviceStr.strip()
> +        spl = s.split()
> +        subVendorID  = spl[0]
> +        subDeviceID  = spl[1]
> +        subDeviceName = s.split("  ")[-1]
> +        devID = "%s:%s" % (subVendorID,subDeviceID)
> +        self.subdevices[devID] = SubDevice(subVendorID,subDeviceID,subDeviceName)
> +
> +    def find_subid(self, subven, subdev):
> +        subven = hex(subven)[2:]
> +        subdev = hex(subdev)[2:]
> +        devid ="%s:%s" % (subven, subdev)
> +
> +        try:
> +            return self.subdevices[devid]
> +        except:
> +            if (subven == "ffff" and subdev == "ffff"):
> +                return SubDevice("ffff", "ffff", "(All Subdevices)");
> +            else:
> +                return SubDevice(subven, subdev, "(Unknown Subdevice)")
> +
> +
> +class SubDevice:
> +    """
> +    Class for subdevices.
> +    """
> +    def __init__(self, vendor, device, name):
> +        """
> +        Class initializes with vendorid, deviceid and name
> +        """
> +        self.vendorID = vendor
> +        self.deviceID = device
> +        self.name = name
> +
> +    def report(self):
> +        print "\t\t%s\t%s\t%s" % (self.vendorID, self.deviceID,self.name)
> +
> +class PCIIds:
> +    """
> +    Top class for all pci.ids entries.
> +    All queries will be asked to this class.
> +    PCIIds.vendors["0e11"].devices["0046"].subdevices["0e11:4091"].name  =  "Smart Array 6i"
> +    """
> +    def __init__(self, filename):
> +        """
> +        Prepares the directories.
> +        Checks local data file.
> +        Tries to load from local, if not found, downloads from web
> +        """
> +        self.version = ""
> +        self.date = ""
> +        self.vendors = {}
> +        self.contents = None
> +        self.readLocal(filename)
> +        self.parse()
> +
> +    def reportVendors(self):
> +        """Reports the vendors
> +        """
> +        for vid, v in self.vendors.items():
> +            print v.ID, v.name
> +
> +    def report(self, vendor = None):
> +        """
> +        Reports everything for all vendors or a specific vendor
> +        PCIIds.report()  reports everything
> +        PCIIDs.report("0e11") reports only "Compaq Computer Corporation"
> +        """
> +        if vendor != None:
> +            self.vendors[vendor].report()
> +        else:
> +            for vID, v in self.vendors.items():
> +                v.report()
> +
> +    def find_vendor(self, vid):
> +        # convert vid to a hex string and remove the 0x
> +        vid = hex(vid)[2:]
> +
> +        try:
> +            return self.vendors[vid]
> +        except:
> +            return Vendor("%s Unknown Vendor" % (vid))
> +
> +    def findDate(self, content):
> +        for l in content:
> +            if l.find("Date:") > -1:
> +                return l.split()[-2].replace("-", "")
> +        return None
> +
> +    def parse(self):
> +        if len(self.contents) < 1:
> +            print "data/%s-pci.ids not found" % self.date
> +        else:
> +            vendorID = ""
> +            deviceID = ""
> +            for l in self.contents:
> +                if l[0] == "#":
> +                    continue
> +                elif len(l.strip()) == 0:
> +                    continue
> +                else:
> +                    if l.find("\t\t") == 0:
> +                        self.vendors[vendorID].devices[deviceID].addSubDevice(l)
> +                    elif l.find("\t") == 0:
> +                        deviceID = l.strip().split()[0]
> +                        self.vendors[vendorID].addDevice(l)
> +                    else:
> +                        vendorID = l.split()[0]
> +                        self.vendors[vendorID] = Vendor(l)
> +
> +    def readLocal(self, filename):
> +        """
> +        Reads the local file
> +        """
> +        self.contents = open(filename).readlines()
> +        self.date = self.findDate(self.contents)
> +
> +    def loadLocal(self):
> +        """
> +        Loads database from local. If there is no file,
> +        it creates a new one from web
> +        """
> +        self.date = idsfile[0].split("/")[1].split("-")[0]
> +        self.readLocal()
> +
> +
> +
> +#=======================================
> +
> +def search_file(filename, search_path):
> +    """ Given a search path, find file with requested name """
> +    for path in string.split(search_path, ":"):
> +        candidate = os.path.join(path, filename)
> +        if os.path.exists(candidate): return os.path.abspath(candidate)
> +    return None
> +
> +class ReadElf(object):
> +    """ display_* methods are used to emit output into the output stream
> +    """
> +    def __init__(self, file, output):
> +        """ file:
> +                stream object with the ELF file to read
> +
> +            output:
> +                output stream to write to
> +        """
> +        self.elffile = ELFFile(file)
> +        self.output = output
> +
> +        # Lazily initialized if a debug dump is requested
> +        self._dwarfinfo = None
> +
> +        self._versioninfo = None
> +
> +    def _section_from_spec(self, spec):
> +        """ Retrieve a section given a "spec" (either number or name).
> +            Return None if no such section exists in the file.
> +        """
> +        try:
> +            num = int(spec)
> +            if num < self.elffile.num_sections():
> +                return self.elffile.get_section(num)
> +            else:
> +                return None
> +        except ValueError:
> +            # Not a number. Must be a name then
> +            return self.elffile.get_section_by_name(str2bytes(spec))
> +
> +    def pretty_print_pmdinfo(self, pmdinfo):
> +        global pcidb
> +
> +        for i in pmdinfo["pci_ids"]:
> +            vendor = pcidb.find_vendor(i[0])
> +            device = vendor.find_device(i[1])
> +            subdev = device.find_subid(i[2], i[3])
> +            print("%s (%s) : %s (%s) %s" % (vendor.name, vendor.ID, device.name, device.ID, subdev.name))
> +
> +    def parse_pmd_info_string(self, mystring):
> +        global raw_output
> +        global pcidb
> +
> +        optional_pmd_info = [{'id': 'params', 'tag' : 'PMD PARAMETERS'}]
> +
> +        i = mystring.index("=");
> +        mystring = mystring[i+2:]
> +        pmdinfo = json.loads(mystring)
> +
> +        if raw_output:
> +            print(pmdinfo)
> +            return
> +
> +        print("PMD NAME: " + pmdinfo["name"])
> +        print("PMD TYPE: " + pmdinfo["type"])
> +        for i in optional_pmd_info:
> +            try:
> +                print("%s: %s" % (i['tag'], pmdinfo[i['id']]))
> +            except KeyError as e:
> +                continue
> +
> +        if (pmdinfo["type"] == "PMD_PDEV"):
> +            print("PMD HW SUPPORT:")
> +            if pcidb != None:
> +                self.pretty_print_pmdinfo(pmdinfo)
> +            else:
> +                print("VENDOR\t DEVICE\t SUBVENDOR\t SUBDEVICE")
> +                for i in pmdinfo["pci_ids"]:
> +                    print("0x%04x\t 0x%04x\t 0x%04x\t\t 0x%04x" % (i[0], i[1], i[2], i[3]))
> +
> +        print("")
> +
> +
> +    def display_pmd_info_strings(self, section_spec):
> +        """ Display a strings dump of a section. section_spec is either a
> +            section number or a name.
> +        """
> +        section = self._section_from_spec(section_spec)
> +        if section is None:
> +            return
> +
> +
> +        data = section.data()
> +        dataptr = 0
> +
> +        while dataptr < len(data):
> +            while ( dataptr < len(data) and
> +                    not (32 <= byte2int(data[dataptr]) <= 127)):
> +                dataptr += 1
> +
> +            if dataptr >= len(data):
> +                break
> +
> +            endptr = dataptr
> +            while endptr < len(data) and byte2int(data[endptr]) != 0:
> +                endptr += 1
> +
> +            mystring = bytes2str(data[dataptr:endptr])
> +            rc = mystring.find("PMD_INFO_STRING")
> +            if (rc != -1):
> +                self.parse_pmd_info_string(mystring)
> +
> +            dataptr = endptr
> +
> +    def search_for_autoload_path(self):
> +        section = self._section_from_spec(".rodata")
> +        if section is None:
> +            return None
> +
> +        data = section.data()
> +        dataptr = 0
> +
> +        while dataptr < len(data):
> +            while ( dataptr < len(data) and
> +                    not (32 <= byte2int(data[dataptr]) <= 127)):
> +                dataptr += 1
> +
> +            if dataptr >= len(data):
> +                break
> +
> +            endptr = dataptr
> +            while endptr < len(data) and byte2int(data[endptr]) != 0:
> +                endptr += 1
> +
> +            mystring = bytes2str(data[dataptr:endptr])
> +            rc = mystring.find("DPDK_PLUGIN_PATH")
> +            if (rc != -1):
> +                rc = mystring.find("=")
> +                return mystring[rc+1:]
> +
> +            dataptr = endptr
> +        return None
> +
> +    def get_dt_runpath(self, dynsec):
> +        for tag in dynsec.iter_tags():
> +            if tag.entry.d_tag == 'DT_RUNPATH':
> +                return tag.runpath
> +        return ""
> +
> +
> +    def process_dt_needed_entries(self):
> +        """ Look to see if there are any DT_NEEDED entries in the binary
> +            And process those if there are
> +        """
> +        global raw_output
> +        runpath = ""
> +        ldlibpath = os.environ.get('LD_LIBRARY_PATH')
> +        if (ldlibpath == None):
> +            ldlibpath = ""
> +
> +        dynsec = self._section_from_spec(".dynamic")
> +        try:
> +            runpath = self.get_dt_runpath(dynsec)
> +        except AttributeError:
> +            # dynsec is None, just return
> +            return
> +
> +        for tag in dynsec.iter_tags():
> +            if tag.entry.d_tag == 'DT_NEEDED':
> +                rc = tag.needed.find("librte_pmd")
> +                if (rc != -1):
> +                    library = search_file(tag.needed,
> +                            runpath + ":" + ldlibpath +
> +                            ":/usr/lib64:/lib64:/usr/lib:/lib")
> +                    if (library != None):
> +                        if (raw_output == False):
> +                            print("Scanning %s for pmd information" % library)
> +                        with open(library, 'rb') as file:
> +                            libelf = ReadElf(file, sys.stdout)
> +                            libelf.process_dt_needed_entries()
> +                            libelf.display_pmd_info_strings(".rodata")
> +                            file.close()
> +
> +def scan_autoload_path(autoload_path):
> +    global raw_output
> +
> +    if os.path.exists(autoload_path) == False:
> +        return
> +
> +    for d in os.listdir(autoload_path):
> +        dpath = os.path.join(autoload_path, d)
> +        if os.path.isdir(dpath):
> +            scan_autoload_path(dpath)
> +        if os.path.isfile(dpath):
> +            if (raw_output == False):
> +                print("Hw Support for library %s" % d)
> +            file = open(dpath, 'rb')
> +            readelf = ReadElf(file, sys.stdout)
> +            readelf.display_pmd_info_strings(".rodata")
> +            file.close()

Might want to wrap ReadElf() in try-except ELFError to avoid traceback 
in case there are other files in the directory. Not that other files are 
expected there, but EAL ignores all non-pmd files there so it'd be good 
to match that.

> +
> +
> +def scan_for_autoload_pmds(dpdk_path):
> +    """
> +    search the specified application or path for a pmd autoload path
> +    then scan said path for pmds and report hw support
> +    """
> +    global raw_output
> +
> +    if (os.path.isfile(dpdk_path) == False):
> +        # Assume this is a directory and we need to scan librte_eal.so
> +        dpdk_path=os.path.join(dpdk_path, "librte_eal.so")

The unversioned .so symlink is not guaranteed to be there, they're 
typically placed into -devel packages which wont be installed for 
"normal use" (developers are another story obviously)

Also assuming there might be multiple versions of DPDK runtime 
parallel-installed, it might point to any of them so its a bit of a lottery.

> +
> +    file = open(dpdk_path, 'rb')
> +    readelf = ReadElf(file, sys.stdout)

Might want to wrap this in try-except ELFError, its a user-entered path.

> +    autoload_path = readelf.search_for_autoload_path()
> +    if (autoload_path == None or autoload_path == ""):
> +        if (raw_output == False):
> +            print("No autoload path configured in %s" % dpdk_path)
> +        return
> +    if (raw_output == False):
> +        print("Found autoload path %s in %s" % (autoload_path, dpdk_path))
> +
> +    file.close()
> +    if (raw_output == False):
> +        print("Discovered Autoload HW Support:")
> +    scan_autoload_path(autoload_path)
> +    return
> +
> +
> +def main(stream=None):
> +    global raw_output
> +    global pcidb
> +
> +    optparser = OptionParser(
> +            usage='usage: %prog [-hrt] [-p <dpdk dir>] [-d <pci id file] <elf-file>',

Minor nit, but with the later additions the usage message isn't accurate 
anymore, <elf-file> could be a directory too so <file|dir> would be 
closer to the mark, and in plugin mode it's not looked at all. Might be 
simpler to both users + and in actual code if plugin mode is just a 
true/false flag and uses the same args[0] as argument as it too handles 
both a file and a directory case.

> +            description="Dump pmd hardware support info",
> +            add_help_option=True,
> +            prog='pmdinfo.py')
> +    optparser.add_option('-r', '--raw',
> +            action='store_true', dest='raw_output',
> +            help='Dump raw json strings')
> +    optparser.add_option("-d", "--pcidb", dest="pcifile",
> +            help="specify a pci database to get vendor names from",
> +            default="/usr/share/hwdata/pci.ids", metavar="FILE")
> +    optparser.add_option("-t", "--table", dest="tblout",
> +            help="output infomation on hw support as a hex table",
> +            action='store_true')
> +    optparser.add_option("-p", "--plugindir", dest="pdir",
> +            help="scan dpdk for autoload plugins", metavar="PATH|APP")
> +
> +    options, args = optparser.parse_args()
> +
> +    if options.raw_output:
> +        raw_output = True
> +
> +    if options.pcifile:
> +        pcidb = PCIIds(options.pcifile)
> +        if pcidb == None:
> +            print("Pci DB file not found")
> +            exit(1)
> +
> +    if options.tblout:
> +        options.pcifile = None
> +        pcidb = None
> +
> +    if options.pdir:
> +        exit (scan_for_autoload_pmds(options.pdir))
> +
> +    ldlibpath = os.environ.get('LD_LIBRARY_PATH')
> +    if (ldlibpath == None):
> +        ldlibpath = ""
> +
> +    if (os.path.exists(args[0]) == True):
> +        myelffile = args[0]
> +    else:
> +        myelffile = search_file(args[0], ldlibpath + ":/usr/lib64:/lib64:/usr/lib:/lib")

Minor nit, but this tracebacks when executing pmdinfo.py without 
arguments. Would be nicer to dump usage instead.

[pmatilai@sopuli dpdk]$ tools/pmdinfo.py
Traceback (most recent call last):
   File "tools/pmdinfo.py", line 543, in <module>
     main()
   File "tools/pmdinfo.py", line 520, in main
     if (os.path.exists(args[0]) == True):
IndexError: list index out of range

	- Panu -

> +
> +    if (myelffile == None):
> +        print("File not found")
> +        sys.exit(1)
> +
> +    with open(myelffile, 'rb') as file:
> +        try:
> +            readelf = ReadElf(file, sys.stdout)
> +            readelf.process_dt_needed_entries()
> +            readelf.display_pmd_info_strings(".rodata")
> +            sys.exit(0)
> +
> +        except ELFError as ex:
> +            sys.stderr.write('ELF error: %s\n' % ex)
> +            sys.exit(1)
> +
> +
> +#-------------------------------------------------------------------------------
> +if __name__ == '__main__':
> +    main()
> +
> +
>

Re: [PATCHv3 5/5] pmdinfo.py: Add tool to query binaries for hw and other support information

[Panu Matilainen]

On 05/20/2016 08:24 PM, Neil Horman wrote:
> This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
> and, if found parses the remainder of the string as a json encoded string,
> outputting the results in either a human readable or raw, script parseable
> format
>
> Note that, in the case of dynamically linked applications, pmdinfo.py will scan
> for implicitly linked PMDs by searching the specified binaries .dynamic section
> for DT_NEEDED entries that contain the substring librte_pmd.  The DT_RUNPATH,
> LD_LIBRARY_PATH, /usr/lib and /lib are searched for these libraries, in that
> order
>
> If a file is specified with no path, it is assumed to be a PMD DSO, and the
> LD_LIBRARY_PATH, /usr/lib[64]/ and /lib[64] is searched for it
>
> Currently the tool can output data in 3 formats:
>
> a) raw, suitable for scripting, where the raw JSON strings are dumped out
> b) table format (default) where hex pci ids are dumped in a table format
> c) pretty, where a user supplied pci.ids file is used to print out vendor and
> device strings
>
> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> CC: Bruce Richardson <bruce.richardson@intel.com>
> CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> CC: Stephen Hemminger <stephen@networkplumber.org>
> CC: Panu Matilainen <pmatilai@redhat.com>
> ---
>  tools/pmdinfo.py | 545 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 545 insertions(+)
>  create mode 100755 tools/pmdinfo.py

Oh, one more thing: assuming there'll be a v4, please add something like 
this to make the tool available in $PATH:

diff --git a/mk/rte.sdkinstall.mk b/mk/rte.sdkinstall.mk
index 68e56b6..5a6a699 100644
--- a/mk/rte.sdkinstall.mk
+++ b/mk/rte.sdkinstall.mk
@@ -126,6 +126,8 @@ install-runtime:
         $(Q)$(call rte_mkdir,      $(DESTDIR)$(sbindir))
         $(Q)$(call rte_symlink, 
$(DESTDIR)$(datadir)/tools/dpdk_nic_bind.py, \
                                    $(DESTDIR)$(sbindir)/dpdk_nic_bind)
+       $(Q)$(call rte_symlink,    $(DESTDIR)$(datadir)/tools/pmdinfo.py, \
+                                  $(DESTDIR)$(bindir)/pmdinfo)

  install-kmod:
  ifneq ($(wildcard $O/kmod/*),)

	- Panu -

Re: [PATCHv3 2/5] drivers: Update driver registration macro usage

[Neil Horman]

On Tue, May 24, 2016 at 09:57:35AM +0300, Panu Matilainen wrote:
> On 05/20/2016 08:24 PM, Neil Horman wrote:
> > Modify the PMD_REGISTER_DRIVER macro, adding a name argument to it.  The
> > addition of a name argument creates a token that can be used for subsequent
> > macros in the creation of unique symbol names to export additional bits of
> > information for use by the pmdinfogen tool.  For example:
> > 
> > PMD_REGISTER_DRIVER(ena_driver, ena);
> > 
> [..]
> > diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c b/drivers/net/bnx2x/bnx2x_ethdev.c
> > index 071b44f..ee9581b 100644
> > --- a/drivers/net/bnx2x/bnx2x_ethdev.c
> > +++ b/drivers/net/bnx2x/bnx2x_ethdev.c
> > @@ -550,5 +550,7 @@ static struct rte_driver rte_bnx2xvf_driver = {
> >  	.init = rte_bnx2xvf_pmd_init,
> >  };
> > 
> > -PMD_REGISTER_DRIVER(rte_bnx2x_driver);
> > -PMD_REGISTER_DRIVER(rte_bnx2xvf_driver);
> > +PMD_REGISTER_DRIVER(rte_bnx2x_driver, pci_id_bnx2x_map, bnx2x);
> > +DRIVER_EXPORT_PCI_TABLE(bnx2x, pci_id_bnx2x_map);
> > +PMD_REGISTER_DRIVER(rte_bnx2xvf_driver, pci_id_bnx2xvf_map, bnx2xvf);
> > +DRIVER_EXPORT_PCI_TABLE(bnx2xvf, pci_id_bnx2xvf_map);
> 
> There are extra arguments to PMD_REGISTER_DRIVER() here, causing build
> failure. Its easy to miss since bnx2x doesn't get built by default, dunno if
> it was in earlier versions already (if it was, sorry for missing it).
> 
Shoot, you're right, the middle argument should be removed, and I didn't catch
it because bnx2x isn't built by default, and I thought it was.  I'll fix that up
Neil

> 	- Panu -
> 

Re: [PATCH 4/4] pmd_hw_support.py: Add tool to query binaries for hw support information

[Neil Horman]

On Tue, May 24, 2016 at 09:15:41AM +0300, Panu Matilainen wrote:
> On 05/23/2016 04:55 PM, Neil Horman wrote:
> > On Mon, May 23, 2016 at 02:56:18PM +0300, Panu Matilainen wrote:
> > > On 05/20/2016 05:06 PM, Neil Horman wrote:
> > > [...]
> > > > Look, I think we're simply not going to agree on this issue at all.  What about
> > > > this in the way of compromise.  I simply am not comfortable with automatically
> > > > trying to guess what hardware support will exist in an application based on the
> > > > transient contents of a plugin directory, because of all the reasons we've
> > > > already gone over, but I do understand the desire to get information about what
> > > > _might_ be automatically loaded for an application.  what if we added a 'plugin
> > > > mode' to pmdinfo. In this mode you would dpecify a dpdk installation directory
> > > > and an appropriate mode option.  When specified pmdinfo would scan librte_eal in
> > > > the specified directory, looking for an exported json string that informs us of
> > > > the configured plugin directory.  If found, we iterate through all the libraries
> > > > there displaying hw support.  That allows you to query the plugin directory for
> > > > available hardware support, while not implying that the application is
> > > > guaranteed to get it (because you didn't specifically state on the command line
> > > > that you wanted to know about the applications hardware support).
> > > 
> > > That brings it all to one tiny step away from what I've been asking: have
> > > the plugin mode automatically locate librte_eal from an executable. So I'm
> > > not quite sure where the compromise is supposed to be here :)
> > The compromise is that I'm not willing to quietly assume that a given
> > application linked to the dpdk library in /usr/lib64/dpdk-<version>, will get
> > hardware support for the cxgb4, mlx5 and ixgbe pmds, because those DSO's are in
> > the exported RTE_EAL_PMD_PATH.
> 
> Why not? I dont get it.
> 
For all the reasons I've stated in the I don't even know how many emails we've
sent back and forth in this thread.  Because, once again, just because a pmd
exists in the auto load path at the time you do the scan, doesn't mean it will
be there at the time you run the application.  By creating this PMD mode, the
users has to recognize implicitly that there is something different about
autoloaded pmds, namely that they are not expressly bound to the application
itself. By specifing a special option they can still figure out what hardware
support exists in the autoload path at that time, but they have to realize that
these pmds are 'different'
 
> > With this method, you at least have to tell the
> > pmdinfo application that I wish to scan that path for pmds and report on
> > hardware support for whatever is found there.  Thats a different operation from
> > getting a report on what hardware an application supports.  i.e. its the
> > difference between asking the questions:
> > 
> > "What hardware does the application /usr/bin/foo support"
> > and
> > "What hardware support is available via the plugin DSO's pointed to by the dpdk
> > version in /usr/lib64/dpdk-2.2"
> 
> Well, for the application to be able to load any PMDs, it will have to be
> linked to some version of librte_eal, which will have to be somewhere in the
> library search path (or use rpath).
> 
Yes. but see my above complaint.  Implicitly tying the application to the PMD
autoload path is trivial, I'm not making that argument, what I'm arguing is
that, the autoload path is transient, and so telling users that application X
supports all the hardware supported by all the PMD's in the autoload path is
erroneous, because whats there the moment you scan, isn't guaranteed to be there
the moment the application runs.  Thats the compromise I've been trying to offer
you.  By creating a separate 'plugin' mode we offer the user the chance to ask
each question above independently, and thereby allow them to draw the conclusion
that tying hardware support in the autoload path is an extra step that they need
to manage (by linking to the right dpdk version, or by setting LD_LIBRARY_PATH
correctly, etc).

> 
> > I feel its important for users to understand that autoloading doesn't not
> > guarantee support for the hardware that is autoloaded to any application.  My
> > compromise is to provide what your asking for, but doing so in a way that
> > attempts to make that differentiation clear.
> 
> I would think requiring a specific option to enable the plugin scan should
> be quite enough to make that point clear.
> 
Yes, that was why I wrote the option. Soo.....Are we in agreement here?

> > > 
> > > I do appreciate wanting to differentiate between "physically" linked-in and
> > > runtime-loaded hardware support, they obviously *are* different from a
> > > technical POV. But its also a difference an average user might not even know
> > > about or understand, let alone care about, they just want to know "will it
> > > work?"
> > > 
> > 
> > Which average user are we talking about here?  Keep in mind the DPDK is
> > anything but mainstream.  Its a toolkit to implement high speed network
> > communications for niche or custom purpose applications.  The users of tools
> > like this are going to be people who understand the nuance of how
> > applications are built and want to tune them to work at their most efficient
> > point, not some person just trying to get firefox to connect to digg.  I think
> > its reasonable to assume that people who are running the tool have sufficient
> > knoweldge to understand that DSO's and static binaries may embody hardware
> > support differently, and that things which are loaded at run time may not be
> > reported at scan time (by way of corressponding example, though its not perfect,
> > people seem to understand that iscsid supports lots of different HBA's, but the
> > specific hardware support for those HBA's isn't codified in /usr/sbin/iscsid,
> > but rather in the modules under /lib/modules/<kversion/...).  I understand thats
> > a lousy comparison, as the notion of static linkage of the kernel doesn't really
> > compare to static application linkage, but still, people can figure out whats
> > going on there pretty readily, and I think they can here too.
> > 
> > As to your other comment, yes, the end users just want to know "will it work".
> > But from a philisophical standpont, I don't agree with your assertion that
> > we can answer this question by doing what you're asking me to do.  The answer to
> > "Will my application support the hardware on this system with the plugins found
> > in RTE_EAL_PMD_PATH?" is "I don't know".  Thats because you don't know what the
> > contents of that directory will be when the application is run later.
> 
> Come on. RTE_EAL_PMD_PATH is expected to point to a system directory owned
> by root or such, stuff just doesn't randomly come and go. Everything is
> subject to root changing system configuration between now and some later
> time.
> 
Sure it can, rpm's/deb/etc are added and removed regularly.  The method of
change doesn't matter, its the result that counts.

> > The only things we can tell at the time we run pmdinfo are:
> > 
> > 1) What the hardware support of a static binary is for its linked in libraries
> > 
> > 2) What the hardware support of a dynamic binary is via its DT_NEEDED entries
> > 
> > 3) What the hardware support of a specific PMD DSO is
> > 
> > I am fundamentally opposed to trying to guess what hardware support will be
> > loaded dynamically via dlopen methods when an application is run at a later
> > time, and firmly believe that it is more consistent to simply not report that
> > information in both the static and dynamic case, and educate the user about how
> > to determine hardware support for dynamically loaded PMD's (perhaps a man page
> > would be worthwhile here)
> 
> Well, with the plugin mode and pmd path export in your v3 patches (thanks
> for that!) all the necessary pieces are there so fishing out the information
> is a rather trivial one-liner of a script now:
> 
> ---
> #!/bin/sh
> 
> /usr/share/dpdk/tools/pmdinfo.py -p $(ldd $1 |awk '/librte_eal/{print $3}')
> ---
> 
> I fail to understand how having a separate script to handle this is better
> than having pmdinfo do it directly when run in the plugin mode, but as you
> wish.
> 
You don't even need a script, you just need to have the rudimentary
understanding that DPDK is architected in such a way that hardware support may
be embodied in multiple binaries, and so depending on your load method (link
time vs. run time), you may need to check a different location.  Its really not
that hard.  You have to understand that thats the case anyway if you use -d on
the command line.

> 	- Panu -
> 
> 

Re: [PATCHv3 5/5] pmdinfo.py: Add tool to query binaries for hw and other support information

[Neil Horman]

On Tue, May 24, 2016 at 10:41:28AM +0300, Panu Matilainen wrote:
> On 05/20/2016 08:24 PM, Neil Horman wrote:
> > This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
> > and, if found parses the remainder of the string as a json encoded string,
> > outputting the results in either a human readable or raw, script parseable
> > format
> > 
> > Note that, in the case of dynamically linked applications, pmdinfo.py will scan
> > for implicitly linked PMDs by searching the specified binaries .dynamic section
> > for DT_NEEDED entries that contain the substring librte_pmd.  The DT_RUNPATH,
> > LD_LIBRARY_PATH, /usr/lib and /lib are searched for these libraries, in that
> > order
> > 
> > If a file is specified with no path, it is assumed to be a PMD DSO, and the
> > LD_LIBRARY_PATH, /usr/lib[64]/ and /lib[64] is searched for it
> > 
> > Currently the tool can output data in 3 formats:
> > 
> > a) raw, suitable for scripting, where the raw JSON strings are dumped out
> > b) table format (default) where hex pci ids are dumped in a table format
> > c) pretty, where a user supplied pci.ids file is used to print out vendor and
> > device strings
> > 
> > Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> > CC: Bruce Richardson <bruce.richardson@intel.com>
> > CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> > CC: Stephen Hemminger <stephen@networkplumber.org>
> > CC: Panu Matilainen <pmatilai@redhat.com>
> > ---
> >  tools/pmdinfo.py | 545 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 545 insertions(+)
> >  create mode 100755 tools/pmdinfo.py
> > 
> > diff --git a/tools/pmdinfo.py b/tools/pmdinfo.py
> > new file mode 100755
> > index 0000000..9b4b4a4
> > --- /dev/null
> > +++ b/tools/pmdinfo.py
> > @@ -0,0 +1,545 @@
> > +#!/usr/bin/python
> > +#-------------------------------------------------------------------------------
> > +# scripts/pmd_hw_support.py
> > +#
> > +# Utility to dump PMD_INFO_STRING support from an object file
> > +#
> > +#-------------------------------------------------------------------------------
> > +import os, sys
> > +from optparse import OptionParser
> > +import string
> > +import json
> > +
> > +# For running from development directory. It should take precedence over the
> > +# installed pyelftools.
> > +sys.path.insert(0, '.')
> > +
> > +
> > +from elftools import __version__
> > +from elftools.common.exceptions import ELFError
> > +from elftools.common.py3compat import (
> > +        ifilter, byte2int, bytes2str, itervalues, str2bytes)
> > +from elftools.elf.elffile import ELFFile
> > +from elftools.elf.dynamic import DynamicSection, DynamicSegment
> > +from elftools.elf.enums import ENUM_D_TAG
> > +from elftools.elf.segments import InterpSegment
> > +from elftools.elf.sections import SymbolTableSection
> > +from elftools.elf.gnuversions import (
> > +    GNUVerSymSection, GNUVerDefSection,
> > +    GNUVerNeedSection,
> > +    )
> > +from elftools.elf.relocation import RelocationSection
> > +from elftools.elf.descriptions import (
> > +    describe_ei_class, describe_ei_data, describe_ei_version,
> > +    describe_ei_osabi, describe_e_type, describe_e_machine,
> > +    describe_e_version_numeric, describe_p_type, describe_p_flags,
> > +    describe_sh_type, describe_sh_flags,
> > +    describe_symbol_type, describe_symbol_bind, describe_symbol_visibility,
> > +    describe_symbol_shndx, describe_reloc_type, describe_dyn_tag,
> > +    describe_ver_flags,
> > +    )
> > +from elftools.elf.constants import E_FLAGS
> > +from elftools.dwarf.dwarfinfo import DWARFInfo
> > +from elftools.dwarf.descriptions import (
> > +    describe_reg_name, describe_attr_value, set_global_machine_arch,
> > +    describe_CFI_instructions, describe_CFI_register_rule,
> > +    describe_CFI_CFA_rule,
> > +    )
> > +from elftools.dwarf.constants import (
> > +    DW_LNS_copy, DW_LNS_set_file, DW_LNE_define_file)
> > +from elftools.dwarf.callframe import CIE, FDE
> > +
> > +raw_output = False
> > +pcidb = None
> > +
> > +#===========================================
> > +
> > +class Vendor:
> > +    """
> > +    Class for vendors. This is the top level class
> > +    for the devices belong to a specific vendor.
> > +    self.devices is the device dictionary
> > +    subdevices are in each device.
> > +    """
> > +    def __init__(self, vendorStr):
> > +        """
> > +        Class initializes with the raw line from pci.ids
> > +        Parsing takes place inside __init__
> > +        """
> > +        self.ID = vendorStr.split()[0]
> > +        self.name = vendorStr.replace("%s " % self.ID,"").rstrip()
> > +        self.devices = {}
> > +
> > +    def addDevice(self, deviceStr):
> > +        """
> > +        Adds a device to self.devices
> > +        takes the raw line from pci.ids
> > +        """
> > +        s = deviceStr.strip()
> > +        devID = s.split()[0]
> > +        if devID in self.devices:
> > +            pass
> > +        else:
> > +            self.devices[devID] = Device(deviceStr)
> > +
> > +    def report(self):
> > +        print self.ID, self.name
> > +        for id, dev in self.devices.items():
> > +            dev.report()
> > +
> > +    def find_device(self, devid):
> > +        # convert to a hex string and remove 0x
> > +        devid = hex(devid)[2:]
> > +        try:
> > +            return self.devices[devid]
> > +        except:
> > +            return Device("%s  Unknown Device" % devid)
> > +
> > +class Device:
> > +    def __init__(self, deviceStr):
> > +        """
> > +        Class for each device.
> > +        Each vendor has its own devices dictionary.
> > +        """
> > +        s = deviceStr.strip()
> > +        self.ID = s.split()[0]
> > +        self.name = s.replace("%s  " % self.ID,"")
> > +        self.subdevices = {}
> > +
> > +    def report(self):
> > +        print "\t%s\t%s" % (self.ID, self.name)
> > +        for subID, subdev in self.subdevices.items():
> > +            subdev.report()
> > +
> > +    def addSubDevice(self, subDeviceStr):
> > +        """
> > +        Adds a subvendor, subdevice to device.
> > +        Uses raw line from pci.ids
> > +        """
> > +        s = subDeviceStr.strip()
> > +        spl = s.split()
> > +        subVendorID  = spl[0]
> > +        subDeviceID  = spl[1]
> > +        subDeviceName = s.split("  ")[-1]
> > +        devID = "%s:%s" % (subVendorID,subDeviceID)
> > +        self.subdevices[devID] = SubDevice(subVendorID,subDeviceID,subDeviceName)
> > +
> > +    def find_subid(self, subven, subdev):
> > +        subven = hex(subven)[2:]
> > +        subdev = hex(subdev)[2:]
> > +        devid ="%s:%s" % (subven, subdev)
> > +
> > +        try:
> > +            return self.subdevices[devid]
> > +        except:
> > +            if (subven == "ffff" and subdev == "ffff"):
> > +                return SubDevice("ffff", "ffff", "(All Subdevices)");
> > +            else:
> > +                return SubDevice(subven, subdev, "(Unknown Subdevice)")
> > +
> > +
> > +class SubDevice:
> > +    """
> > +    Class for subdevices.
> > +    """
> > +    def __init__(self, vendor, device, name):
> > +        """
> > +        Class initializes with vendorid, deviceid and name
> > +        """
> > +        self.vendorID = vendor
> > +        self.deviceID = device
> > +        self.name = name
> > +
> > +    def report(self):
> > +        print "\t\t%s\t%s\t%s" % (self.vendorID, self.deviceID,self.name)
> > +
> > +class PCIIds:
> > +    """
> > +    Top class for all pci.ids entries.
> > +    All queries will be asked to this class.
> > +    PCIIds.vendors["0e11"].devices["0046"].subdevices["0e11:4091"].name  =  "Smart Array 6i"
> > +    """
> > +    def __init__(self, filename):
> > +        """
> > +        Prepares the directories.
> > +        Checks local data file.
> > +        Tries to load from local, if not found, downloads from web
> > +        """
> > +        self.version = ""
> > +        self.date = ""
> > +        self.vendors = {}
> > +        self.contents = None
> > +        self.readLocal(filename)
> > +        self.parse()
> > +
> > +    def reportVendors(self):
> > +        """Reports the vendors
> > +        """
> > +        for vid, v in self.vendors.items():
> > +            print v.ID, v.name
> > +
> > +    def report(self, vendor = None):
> > +        """
> > +        Reports everything for all vendors or a specific vendor
> > +        PCIIds.report()  reports everything
> > +        PCIIDs.report("0e11") reports only "Compaq Computer Corporation"
> > +        """
> > +        if vendor != None:
> > +            self.vendors[vendor].report()
> > +        else:
> > +            for vID, v in self.vendors.items():
> > +                v.report()
> > +
> > +    def find_vendor(self, vid):
> > +        # convert vid to a hex string and remove the 0x
> > +        vid = hex(vid)[2:]
> > +
> > +        try:
> > +            return self.vendors[vid]
> > +        except:
> > +            return Vendor("%s Unknown Vendor" % (vid))
> > +
> > +    def findDate(self, content):
> > +        for l in content:
> > +            if l.find("Date:") > -1:
> > +                return l.split()[-2].replace("-", "")
> > +        return None
> > +
> > +    def parse(self):
> > +        if len(self.contents) < 1:
> > +            print "data/%s-pci.ids not found" % self.date
> > +        else:
> > +            vendorID = ""
> > +            deviceID = ""
> > +            for l in self.contents:
> > +                if l[0] == "#":
> > +                    continue
> > +                elif len(l.strip()) == 0:
> > +                    continue
> > +                else:
> > +                    if l.find("\t\t") == 0:
> > +                        self.vendors[vendorID].devices[deviceID].addSubDevice(l)
> > +                    elif l.find("\t") == 0:
> > +                        deviceID = l.strip().split()[0]
> > +                        self.vendors[vendorID].addDevice(l)
> > +                    else:
> > +                        vendorID = l.split()[0]
> > +                        self.vendors[vendorID] = Vendor(l)
> > +
> > +    def readLocal(self, filename):
> > +        """
> > +        Reads the local file
> > +        """
> > +        self.contents = open(filename).readlines()
> > +        self.date = self.findDate(self.contents)
> > +
> > +    def loadLocal(self):
> > +        """
> > +        Loads database from local. If there is no file,
> > +        it creates a new one from web
> > +        """
> > +        self.date = idsfile[0].split("/")[1].split("-")[0]
> > +        self.readLocal()
> > +
> > +
> > +
> > +#=======================================
> > +
> > +def search_file(filename, search_path):
> > +    """ Given a search path, find file with requested name """
> > +    for path in string.split(search_path, ":"):
> > +        candidate = os.path.join(path, filename)
> > +        if os.path.exists(candidate): return os.path.abspath(candidate)
> > +    return None
> > +
> > +class ReadElf(object):
> > +    """ display_* methods are used to emit output into the output stream
> > +    """
> > +    def __init__(self, file, output):
> > +        """ file:
> > +                stream object with the ELF file to read
> > +
> > +            output:
> > +                output stream to write to
> > +        """
> > +        self.elffile = ELFFile(file)
> > +        self.output = output
> > +
> > +        # Lazily initialized if a debug dump is requested
> > +        self._dwarfinfo = None
> > +
> > +        self._versioninfo = None
> > +
> > +    def _section_from_spec(self, spec):
> > +        """ Retrieve a section given a "spec" (either number or name).
> > +            Return None if no such section exists in the file.
> > +        """
> > +        try:
> > +            num = int(spec)
> > +            if num < self.elffile.num_sections():
> > +                return self.elffile.get_section(num)
> > +            else:
> > +                return None
> > +        except ValueError:
> > +            # Not a number. Must be a name then
> > +            return self.elffile.get_section_by_name(str2bytes(spec))
> > +
> > +    def pretty_print_pmdinfo(self, pmdinfo):
> > +        global pcidb
> > +
> > +        for i in pmdinfo["pci_ids"]:
> > +            vendor = pcidb.find_vendor(i[0])
> > +            device = vendor.find_device(i[1])
> > +            subdev = device.find_subid(i[2], i[3])
> > +            print("%s (%s) : %s (%s) %s" % (vendor.name, vendor.ID, device.name, device.ID, subdev.name))
> > +
> > +    def parse_pmd_info_string(self, mystring):
> > +        global raw_output
> > +        global pcidb
> > +
> > +        optional_pmd_info = [{'id': 'params', 'tag' : 'PMD PARAMETERS'}]
> > +
> > +        i = mystring.index("=");
> > +        mystring = mystring[i+2:]
> > +        pmdinfo = json.loads(mystring)
> > +
> > +        if raw_output:
> > +            print(pmdinfo)
> > +            return
> > +
> > +        print("PMD NAME: " + pmdinfo["name"])
> > +        print("PMD TYPE: " + pmdinfo["type"])
> > +        for i in optional_pmd_info:
> > +            try:
> > +                print("%s: %s" % (i['tag'], pmdinfo[i['id']]))
> > +            except KeyError as e:
> > +                continue
> > +
> > +        if (pmdinfo["type"] == "PMD_PDEV"):
> > +            print("PMD HW SUPPORT:")
> > +            if pcidb != None:
> > +                self.pretty_print_pmdinfo(pmdinfo)
> > +            else:
> > +                print("VENDOR\t DEVICE\t SUBVENDOR\t SUBDEVICE")
> > +                for i in pmdinfo["pci_ids"]:
> > +                    print("0x%04x\t 0x%04x\t 0x%04x\t\t 0x%04x" % (i[0], i[1], i[2], i[3]))
> > +
> > +        print("")
> > +
> > +
> > +    def display_pmd_info_strings(self, section_spec):
> > +        """ Display a strings dump of a section. section_spec is either a
> > +            section number or a name.
> > +        """
> > +        section = self._section_from_spec(section_spec)
> > +        if section is None:
> > +            return
> > +
> > +
> > +        data = section.data()
> > +        dataptr = 0
> > +
> > +        while dataptr < len(data):
> > +            while ( dataptr < len(data) and
> > +                    not (32 <= byte2int(data[dataptr]) <= 127)):
> > +                dataptr += 1
> > +
> > +            if dataptr >= len(data):
> > +                break
> > +
> > +            endptr = dataptr
> > +            while endptr < len(data) and byte2int(data[endptr]) != 0:
> > +                endptr += 1
> > +
> > +            mystring = bytes2str(data[dataptr:endptr])
> > +            rc = mystring.find("PMD_INFO_STRING")
> > +            if (rc != -1):
> > +                self.parse_pmd_info_string(mystring)
> > +
> > +            dataptr = endptr
> > +
> > +    def search_for_autoload_path(self):
> > +        section = self._section_from_spec(".rodata")
> > +        if section is None:
> > +            return None
> > +
> > +        data = section.data()
> > +        dataptr = 0
> > +
> > +        while dataptr < len(data):
> > +            while ( dataptr < len(data) and
> > +                    not (32 <= byte2int(data[dataptr]) <= 127)):
> > +                dataptr += 1
> > +
> > +            if dataptr >= len(data):
> > +                break
> > +
> > +            endptr = dataptr
> > +            while endptr < len(data) and byte2int(data[endptr]) != 0:
> > +                endptr += 1
> > +
> > +            mystring = bytes2str(data[dataptr:endptr])
> > +            rc = mystring.find("DPDK_PLUGIN_PATH")
> > +            if (rc != -1):
> > +                rc = mystring.find("=")
> > +                return mystring[rc+1:]
> > +
> > +            dataptr = endptr
> > +        return None
> > +
> > +    def get_dt_runpath(self, dynsec):
> > +        for tag in dynsec.iter_tags():
> > +            if tag.entry.d_tag == 'DT_RUNPATH':
> > +                return tag.runpath
> > +        return ""
> > +
> > +
> > +    def process_dt_needed_entries(self):
> > +        """ Look to see if there are any DT_NEEDED entries in the binary
> > +            And process those if there are
> > +        """
> > +        global raw_output
> > +        runpath = ""
> > +        ldlibpath = os.environ.get('LD_LIBRARY_PATH')
> > +        if (ldlibpath == None):
> > +            ldlibpath = ""
> > +
> > +        dynsec = self._section_from_spec(".dynamic")
> > +        try:
> > +            runpath = self.get_dt_runpath(dynsec)
> > +        except AttributeError:
> > +            # dynsec is None, just return
> > +            return
> > +
> > +        for tag in dynsec.iter_tags():
> > +            if tag.entry.d_tag == 'DT_NEEDED':
> > +                rc = tag.needed.find("librte_pmd")
> > +                if (rc != -1):
> > +                    library = search_file(tag.needed,
> > +                            runpath + ":" + ldlibpath +
> > +                            ":/usr/lib64:/lib64:/usr/lib:/lib")
> > +                    if (library != None):
> > +                        if (raw_output == False):
> > +                            print("Scanning %s for pmd information" % library)
> > +                        with open(library, 'rb') as file:
> > +                            libelf = ReadElf(file, sys.stdout)
> > +                            libelf.process_dt_needed_entries()
> > +                            libelf.display_pmd_info_strings(".rodata")
> > +                            file.close()
> > +
> > +def scan_autoload_path(autoload_path):
> > +    global raw_output
> > +
> > +    if os.path.exists(autoload_path) == False:
> > +        return
> > +
> > +    for d in os.listdir(autoload_path):
> > +        dpath = os.path.join(autoload_path, d)
> > +        if os.path.isdir(dpath):
> > +            scan_autoload_path(dpath)
> > +        if os.path.isfile(dpath):
> > +            if (raw_output == False):
> > +                print("Hw Support for library %s" % d)
> > +            file = open(dpath, 'rb')
> > +            readelf = ReadElf(file, sys.stdout)
> > +            readelf.display_pmd_info_strings(".rodata")
> > +            file.close()
> 
> Might want to wrap ReadElf() in try-except ELFError to avoid traceback in
> case there are other files in the directory. Not that other files are
> expected there, but EAL ignores all non-pmd files there so it'd be good to
> match that.
> 
Yup, I can do that.

> > +
> > +
> > +def scan_for_autoload_pmds(dpdk_path):
> > +    """
> > +    search the specified application or path for a pmd autoload path
> > +    then scan said path for pmds and report hw support
> > +    """
> > +    global raw_output
> > +
> > +    if (os.path.isfile(dpdk_path) == False):
> > +        # Assume this is a directory and we need to scan librte_eal.so
> > +        dpdk_path=os.path.join(dpdk_path, "librte_eal.so")
> 
> The unversioned .so symlink is not guaranteed to be there, they're typically
> placed into -devel packages which wont be installed for "normal use"
> (developers are another story obviously)
> 
> Also assuming there might be multiple versions of DPDK runtime
> parallel-installed, it might point to any of them so its a bit of a lottery.
> 
Crap, thats right, if they're in the same directory path, then we have no way to
disambiguate that.  Maybe we do need to just specify the binary, as the
DT_NEEDED entry points to the specific version the application will use.


> > +
> > +    file = open(dpdk_path, 'rb')
> > +    readelf = ReadElf(file, sys.stdout)
> 
> Might want to wrap this in try-except ELFError, its a user-entered path.
> 
yeah, as a rule, we should. I will

> > +    autoload_path = readelf.search_for_autoload_path()
> > +    if (autoload_path == None or autoload_path == ""):
> > +        if (raw_output == False):
> > +            print("No autoload path configured in %s" % dpdk_path)
> > +        return
> > +    if (raw_output == False):
> > +        print("Found autoload path %s in %s" % (autoload_path, dpdk_path))
> > +
> > +    file.close()
> > +    if (raw_output == False):
> > +        print("Discovered Autoload HW Support:")
> > +    scan_autoload_path(autoload_path)
> > +    return
> > +
> > +
> > +def main(stream=None):
> > +    global raw_output
> > +    global pcidb
> > +
> > +    optparser = OptionParser(
> > +            usage='usage: %prog [-hrt] [-p <dpdk dir>] [-d <pci id file] <elf-file>',
> 
> Minor nit, but with the later additions the usage message isn't accurate
> anymore, <elf-file> could be a directory too so <file|dir> would be closer
> to the mark, and in plugin mode it's not looked at all. Might be simpler to
> both users + and in actual code if plugin mode is just a true/false flag and
> uses the same args[0] as argument as it too handles both a file and a
> directory case.
> 
I'll clean this up as I finish the above

> > +            description="Dump pmd hardware support info",
> > +            add_help_option=True,
> > +            prog='pmdinfo.py')
> > +    optparser.add_option('-r', '--raw',
> > +            action='store_true', dest='raw_output',
> > +            help='Dump raw json strings')
> > +    optparser.add_option("-d", "--pcidb", dest="pcifile",
> > +            help="specify a pci database to get vendor names from",
> > +            default="/usr/share/hwdata/pci.ids", metavar="FILE")
> > +    optparser.add_option("-t", "--table", dest="tblout",
> > +            help="output infomation on hw support as a hex table",
> > +            action='store_true')
> > +    optparser.add_option("-p", "--plugindir", dest="pdir",
> > +            help="scan dpdk for autoload plugins", metavar="PATH|APP")
> > +
> > +    options, args = optparser.parse_args()
> > +
> > +    if options.raw_output:
> > +        raw_output = True
> > +
> > +    if options.pcifile:
> > +        pcidb = PCIIds(options.pcifile)
> > +        if pcidb == None:
> > +            print("Pci DB file not found")
> > +            exit(1)
> > +
> > +    if options.tblout:
> > +        options.pcifile = None
> > +        pcidb = None
> > +
> > +    if options.pdir:
> > +        exit (scan_for_autoload_pmds(options.pdir))
> > +
> > +    ldlibpath = os.environ.get('LD_LIBRARY_PATH')
> > +    if (ldlibpath == None):
> > +        ldlibpath = ""
> > +
> > +    if (os.path.exists(args[0]) == True):
> > +        myelffile = args[0]
> > +    else:
> > +        myelffile = search_file(args[0], ldlibpath + ":/usr/lib64:/lib64:/usr/lib:/lib")
> 
> Minor nit, but this tracebacks when executing pmdinfo.py without arguments.
> Would be nicer to dump usage instead.
> 
Yup

> [pmatilai@sopuli dpdk]$ tools/pmdinfo.py
> Traceback (most recent call last):
>   File "tools/pmdinfo.py", line 543, in <module>
>     main()
>   File "tools/pmdinfo.py", line 520, in main
>     if (os.path.exists(args[0]) == True):
> IndexError: list index out of range
> 
> 	- Panu -
> 
> > +
> > +    if (myelffile == None):
> > +        print("File not found")
> > +        sys.exit(1)
> > +
> > +    with open(myelffile, 'rb') as file:
> > +        try:
> > +            readelf = ReadElf(file, sys.stdout)
> > +            readelf.process_dt_needed_entries()
> > +            readelf.display_pmd_info_strings(".rodata")
> > +            sys.exit(0)
> > +
> > +        except ELFError as ex:
> > +            sys.stderr.write('ELF error: %s\n' % ex)
> > +            sys.exit(1)
> > +
> > +
> > +#-------------------------------------------------------------------------------
> > +if __name__ == '__main__':
> > +    main()
> > +
> > +
> > 
> 
> 

[PATCHv4 0/5] Implement pmd hardware support exports

[Neil Horman]

Hey all-
	So heres attempt number 2 at a method for exporting PMD hardware support
information.  As we discussed previously, the consensus seems to be that pmd
information should be:

1) Able to be interrogated on any ELF binary (application binary or individual
DSO)
2) Equally functional on statically linked applications or on DSO's
3) Resilient to symbol stripping
4) Script friendly
5) Show kernel dependencies
6) List driver options
7) Show driver name
8) Offer human readable output
9) Show DPDK version
10) Show driver version
11) Allow for expansion
12) Not place additional build environment dependencies on an application

I added item 12 myself, because I don't think its reasonable to require
applications to use a specific linker script to get hardware support information
(which precludes the use of a special .modinfo section like the kernel uses)

However, we still can use some tricks from the kernel to make this work.  In
this approach, what I've done is the following:

A) Modified the driver registration macro to also define a variable:
this_pmd_name<n>= "name"

Based on the unique name string pointed to by the above variable, we can
query for an arbitrary number of other symbols following the pattern:
<name>_<tag>

Where tag is some well known identifier for information we wish to export

B) Added a utility called pmdinfogen.  This utility is not meant for general use,
but rather is used by the dpdk build environment itself when compiling pmds.
for each object that uses the PMD_REGISTER_DRIVER macro, this utiity is run.  It
searches for the symbols in (A), and using those, extracts the hardware support
info, and module name from the object, using that to produce a new c file
containing a single variable in the following format:

static const char <name>[] __attribute__((used)) = "PMD_DRIVER_INFO=<json>";

The <name> is arbitrary, as its static and not referenced.  The relevant bit is
the string value assigned to it.  The <json> is a json encoded string of the
extracted hardware support information pmdinfo found for the corresponding
object.  This C file is suitable for compilation and relocatable linking back
into the parent object file.  The result of this operation is that the object
string table now contains a string that will not e removed by stripping, whos
leading text (PMD_DRIVER_INFO) can be easily searched for at any time weather
the symbol referring to it is stripped or not.

C) Added a utilty called pmdinfo.py.  This python script, searches the
string table of the .rodata section of any provided ELF file looking for the
PMD_DRIVER_INFO prefix on a string.  When found, it will interpret the remainder
of the string as json, and output the hardware support for that ELF file (if
any).


This approach ticks most of the above boxes:
1) Impervious to stripping
2) Works on static and dynamic binaries
3) Human and script friendly
4) Allows for expansion

Because of (4) the other items should be pretty easy to implement, as its just a
matter of modifying the macros to export the info, pmdinfo to encode it to json,
and pmd_hw_support.py to read it.  I'm not adding them now, as theres alot of
rote work to do to get some of them in place (e.g. DPDK has no current global
VERSION macro, drivers don't have a consistent version scheme, command line
strings have to be built for each driver, etc).  But once this is accepted,
those items can be done as time allows and should be fairly easy to implement.

Change Notes:
v2)
 * Made the export macros a bit easier to expand
 * Added a macro to optionally export command line strings
 * Renamed utilties to be more appropriate
   (pmdinfo -> pmdinfogen, pmd_hw_support.py -> pmdinfo.py)
 * Added search capabilities to pmdinfo.py so that we search for libraries
   linked using DT_NEEDINFO entries.  We search DT_RUNPATH, LD_LIBRARY_PATH,
   /usr/lib and /lib
 * Added serch abilities if the specified binary to pmdinfo isn't found, we
   search LD_LIBRARY_PATH, /usr/lib and /lib
 * Added an option to pmdinfo.py to pretty-print hardware support info using the
   pci.ids database

v3)
 * Made table mode printing the default mode
 * Added a default path to the pci.ids file
 * Modifed pmdinfo to use python rather than python3
 * Cleaned up some changelog entries
 * Added an export for RTE_EAL_PMD_PATH
 * Added a plugin option to pmdinfo to scan for autoloaded DSO's

v4)
 * Modified the operation of the -p option. As much as I don't like implying
that autoloaded pmds are guaranteed to be there at run time, I'm having a hard
time seeing how we can avoid specifying the application file to scan for the
autoload directory.  Without it we can't determine which library the user means
in a multiversion installation
 * Cleaned up the help text
 * Added a rule for an install target for pmdinfo
 * Guarded against some tracebacks in pmdinfo
 * Use DT_NEEDED entries to get versioned libraries in -p mode
 * Fixed traceback that occurs on lack of input arguments
 * Fixed some erroneous macro usage in drivers that aren't in the default build

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>

[PATCHv4 1/5] pmdinfogen: Add buildtools and pmdinfogen utility

[Neil Horman]

pmdinfogen is a tool used to parse object files and build json strings for use in
later determining hardware support in a dso or application binary.  pmdinfo
looks for the non-exported symbol names this_pmd_name<n> and this_pmd_tbl<n>
(where n is a integer counter).  It records the name of each of these tuples,
using the later to find the symbolic name of the pci_table for physical devices
that the object supports.  With this information, it outputs a C file with a
single line of the form:

static char *<pmd_name>_driver_info[] __attribute__((used)) = " \
	PMD_DRIVER_INFO=<json string>";

Where <pmd_name> is the arbitrary name of the pmd, and <json_string> is the json
encoded string that hold relevant pmd information, including the pmd name, type
and optional array of pci device/vendor ids that the driver supports.

This c file is suitable for compiling to object code, then relocatably linking
into the parent file from which the C was generated.  This creates an entry in
the string table of the object that can inform a later tool about hardware
support.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 GNUmakefile                        |   2 +-
 buildtools/Makefile                |  36 ++++
 buildtools/pmdinfogen/Makefile     |  48 +++++
 buildtools/pmdinfogen/pmdinfogen.c | 413 +++++++++++++++++++++++++++++++++++++
 buildtools/pmdinfogen/pmdinfogen.h |  83 ++++++++
 mk/rte.buildtools.mk               | 148 +++++++++++++
 mk/rte.sdkbuild.mk                 |   3 +-
 7 files changed, 731 insertions(+), 2 deletions(-)
 create mode 100644 buildtools/Makefile
 create mode 100644 buildtools/pmdinfogen/Makefile
 create mode 100644 buildtools/pmdinfogen/pmdinfogen.c
 create mode 100644 buildtools/pmdinfogen/pmdinfogen.h
 create mode 100644 mk/rte.buildtools.mk

diff --git a/GNUmakefile b/GNUmakefile
index b59e4b6..00fe0db 100644
--- a/GNUmakefile
+++ b/GNUmakefile
@@ -40,6 +40,6 @@ export RTE_SDK
 # directory list
 #
 
-ROOTDIRS-y := lib drivers app
+ROOTDIRS-y := buildtools lib drivers app
 
 include $(RTE_SDK)/mk/rte.sdkroot.mk
diff --git a/buildtools/Makefile b/buildtools/Makefile
new file mode 100644
index 0000000..eb565eb
--- /dev/null
+++ b/buildtools/Makefile
@@ -0,0 +1,36 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+DIRS-y += pmdinfogen
+
+include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/buildtools/pmdinfogen/Makefile b/buildtools/pmdinfogen/Makefile
new file mode 100644
index 0000000..4de9506
--- /dev/null
+++ b/buildtools/pmdinfogen/Makefile
@@ -0,0 +1,48 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+#
+# library name
+#
+APP = pmdinfogen
+
+#
+# all sources are stored in SRCS-y
+#
+SRCS-y += pmdinfogen.c
+
+#CFLAGS += $(WERROR_FLAGS) -g
+CFLAGS += -g
+
+include $(RTE_SDK)/mk/rte.buildtools.mk
+
diff --git a/buildtools/pmdinfogen/pmdinfogen.c b/buildtools/pmdinfogen/pmdinfogen.c
new file mode 100644
index 0000000..af0b2df
--- /dev/null
+++ b/buildtools/pmdinfogen/pmdinfogen.c
@@ -0,0 +1,413 @@
+/* Postprocess pmd object files to export hw support 
+ *
+ * Copyright 2016 Neil Horman <nhorman@tuxdriver.com>
+ * Based in part on modpost.c from the linux kernel
+ *
+ * This software may be used and distributed according to the terms
+ * of the GNU General Public License V2, incorporated herein by reference.
+ *
+ */
+
+#define _GNU_SOURCE
+#include <stdio.h>
+#include <ctype.h>
+#include <string.h>
+#include <limits.h>
+#include <stdbool.h>
+#include <errno.h>
+#include "pmdinfogen.h"
+
+
+static const char *sym_name(struct elf_info *elf, Elf_Sym *sym)
+{
+	if (sym)
+		return elf->strtab + sym->st_name;
+	else
+		return "(unknown)";
+}
+
+void *grab_file(const char *filename, unsigned long *size)
+{
+	struct stat st;
+	void *map = MAP_FAILED;
+	int fd;
+
+	fd = open(filename, O_RDONLY);
+	if (fd < 0)
+		return NULL;
+	if (fstat(fd, &st))
+		goto failed;
+
+	*size = st.st_size;
+	map = mmap(NULL, *size, PROT_READ|PROT_WRITE, MAP_PRIVATE, fd, 0);
+
+failed:
+	close(fd);
+	if (map == MAP_FAILED)
+		return NULL;
+	return map;
+}
+
+/**
+  * Return a copy of the next line in a mmap'ed file.
+  * spaces in the beginning of the line is trimmed away.
+  * Return a pointer to a static buffer.
+  **/
+void release_file(void *file, unsigned long size)
+{
+	munmap(file, size);
+}
+
+
+static void *get_sym_value(struct elf_info *info, const Elf_Sym *sym)
+{
+	void *ptr = (void *)info->hdr + info->sechdrs[sym->st_shndx].sh_offset;
+
+	return (void *)(ptr + sym->st_value);
+}
+
+static Elf_Sym *find_sym_in_symtab(struct elf_info *info, 
+				   const char *name, Elf_Sym *last)
+{
+	Elf_Sym *idx;
+	if (last)
+		idx = last+1;
+	else
+		idx = info->symtab_start;
+
+	for(; idx < info->symtab_stop; idx++) {
+		const char *n = sym_name(info, idx);
+		if (!strncmp(n, name, strlen(name)))
+			return idx;
+	}
+	return NULL;
+}
+
+static int parse_elf(struct elf_info *info, const char *filename)
+{
+	unsigned int i;
+	Elf_Ehdr *hdr;
+	Elf_Shdr *sechdrs;
+	Elf_Sym  *sym;
+	const char *secstrings;
+	unsigned int symtab_idx = ~0U, symtab_shndx_idx = ~0U;
+
+	hdr = grab_file(filename, &info->size);
+	if (!hdr) {
+		perror(filename);
+		exit(1);
+	}
+	info->hdr = hdr;
+	if (info->size < sizeof(*hdr)) {
+		/* file too small, assume this is an empty .o file */
+		return 0;
+	}
+	/* Is this a valid ELF file? */
+	if ((hdr->e_ident[EI_MAG0] != ELFMAG0) ||
+	    (hdr->e_ident[EI_MAG1] != ELFMAG1) ||
+	    (hdr->e_ident[EI_MAG2] != ELFMAG2) ||
+	    (hdr->e_ident[EI_MAG3] != ELFMAG3)) {
+		/* Not an ELF file - silently ignore it */
+		return 0;
+	}
+	/* Fix endianness in ELF header */
+	hdr->e_type      = TO_NATIVE(hdr->e_type);
+	hdr->e_machine   = TO_NATIVE(hdr->e_machine);
+	hdr->e_version   = TO_NATIVE(hdr->e_version);
+	hdr->e_entry     = TO_NATIVE(hdr->e_entry);
+	hdr->e_phoff     = TO_NATIVE(hdr->e_phoff);
+	hdr->e_shoff     = TO_NATIVE(hdr->e_shoff);
+	hdr->e_flags     = TO_NATIVE(hdr->e_flags);
+	hdr->e_ehsize    = TO_NATIVE(hdr->e_ehsize);
+	hdr->e_phentsize = TO_NATIVE(hdr->e_phentsize);
+	hdr->e_phnum     = TO_NATIVE(hdr->e_phnum);
+	hdr->e_shentsize = TO_NATIVE(hdr->e_shentsize);
+	hdr->e_shnum     = TO_NATIVE(hdr->e_shnum);
+	hdr->e_shstrndx  = TO_NATIVE(hdr->e_shstrndx);
+	sechdrs = (void *)hdr + hdr->e_shoff;
+	info->sechdrs = sechdrs;
+
+	/* Check if file offset is correct */
+	if (hdr->e_shoff > info->size) {
+		fprintf(stderr, "section header offset=%lu in file '%s' is bigger than "
+		      "filesize=%lu\n", (unsigned long)hdr->e_shoff,
+		      filename, info->size);
+		return 0;
+	}
+
+	if (hdr->e_shnum == SHN_UNDEF) {
+		/*
+		 * There are more than 64k sections,
+		 * read count from .sh_size.
+		 */
+		info->num_sections = TO_NATIVE(sechdrs[0].sh_size);
+	}
+	else {
+		info->num_sections = hdr->e_shnum;
+	}
+	if (hdr->e_shstrndx == SHN_XINDEX) {
+		info->secindex_strings = TO_NATIVE(sechdrs[0].sh_link);
+	}
+	else {
+		info->secindex_strings = hdr->e_shstrndx;
+	}
+
+	/* Fix endianness in section headers */
+	for (i = 0; i < info->num_sections; i++) {
+		sechdrs[i].sh_name      = TO_NATIVE(sechdrs[i].sh_name);
+		sechdrs[i].sh_type      = TO_NATIVE(sechdrs[i].sh_type);
+		sechdrs[i].sh_flags     = TO_NATIVE(sechdrs[i].sh_flags);
+		sechdrs[i].sh_addr      = TO_NATIVE(sechdrs[i].sh_addr);
+		sechdrs[i].sh_offset    = TO_NATIVE(sechdrs[i].sh_offset);
+		sechdrs[i].sh_size      = TO_NATIVE(sechdrs[i].sh_size);
+		sechdrs[i].sh_link      = TO_NATIVE(sechdrs[i].sh_link);
+		sechdrs[i].sh_info      = TO_NATIVE(sechdrs[i].sh_info);
+		sechdrs[i].sh_addralign = TO_NATIVE(sechdrs[i].sh_addralign);
+		sechdrs[i].sh_entsize   = TO_NATIVE(sechdrs[i].sh_entsize);
+	}
+	/* Find symbol table. */
+	secstrings = (void *)hdr + sechdrs[info->secindex_strings].sh_offset;
+	for (i = 1; i < info->num_sections; i++) {
+		int nobits = sechdrs[i].sh_type == SHT_NOBITS;
+
+		if (!nobits && sechdrs[i].sh_offset > info->size) {
+			fprintf(stderr, "%s is truncated. sechdrs[i].sh_offset=%lu > "
+			      "sizeof(*hrd)=%zu\n", filename,
+			      (unsigned long)sechdrs[i].sh_offset,
+			      sizeof(*hdr));
+			return 0;
+		}
+
+		if (sechdrs[i].sh_type == SHT_SYMTAB) {
+			unsigned int sh_link_idx;
+			symtab_idx = i;
+			info->symtab_start = (void *)hdr +
+			    sechdrs[i].sh_offset;
+			info->symtab_stop  = (void *)hdr +
+			    sechdrs[i].sh_offset + sechdrs[i].sh_size;
+			sh_link_idx = sechdrs[i].sh_link;
+			info->strtab       = (void *)hdr +
+			    sechdrs[sh_link_idx].sh_offset;
+		}
+
+		/* 32bit section no. table? ("more than 64k sections") */
+		if (sechdrs[i].sh_type == SHT_SYMTAB_SHNDX) {
+			symtab_shndx_idx = i;
+			info->symtab_shndx_start = (void *)hdr +
+			    sechdrs[i].sh_offset;
+			info->symtab_shndx_stop  = (void *)hdr +
+			    sechdrs[i].sh_offset + sechdrs[i].sh_size;
+		}
+	}
+	if (!info->symtab_start)
+		fprintf(stderr, "%s has no symtab?\n", filename);
+
+	/* Fix endianness in symbols */
+	for (sym = info->symtab_start; sym < info->symtab_stop; sym++) {
+		sym->st_shndx = TO_NATIVE(sym->st_shndx);
+		sym->st_name  = TO_NATIVE(sym->st_name);
+		sym->st_value = TO_NATIVE(sym->st_value);
+		sym->st_size  = TO_NATIVE(sym->st_size);
+	}
+
+	if (symtab_shndx_idx != ~0U) {
+		Elf32_Word *p;
+		if (symtab_idx != sechdrs[symtab_shndx_idx].sh_link)
+			fprintf(stderr, "%s: SYMTAB_SHNDX has bad sh_link: %u!=%u\n",
+			      filename, sechdrs[symtab_shndx_idx].sh_link,
+			      symtab_idx);
+		/* Fix endianness */
+		for (p = info->symtab_shndx_start; p < info->symtab_shndx_stop;
+		     p++)
+			*p = TO_NATIVE(*p);
+	}
+
+	return 1;
+}
+
+static void parse_elf_finish(struct elf_info *info)
+{
+	struct pmd_driver *tmp, *idx = info->drivers;
+	release_file(info->hdr, info->size);
+	while (idx) {
+		tmp = idx->next;
+		free(idx);
+		idx = tmp;
+	}
+}
+
+static const char *sec_name(struct elf_info *elf, int secindex)
+{
+	Elf_Shdr *sechdrs = elf->sechdrs;
+	return (void *)elf->hdr +
+		elf->sechdrs[elf->secindex_strings].sh_offset +
+		sechdrs[secindex].sh_name;
+}
+
+static int get_symbol_index(struct elf_info *info, Elf64_Sym *sym)
+{
+	const char *name =  sym_name(info, sym);
+	const char *idx;
+
+	idx = name;
+	while (idx) {
+		if (isdigit(*idx))
+			return atoi(idx);
+		idx++;
+	}
+	return -1;
+}
+
+struct opt_tag {
+	const char* suffix;
+	const char* json_id;
+};
+
+static const struct opt_tag opt_tags[] = {
+	{"_param_string_export", "params"},
+};
+
+static int complete_pmd_entry(struct elf_info *info, struct pmd_driver *drv)
+{
+	const char *tname;
+	int i;
+	char tmpsymname[128];
+	Elf_Sym *tmpsym;
+	
+
+	drv->name = get_sym_value(info, drv->name_sym);
+
+	for (i=0; i<PMD_OPT_MAX; i++) {
+		memset(tmpsymname, 0, 128);
+		sprintf(tmpsymname, "__%s%s", drv->name, opt_tags[i].suffix);
+		tmpsym = find_sym_in_symtab(info, tmpsymname, NULL);
+		if (!tmpsym)
+			continue;
+		drv->opt_vals[i] = get_sym_value(info, tmpsym);
+	}
+
+	memset(tmpsymname, 0, 128);
+	sprintf(tmpsymname, "__%s_pci_tbl_export", drv->name);
+
+	tmpsym = find_sym_in_symtab(info, tmpsymname, NULL);
+
+
+	/*
+ 	 * If this returns NULL, then this is a PMD_VDEV, because
+ 	 * it has no pci table reference
+ 	 */
+	if (!tmpsym) {
+		drv->pci_tbl = NULL;
+		return 0;
+	}
+
+	tname = get_sym_value(info, tmpsym);
+	tmpsym = find_sym_in_symtab(info, tname, NULL);
+	if (!tmpsym)
+		return -ENOENT;
+
+	drv->pci_tbl = (struct rte_pci_id *)get_sym_value(info, tmpsym);
+	if (!drv->pci_tbl)
+		return -ENOENT;
+
+
+	return 0;
+	
+}
+
+static int locate_pmd_entries(struct elf_info *info)
+{
+	Elf_Sym *last = NULL;
+	struct pmd_driver *new;
+
+	info->drivers = NULL;
+
+	do {
+		new = calloc(sizeof(struct pmd_driver), 1);
+		new->name_sym = find_sym_in_symtab(info, "this_pmd_name", last);
+		last = new->name_sym;
+		if (!new->name_sym)
+			free(new);
+		else {
+			if (complete_pmd_entry(info, new)) {
+				fprintf(stderr, "Failed to complete pmd entry\n");
+				free(new);
+			} else {
+				new->next = info->drivers;
+				info->drivers = new;
+			}
+		}
+	} while (last);
+}
+
+static void output_pmd_info_string(struct elf_info *info, char *outfile)
+{
+	FILE *ofd;
+	struct pmd_driver *drv;
+	struct rte_pci_id *pci_ids;
+	int idx = 0;
+
+	ofd = fopen(outfile, "w+");
+	if (!ofd) {
+		fprintf(stderr, "Unable to open output file\n");
+		return;
+	}
+
+	drv = info->drivers;
+
+	while (drv) {
+		fprintf(ofd, "const char %s_pmd_info[] __attribute__((used)) = \"PMD_INFO_STRING= {",
+			drv->name);
+		fprintf(ofd,"\\\"name\\\" : \\\"%s\\\", ", drv->name);
+		fprintf(ofd,"\\\"type\\\" : \\\"%s\\\", ", drv->pci_tbl ? "PMD_PDEV" : "PMD_VDEV");
+
+		for(idx=0; idx<PMD_OPT_MAX; idx++) {
+			if (drv->opt_vals[idx])
+				fprintf(ofd,"\\\"%s\\\" : \\\"%s\\\", ", opt_tags[idx].json_id,
+					drv->opt_vals[idx]);
+		}
+
+		pci_ids = drv->pci_tbl;
+		fprintf(ofd, "\\\"pci_ids\\\" : [");
+
+		while (pci_ids && pci_ids->device_id) {
+			fprintf(ofd, "[%d, %d, %d, %d]",
+				pci_ids->vendor_id, pci_ids->device_id,
+				pci_ids->subsystem_vendor_id,
+				pci_ids->subsystem_device_id);
+			pci_ids++;
+			if (pci_ids->device_id)
+				fprintf(ofd, ",");
+			else
+				fprintf(ofd, " ");
+		}
+		fprintf(ofd, "]}\";");
+		drv = drv->next;
+	}
+
+	fclose(ofd);
+}
+
+int main(int argc, char **argv)
+{
+	struct elf_info info;
+	int rc = 1;
+
+	if (argc < 3) {
+		fprintf(stderr, "usage: pmdinfo <object file> <c output file>\n");
+		exit(127);
+	}
+	parse_elf(&info, argv[1]);
+
+	locate_pmd_entries(&info);
+
+	if (info.drivers) {
+		output_pmd_info_string(&info, argv[2]);
+		rc = 0;
+	} else {
+		fprintf(stderr, "Hmm, Appears to be a driver but no drivers registered\n");
+	}
+
+	parse_elf_finish(&info);
+	exit(rc);
+}
diff --git a/buildtools/pmdinfogen/pmdinfogen.h b/buildtools/pmdinfogen/pmdinfogen.h
new file mode 100644
index 0000000..580ed9f
--- /dev/null
+++ b/buildtools/pmdinfogen/pmdinfogen.h
@@ -0,0 +1,83 @@
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdarg.h>
+#include <string.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/mman.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <elf.h>
+
+
+/* On BSD-alike OSes elf.h defines these according to host's word size */
+#undef ELF_ST_BIND
+#undef ELF_ST_TYPE
+#undef ELF_R_SYM
+#undef ELF_R_TYPE
+
+#define Elf_Ehdr    Elf64_Ehdr
+#define Elf_Shdr    Elf64_Shdr
+#define Elf_Sym     Elf64_Sym
+#define Elf_Addr    Elf64_Addr
+#define Elf_Sword   Elf64_Sxword
+#define Elf_Section Elf64_Half
+#define ELF_ST_BIND ELF64_ST_BIND
+#define ELF_ST_TYPE ELF64_ST_TYPE
+
+#define Elf_Rel     Elf64_Rel
+#define Elf_Rela    Elf64_Rela
+#define ELF_R_SYM   ELF64_R_SYM
+#define ELF_R_TYPE  ELF64_R_TYPE
+
+#define TO_NATIVE(x) (x)
+
+
+struct rte_pci_id {
+	uint16_t vendor_id;           /**< Vendor ID or PCI_ANY_ID. */
+	uint16_t device_id;           /**< Device ID or PCI_ANY_ID. */
+	uint16_t subsystem_vendor_id; /**< Subsystem vendor ID or PCI_ANY_ID. */
+	uint16_t subsystem_device_id; /**< Subsystem device ID or PCI_ANY_ID. */
+};
+
+enum opt_params {
+	PMD_PARAM_STRING = 0,
+	PMD_OPT_MAX
+};
+
+struct pmd_driver {
+	Elf_Sym *name_sym;
+	const char *name;
+	struct rte_pci_id *pci_tbl;
+	struct pmd_driver *next;
+
+	const char* opt_vals[PMD_OPT_MAX];
+};
+
+struct elf_info {
+	unsigned long size;
+	Elf_Ehdr     *hdr;
+	Elf_Shdr     *sechdrs;
+	Elf_Sym      *symtab_start;
+	Elf_Sym      *symtab_stop;
+	Elf_Section  export_sec;
+	Elf_Section  export_unused_sec;
+	Elf_Section  export_gpl_sec;
+	Elf_Section  export_unused_gpl_sec;
+	Elf_Section  export_gpl_future_sec;
+	char         *strtab;
+	char	     *modinfo;
+	unsigned int modinfo_len;
+
+	/* support for 32bit section numbers */
+
+	unsigned int num_sections; /* max_secindex + 1 */
+	unsigned int secindex_strings;
+	/* if Nth symbol table entry has .st_shndx = SHN_XINDEX,
+	 * take shndx from symtab_shndx_start[N] instead */
+	Elf32_Word   *symtab_shndx_start;
+	Elf32_Word   *symtab_shndx_stop;
+
+	struct pmd_driver *drivers;
+};
+
diff --git a/mk/rte.buildtools.mk b/mk/rte.buildtools.mk
new file mode 100644
index 0000000..e8bfcef
--- /dev/null
+++ b/mk/rte.buildtools.mk
@@ -0,0 +1,148 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
+#   Copyright(c) 2014-2015 6WIND S.A.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/internal/rte.compile-pre.mk
+include $(RTE_SDK)/mk/internal/rte.install-pre.mk
+include $(RTE_SDK)/mk/internal/rte.clean-pre.mk
+include $(RTE_SDK)/mk/internal/rte.build-pre.mk
+include $(RTE_SDK)/mk/internal/rte.depdirs-pre.mk
+
+# VPATH contains at least SRCDIR
+VPATH += $(SRCDIR)
+
+_BUILD = $(APP)
+_INSTALL = $(INSTALL-FILES-y) $(SYMLINK-FILES-y)
+_INSTALL += $(RTE_OUTPUT)/buildtools/$(APP) $(RTE_OUTPUT)/buildtools/$(APP).map
+POSTINSTALL += target-appinstall
+_CLEAN = doclean
+POSTCLEAN += target-appclean
+
+.PHONY: all
+all: install
+
+.PHONY: install
+install: build _postinstall
+
+_postinstall: build
+
+.PHONY: build
+build: _postbuild
+
+exe2cmd = $(strip $(call dotfile,$(patsubst %,%.cmd,$(1))))
+
+ifeq ($(LINK_USING_CC),1)
+override EXTRA_LDFLAGS := $(call linkerprefix,$(EXTRA_LDFLAGS))
+O_TO_EXE = $(CC) $(CFLAGS) $(LDFLAGS_$(@)) \
+	-Wl,-Map=$(@).map,--cref -o $@ $(OBJS-y) $(call linkerprefix,$(LDFLAGS)) \
+	$(EXTRA_LDFLAGS) $(call linkerprefix,$(LDLIBS))
+else
+O_TO_EXE = $(LD) $(LDFLAGS) $(LDFLAGS_$(@)) $(EXTRA_LDFLAGS) \
+	-Map=$(@).map --cref -o $@ $(OBJS-y) $(LDLIBS)
+endif
+O_TO_EXE_STR = $(subst ','\'',$(O_TO_EXE)) #'# fix syntax highlight
+O_TO_EXE_DISP = $(if $(V),"$(O_TO_EXE_STR)","  LD $(@)")
+O_TO_EXE_CMD = "cmd_$@ = $(O_TO_EXE_STR)"
+O_TO_EXE_DO = @set -e; \
+	echo $(O_TO_EXE_DISP); \
+	$(O_TO_EXE) && \
+	echo $(O_TO_EXE_CMD) > $(call exe2cmd,$(@))
+
+-include .$(APP).cmd
+
+# path where libraries are retrieved
+LDLIBS_PATH := $(subst -Wl$(comma)-L,,$(filter -Wl$(comma)-L%,$(LDLIBS)))
+LDLIBS_PATH += $(subst -L,,$(filter -L%,$(LDLIBS)))
+
+# list of .a files that are linked to this application
+LDLIBS_NAMES := $(patsubst -l%,lib%.a,$(filter -l%,$(LDLIBS)))
+LDLIBS_NAMES += $(patsubst -Wl$(comma)-l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIBS)))
+
+# list of found libraries files (useful for deps). If not found, the
+# library is silently ignored and dep won't be checked
+LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
+	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
+
+#
+# Compile executable file if needed
+#
+$(APP): $(OBJS-y) $(LDLIBS_FILES) $(DEP_$(APP)) $(LDSCRIPT) FORCE
+	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
+	$(if $(D),\
+		@echo -n "$< -> $@ " ; \
+		echo -n "file_missing=$(call boolean,$(file_missing)) " ; \
+		echo -n "cmdline_changed=$(call boolean,$(call cmdline_changed,$(O_TO_EXE_STR))) " ; \
+		echo -n "depfile_missing=$(call boolean,$(depfile_missing)) " ; \
+		echo "depfile_newer=$(call boolean,$(depfile_newer)) ")
+	$(if $(or \
+		$(file_missing),\
+		$(call cmdline_changed,$(O_TO_EXE_STR)),\
+		$(depfile_missing),\
+		$(depfile_newer)),\
+		$(O_TO_EXE_DO))
+
+#
+# install app in $(RTE_OUTPUT)/app
+#
+$(RTE_OUTPUT)/buildtools/$(APP): $(APP)
+	@echo "  INSTALL-APP $(APP)"
+	@[ -d $(RTE_OUTPUT)/buildtools ] || mkdir -p $(RTE_OUTPUT)/buildtools
+	$(Q)cp -f $(APP) $(RTE_OUTPUT)/buildtools
+
+#
+# install app map file in $(RTE_OUTPUT)/app
+#
+$(RTE_OUTPUT)/buildtools/$(APP).map: $(APP)
+	@echo "  INSTALL-MAP $(APP).map"
+	@[ -d $(RTE_OUTPUT)/buildtools ] || mkdir -p $(RTE_OUTPUT)/buildtools
+	$(Q)cp -f $(APP).map $(RTE_OUTPUT)/buildtools
+
+#
+# Clean all generated files
+#
+.PHONY: clean
+clean: _postclean
+	$(Q)rm -f $(_BUILD_TARGETS) $(_INSTALL_TARGETS) $(_CLEAN_TARGETS)
+
+.PHONY: doclean
+doclean:
+	$(Q)rm -rf $(APP) $(OBJS-all) $(DEPS-all) $(DEPSTMP-all) \
+	  $(CMDS-all) $(INSTALL-FILES-all) .$(APP).cmd
+
+
+include $(RTE_SDK)/mk/internal/rte.compile-post.mk
+include $(RTE_SDK)/mk/internal/rte.install-post.mk
+include $(RTE_SDK)/mk/internal/rte.clean-post.mk
+include $(RTE_SDK)/mk/internal/rte.build-post.mk
+include $(RTE_SDK)/mk/internal/rte.depdirs-post.mk
+
+.PHONY: FORCE
+FORCE:
diff --git a/mk/rte.sdkbuild.mk b/mk/rte.sdkbuild.mk
index eec5241..fb68af2 100644
--- a/mk/rte.sdkbuild.mk
+++ b/mk/rte.sdkbuild.mk
@@ -64,7 +64,8 @@ build: $(ROOTDIRS-y)
 clean: $(CLEANDIRS)
 	@rm -rf $(RTE_OUTPUT)/include $(RTE_OUTPUT)/app \
 		$(RTE_OUTPUT)/hostapp $(RTE_OUTPUT)/lib \
-		$(RTE_OUTPUT)/hostlib $(RTE_OUTPUT)/kmod
+		$(RTE_OUTPUT)/hostlib $(RTE_OUTPUT)/kmod \
+		$(RTE_OUTPUT)/buildtools
 	@[ -d $(RTE_OUTPUT)/include ] || mkdir -p $(RTE_OUTPUT)/include
 	@$(RTE_SDK)/scripts/gen-config-h.sh $(RTE_OUTPUT)/.config \
 		> $(RTE_OUTPUT)/include/rte_config.h
-- 
2.5.5

[PATCHv4 2/5] drivers: Update driver registration macro usage

[Neil Horman]

Modify the PMD_REGISTER_DRIVER macro, adding a name argument to it.  The
addition of a name argument creates a token that can be used for subsequent
macros in the creation of unique symbol names to export additional bits of
information for use by the pmdinfogen tool.  For example:

PMD_REGISTER_DRIVER(ena_driver, ena);

registers the ena_driver struct as it always did, and creates a symbol
const char this_pmd_name0[] __attribute__((used)) = "ena";

which pmdinfogen can search for and extract.  The subsequent macro

DRIVER_REGISTER_PCI_TABLE(ena, ena_pci_id_map);

creates a symbol
const char ena_pci_tbl_export[] __attribute__((used)) = "ena_pci_id_map";

Which allows pmdinfogen to find the pci table of this driver

Using this pattern, we can export arbitrary bits of information.

pmdinfo uses this information to extract hardware support from an object file
and create a json string to make hardware support info discoverable later.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 drivers/Makefile                           |  2 ++
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c   |  4 +++-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c |  4 +++-
 drivers/crypto/null/null_crypto_pmd.c      |  4 +++-
 drivers/crypto/qat/rte_qat_cryptodev.c     |  4 +++-
 drivers/crypto/snow3g/rte_snow3g_pmd.c     |  4 +++-
 drivers/net/af_packet/rte_eth_af_packet.c  |  4 +++-
 drivers/net/bnx2x/bnx2x_ethdev.c           |  6 ++++--
 drivers/net/bonding/rte_eth_bond_pmd.c     |  7 ++++++-
 drivers/net/cxgbe/cxgbe_ethdev.c           |  4 +++-
 drivers/net/e1000/em_ethdev.c              |  3 ++-
 drivers/net/e1000/igb_ethdev.c             |  6 ++++--
 drivers/net/ena/ena_ethdev.c               |  3 ++-
 drivers/net/enic/enic_ethdev.c             |  3 ++-
 drivers/net/fm10k/fm10k_ethdev.c           |  3 ++-
 drivers/net/i40e/i40e_ethdev.c             |  3 ++-
 drivers/net/i40e/i40e_ethdev_vf.c          |  3 ++-
 drivers/net/ixgbe/ixgbe_ethdev.c           |  6 ++++--
 drivers/net/mlx4/mlx4.c                    |  3 ++-
 drivers/net/mlx5/mlx5.c                    |  3 ++-
 drivers/net/mpipe/mpipe_tilegx.c           |  4 ++--
 drivers/net/nfp/nfp_net.c                  |  3 ++-
 drivers/net/null/rte_eth_null.c            |  3 ++-
 drivers/net/pcap/rte_eth_pcap.c            |  4 +++-
 drivers/net/qede/qede_ethdev.c             |  6 ++++--
 drivers/net/ring/rte_eth_ring.c            |  3 ++-
 drivers/net/szedata2/rte_eth_szedata2.c    |  3 ++-
 drivers/net/vhost/rte_eth_vhost.c          |  3 ++-
 drivers/net/virtio/virtio_ethdev.c         |  3 ++-
 drivers/net/vmxnet3/vmxnet3_ethdev.c       |  3 ++-
 drivers/net/xenvirt/rte_eth_xenvirt.c      |  2 +-
 lib/librte_eal/common/include/rte_dev.h    | 23 +++++++++++++++++++----
 32 files changed, 100 insertions(+), 39 deletions(-)

diff --git a/drivers/Makefile b/drivers/Makefile
index 81c03a8..75a3168 100644
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -34,4 +34,6 @@ include $(RTE_SDK)/mk/rte.vars.mk
 DIRS-y += net
 DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += crypto
 
+DEPDIRS-y += buildtools/pmdinfo
+
 include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
index 2987ef6..593769f 100644
--- a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
+++ b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
@@ -521,4 +521,6 @@ static struct rte_driver aesni_gcm_pmd_drv = {
 	.uninit = aesni_gcm_uninit
 };
 
-PMD_REGISTER_DRIVER(aesni_gcm_pmd_drv);
+PMD_REGISTER_DRIVER(aesni_gcm_pmd_drv, aesni_gcm);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> \
+max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
index 3415ac1..2e30939 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
@@ -716,4 +716,6 @@ static struct rte_driver cryptodev_aesni_mb_pmd_drv = {
 	.uninit = cryptodev_aesni_mb_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drv);
+PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drvi, aesni_mb);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> \
+max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/null/null_crypto_pmd.c b/drivers/crypto/null/null_crypto_pmd.c
index bdaf13c..44c4fba 100644
--- a/drivers/crypto/null/null_crypto_pmd.c
+++ b/drivers/crypto/null/null_crypto_pmd.c
@@ -275,4 +275,6 @@ static struct rte_driver cryptodev_null_pmd_drv = {
 	.uninit = cryptodev_null_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_null_pmd_drv);
+PMD_REGISTER_DRIVER(cryptodev_null_pmd_drv, cryptodev_null_pmd);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> \
+max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index a7912f5..d2de6a6 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -137,4 +137,6 @@ static struct rte_driver pmd_qat_drv = {
 	.init = rte_qat_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(pmd_qat_drv);
+PMD_REGISTER_DRIVER(pmd_qat_drv, qat);
+DRIVER_REGISTER_PCI_TABLE(qat, pci_id_qat_map);
+
diff --git a/drivers/crypto/snow3g/rte_snow3g_pmd.c b/drivers/crypto/snow3g/rte_snow3g_pmd.c
index f3e0e66..eee2544 100644
--- a/drivers/crypto/snow3g/rte_snow3g_pmd.c
+++ b/drivers/crypto/snow3g/rte_snow3g_pmd.c
@@ -548,4 +548,6 @@ static struct rte_driver cryptodev_snow3g_pmd_drv = {
 	.uninit = cryptodev_snow3g_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_snow3g_pmd_drv);
+PMD_REGISTER_DRIVER(cryptodev_snow3g_pmd_drv, snow3g);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> \
+max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index f17bd7e..e960512 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -878,4 +878,6 @@ static struct rte_driver pmd_af_packet_drv = {
 	.uninit = rte_pmd_af_packet_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_af_packet_drv);
+PMD_REGISTER_DRIVER(pmd_af_packet_drv, eth_af_packet);
+DRIVER_REGISTER_PARAM_STRING(eth_af_packet, "iface=<string> \
+qpairs=<int> blocksz=<int> framesz=<int> framecnt=<int>");
diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c b/drivers/net/bnx2x/bnx2x_ethdev.c
index 071b44f..2b88324 100644
--- a/drivers/net/bnx2x/bnx2x_ethdev.c
+++ b/drivers/net/bnx2x/bnx2x_ethdev.c
@@ -550,5 +550,7 @@ static struct rte_driver rte_bnx2xvf_driver = {
 	.init = rte_bnx2xvf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_bnx2x_driver);
-PMD_REGISTER_DRIVER(rte_bnx2xvf_driver);
+PMD_REGISTER_DRIVER(rte_bnx2x_driver, bnx2x);
+DRIVER_REGISTER_PCI_TABLE(bnx2x, pci_id_bnx2x_map);
+PMD_REGISTER_DRIVER(rte_bnx2xvf_driver, bnx2xvf);
+DRIVER_REGISTER_PCI_TABLE(bnx2xvf, pci_id_bnx2xvf_map);
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 129f04b..b4556be 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -2511,4 +2511,9 @@ static struct rte_driver bond_drv = {
 	.uninit = bond_uninit,
 };
 
-PMD_REGISTER_DRIVER(bond_drv);
+PMD_REGISTER_DRIVER(bond_drv, bonding);
+
+DRIVER_REGISTER_PARAM_STRING(bonding, "slave=<ifc> primary=<ifc> mode=[0-4] \
+xmit_policy=[l2 | l23 | l34] socket_id=<int> mac=<mac addr> \
+lsc_poll_period_ms=<int> up_delay=<int> down_delay=<int>");
+
diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index 04eddaf..e84e48b 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -894,4 +894,6 @@ static struct rte_driver rte_cxgbe_driver = {
 	.init = rte_cxgbe_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_cxgbe_driver);
+PMD_REGISTER_DRIVER(rte_cxgbe_driver, cxgb4);
+DRIVER_REGISTER_PCI_TABLE(cxgb4, cxgb4_pci_tbl);
+
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index 653be09..4de5eb2 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -1777,4 +1777,5 @@ struct rte_driver em_pmd_drv = {
 	.init = rte_em_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(em_pmd_drv);
+PMD_REGISTER_DRIVER(em_pmd_drv, em);
+DRIVER_REGISTER_PCI_TABLE(em, pci_id_em_map);
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index f0921ee..f49aae7 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -4973,5 +4973,7 @@ eth_igb_configure_msix_intr(struct rte_eth_dev *dev)
 	E1000_WRITE_FLUSH(hw);
 }
 
-PMD_REGISTER_DRIVER(pmd_igb_drv);
-PMD_REGISTER_DRIVER(pmd_igbvf_drv);
+PMD_REGISTER_DRIVER(pmd_igb_drv, igb);
+DRIVER_REGISTER_PCI_TABLE(igb, pci_id_igb_map);
+PMD_REGISTER_DRIVER(pmd_igbvf_drv, igbvf);
+DRIVER_REGISTER_PCI_TABLE(igbvf, pci_id_igbvf_map);
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index e157587..702289b 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -1450,4 +1450,5 @@ struct rte_driver ena_pmd_drv = {
 	.init = rte_ena_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(ena_pmd_drv);
+PMD_REGISTER_DRIVER(ena_pmd_drv, ena);
+DRIVER_REGISTER_PCI_TABLE(ena, pci_id_ena_map);
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index 6bea940..36ee7ca 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -684,4 +684,5 @@ static struct rte_driver rte_enic_driver = {
 	.init = rte_enic_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_enic_driver);
+PMD_REGISTER_DRIVER(rte_enic_driver, enic);
+DRIVER_REGISTER_PCI_TABLE(enic, pci_id_enic_map);
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index c2d377f..9b2a7ee 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -3055,4 +3055,5 @@ static struct rte_driver rte_fm10k_driver = {
 	.init = rte_pmd_fm10k_init,
 };
 
-PMD_REGISTER_DRIVER(rte_fm10k_driver);
+PMD_REGISTER_DRIVER(rte_fm10k_driver, fm10k);
+DRIVER_REGISTER_PCI_TABLE(fm10k, pci_id_fm10k_map);
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 24777d5..3b6c661 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -695,7 +695,8 @@ static struct rte_driver rte_i40e_driver = {
 	.init = rte_i40e_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_i40e_driver);
+PMD_REGISTER_DRIVER(rte_i40e_driver, i40e);
+DRIVER_REGISTER_PCI_TABLE(i40e, pci_id_i40e_map);
 
 /*
  * Initialize registers for flexible payload, which should be set by NVM.
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 90682ac..e94a459 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1563,7 +1563,8 @@ static struct rte_driver rte_i40evf_driver = {
 	.init = rte_i40evf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_i40evf_driver);
+PMD_REGISTER_DRIVER(rte_i40evf_driver, i40evf);
+DRIVER_REGISTER_PCI_TABLE(i40evf, pci_id_i40evf_map);
 
 static int
 i40evf_dev_configure(struct rte_eth_dev *dev)
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index a2b170b..af1ca56 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -7164,5 +7164,7 @@ static struct rte_driver rte_ixgbevf_driver = {
 	.init = rte_ixgbevf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_ixgbe_driver);
-PMD_REGISTER_DRIVER(rte_ixgbevf_driver);
+PMD_REGISTER_DRIVER(rte_ixgbe_driver, ixgbe);
+DRIVER_REGISTER_PCI_TABLE(ixgbe, pci_id_ixgbe_map);
+PMD_REGISTER_DRIVER(rte_ixgbevf_driver, ixgbevf);
+DRIVER_REGISTER_PCI_TABLE(ixgbevf, pci_id_ixgbevf_map);
diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index c5d8535..87add1f 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -5817,4 +5817,5 @@ static struct rte_driver rte_mlx4_driver = {
 	.init = rte_mlx4_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_mlx4_driver)
+PMD_REGISTER_DRIVER(rte_mlx4_driveri, mlx4)
+DRIVER_REGISTER_PCI_TABLE(mlx4, mlx4_pci_id_map);
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 041cfc3..5f2d6c8 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -672,4 +672,5 @@ static struct rte_driver rte_mlx5_driver = {
 	.init = rte_mlx5_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_mlx5_driver)
+PMD_REGISTER_DRIVER(rte_mlx5_driveri, mlx5)
+DRIVER_REGISTER_PCI_TABLE(mlx5, mlx5_pci_id_map);
diff --git a/drivers/net/mpipe/mpipe_tilegx.c b/drivers/net/mpipe/mpipe_tilegx.c
index 26e1424..4a2a905 100644
--- a/drivers/net/mpipe/mpipe_tilegx.c
+++ b/drivers/net/mpipe/mpipe_tilegx.c
@@ -1635,8 +1635,8 @@ static struct rte_driver pmd_mpipe_gbe_drv = {
 	.init = rte_pmd_mpipe_devinit,
 };
 
-PMD_REGISTER_DRIVER(pmd_mpipe_xgbe_drv);
-PMD_REGISTER_DRIVER(pmd_mpipe_gbe_drv);
+PMD_REGISTER_DRIVER(pmd_mpipe_xgbe_drv, mpipe_xgbe);
+PMD_REGISTER_DRIVER(pmd_mpipe_gbe_drv, mpipe_gbe);
 
 static void __attribute__((constructor, used))
 mpipe_init_contexts(void)
diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
index ea5a2a3..78394a1 100644
--- a/drivers/net/nfp/nfp_net.c
+++ b/drivers/net/nfp/nfp_net.c
@@ -2490,7 +2490,8 @@ static struct rte_driver rte_nfp_net_driver = {
 	.init = nfp_net_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_nfp_net_driver);
+PMD_REGISTER_DRIVER(rte_nfp_net_driver, nfp);
+DRIVER_REGISTER_PCI_TABLE(nfp, pci_id_nfp_net_map);
 
 /*
  * Local variables:
diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index 5e8e203..88eb582 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -689,4 +689,5 @@ static struct rte_driver pmd_null_drv = {
 	.uninit = rte_pmd_null_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_null_drv);
+PMD_REGISTER_DRIVER(pmd_null_drv, eth_null);
+DRIVER_REGISTER_PARAM_STRING(eth_null, "size=<int> copy=<int>");
diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
index c98e234..397f01c 100644
--- a/drivers/net/pcap/rte_eth_pcap.c
+++ b/drivers/net/pcap/rte_eth_pcap.c
@@ -1090,4 +1090,6 @@ static struct rte_driver pmd_pcap_drv = {
 	.uninit = rte_pmd_pcap_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_pcap_drv);
+PMD_REGISTER_DRIVER(pmd_pcap_drv, pcap);
+DRIVER_REGISTER_PARAM_STRING(pcap, "rx_pcap=<string> tx_pcap=<string> \
+rx_iface=<ifc> tx_iface=<ifc> iface=<ifc>");
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index 1273fd3..187ccf1 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -1102,5 +1102,7 @@ static struct rte_driver rte_qede_driver = {
 	.init = rte_qedevf_pmd_init
 };
 
-PMD_REGISTER_DRIVER(rte_qede_driver);
-PMD_REGISTER_DRIVER(rte_qedevf_driver);
+PMD_REGISTER_DRIVER(rte_qede_driver, qede);
+DRIVER_REGISTER_PCI_TABLE(qede, pci_id_qede_map);
+PMD_REGISTER_DRIVER(rte_qedevf_driver, qedevf);
+DRIVER_REGISTER_PCI_TABLE(qedevf, pci_id_qedevf_map);
diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index b1783c3..f2f5217 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -630,4 +630,5 @@ static struct rte_driver pmd_ring_drv = {
 	.uninit = rte_pmd_ring_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_ring_drv);
+PMD_REGISTER_DRIVER(pmd_ring_drv, eth_ring);
+DRIVER_REGISTER_PARAM_STRING(eth_ring, "nodeaction=[attach|detach]");
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 78c43b0..abbc907 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1601,4 +1601,5 @@ static struct rte_driver rte_szedata2_driver = {
 	.uninit = rte_szedata2_uninit,
 };
 
-PMD_REGISTER_DRIVER(rte_szedata2_driver);
+PMD_REGISTER_DRIVER(rte_szedata2_driver, rte_szedata2_pmd);
+DRIVER_REGISTER_PCI_TABLE(rte_szedata2_pmd, rte_szedata2_pci_id_table);
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index 310cbef..d321c3e 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -924,4 +924,5 @@ static struct rte_driver pmd_vhost_drv = {
 	.uninit = rte_pmd_vhost_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_vhost_drv);
+PMD_REGISTER_DRIVER(pmd_vhost_drv, eth_vhost);
+DRIVER_REGISTER_PARAM_STRING(eth_vhost, "iface=<ifc> queues=<int>");
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index c3fb628..24d387f 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -1459,4 +1459,5 @@ static struct rte_driver rte_virtio_driver = {
 	.init = rte_virtio_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_virtio_driver);
+PMD_REGISTER_DRIVER(rte_virtio_driver, virtio_net);
+DRIVER_REGISTER_PCI_TABLE(virtio_net, pci_id_virtio_map);
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index 29b469c..8da4449 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -954,4 +954,5 @@ static struct rte_driver rte_vmxnet3_driver = {
 	.init = rte_vmxnet3_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_vmxnet3_driver);
+PMD_REGISTER_DRIVER(rte_vmxnet3_driver, vmxnet3);
+DRIVER_REGISTER_PCI_TABLE(vmxnet3, pci_id_vmxnet3_map);
diff --git a/drivers/net/xenvirt/rte_eth_xenvirt.c b/drivers/net/xenvirt/rte_eth_xenvirt.c
index 1adeb5b..81f8af9 100644
--- a/drivers/net/xenvirt/rte_eth_xenvirt.c
+++ b/drivers/net/xenvirt/rte_eth_xenvirt.c
@@ -763,4 +763,4 @@ static struct rte_driver pmd_xenvirt_drv = {
 	.uninit = rte_pmd_xenvirt_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_xenvirt_drv);
+PMD_REGISTER_DRIVER(pmd_xenvirt_drv, xenvirt);
diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
index f1b5507..e608ff7 100644
--- a/lib/librte_eal/common/include/rte_dev.h
+++ b/lib/librte_eal/common/include/rte_dev.h
@@ -48,7 +48,7 @@ extern "C" {
 
 #include <stdio.h>
 #include <sys/queue.h>
-
+#include <rte_pci.h>
 #include <rte_log.h>
 
 __attribute__((format(printf, 2, 0)))
@@ -178,12 +178,27 @@ int rte_eal_vdev_init(const char *name, const char *args);
  */
 int rte_eal_vdev_uninit(const char *name);
 
-#define PMD_REGISTER_DRIVER(d)\
+#define DRIVER_EXPORT_NAME_ARRAY(n, idx) n##idx[] __attribute__((used))
+
+#define DRIVER_EXPORT_NAME(d, idx) \
+static const char DRIVER_EXPORT_NAME_ARRAY(this_pmd_name, idx) = RTE_STR(d);\
+ 
+#define PMD_REGISTER_DRIVER(d, n)\
 void devinitfn_ ##d(void);\
 void __attribute__((constructor, used)) devinitfn_ ##d(void)\
 {\
-	rte_eal_driver_register(&d);\
-}
+	(d).name = RTE_STR(n);\
+        rte_eal_driver_register(&d);\
+}\
+DRIVER_EXPORT_NAME(n, __COUNTER__)
+
+#define DRV_EXP_TAG(n, t) __##n##_##t
+
+#define DRIVER_REGISTER_PCI_TABLE(n, t) \
+static const char DRV_EXP_TAG(n, pci_tbl_export)[] __attribute__((used)) = RTE_STR(t)
+
+#define DRIVER_REGISTER_PARAM_STRING(n, s) \
+static const char DRV_EXP_TAG(n, param_string_export)[] __attribute__((used)) = s
 
 #ifdef __cplusplus
 }
-- 
2.5.5

[PATCHv4 3/5] eal: Add an export symbol to expose the autoload path to external tools

[Neil Horman]

Export a symbol containing the string:
DPDK_PLUGIN_PATH="$(CONFIG_RTE_EAL_PMD_PATH)"

Where the latter half of the string is set at build time to a location from
which autoloaded DSO's will be found.  This string is used by pmdinfo in
'plugin' mode, whereby a user can specify a dpdk installation directory (or
static binary), and scan the associated path (if found) for pmd DSO's and report
on their hardware support.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 lib/librte_eal/common/eal_common_options.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c
index 3efc90f..1af0fb3 100644
--- a/lib/librte_eal/common/eal_common_options.c
+++ b/lib/librte_eal/common/eal_common_options.c
@@ -115,6 +115,15 @@ TAILQ_HEAD_INITIALIZER(solib_list);
 /* Default path of external loadable drivers */
 static const char *default_solib_dir = RTE_EAL_PMD_PATH;
 
+/*
+ * Stringified version of solib path used by pmdinfo.py 
+ * Note: PLEASE DO NOT ALTER THIS without making a corresponding
+ * change to tools/pmdinfo.py
+ */
+static const char dpdk_solib_path[] __attribute__((used)) = \
+"DPDK_PLUGIN_PATH=" RTE_EAL_PMD_PATH;
+
+
 static int master_lcore_parsed;
 static int mem_parsed;
 
-- 
2.5.5

[PATCHv4 4/5] Makefile: Do post processing on objects that register a driver

[Neil Horman]

Modify the compilation makefile to identify C files that export PMD information,
and use that to trigger execution of the pmdinfo binary.  If the execution of
pmdinfo is successful, compile the output C file to an object, and use the
linker to do relocatable linking on the resultant object file into the parent
object that it came from.  This effectively just adds the json string into the
string table of the object that defines the PMD to the outside world.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 mk/internal/rte.compile-pre.mk | 19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/mk/internal/rte.compile-pre.mk b/mk/internal/rte.compile-pre.mk
index b9bff4a..6fa4042 100644
--- a/mk/internal/rte.compile-pre.mk
+++ b/mk/internal/rte.compile-pre.mk
@@ -80,7 +80,8 @@ C_TO_O_STR = $(subst ','\'',$(C_TO_O)) #'# fix syntax highlight
 C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)","  HOSTCC $(@)")
 else
 C_TO_O = $(CC) -Wp,-MD,$(call obj2dep,$(@)).tmp $(CFLAGS) \
-	$(CFLAGS_$(@)) $(EXTRA_CFLAGS) -o $@ -c $<
+	 $(CFLAGS_$(@)) $(EXTRA_CFLAGS) -o $@ -c $<
+
 C_TO_O_STR = $(subst ','\'',$(C_TO_O)) #'# fix syntax highlight
 C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)","  CC $(@)")
 endif
@@ -88,10 +89,26 @@ C_TO_O_CMD = 'cmd_$@ = $(C_TO_O_STR)'
 C_TO_O_DO = @set -e; \
 	echo $(C_TO_O_DISP); \
 	$(C_TO_O) && \
+	sh -c "grep -q \"PMD_REGISTER_DRIVER(.*)\" $<; \
+	if [ \$$? -eq 0 ]; \
+	then \
+		echo MODGEN $@; \
+		OBJF=`readlink -f $@`; \
+		${RTE_OUTPUT}/buildtools/pmdinfogen \$$OBJF \$$OBJF.mod.c; \
+		if [ \$$? -eq 0 ]; \
+		then \
+			echo MODBUILD $@; \
+			$(CC) -c -o \$$OBJF.mod.o \$$OBJF.mod.c; \
+			$(CROSS)ld -r -o \$$OBJF.o \$$OBJF.mod.o \$$OBJF; \
+			mv -f \$$OBJF.o \$$OBJF; \
+		fi; \
+	fi; \
+	true" && \
 	echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \
 	sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \
 	rm -f $(call obj2dep,$(@)).tmp
 
+
 # return an empty string if string are equal
 compare = $(strip $(subst $(1),,$(2)) $(subst $(2),,$(1)))
 
-- 
2.5.5

[PATCHv4 5/5] pmdinfo.py: Add tool to query binaries for hw and other support information

[Neil Horman]

This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
and, if found parses the remainder of the string as a json encoded string,
outputting the results in either a human readable or raw, script parseable
format

Note that, in the case of dynamically linked applications, pmdinfo.py will scan
for implicitly linked PMDs by searching the specified binaries .dynamic section
for DT_NEEDED entries that contain the substring librte_pmd.  The DT_RUNPATH,
LD_LIBRARY_PATH, /usr/lib and /lib are searched for these libraries, in that
order

If a file is specified with no path, it is assumed to be a PMD DSO, and the
LD_LIBRARY_PATH, /usr/lib[64]/ and /lib[64] is searched for it

Currently the tool can output data in 3 formats:

a) raw, suitable for scripting, where the raw JSON strings are dumped out
b) table format (default) where hex pci ids are dumped in a table format
c) pretty, where a user supplied pci.ids file is used to print out vendor and
device strings

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 mk/rte.sdkinstall.mk |   2 +
 tools/pmdinfo.py     | 618 +++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 620 insertions(+)
 create mode 100755 tools/pmdinfo.py

diff --git a/mk/rte.sdkinstall.mk b/mk/rte.sdkinstall.mk
index 68e56b6..119129a 100644
--- a/mk/rte.sdkinstall.mk
+++ b/mk/rte.sdkinstall.mk
@@ -126,6 +126,8 @@ install-runtime:
 	$(Q)$(call rte_mkdir,      $(DESTDIR)$(sbindir))
 	$(Q)$(call rte_symlink,    $(DESTDIR)$(datadir)/tools/dpdk_nic_bind.py, \
 	                           $(DESTDIR)$(sbindir)/dpdk_nic_bind)
+	$(Q)$(call rte_symlink,    $(DESTDIR)$(datadir)/tools/pmdinfo.py, \
+				   $(DESTDIR)$(bindir)/pmdinfo)
 
 install-kmod:
 ifneq ($(wildcard $O/kmod/*),)
diff --git a/tools/pmdinfo.py b/tools/pmdinfo.py
new file mode 100755
index 0000000..e2b410d
--- /dev/null
+++ b/tools/pmdinfo.py
@@ -0,0 +1,618 @@
+#!/usr/bin/python
+#-------------------------------------------------------------------------------
+# scripts/pmd_hw_support.py
+#
+# Utility to dump PMD_INFO_STRING support from an object file
+#
+#-------------------------------------------------------------------------------
+import os, sys
+from optparse import OptionParser
+import string
+import json
+
+# For running from development directory. It should take precedence over the
+# installed pyelftools.
+sys.path.insert(0, '.')
+
+
+from elftools import __version__
+from elftools.common.exceptions import ELFError
+from elftools.common.py3compat import (
+        ifilter, byte2int, bytes2str, itervalues, str2bytes)
+from elftools.elf.elffile import ELFFile
+from elftools.elf.dynamic import DynamicSection, DynamicSegment
+from elftools.elf.enums import ENUM_D_TAG
+from elftools.elf.segments import InterpSegment
+from elftools.elf.sections import SymbolTableSection
+from elftools.elf.gnuversions import (
+    GNUVerSymSection, GNUVerDefSection,
+    GNUVerNeedSection,
+    )
+from elftools.elf.relocation import RelocationSection
+from elftools.elf.descriptions import (
+    describe_ei_class, describe_ei_data, describe_ei_version,
+    describe_ei_osabi, describe_e_type, describe_e_machine,
+    describe_e_version_numeric, describe_p_type, describe_p_flags,
+    describe_sh_type, describe_sh_flags,
+    describe_symbol_type, describe_symbol_bind, describe_symbol_visibility,
+    describe_symbol_shndx, describe_reloc_type, describe_dyn_tag,
+    describe_ver_flags,
+    )
+from elftools.elf.constants import E_FLAGS
+from elftools.dwarf.dwarfinfo import DWARFInfo
+from elftools.dwarf.descriptions import (
+    describe_reg_name, describe_attr_value, set_global_machine_arch,
+    describe_CFI_instructions, describe_CFI_register_rule,
+    describe_CFI_CFA_rule,
+    )
+from elftools.dwarf.constants import (
+    DW_LNS_copy, DW_LNS_set_file, DW_LNE_define_file)
+from elftools.dwarf.callframe import CIE, FDE
+
+raw_output = False
+pcidb = None
+
+#===========================================
+
+class Vendor:
+    """
+    Class for vendors. This is the top level class
+    for the devices belong to a specific vendor.
+    self.devices is the device dictionary
+    subdevices are in each device.
+    """
+    def __init__(self, vendorStr):
+        """
+        Class initializes with the raw line from pci.ids
+        Parsing takes place inside __init__
+        """
+        self.ID = vendorStr.split()[0]
+        self.name = vendorStr.replace("%s " % self.ID,"").rstrip()
+        self.devices = {}
+
+    def addDevice(self, deviceStr):
+        """
+        Adds a device to self.devices
+        takes the raw line from pci.ids
+        """
+        s = deviceStr.strip()
+        devID = s.split()[0]
+        if devID in self.devices:
+            pass
+        else:
+            self.devices[devID] = Device(deviceStr)
+
+    def report(self):
+        print self.ID, self.name
+        for id, dev in self.devices.items():
+            dev.report()
+
+    def find_device(self, devid):
+        # convert to a hex string and remove 0x
+        devid = hex(devid)[2:]
+        try:
+            return self.devices[devid]
+        except:
+            return Device("%s  Unknown Device" % devid) 
+
+class Device:
+    def __init__(self, deviceStr):
+        """
+        Class for each device.
+        Each vendor has its own devices dictionary.
+        """
+        s = deviceStr.strip()
+        self.ID = s.split()[0]
+        self.name = s.replace("%s  " % self.ID,"")
+        self.subdevices = {}
+
+    def report(self):
+        print "\t%s\t%s" % (self.ID, self.name)
+        for subID, subdev in self.subdevices.items():
+            subdev.report()
+
+    def addSubDevice(self, subDeviceStr):
+        """
+        Adds a subvendor, subdevice to device.
+        Uses raw line from pci.ids
+        """
+        s = subDeviceStr.strip()
+        spl = s.split()
+        subVendorID  = spl[0]
+        subDeviceID  = spl[1]
+        subDeviceName = s.split("  ")[-1]
+        devID = "%s:%s" % (subVendorID,subDeviceID)
+        self.subdevices[devID] = SubDevice(subVendorID,subDeviceID,subDeviceName)
+
+    def find_subid(self, subven, subdev):
+        subven = hex(subven)[2:]
+        subdev = hex(subdev)[2:]
+        devid ="%s:%s" % (subven, subdev)
+
+        try:
+            return self.subdevices[devid]
+        except:
+            if (subven == "ffff" and subdev == "ffff"):
+                return SubDevice("ffff", "ffff", "(All Subdevices)");
+            else:
+                return SubDevice(subven, subdev, "(Unknown Subdevice)")
+        
+
+class SubDevice:
+    """
+    Class for subdevices.
+    """
+    def __init__(self, vendor, device, name):
+        """
+        Class initializes with vendorid, deviceid and name
+        """
+        self.vendorID = vendor
+        self.deviceID = device
+        self.name = name
+
+    def report(self):
+        print "\t\t%s\t%s\t%s" % (self.vendorID, self.deviceID,self.name)
+
+class PCIIds:
+    """
+    Top class for all pci.ids entries.
+    All queries will be asked to this class.
+    PCIIds.vendors["0e11"].devices["0046"].subdevices["0e11:4091"].name  =  "Smart Array 6i"
+    """
+    def __init__(self, filename):
+        """
+        Prepares the directories.
+        Checks local data file.
+        Tries to load from local, if not found, downloads from web
+        """
+        self.version = ""
+        self.date = ""
+        self.vendors = {}
+        self.contents = None
+        self.readLocal(filename)
+        self.parse()
+
+    def reportVendors(self):
+        """Reports the vendors
+        """
+        for vid, v in self.vendors.items():
+            print v.ID, v.name
+
+    def report(self, vendor = None):
+        """
+        Reports everything for all vendors or a specific vendor
+        PCIIds.report()  reports everything
+        PCIIDs.report("0e11") reports only "Compaq Computer Corporation"
+        """
+        if vendor != None:
+            self.vendors[vendor].report()
+        else:
+            for vID, v in self.vendors.items():
+                v.report()
+
+    def find_vendor(self, vid):
+        # convert vid to a hex string and remove the 0x
+        vid = hex(vid)[2:]
+
+        try:
+            return self.vendors[vid]
+        except: 
+            return Vendor("%s Unknown Vendor" % (vid))
+
+    def findDate(self, content):
+        for l in content:
+            if l.find("Date:") > -1:
+                return l.split()[-2].replace("-", "")
+        return None
+
+    def parse(self):
+        if len(self.contents) < 1:
+            print "data/%s-pci.ids not found" % self.date
+        else:
+            vendorID = ""
+            deviceID = ""
+            for l in self.contents:
+                if l[0] == "#":
+                    continue
+                elif len(l.strip()) == 0:
+                    continue
+                else:
+                    if l.find("\t\t") == 0:
+                        self.vendors[vendorID].devices[deviceID].addSubDevice(l)
+                    elif l.find("\t") == 0:
+                        deviceID = l.strip().split()[0]
+                        self.vendors[vendorID].addDevice(l)
+                    else:
+                        vendorID = l.split()[0]
+                        self.vendors[vendorID] = Vendor(l)
+
+    def readLocal(self, filename):
+        """
+        Reads the local file
+        """
+        self.contents = open(filename).readlines()
+        self.date = self.findDate(self.contents)
+
+    def loadLocal(self):
+        """
+        Loads database from local. If there is no file,
+        it creates a new one from web
+        """
+        self.date = idsfile[0].split("/")[1].split("-")[0]
+        self.readLocal()
+
+
+
+#=======================================
+
+def search_file(filename, search_path):
+    """ Given a search path, find file with requested name """
+    for path in string.split(search_path, ":"):
+        candidate = os.path.join(path, filename)
+        if os.path.exists(candidate): return os.path.abspath(candidate)
+    return None
+
+class ReadElf(object):
+    """ display_* methods are used to emit output into the output stream
+    """
+    def __init__(self, file, output):
+        """ file:
+                stream object with the ELF file to read
+
+            output:
+                output stream to write to
+        """
+        self.elffile = ELFFile(file)
+        self.output = output
+
+        # Lazily initialized if a debug dump is requested
+        self._dwarfinfo = None
+
+        self._versioninfo = None
+
+    def _section_from_spec(self, spec):
+        """ Retrieve a section given a "spec" (either number or name).
+            Return None if no such section exists in the file.
+        """
+        try:
+            num = int(spec)
+            if num < self.elffile.num_sections():
+                return self.elffile.get_section(num)
+            else:
+                return None
+        except ValueError:
+            # Not a number. Must be a name then
+            return self.elffile.get_section_by_name(str2bytes(spec))
+
+    def pretty_print_pmdinfo(self, pmdinfo):
+        global pcidb
+
+        for i in pmdinfo["pci_ids"]:
+            vendor = pcidb.find_vendor(i[0])
+            device = vendor.find_device(i[1])
+            subdev = device.find_subid(i[2], i[3])
+            print("%s (%s) : %s (%s) %s" % (vendor.name, vendor.ID, device.name, device.ID, subdev.name))
+   
+    def parse_pmd_info_string(self, mystring):
+        global raw_output
+        global pcidb
+
+        optional_pmd_info = [{'id': 'params', 'tag' : 'PMD PARAMETERS'}]
+
+        i = mystring.index("=");
+        mystring = mystring[i+2:]
+        pmdinfo = json.loads(mystring)
+
+        if raw_output:
+            print(pmdinfo)
+            return
+
+        print("PMD NAME: " + pmdinfo["name"])
+        print("PMD TYPE: " + pmdinfo["type"])
+        for i in optional_pmd_info:
+            try:
+                print("%s: %s" % (i['tag'], pmdinfo[i['id']]))
+            except KeyError as e:
+                continue
+
+        if (pmdinfo["type"] == "PMD_PDEV"):
+            print("PMD HW SUPPORT:")
+            if pcidb != None:
+                self.pretty_print_pmdinfo(pmdinfo)
+            else:
+                print("VENDOR\t DEVICE\t SUBVENDOR\t SUBDEVICE")
+                for i in pmdinfo["pci_ids"]:
+                    print("0x%04x\t 0x%04x\t 0x%04x\t\t 0x%04x" % (i[0], i[1], i[2], i[3]))
+
+        print("")
+
+
+    def display_pmd_info_strings(self, section_spec):
+        """ Display a strings dump of a section. section_spec is either a
+            section number or a name.
+        """
+        section = self._section_from_spec(section_spec)
+        if section is None:
+            return
+
+
+        data = section.data()
+        dataptr = 0
+
+        while dataptr < len(data):
+            while ( dataptr < len(data) and
+                    not (32 <= byte2int(data[dataptr]) <= 127)):
+                dataptr += 1
+
+            if dataptr >= len(data):
+                break
+
+            endptr = dataptr
+            while endptr < len(data) and byte2int(data[endptr]) != 0:
+                endptr += 1
+
+            mystring = bytes2str(data[dataptr:endptr])
+            rc = mystring.find("PMD_INFO_STRING")
+            if (rc != -1):
+                self.parse_pmd_info_string(mystring)
+
+            dataptr = endptr
+
+    def find_librte_eal(self, section):
+        for tag in section.iter_tags():
+            if tag.entry.d_tag == 'DT_NEEDED':
+                if "librte_eal" in tag.needed:
+                    return tag.needed
+        return None 
+
+        
+    def search_for_autoload_path(self):
+        scanelf = self
+        scanfile = None
+        library = None
+
+        section = self._section_from_spec(".dynamic")
+        try:
+            eallib = self.find_librte_eal(section)
+            if (eallib != None):
+                ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+                if (ldlibpath == None):
+                    ldlibpath = ""
+                dtr = self.get_dt_runpath(section)
+                library = search_file(eallib,
+                                      dtr + ":" + ldlibpath +
+                                      ":/usr/lib64:/lib64:/usr/lib:/lib")
+                if (library == None):
+                    return (None, None)
+                if (raw_output == False):
+                    print("Scanning for autoload path in %s" % library)
+                scanfile = open(library, 'rb')
+                scanelf = ReadElf(scanfile, sys.stdout)
+        except AttributeError:
+            # Not a dynamic binary
+            pass
+        except ELFError:
+            scanfile.close()
+            return (None, None)
+
+        section = scanelf._section_from_spec(".rodata")
+        if section is None:
+            if (scanfile != None):
+                scanfile.close()
+            return (None, None)
+
+        data = section.data()
+        dataptr = 0
+
+        while dataptr < len(data):
+            while ( dataptr < len(data) and
+                    not (32 <= byte2int(data[dataptr]) <= 127)):
+                dataptr += 1
+
+            if dataptr >= len(data):
+                break
+
+            endptr = dataptr
+            while endptr < len(data) and byte2int(data[endptr]) != 0:
+                endptr += 1
+
+            mystring = bytes2str(data[dataptr:endptr])
+            rc = mystring.find("DPDK_PLUGIN_PATH")
+            if (rc != -1):
+                rc = mystring.find("=")
+                return (mystring[rc+1:], library)
+
+            dataptr = endptr
+        if (scanfile != None):
+            scanfile.close()
+        return (None, None)
+
+    def get_dt_runpath(self, dynsec):
+        for tag in dynsec.iter_tags():
+            if tag.entry.d_tag == 'DT_RUNPATH':
+                return tag.runpath
+        return ""
+
+
+    def process_dt_needed_entries(self):
+        """ Look to see if there are any DT_NEEDED entries in the binary
+            And process those if there are
+        """
+        global raw_output
+        runpath = ""
+        ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+        if (ldlibpath == None):
+            ldlibpath = ""
+
+        dynsec = self._section_from_spec(".dynamic")
+        try:
+            runpath = self.get_dt_runpath(dynsec)
+        except AttributeError:
+            # dynsec is None, just return
+            return
+
+        for tag in dynsec.iter_tags():
+            if tag.entry.d_tag == 'DT_NEEDED':
+                rc = tag.needed.find("librte_pmd")
+                if (rc != -1):
+                    library = search_file(tag.needed,
+                            runpath + ":" + ldlibpath +
+                            ":/usr/lib64:/lib64:/usr/lib:/lib")
+                    if (library != None):
+                        if (raw_output == False):
+                            print("Scanning %s for pmd information" % library)
+                        with open(library, 'rb') as file:
+                            try:
+                                libelf = ReadElf(file, sys.stdout)
+                            except ELFError as e:
+                                print("%s is no an ELF file" % library)
+                                continue
+                            libelf.process_dt_needed_entries()
+                            libelf.display_pmd_info_strings(".rodata")
+                            file.close()
+
+def scan_autoload_path(autoload_path):
+    global raw_output
+
+    if os.path.exists(autoload_path) == False:
+        return
+
+    try:
+        dirs = os.listdir(autoload_path)
+    except OSError as e:
+        # Couldn't read the directory, give up
+        return
+
+    for d in dirs:
+        dpath = os.path.join(autoload_path, d)
+        if os.path.isdir(dpath):
+            scan_autoload_path(dpath)
+        if os.path.isfile(dpath):
+            try:
+                file = open(dpath, 'rb')
+                readelf = ReadElf(file, sys.stdout)
+            except ELFError as e:
+                # this is likely not an elf file, skip it
+               continue
+            except IOError as e:
+                # No permission to read the file, skip it
+                continue
+
+            if (raw_output == False):
+                print("Hw Support for library %s" % d)
+            readelf.display_pmd_info_strings(".rodata")
+            file.close()
+
+
+
+def scan_for_autoload_pmds(dpdk_path):
+    """
+    search the specified application or path for a pmd autoload path
+    then scan said path for pmds and report hw support
+    """
+    global raw_output
+
+    if (os.path.isfile(dpdk_path) == False):
+        if (raw_output == False):
+            print("Must specify a file name")
+        return
+        
+
+    file = open(dpdk_path, 'rb')
+    try:
+        readelf = ReadElf(file, sys.stdout)
+    except ElfError as e:
+        if (raw_output == False):
+            print("Unable to parse %s" % file)
+        return
+
+    (autoload_path, scannedfile) = readelf.search_for_autoload_path()
+    if (autoload_path == None or autoload_path == ""):
+        if (raw_output == False):
+            print("No autoload path configured in %s" % dpdk_path)
+        return
+    if (raw_output == False):
+        if (scannedfile == None):
+            scannedfile = dpdk_path
+        print("Found autoload path %s in %s" % (autoload_path, scannedfile))
+
+    file.close()
+    if (raw_output == False):
+        print("Discovered Autoload HW Support:") 
+    scan_autoload_path(autoload_path) 
+    return
+    
+    
+def main(stream=None):
+    global raw_output
+    global pcidb
+
+    optparser = OptionParser(
+            usage='usage: %prog [-hrtp] [-d <pci id file] <elf-file>',
+            description="Dump pmd hardware support info",
+            add_help_option=True,
+            prog='pmdinfo.py')
+    optparser.add_option('-r', '--raw',
+            action='store_true', dest='raw_output',
+            help='Dump raw json strings')
+    optparser.add_option("-d", "--pcidb", dest="pcifile",
+            help="specify a pci database to get vendor names from",
+            default="/usr/share/hwdata/pci.ids", metavar="FILE")
+    optparser.add_option("-t", "--table", dest="tblout",
+            help="output infomation on hw support as a hex table",
+            action='store_true')
+    optparser.add_option("-p", "--plugindir", dest="pdir",
+            help="scan dpdk for autoload plugins", action='store_true')
+
+    options, args = optparser.parse_args()
+
+    if options.raw_output:
+        raw_output = True
+
+    if options.pcifile:
+        pcidb = PCIIds(options.pcifile)
+        if pcidb == None:
+            print("Pci DB file not found")
+            exit(1)
+
+    if options.tblout:   
+        options.pcifile = None
+        pcidb = None
+
+    if (len(args) == 0):
+        optparser.print_usage()
+        exit(1)
+
+    if options.pdir == True:
+        exit (scan_for_autoload_pmds(args[0]))
+
+    ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+    if (ldlibpath == None):
+        ldlibpath = ""
+
+    if (os.path.exists(args[0]) == True):
+        myelffile = args[0]
+    else:
+        myelffile = search_file(args[0], ldlibpath + ":/usr/lib64:/lib64:/usr/lib:/lib")
+
+    if (myelffile == None):
+        print("File not found")
+        sys.exit(1)
+
+    with open(myelffile, 'rb') as file:
+        try:
+            readelf = ReadElf(file, sys.stdout)
+            readelf.process_dt_needed_entries()
+            readelf.display_pmd_info_strings(".rodata") 
+            sys.exit(0)
+ 
+        except ELFError as ex:
+            sys.stderr.write('ELF error: %s\n' % ex)
+            sys.exit(1)
+
+
+#-------------------------------------------------------------------------------
+if __name__ == '__main__':
+    main()
+
+
-- 
2.5.5

Re: [PATCHv4 0/5] Implement pmd hardware support exports

[Panu Matilainen]

On 05/24/2016 10:41 PM, Neil Horman wrote:
> Hey all-
> 	So heres attempt number 2 at a method for exporting PMD hardware support
> information.  As we discussed previously, the consensus seems to be that pmd
> information should be:
>
> 1) Able to be interrogated on any ELF binary (application binary or individual
> DSO)
> 2) Equally functional on statically linked applications or on DSO's
> 3) Resilient to symbol stripping
> 4) Script friendly
> 5) Show kernel dependencies
> 6) List driver options
> 7) Show driver name
> 8) Offer human readable output
> 9) Show DPDK version
> 10) Show driver version
> 11) Allow for expansion
> 12) Not place additional build environment dependencies on an application
>
[...]
> v4)
>  * Modified the operation of the -p option. As much as I don't like implying
> that autoloaded pmds are guaranteed to be there at run time, I'm having a hard
> time seeing how we can avoid specifying the application file to scan for the
> autoload directory.  Without it we can't determine which library the user means
> in a multiversion installation
>  * Cleaned up the help text
>  * Added a rule for an install target for pmdinfo
>  * Guarded against some tracebacks in pmdinfo
>  * Use DT_NEEDED entries to get versioned libraries in -p mode

Thank you! That's exactly what I've been asking for all along.

>  * Fixed traceback that occurs on lack of input arguments
>  * Fixed some erroneous macro usage in drivers that aren't in the default build
>
> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> CC: Bruce Richardson <bruce.richardson@intel.com>
> CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> CC: Stephen Hemminger <stephen@networkplumber.org>
> CC: Panu Matilainen <pmatilai@redhat.com>

/me happy now, so:

Acked-by: Panu Matilainen <pmatilai@redhat.com>

As always there might be some refining to do as we get more experience 
with it but it seems like a fine starting point to me.

	- Panu -

Re: [PATCHv4 0/5] Implement pmd hardware support exports

[Neil Horman]

On Wed, May 25, 2016 at 11:32:06AM +0300, Panu Matilainen wrote:
> On 05/24/2016 10:41 PM, Neil Horman wrote:
> > Hey all-
> > 	So heres attempt number 2 at a method for exporting PMD hardware support
> > information.  As we discussed previously, the consensus seems to be that pmd
> > information should be:
> > 
> > 1) Able to be interrogated on any ELF binary (application binary or individual
> > DSO)
> > 2) Equally functional on statically linked applications or on DSO's
> > 3) Resilient to symbol stripping
> > 4) Script friendly
> > 5) Show kernel dependencies
> > 6) List driver options
> > 7) Show driver name
> > 8) Offer human readable output
> > 9) Show DPDK version
> > 10) Show driver version
> > 11) Allow for expansion
> > 12) Not place additional build environment dependencies on an application
> > 
> [...]
> > v4)
> >  * Modified the operation of the -p option. As much as I don't like implying
> > that autoloaded pmds are guaranteed to be there at run time, I'm having a hard
> > time seeing how we can avoid specifying the application file to scan for the
> > autoload directory.  Without it we can't determine which library the user means
> > in a multiversion installation
> >  * Cleaned up the help text
> >  * Added a rule for an install target for pmdinfo
> >  * Guarded against some tracebacks in pmdinfo
> >  * Use DT_NEEDED entries to get versioned libraries in -p mode
> 
> Thank you! That's exactly what I've been asking for all along.
> 
Well, don't thank me, I'm not a big fan of it, I just don't see a way around it
at this point, not without some heuristic thats going to be wrong half the time.

> >  * Fixed traceback that occurs on lack of input arguments
> >  * Fixed some erroneous macro usage in drivers that aren't in the default build
> > 
> > Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> > CC: Bruce Richardson <bruce.richardson@intel.com>
> > CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> > CC: Stephen Hemminger <stephen@networkplumber.org>
> > CC: Panu Matilainen <pmatilai@redhat.com>
> 
> /me happy now, so:
> 
> Acked-by: Panu Matilainen <pmatilai@redhat.com>
> 
thanks
Neil

> As always there might be some refining to do as we get more experience with
> it but it seems like a fine starting point to me.
> 
> 	- Panu -
> 

Re: [PATCHv4 1/5] pmdinfogen: Add buildtools and pmdinfogen utility

[Thomas Monjalon]

2016-05-24 15:41, Neil Horman:
> pmdinfogen is a tool used to parse object files and build json strings for use in
> later determining hardware support in a dso or application binary.  pmdinfo
> looks for the non-exported symbol names this_pmd_name<n> and this_pmd_tbl<n>
> (where n is a integer counter).  It records the name of each of these tuples,
> using the later to find the symbolic name of the pci_table for physical devices
> that the object supports.  With this information, it outputs a C file with a
> single line of the form:
> 
> static char *<pmd_name>_driver_info[] __attribute__((used)) = " \
> 	PMD_DRIVER_INFO=<json string>";
> 
> Where <pmd_name> is the arbitrary name of the pmd, and <json_string> is the json
> encoded string that hold relevant pmd information, including the pmd name, type
> and optional array of pci device/vendor ids that the driver supports.
> 
> This c file is suitable for compiling to object code, then relocatably linking
> into the parent file from which the C was generated.  This creates an entry in
> the string table of the object that can inform a later tool about hardware
> support.

This description is helpful and should be in the doc:
	doc/guides/prog_guide/dev_kit_build_system.rst

> --- a/GNUmakefile
> +++ b/GNUmakefile
> -ROOTDIRS-y := lib drivers app
> +ROOTDIRS-y := buildtools lib drivers app

Why a new directory?
It is not a script nor an end-user tool, I guess.

I feel strange to build an app for the build system.
For information, do you know some Python lib to do this kind of tool?

> +++ b/buildtools/pmdinfogen/Makefile
> +#CFLAGS += $(WERROR_FLAGS) -g
> +CFLAGS += -g

Please choose one line or the other or none of them.

> +include $(RTE_SDK)/mk/rte.buildtools.mk

Why a new Makefile? Can you use rte.hostapp.mk?

> +++ b/buildtools/pmdinfogen/pmdinfogen.c
[...]
> +	/*
> + 	 * If this returns NULL, then this is a PMD_VDEV, because
> + 	 * it has no pci table reference
> + 	 */

We can imagine physical PMD not using PCI.
I think this comment should be removed.

> +	if (!tmpsym) {
> +		drv->pci_tbl = NULL;
> +		return 0;
> +	}
[...]
> +
> +
> +	return 0;
> +	
> +}

That's a lot of blank lines ;)

[...]
> +		fprintf(ofd,"\\\"type\\\" : \\\"%s\\\", ", drv->pci_tbl ? "PMD_PDEV" : "PMD_VDEV");

Please forget the naming PDEV/VDEV.

[...]
> +	if (info.drivers) {
> +		output_pmd_info_string(&info, argv[2]);
> +		rc = 0;
> +	} else {
> +		fprintf(stderr, "Hmm, Appears to be a driver but no drivers registered\n");

Why it appears to be a driver?
What means "no drivers registered" exactly?

> +++ b/buildtools/pmdinfogen/pmdinfogen.h
[...]
> +#define Elf_Ehdr    Elf64_Ehdr
> +#define Elf_Shdr    Elf64_Shdr
> +#define Elf_Sym     Elf64_Sym
> +#define Elf_Addr    Elf64_Addr
> +#define Elf_Sword   Elf64_Sxword
> +#define Elf_Section Elf64_Half
> +#define ELF_ST_BIND ELF64_ST_BIND
> +#define ELF_ST_TYPE ELF64_ST_TYPE
> +
> +#define Elf_Rel     Elf64_Rel
> +#define Elf_Rela    Elf64_Rela
> +#define ELF_R_SYM   ELF64_R_SYM
> +#define ELF_R_TYPE  ELF64_R_TYPE

Why these defines are needed?

> +#define TO_NATIVE(x) (x)

Nice :) Why?

> +struct rte_pci_id {
> +	uint16_t vendor_id;           /**< Vendor ID or PCI_ANY_ID. */
> +	uint16_t device_id;           /**< Device ID or PCI_ANY_ID. */
> +	uint16_t subsystem_vendor_id; /**< Subsystem vendor ID or PCI_ANY_ID. */
> +	uint16_t subsystem_device_id; /**< Subsystem device ID or PCI_ANY_ID. */
> +};
[...]
> +struct pmd_driver {
> +	Elf_Sym *name_sym;
> +	const char *name;
> +	struct rte_pci_id *pci_tbl;
> +	struct pmd_driver *next;
> +
> +	const char* opt_vals[PMD_OPT_MAX];
> +};

Are you duplicating some structures from EAL?
It will be out of sync easily.

> +struct elf_info {
> +	unsigned long size;
> +	Elf_Ehdr     *hdr;
> +	Elf_Shdr     *sechdrs;
> +	Elf_Sym      *symtab_start;
> +	Elf_Sym      *symtab_stop;
> +	Elf_Section  export_sec;
> +	Elf_Section  export_unused_sec;
> +	Elf_Section  export_gpl_sec;
> +	Elf_Section  export_unused_gpl_sec;
> +	Elf_Section  export_gpl_future_sec;
> +	char         *strtab;
> +	char	     *modinfo;
> +	unsigned int modinfo_len;

Why these fields?

> +++ b/mk/rte.buildtools.mk

This file must be removed I think.
We are going to be sick after digesting so much makefiles ;)

Last comment,
The MAINTAINERS file must be updated for this tool.

Thanks for taking care of tooling.

Re: [PATCHv4 2/5] drivers: Update driver registration macro usage

[Thomas Monjalon]

2016-05-24 15:41, Neil Horman:
> Modify the PMD_REGISTER_DRIVER macro, adding a name argument to it.  The
> addition of a name argument creates a token that can be used for subsequent
> macros in the creation of unique symbol names to export additional bits of
> information for use by the pmdinfogen tool.  For example:
> 
> PMD_REGISTER_DRIVER(ena_driver, ena);
> 
> registers the ena_driver struct as it always did, and creates a symbol
> const char this_pmd_name0[] __attribute__((used)) = "ena";
> 
> which pmdinfogen can search for and extract.

The EAL rework (http://dpdk.org/ml/archives/dev/2016-April/037691.html)
integrates already a name:

+#define RTE_EAL_PCI_REGISTER(name, d) \
+RTE_INIT(pciinitfn_ ##name); \
+static void pciinitfn_ ##name(void) \
+{ \
+	rte_eal_pci_register(&d); \
+}

I think it would be better to rebase on top of it.

> The subsequent macro
> 
> DRIVER_REGISTER_PCI_TABLE(ena, ena_pci_id_map);
> 
> creates a symbol
> const char ena_pci_tbl_export[] __attribute__((used)) = "ena_pci_id_map";
> 
> Which allows pmdinfogen to find the pci table of this driver
> 
> Using this pattern, we can export arbitrary bits of information.
> 
> pmdinfo uses this information to extract hardware support from an object file
> and create a json string to make hardware support info discoverable later.

> --- a/drivers/Makefile
> +++ b/drivers/Makefile
> @@ -34,4 +34,6 @@ include $(RTE_SDK)/mk/rte.vars.mk
>  DIRS-y += net
>  DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += crypto
>  
> +DEPDIRS-y += buildtools/pmdinfo

Why pmdinfo is a build dependency?

> --- a/lib/librte_eal/common/include/rte_dev.h
> +++ b/lib/librte_eal/common/include/rte_dev.h
> @@ -48,7 +48,7 @@ extern "C" {
>  
>  #include <stdio.h>
>  #include <sys/queue.h>
> -
> +#include <rte_pci.h>

Why not keep PCI stuff in rte_pci.h?

> +#define DRV_EXP_TAG(n, t) __##n##_##t
> +
> +#define DRIVER_REGISTER_PCI_TABLE(n, t) \
> +static const char DRV_EXP_TAG(n, pci_tbl_export)[] __attribute__((used)) = RTE_STR(t)

I really dislike one-char variables, especially when there is no comment.
Please choose comments or explicit variables.

Re: [PATCHv4 4/5] Makefile: Do post processing on objects that register a driver

[Thomas Monjalon]

2016-05-24 15:41, Neil Horman:
> --- a/mk/internal/rte.compile-pre.mk
> +++ b/mk/internal/rte.compile-pre.mk
> @@ -80,7 +80,8 @@ C_TO_O_STR = $(subst ','\'',$(C_TO_O)) #'# fix syntax highlight
>  C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)","  HOSTCC $(@)")
>  else
>  C_TO_O = $(CC) -Wp,-MD,$(call obj2dep,$(@)).tmp $(CFLAGS) \
> -	$(CFLAGS_$(@)) $(EXTRA_CFLAGS) -o $@ -c $<
> +	 $(CFLAGS_$(@)) $(EXTRA_CFLAGS) -o $@ -c $<
> +

whitespace change?

>  C_TO_O_STR = $(subst ','\'',$(C_TO_O)) #'# fix syntax highlight
>  C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)","  CC $(@)")
>  endif
> @@ -88,10 +89,26 @@ C_TO_O_CMD = 'cmd_$@ = $(C_TO_O_STR)'
>  C_TO_O_DO = @set -e; \
>  	echo $(C_TO_O_DISP); \
>  	$(C_TO_O) && \
> +	sh -c "grep -q \"PMD_REGISTER_DRIVER(.*)\" $<; \
> +	if [ \$$? -eq 0 ]; \
> +	then \

It is preferred to keep "then" at the end of the previous line.

> +		echo MODGEN $@; \
> +		OBJF=`readlink -f $@`; \
> +		${RTE_OUTPUT}/buildtools/pmdinfogen \$$OBJF \$$OBJF.mod.c; \

Maybe .pmd.c would be more appropriate than .mod.c?
What means mod/MODGEN/MODBUILD?

> +		if [ \$$? -eq 0 ]; \
> +		then \
> +			echo MODBUILD $@; \
> +			$(CC) -c -o \$$OBJF.mod.o \$$OBJF.mod.c; \
> +			$(CROSS)ld -r -o \$$OBJF.o \$$OBJF.mod.o \$$OBJF; \
> +			mv -f \$$OBJF.o \$$OBJF; \
> +		fi; \
> +	fi; \
> +	true" && \

Why "true"?

It deserves to be in a shell script, at least to ease testing.

Re: [PATCHv4 1/5] pmdinfogen: Add buildtools and pmdinfogen utility

[Neil Horman]

On Wed, May 25, 2016 at 03:21:19PM +0200, Thomas Monjalon wrote:
> 2016-05-24 15:41, Neil Horman:
> > pmdinfogen is a tool used to parse object files and build json strings for use in
> > later determining hardware support in a dso or application binary.  pmdinfo
> > looks for the non-exported symbol names this_pmd_name<n> and this_pmd_tbl<n>
> > (where n is a integer counter).  It records the name of each of these tuples,
> > using the later to find the symbolic name of the pci_table for physical devices
> > that the object supports.  With this information, it outputs a C file with a
> > single line of the form:
> > 
> > static char *<pmd_name>_driver_info[] __attribute__((used)) = " \
> > 	PMD_DRIVER_INFO=<json string>";
> > 
> > Where <pmd_name> is the arbitrary name of the pmd, and <json_string> is the json
> > encoded string that hold relevant pmd information, including the pmd name, type
> > and optional array of pci device/vendor ids that the driver supports.
> > 
> > This c file is suitable for compiling to object code, then relocatably linking
> > into the parent file from which the C was generated.  This creates an entry in
> > the string table of the object that can inform a later tool about hardware
> > support.
> 
> This description is helpful and should be in the doc:
> 	doc/guides/prog_guide/dev_kit_build_system.rst
Yeah, ok I can add that. 

> 
> > --- a/GNUmakefile
> > +++ b/GNUmakefile
> > -ROOTDIRS-y := lib drivers app
> > +ROOTDIRS-y := buildtools lib drivers app
> 
> Why a new directory?
> It is not a script nor an end-user tool, I guess.
Dependencies.  This tool has to be built prior to the rest of the dpdk, but app
already relies on dpdk libraries to be built, so you get circular dependencies.
I could have put it in scripts I guess, but its not a script.  Its own directory
seemed to make the most sense, given those two points

> 
> I feel strange to build an app for the build system.
Why?  I agree its not overly common, but theres lots of precident for it.
The linux and bsd kernels obviously do this for modules, and there are lots of
tools that convert generic descriptions in xml into platform native source code
prior to compilation.

> For information, do you know some Python lib to do this kind of tool?
> 
No, if there was I would have used it, but this sort of thing is project
specific, theres no 'generic' symbol stringification solution available.

> > +++ b/buildtools/pmdinfogen/Makefile
> > +#CFLAGS += $(WERROR_FLAGS) -g
> > +CFLAGS += -g
> 
> Please choose one line or the other or none of them.
> 
Oh, thats a debug error, I can fix that.

> > +include $(RTE_SDK)/mk/rte.buildtools.mk
> 
> Why a new Makefile? Can you use rte.hostapp.mk?
> 
I don't know, maybe.  Nothing else currently uses rte.hostapp.mk, so I missed
its existance.  I make the argument that, that being the case, we should stick
with the Makefile I just tested with, and remove the rte.hostapp.mk file


> > +++ b/buildtools/pmdinfogen/pmdinfogen.c
> [...]
> > +	/*
> > + 	 * If this returns NULL, then this is a PMD_VDEV, because
> > + 	 * it has no pci table reference
> > + 	 */
> 
> We can imagine physical PMD not using PCI.
> I think this comment should be removed.
We can, but currently its a true statement.  we have two types of PMDs, a PDEV
and a VDEV, the former is a pci device, and the latter is a virtual device, so
you can imply the PDEV type from the presence of pci entries, and VDEV from the
alternative.  If we were to do something, I would recommend adding a macro to
explicitly ennumerate each pmds type.  I would prefer to wait until that was a
need however, as it can be done invisibly to the user.

> 
> > +	if (!tmpsym) {
> > +		drv->pci_tbl = NULL;
> > +		return 0;
> > +	}
> [...]
> > +
> > +
> > +	return 0;
> > +	
> > +}
> 
> That's a lot of blank lines ;)
> 
My eyes were getting tired :)

> [...]
> > +		fprintf(ofd,"\\\"type\\\" : \\\"%s\\\", ", drv->pci_tbl ? "PMD_PDEV" : "PMD_VDEV");
> 
> Please forget the naming PDEV/VDEV.
> 
I don't know what you mean here, you would rather they be named PCI and Virtual,
or something else?


> [...]
> > +	if (info.drivers) {
> > +		output_pmd_info_string(&info, argv[2]);
> > +		rc = 0;
> > +	} else {
> > +		fprintf(stderr, "Hmm, Appears to be a driver but no drivers registered\n");
> 
> Why it appears to be a driver?
> What means "no drivers registered" exactly?
> 
It means that the tool has identified this file as a driver based on some
criteria (in this case the source code contained a use of the
PMD_REGISTER_DRIVER macro, but for whatever reason, when this tool scanned it,
it never located the pmd_driver_name<n> symbol.  It should never happen, and
serves as a indicator to the developer that they need to investigate either the
construction of the driver or the use of this tool.



> > +++ b/buildtools/pmdinfogen/pmdinfogen.h
> [...]
> > +#define Elf_Ehdr    Elf64_Ehdr
> > +#define Elf_Shdr    Elf64_Shdr
> > +#define Elf_Sym     Elf64_Sym
> > +#define Elf_Addr    Elf64_Addr
> > +#define Elf_Sword   Elf64_Sxword
> > +#define Elf_Section Elf64_Half
> > +#define ELF_ST_BIND ELF64_ST_BIND
> > +#define ELF_ST_TYPE ELF64_ST_TYPE
> > +
> > +#define Elf_Rel     Elf64_Rel
> > +#define Elf_Rela    Elf64_Rela
> > +#define ELF_R_SYM   ELF64_R_SYM
> > +#define ELF_R_TYPE  ELF64_R_TYPE
> 
> Why these defines are needed?
> 
Because I borrowed the code from modpost.c, which allows for both ELF32 and
ELF64 compilation.  I wanted to keep it in place should DPDK ever target
different sized architectures.

> > +#define TO_NATIVE(x) (x)
> 
> Nice :) Why?
> 
Because there is more than intel in the world :).  For alternate endian
machines, this can easily be redefined to accomodate that.


> > +struct rte_pci_id {
> > +	uint16_t vendor_id;           /**< Vendor ID or PCI_ANY_ID. */
> > +	uint16_t device_id;           /**< Device ID or PCI_ANY_ID. */
> > +	uint16_t subsystem_vendor_id; /**< Subsystem vendor ID or PCI_ANY_ID. */
> > +	uint16_t subsystem_device_id; /**< Subsystem device ID or PCI_ANY_ID. */
> > +};
> [...]
> > +struct pmd_driver {
> > +	Elf_Sym *name_sym;
> > +	const char *name;
> > +	struct rte_pci_id *pci_tbl;
> > +	struct pmd_driver *next;
> > +
> > +	const char* opt_vals[PMD_OPT_MAX];
> > +};
> 
> Are you duplicating some structures from EAL?
> It will be out of sync easily.
> 
Only the rte_pci_id, which hasn't changed since the initial public release of
the DPDK.  We can clean this up later if you like, but I'm really not too
worried about it.

> > +struct elf_info {
> > +	unsigned long size;
> > +	Elf_Ehdr     *hdr;
> > +	Elf_Shdr     *sechdrs;
> > +	Elf_Sym      *symtab_start;
> > +	Elf_Sym      *symtab_stop;
> > +	Elf_Section  export_sec;
> > +	Elf_Section  export_unused_sec;
> > +	Elf_Section  export_gpl_sec;
> > +	Elf_Section  export_unused_gpl_sec;
> > +	Elf_Section  export_gpl_future_sec;
> > +	char         *strtab;
> > +	char	     *modinfo;
> > +	unsigned int modinfo_len;
> 
> Why these fields?
> 
Because thats how you parse an ELF file and look up the information you need to
extract the data this tool then exports.  I don't mean to sound short, but your
question doesn't really make any sense.  The members of this structure are
needed to extract the info from object files to build the export strings that
pmdinfo.py needs later.


> > +++ b/mk/rte.buildtools.mk
> 
> This file must be removed I think.
> We are going to be sick after digesting so much makefiles ;)
> 
See above, given that I just tested this, and rte.hostapp.mk isn't used, I'd
recommend deleting the latter, rather than deleting this one and moving to the
old one.

> Last comment,
> The MAINTAINERS file must be updated for this tool.
> 
Yeah, I can do that.

> Thanks for taking care of tooling.
> 
Yup

Re: [PATCHv4 5/5] pmdinfo.py: Add tool to query binaries for hw and other support information

[Thomas Monjalon]

2016-05-24 15:41, Neil Horman:
> Note that, in the case of dynamically linked applications, pmdinfo.py will scan
> for implicitly linked PMDs by searching the specified binaries .dynamic section
> for DT_NEEDED entries that contain the substring librte_pmd.

I don't know any DPDK app dynamically linked with a PMD (with DT_NEEDED).
However it is a good idea to handle this case.
But relying on the name assumption "librte_pmd" is really weak.

> +	$(Q)$(call rte_symlink,    $(DESTDIR)$(datadir)/tools/pmdinfo.py, \
> +				   $(DESTDIR)$(bindir)/pmdinfo)

I think we must prefix the tool name with dpdk.
What about dpdk-objinfo or dpdk-pmdinfo?

> +from elftools.elf.elffile import ELFFile
> +from elftools.elf.dynamic import DynamicSection, DynamicSegment
> +from elftools.elf.enums import ENUM_D_TAG
> +from elftools.elf.segments import InterpSegment
> +from elftools.elf.sections import SymbolTableSection

Should it be possible to implement pmdinfogen with this
Python library?

I'll probably comment on the pmdinfo script details later.
Just knowing you did a tool is enough to assert that it is a good step :)
Thanks

Re: [PATCHv4 2/5] drivers: Update driver registration macro usage

[Neil Horman]

On Wed, May 25, 2016 at 06:20:06PM +0200, Thomas Monjalon wrote:
> 2016-05-24 15:41, Neil Horman:
> > Modify the PMD_REGISTER_DRIVER macro, adding a name argument to it.  The
> > addition of a name argument creates a token that can be used for subsequent
> > macros in the creation of unique symbol names to export additional bits of
> > information for use by the pmdinfogen tool.  For example:
> > 
> > PMD_REGISTER_DRIVER(ena_driver, ena);
> > 
> > registers the ena_driver struct as it always did, and creates a symbol
> > const char this_pmd_name0[] __attribute__((used)) = "ena";
> > 
> > which pmdinfogen can search for and extract.
> 
> The EAL rework (http://dpdk.org/ml/archives/dev/2016-April/037691.html)
> integrates already a name:
> 
> +#define RTE_EAL_PCI_REGISTER(name, d) \
> +RTE_INIT(pciinitfn_ ##name); \
> +static void pciinitfn_ ##name(void) \
> +{ \
> +	rte_eal_pci_register(&d); \
> +}
> 
> I think it would be better to rebase on top of it.
> 
Those patches are over a month old and still in the new state according to
patchwork.  I'm not very comfortable rebasing (and implicitly blocking)
acceptance of this patch on that one.  Its really a just two lines of conflict.  I
would suggest that, whichever patch gets integrated first, the other series can
rebase on the new head.  It should be a pretty easy fix either way.

> > The subsequent macro
> > 
> > DRIVER_REGISTER_PCI_TABLE(ena, ena_pci_id_map);
> > 
> > creates a symbol
> > const char ena_pci_tbl_export[] __attribute__((used)) = "ena_pci_id_map";
> > 
> > Which allows pmdinfogen to find the pci table of this driver
> > 
> > Using this pattern, we can export arbitrary bits of information.
> > 
> > pmdinfo uses this information to extract hardware support from an object file
> > and create a json string to make hardware support info discoverable later.
> 
> > --- a/drivers/Makefile
> > +++ b/drivers/Makefile
> > @@ -34,4 +34,6 @@ include $(RTE_SDK)/mk/rte.vars.mk
> >  DIRS-y += net
> >  DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += crypto
> >  
> > +DEPDIRS-y += buildtools/pmdinfo
> 
> Why pmdinfo is a build dependency?
> 
beause pmdinfogen has to be built and available for use prior to compiling the
rest of the dpdk.  I suppose we could build it after, and then go back through
and check all the objects for driver info, but I'd rather build it first, and
search the objects as they are built.

> > --- a/lib/librte_eal/common/include/rte_dev.h
> > +++ b/lib/librte_eal/common/include/rte_dev.h
> > @@ -48,7 +48,7 @@ extern "C" {
> >  
> >  #include <stdio.h>
> >  #include <sys/queue.h>
> > -
> > +#include <rte_pci.h>
> 
> Why not keep PCI stuff in rte_pci.h?
> 
I am.

> > +#define DRV_EXP_TAG(n, t) __##n##_##t
> > +
> > +#define DRIVER_REGISTER_PCI_TABLE(n, t) \
> > +static const char DRV_EXP_TAG(n, pci_tbl_export)[] __attribute__((used)) = RTE_STR(t)
> 
> I really dislike one-char variables, especially when there is no comment.
> Please choose comments or explicit variables.
> 

You mean you want the macro variables to be longer/more descriptive?  I suppose,
but in fairness, we have lots of macros that use single letter variables, I'm
not sure why your concerned about these specifically.  I'll change it though.

Neil

Re: [PATCHv4 1/5] pmdinfogen: Add buildtools and pmdinfogen utility

[Thomas Monjalon]

2016-05-25 13:22, Neil Horman:
> On Wed, May 25, 2016 at 03:21:19PM +0200, Thomas Monjalon wrote:
> > 2016-05-24 15:41, Neil Horman:
> > > --- a/GNUmakefile
> > > +++ b/GNUmakefile
> > > -ROOTDIRS-y := lib drivers app
> > > +ROOTDIRS-y := buildtools lib drivers app
> > 
> > Why a new directory?
> > It is not a script nor an end-user tool, I guess.
> Dependencies.  This tool has to be built prior to the rest of the dpdk, but app
> already relies on dpdk libraries to be built, so you get circular dependencies.
> I could have put it in scripts I guess, but its not a script.  Its own directory
> seemed to make the most sense, given those two points

OK

> > > +include $(RTE_SDK)/mk/rte.buildtools.mk
> > 
> > Why a new Makefile? Can you use rte.hostapp.mk?
> > 
> I don't know, maybe.  Nothing else currently uses rte.hostapp.mk, so I missed
> its existance.  I make the argument that, that being the case, we should stick
> with the Makefile I just tested with, and remove the rte.hostapp.mk file

No, rte.hostapp.mk has been used and tested in the history of the project.
Please try it.

> > > +++ b/buildtools/pmdinfogen/pmdinfogen.c
> > [...]
> > > +	/*
> > > + 	 * If this returns NULL, then this is a PMD_VDEV, because
> > > + 	 * it has no pci table reference
> > > + 	 */
> > 
> > We can imagine physical PMD not using PCI.
> > I think this comment should be removed.
> We can, but currently its a true statement.  we have two types of PMDs, a PDEV
> and a VDEV, the former is a pci device, and the latter is a virtual device, so
> you can imply the PDEV type from the presence of pci entries, and VDEV from the
> alternative.  If we were to do something, I would recommend adding a macro to
> explicitly ennumerate each pmds type.  I would prefer to wait until that was a
> need however, as it can be done invisibly to the user.

We are removing the PMD types in the EAL rework.
So this comment will be outdated. Better to remove now.

> > [...]
> > > +		fprintf(ofd,"\\\"type\\\" : \\\"%s\\\", ", drv->pci_tbl ? "PMD_PDEV" : "PMD_VDEV");
> > 
> > Please forget the naming PDEV/VDEV.
> > 
> I don't know what you mean here, you would rather they be named PCI and Virtual,
> or something else?

Yes please.

> > [...]
> > > +	if (info.drivers) {
> > > +		output_pmd_info_string(&info, argv[2]);
> > > +		rc = 0;
> > > +	} else {
> > > +		fprintf(stderr, "Hmm, Appears to be a driver but no drivers registered\n");
> > 
> > Why it appears to be a driver?
> > What means "no drivers registered" exactly?
> > 
> It means that the tool has identified this file as a driver based on some
> criteria (in this case the source code contained a use of the
> PMD_REGISTER_DRIVER macro, but for whatever reason, when this tool scanned it,
> it never located the pmd_driver_name<n> symbol.  It should never happen, and
> serves as a indicator to the developer that they need to investigate either the
> construction of the driver or the use of this tool.

OK

> > > +++ b/buildtools/pmdinfogen/pmdinfogen.h
> > [...]
> > > +#define Elf_Ehdr    Elf64_Ehdr
> > > +#define Elf_Shdr    Elf64_Shdr
> > > +#define Elf_Sym     Elf64_Sym
> > > +#define Elf_Addr    Elf64_Addr
> > > +#define Elf_Sword   Elf64_Sxword
> > > +#define Elf_Section Elf64_Half
> > > +#define ELF_ST_BIND ELF64_ST_BIND
> > > +#define ELF_ST_TYPE ELF64_ST_TYPE
> > > +
> > > +#define Elf_Rel     Elf64_Rel
> > > +#define Elf_Rela    Elf64_Rela
> > > +#define ELF_R_SYM   ELF64_R_SYM
> > > +#define ELF_R_TYPE  ELF64_R_TYPE
> > 
> > Why these defines are needed?
> > 
> Because I borrowed the code from modpost.c, which allows for both ELF32 and
> ELF64 compilation.  I wanted to keep it in place should DPDK ever target
> different sized architectures.

Maybe a comment is needed.
Is ELF32 used on 32-bit archs like i686 or ARMv7?

> > > +struct rte_pci_id {
> > > +	uint16_t vendor_id;           /**< Vendor ID or PCI_ANY_ID. */
> > > +	uint16_t device_id;           /**< Device ID or PCI_ANY_ID. */
> > > +	uint16_t subsystem_vendor_id; /**< Subsystem vendor ID or PCI_ANY_ID. */
> > > +	uint16_t subsystem_device_id; /**< Subsystem device ID or PCI_ANY_ID. */
> > > +};
> > [...]
> > > +struct pmd_driver {
> > > +	Elf_Sym *name_sym;
> > > +	const char *name;
> > > +	struct rte_pci_id *pci_tbl;
> > > +	struct pmd_driver *next;
> > > +
> > > +	const char* opt_vals[PMD_OPT_MAX];
> > > +};
> > 
> > Are you duplicating some structures from EAL?
> > It will be out of sync easily.
> > 
> Only the rte_pci_id, which hasn't changed since the initial public release of
> the DPDK.  We can clean this up later if you like, but I'm really not too
> worried about it.

I would prefer an include if possible.
rte_pci_id is changing in 16.07 ;)

> > > +struct elf_info {
> > > +	unsigned long size;
> > > +	Elf_Ehdr     *hdr;
> > > +	Elf_Shdr     *sechdrs;
> > > +	Elf_Sym      *symtab_start;
> > > +	Elf_Sym      *symtab_stop;
> > > +	Elf_Section  export_sec;
> > > +	Elf_Section  export_unused_sec;
> > > +	Elf_Section  export_gpl_sec;
> > > +	Elf_Section  export_unused_gpl_sec;
> > > +	Elf_Section  export_gpl_future_sec;
> > > +	char         *strtab;
> > > +	char	     *modinfo;
> > > +	unsigned int modinfo_len;
> > 
> > Why these fields?
> > 
> Because thats how you parse an ELF file and look up the information you need to
> extract the data this tool then exports.  I don't mean to sound short, but your
> question doesn't really make any sense.  The members of this structure are
> needed to extract the info from object files to build the export strings that
> pmdinfo.py needs later.

Sorry, I haven't parse the whole code but some fields are unused.
And modinfo looks wrong.

> > > +++ b/mk/rte.buildtools.mk
> > 
> > This file must be removed I think.
> > We are going to be sick after digesting so much makefiles ;)
> > 
> See above, given that I just tested this, and rte.hostapp.mk isn't used, I'd
> recommend deleting the latter, rather than deleting this one and moving to the
> old one.

See above, I do not agree :)

Re: [PATCHv4 4/5] Makefile: Do post processing on objects that register a driver

[Neil Horman]

On Wed, May 25, 2016 at 07:08:19PM +0200, Thomas Monjalon wrote:
> 2016-05-24 15:41, Neil Horman:
> > --- a/mk/internal/rte.compile-pre.mk
> > +++ b/mk/internal/rte.compile-pre.mk
> > @@ -80,7 +80,8 @@ C_TO_O_STR = $(subst ','\'',$(C_TO_O)) #'# fix syntax highlight
> >  C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)","  HOSTCC $(@)")
> >  else
> >  C_TO_O = $(CC) -Wp,-MD,$(call obj2dep,$(@)).tmp $(CFLAGS) \
> > -	$(CFLAGS_$(@)) $(EXTRA_CFLAGS) -o $@ -c $<
> > +	 $(CFLAGS_$(@)) $(EXTRA_CFLAGS) -o $@ -c $<
> > +
> 
> whitespace change?
> 

Looks like, I'll remove it

> >  C_TO_O_STR = $(subst ','\'',$(C_TO_O)) #'# fix syntax highlight
> >  C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)","  CC $(@)")
> >  endif
> > @@ -88,10 +89,26 @@ C_TO_O_CMD = 'cmd_$@ = $(C_TO_O_STR)'
> >  C_TO_O_DO = @set -e; \
> >  	echo $(C_TO_O_DISP); \
> >  	$(C_TO_O) && \
> > +	sh -c "grep -q \"PMD_REGISTER_DRIVER(.*)\" $<; \
> > +	if [ \$$? -eq 0 ]; \
> > +	then \
> 
> It is preferred to keep "then" at the end of the previous line.
Very well.

> 
> > +		echo MODGEN $@; \
> > +		OBJF=`readlink -f $@`; \
> > +		${RTE_OUTPUT}/buildtools/pmdinfogen \$$OBJF \$$OBJF.mod.c; \
> 
> Maybe .pmd.c would be more appropriate than .mod.c?
fine
> What means mod/MODGEN/MODBUILD?
GENerate Module information & BUILD module information.

> 
> > +		if [ \$$? -eq 0 ]; \
> > +		then \
> > +			echo MODBUILD $@; \
> > +			$(CC) -c -o \$$OBJF.mod.o \$$OBJF.mod.c; \
> > +			$(CROSS)ld -r -o \$$OBJF.o \$$OBJF.mod.o \$$OBJF; \
> > +			mv -f \$$OBJF.o \$$OBJF; \
> > +		fi; \
> > +	fi; \
> > +	true" && \
> 
> Why "true"?
Debugging statement, I'll remove it.

> 
> It deserves to be in a shell script, at least to ease testing.
What do you mean by "it" and why would it be easier to test in a shell script?

> 
> 

Re: [PATCHv4 5/5] pmdinfo.py: Add tool to query binaries for hw and other support information

[Neil Horman]

On Wed, May 25, 2016 at 07:22:39PM +0200, Thomas Monjalon wrote:
> 2016-05-24 15:41, Neil Horman:
> > Note that, in the case of dynamically linked applications, pmdinfo.py will scan
> > for implicitly linked PMDs by searching the specified binaries .dynamic section
> > for DT_NEEDED entries that contain the substring librte_pmd.
> 
> I don't know any DPDK app dynamically linked with a PMD (with DT_NEEDED).
I know lots of them, they're all in the dpdk.  everything under app that uses a
virutal device links at link time to librte_pmd_bonding and librte_pmd_pipe (and
a few others), because they have additional apis that they need to resolve at
load time.

> However it is a good idea to handle this case.
> But relying on the name assumption "librte_pmd" is really weak.
> 
> > +	$(Q)$(call rte_symlink,    $(DESTDIR)$(datadir)/tools/pmdinfo.py, \
> > +				   $(DESTDIR)$(bindir)/pmdinfo)
> 
> I think we must prefix the tool name with dpdk.
> What about dpdk-objinfo or dpdk-pmdinfo?
> 
> > +from elftools.elf.elffile import ELFFile
> > +from elftools.elf.dynamic import DynamicSection, DynamicSegment
> > +from elftools.elf.enums import ENUM_D_TAG
> > +from elftools.elf.segments import InterpSegment
> > +from elftools.elf.sections import SymbolTableSection
> 
> Should it be possible to implement pmdinfogen with this
> Python library?
> 
Sure, but that really doesn't buy us anything, as its already implemented in C.
In fact, I would assert its harmful, because it implies that the build
environment needs to have python installed, as well as the pyelftools library,
which we don't need if we build from C.

> I'll probably comment on the pmdinfo script details later.
> Just knowing you did a tool is enough to assert that it is a good step :)
> Thanks
> 

Re: [PATCHv4 4/5] Makefile: Do post processing on objects that register a driver

[Thomas Monjalon]

2016-05-25 13:40, Neil Horman:
> On Wed, May 25, 2016 at 07:08:19PM +0200, Thomas Monjalon wrote:
> > 2016-05-24 15:41, Neil Horman:
> > > +		echo MODGEN $@; \
> > > +		OBJF=`readlink -f $@`; \
> > > +		${RTE_OUTPUT}/buildtools/pmdinfogen \$$OBJF \$$OBJF.mod.c; \
> > 
> > Maybe .pmd.c would be more appropriate than .mod.c?
> fine
> > What means mod/MODGEN/MODBUILD?
> GENerate Module information & BUILD module information.

I think "module" is not appropriate here.

> > It deserves to be in a shell script, at least to ease testing.
> What do you mean by "it" and why would it be easier to test in a shell script?

"it" is mostly this whole patch.
With a shell script, we can test the behaviour on one file easily.
Maybe I'm wrong, but I don't like having too much lines in a Makefile rule.
We probably need more opinions.

Re: [PATCHv4 5/5] pmdinfo.py: Add tool to query binaries for hw and other support information

[Thomas Monjalon]

2016-05-25 13:47, Neil Horman:
> On Wed, May 25, 2016 at 07:22:39PM +0200, Thomas Monjalon wrote:
> > 2016-05-24 15:41, Neil Horman:
> > > Note that, in the case of dynamically linked applications, pmdinfo.py will scan
> > > for implicitly linked PMDs by searching the specified binaries .dynamic section
> > > for DT_NEEDED entries that contain the substring librte_pmd.
> > 
> > I don't know any DPDK app dynamically linked with a PMD (with DT_NEEDED).
> I know lots of them, they're all in the dpdk.  everything under app that uses a
> virutal device links at link time to librte_pmd_bonding and librte_pmd_pipe (and
> a few others), because they have additional apis that they need to resolve at
> load time.

Oh yes! you are right.

> > However it is a good idea to handle this case.
> > But relying on the name assumption "librte_pmd" is really weak.
> > 
> > > +	$(Q)$(call rte_symlink,    $(DESTDIR)$(datadir)/tools/pmdinfo.py, \
> > > +				   $(DESTDIR)$(bindir)/pmdinfo)
> > 
> > I think we must prefix the tool name with dpdk.
> > What about dpdk-objinfo or dpdk-pmdinfo?
> > 
> > > +from elftools.elf.elffile import ELFFile
> > > +from elftools.elf.dynamic import DynamicSection, DynamicSegment
> > > +from elftools.elf.enums import ENUM_D_TAG
> > > +from elftools.elf.segments import InterpSegment
> > > +from elftools.elf.sections import SymbolTableSection
> > 
> > Should it be possible to implement pmdinfogen with this
> > Python library?
> > 
> Sure, but that really doesn't buy us anything, as its already implemented in C.
> In fact, I would assert its harmful, because it implies that the build
> environment needs to have python installed, as well as the pyelftools library,
> which we don't need if we build from C.

Right

Re: [PATCHv4 1/5] pmdinfogen: Add buildtools and pmdinfogen utility

[Neil Horman]

On Wed, May 25, 2016 at 07:39:30PM +0200, Thomas Monjalon wrote:
> 2016-05-25 13:22, Neil Horman:
> > On Wed, May 25, 2016 at 03:21:19PM +0200, Thomas Monjalon wrote:
> > > 2016-05-24 15:41, Neil Horman:
> > > > --- a/GNUmakefile
> > > > +++ b/GNUmakefile
> > > > -ROOTDIRS-y := lib drivers app
> > > > +ROOTDIRS-y := buildtools lib drivers app
> > > 
> > > Why a new directory?
> > > It is not a script nor an end-user tool, I guess.
> > Dependencies.  This tool has to be built prior to the rest of the dpdk, but app
> > already relies on dpdk libraries to be built, so you get circular dependencies.
> > I could have put it in scripts I guess, but its not a script.  Its own directory
> > seemed to make the most sense, given those two points
> 
> OK
> 
> > > > +include $(RTE_SDK)/mk/rte.buildtools.mk
> > > 
> > > Why a new Makefile? Can you use rte.hostapp.mk?
> > > 
> > I don't know, maybe.  Nothing else currently uses rte.hostapp.mk, so I missed
> > its existance.  I make the argument that, that being the case, we should stick
> > with the Makefile I just tested with, and remove the rte.hostapp.mk file
> 
> No, rte.hostapp.mk has been used and tested in the history of the project.
> Please try it.
> 
It works, but its really ugly (as it means that the buildtools directory gets
install to the hostapp directory under the build).  I could move that of course,
but at this point, you are asking me to remove a working makefile to replace it
with another makefile that, by all rights should have been removed as part of
commit efa2084a840fb83fd9be83adca57e5f23d3fa9fe:
Author: Thomas Monjalon <thomas.monjalon@6wind.com>
Date:   Tue Mar 10 17:55:25 2015 +0100

    scripts: remove useless build tools
    
    test-framework.sh is an old script to check building of some dependencies.
    testhost is an old app used to check HOSTCC.
    
    Let's clean the scripts directory.

Here you removed the only user of rte.hostapp.mk, but neglected to remove
hostapp.mk itself.  I really fail to see why making me rework my current
makefile setup, that matches the purpose of the tool is a superior solution to
just getting rid of the unused makefile thats there right now.

> > > > +++ b/buildtools/pmdinfogen/pmdinfogen.c
> > > [...]
> > > > +	/*
> > > > + 	 * If this returns NULL, then this is a PMD_VDEV, because
> > > > + 	 * it has no pci table reference
> > > > + 	 */
> > > 
> > > We can imagine physical PMD not using PCI.
> > > I think this comment should be removed.
> > We can, but currently its a true statement.  we have two types of PMDs, a PDEV
> > and a VDEV, the former is a pci device, and the latter is a virtual device, so
> > you can imply the PDEV type from the presence of pci entries, and VDEV from the
> > alternative.  If we were to do something, I would recommend adding a macro to
> > explicitly ennumerate each pmds type.  I would prefer to wait until that was a
> > need however, as it can be done invisibly to the user.
> 
> We are removing the PMD types in the EAL rework.
> So this comment will be outdated. Better to remove now.
> 
Then, I'm just not going to enumerate the type of driver at all, I'll remove
that attribute entirely.  But I really don't like to write code for things that
are 'predictive'.

> > > [...]
> > > > +		fprintf(ofd,"\\\"type\\\" : \\\"%s\\\", ", drv->pci_tbl ? "PMD_PDEV" : "PMD_VDEV");
> > > 
> > > Please forget the naming PDEV/VDEV.
> > > 
> > I don't know what you mean here, you would rather they be named PCI and Virtual,
> > or something else?
> 
> Yes please.
> 
No, If you're removing the types, and you're sure of that, I'm just going to
remove the description entirely.  If you're unsure about exactly whats going to
happen, we should reflect the state of the build now, and make the appropriate
change when it lands.


> > > [...]
> > > > +	if (info.drivers) {
> > > > +		output_pmd_info_string(&info, argv[2]);
> > > > +		rc = 0;
> > > > +	} else {
> > > > +		fprintf(stderr, "Hmm, Appears to be a driver but no drivers registered\n");
> > > 
> > > Why it appears to be a driver?
> > > What means "no drivers registered" exactly?
> > > 
> > It means that the tool has identified this file as a driver based on some
> > criteria (in this case the source code contained a use of the
> > PMD_REGISTER_DRIVER macro, but for whatever reason, when this tool scanned it,
> > it never located the pmd_driver_name<n> symbol.  It should never happen, and
> > serves as a indicator to the developer that they need to investigate either the
> > construction of the driver or the use of this tool.
> 
> OK
> 
> > > > +++ b/buildtools/pmdinfogen/pmdinfogen.h
> > > [...]
> > > > +#define Elf_Ehdr    Elf64_Ehdr
> > > > +#define Elf_Shdr    Elf64_Shdr
> > > > +#define Elf_Sym     Elf64_Sym
> > > > +#define Elf_Addr    Elf64_Addr
> > > > +#define Elf_Sword   Elf64_Sxword
> > > > +#define Elf_Section Elf64_Half
> > > > +#define ELF_ST_BIND ELF64_ST_BIND
> > > > +#define ELF_ST_TYPE ELF64_ST_TYPE
> > > > +
> > > > +#define Elf_Rel     Elf64_Rel
> > > > +#define Elf_Rela    Elf64_Rela
> > > > +#define ELF_R_SYM   ELF64_R_SYM
> > > > +#define ELF_R_TYPE  ELF64_R_TYPE
> > > 
> > > Why these defines are needed?
> > > 
> > Because I borrowed the code from modpost.c, which allows for both ELF32 and
> > ELF64 compilation.  I wanted to keep it in place should DPDK ever target
> > different sized architectures.
> 
> Maybe a comment is needed.
> Is ELF32 used on 32-bit archs like i686 or ARMv7?
It depends on exactly how its built, but that would be a common use, yes.

> 
> > > > +struct rte_pci_id {
> > > > +	uint16_t vendor_id;           /**< Vendor ID or PCI_ANY_ID. */
> > > > +	uint16_t device_id;           /**< Device ID or PCI_ANY_ID. */
> > > > +	uint16_t subsystem_vendor_id; /**< Subsystem vendor ID or PCI_ANY_ID. */
> > > > +	uint16_t subsystem_device_id; /**< Subsystem device ID or PCI_ANY_ID. */
> > > > +};
> > > [...]
> > > > +struct pmd_driver {
> > > > +	Elf_Sym *name_sym;
> > > > +	const char *name;
> > > > +	struct rte_pci_id *pci_tbl;
> > > > +	struct pmd_driver *next;
> > > > +
> > > > +	const char* opt_vals[PMD_OPT_MAX];
> > > > +};
> > > 
> > > Are you duplicating some structures from EAL?
> > > It will be out of sync easily.
> > > 
> > Only the rte_pci_id, which hasn't changed since the initial public release of
> > the DPDK.  We can clean this up later if you like, but I'm really not too
> > worried about it.
> 
> I would prefer an include if possible.
> rte_pci_id is changing in 16.07 ;)
> 
So, we've had this discussion before :).  Its really not fair to ask anyone to
write code based on predictive changes.  If theres some patch out there thats
planning on making a change, we can't be expected to write with it in mind.  If
you want people to use it, then get it merged.  I understand thats not really
the issue here, and I'm making the change because you're right, we should avoid
duplicating the structures if we can, but please understand that its impossible
to write for change thats predicted to come at a later date.

> > > > +struct elf_info {
> > > > +	unsigned long size;
> > > > +	Elf_Ehdr     *hdr;
> > > > +	Elf_Shdr     *sechdrs;
> > > > +	Elf_Sym      *symtab_start;
> > > > +	Elf_Sym      *symtab_stop;
> > > > +	Elf_Section  export_sec;
> > > > +	Elf_Section  export_unused_sec;
> > > > +	Elf_Section  export_gpl_sec;
> > > > +	Elf_Section  export_unused_gpl_sec;
> > > > +	Elf_Section  export_gpl_future_sec;
> > > > +	char         *strtab;
> > > > +	char	     *modinfo;
> > > > +	unsigned int modinfo_len;
> > > 
> > > Why these fields?
> > > 
> > Because thats how you parse an ELF file and look up the information you need to
> > extract the data this tool then exports.  I don't mean to sound short, but your
> > question doesn't really make any sense.  The members of this structure are
> > needed to extract the info from object files to build the export strings that
> > pmdinfo.py needs later.
> 
> Sorry, I haven't parse the whole code but some fields are unused.
> And modinfo looks wrong.
> 
Yup, your right, modifo did get orphaned when I was doing previous cleanup, and
can be removed.

> > > > +++ b/mk/rte.buildtools.mk
> > > 
> > > This file must be removed I think.
> > > We are going to be sick after digesting so much makefiles ;)
> > > 
> > See above, given that I just tested this, and rte.hostapp.mk isn't used, I'd
> > recommend deleting the latter, rather than deleting this one and moving to the
> > old one.
> 
> See above, I do not agree :)
> 
Then we're not going to agree about this :).  I'll re-iterate my stance.  Moving to
use rte.hotapp.mk, causes alot more work for me, makes the use of the tool
somewhat uglier, and by all rights shouldn't be there at all, due to your
previously mentioned commit. It just makes more sense to use the buildtools
makefile and remove the vesitgial rte.hostapp.mk makefile.

Neil

> 

Re: [PATCHv4 1/5] pmdinfogen: Add buildtools and pmdinfogen utility

[Thomas Monjalon]

2016-05-25 15:13, Neil Horman:
> On Wed, May 25, 2016 at 07:39:30PM +0200, Thomas Monjalon wrote:
> > 2016-05-25 13:22, Neil Horman:
> > > On Wed, May 25, 2016 at 03:21:19PM +0200, Thomas Monjalon wrote:
> > > > 2016-05-24 15:41, Neil Horman:
> > > > > +include $(RTE_SDK)/mk/rte.buildtools.mk
> > > > 
> > > > Why a new Makefile? Can you use rte.hostapp.mk?
> > > > 
> > > I don't know, maybe.  Nothing else currently uses rte.hostapp.mk, so I missed
> > > its existance.  I make the argument that, that being the case, we should stick
> > > with the Makefile I just tested with, and remove the rte.hostapp.mk file
> > 
> > No, rte.hostapp.mk has been used and tested in the history of the project.
> > Please try it.
> > 
> It works, but its really ugly (as it means that the buildtools directory gets
> install to the hostapp directory under the build).  I could move that of course,
> but at this point, you are asking me to remove a working makefile to replace it
> with another makefile that, by all rights should have been removed as part of
> commit efa2084a840fb83fd9be83adca57e5f23d3fa9fe:
> Author: Thomas Monjalon <thomas.monjalon@6wind.com>
> Date:   Tue Mar 10 17:55:25 2015 +0100
> 
>     scripts: remove useless build tools
>     
>     test-framework.sh is an old script to check building of some dependencies.
>     testhost is an old app used to check HOSTCC.
>     
>     Let's clean the scripts directory.
> 
> Here you removed the only user of rte.hostapp.mk, but neglected to remove
> hostapp.mk itself.

Yes. I didn't really neglect to remove it. I thought it would be used later.

> I really fail to see why making me rework my current
> makefile setup, that matches the purpose of the tool is a superior solution to
> just getting rid of the unused makefile thats there right now.

I'm just trying to avoid creating a new makefile for each tool.
Is it possible to fix the directory in rte.hostapp.mk?
Every apps use the same makefile rte.app.mk. I think it should be the same
for host apps.

> > > > > +++ b/buildtools/pmdinfogen/pmdinfogen.c
> > > > [...]
> > > > > +	/*
> > > > > + 	 * If this returns NULL, then this is a PMD_VDEV, because
> > > > > + 	 * it has no pci table reference
> > > > > + 	 */
> > > > 
> > > > We can imagine physical PMD not using PCI.
> > > > I think this comment should be removed.
> > > We can, but currently its a true statement.  we have two types of PMDs, a PDEV
> > > and a VDEV, the former is a pci device, and the latter is a virtual device, so
> > > you can imply the PDEV type from the presence of pci entries, and VDEV from the
> > > alternative.  If we were to do something, I would recommend adding a macro to
> > > explicitly ennumerate each pmds type.  I would prefer to wait until that was a
> > > need however, as it can be done invisibly to the user.
> > 
> > We are removing the PMD types in the EAL rework.
> > So this comment will be outdated. Better to remove now.
> > 
> Then, I'm just not going to enumerate the type of driver at all, I'll remove
> that attribute entirely.

OK

> But I really don't like to write code for things that are 'predictive'.

Not really predictive as it is an older patch.

> > > > [...]
> > > > > +		fprintf(ofd,"\\\"type\\\" : \\\"%s\\\", ", drv->pci_tbl ? "PMD_PDEV" : "PMD_VDEV");
> > > > 
> > > > Please forget the naming PDEV/VDEV.
> > > > 
> > > I don't know what you mean here, you would rather they be named PCI and Virtual,
> > > or something else?
> > 
> > Yes please.
> > 
> No, If you're removing the types, and you're sure of that, I'm just going to
> remove the description entirely.  If you're unsure about exactly whats going to
> happen, we should reflect the state of the build now, and make the appropriate
> change when it lands.

OK to remove the type description.

> > > > > +++ b/buildtools/pmdinfogen/pmdinfogen.h
> > > > [...]
> > > > > +#define Elf_Ehdr    Elf64_Ehdr
> > > > > +#define Elf_Shdr    Elf64_Shdr
> > > > > +#define Elf_Sym     Elf64_Sym
> > > > > +#define Elf_Addr    Elf64_Addr
> > > > > +#define Elf_Sword   Elf64_Sxword
> > > > > +#define Elf_Section Elf64_Half
> > > > > +#define ELF_ST_BIND ELF64_ST_BIND
> > > > > +#define ELF_ST_TYPE ELF64_ST_TYPE
> > > > > +
> > > > > +#define Elf_Rel     Elf64_Rel
> > > > > +#define Elf_Rela    Elf64_Rela
> > > > > +#define ELF_R_SYM   ELF64_R_SYM
> > > > > +#define ELF_R_TYPE  ELF64_R_TYPE
> > > > 
> > > > Why these defines are needed?
> > > > 
> > > Because I borrowed the code from modpost.c, which allows for both ELF32 and
> > > ELF64 compilation.  I wanted to keep it in place should DPDK ever target
> > > different sized architectures.
> > 
> > Maybe a comment is needed.
> > Is ELF32 used on 32-bit archs like i686 or ARMv7?
> It depends on exactly how its built, but that would be a common use, yes.

We have such 32-bit archs in DPDK. Is pmdinfogen working for them?

> > > > > +struct rte_pci_id {
> > > > > +	uint16_t vendor_id;           /**< Vendor ID or PCI_ANY_ID. */
> > > > > +	uint16_t device_id;           /**< Device ID or PCI_ANY_ID. */
> > > > > +	uint16_t subsystem_vendor_id; /**< Subsystem vendor ID or PCI_ANY_ID. */
> > > > > +	uint16_t subsystem_device_id; /**< Subsystem device ID or PCI_ANY_ID. */
> > > > > +};
> > > > [...]
> > > > > +struct pmd_driver {
> > > > > +	Elf_Sym *name_sym;
> > > > > +	const char *name;
> > > > > +	struct rte_pci_id *pci_tbl;
> > > > > +	struct pmd_driver *next;
> > > > > +
> > > > > +	const char* opt_vals[PMD_OPT_MAX];
> > > > > +};
> > > > 
> > > > Are you duplicating some structures from EAL?
> > > > It will be out of sync easily.
> > > > 
> > > Only the rte_pci_id, which hasn't changed since the initial public release of
> > > the DPDK.  We can clean this up later if you like, but I'm really not too
> > > worried about it.
> > 
> > I would prefer an include if possible.
> > rte_pci_id is changing in 16.07 ;)
> > 
> So, we've had this discussion before :).  Its really not fair to ask anyone to
> write code based on predictive changes.  If theres some patch out there thats
> planning on making a change, we can't be expected to write with it in mind.  If
> you want people to use it, then get it merged.  I understand thats not really
> the issue here, and I'm making the change because you're right, we should avoid
> duplicating the structures if we can, but please understand that its impossible
> to write for change thats predicted to come at a later date.

I understand your point.
The rte_pci_id change has been reviewed several times already and should be
applied very soon.

> > > > > +++ b/mk/rte.buildtools.mk
> > > > 
> > > > This file must be removed I think.
> > > > We are going to be sick after digesting so much makefiles ;)
> > > > 
> > > See above, given that I just tested this, and rte.hostapp.mk isn't used, I'd
> > > recommend deleting the latter, rather than deleting this one and moving to the
> > > old one.
> > 
> > See above, I do not agree :)
> > 
> Then we're not going to agree about this :).  I'll re-iterate my stance.  Moving to
> use rte.hotapp.mk, causes alot more work for me, makes the use of the tool
> somewhat uglier, and by all rights shouldn't be there at all, due to your
> previously mentioned commit. It just makes more sense to use the buildtools
> makefile and remove the vesitgial rte.hostapp.mk makefile.

Re: [PATCHv4 4/5] Makefile: Do post processing on objects that register a driver

[Neil Horman]

On Wed, May 25, 2016 at 08:56:25PM +0200, Thomas Monjalon wrote:
> 2016-05-25 13:40, Neil Horman:
> > On Wed, May 25, 2016 at 07:08:19PM +0200, Thomas Monjalon wrote:
> > > 2016-05-24 15:41, Neil Horman:
> > > > +		echo MODGEN $@; \
> > > > +		OBJF=`readlink -f $@`; \
> > > > +		${RTE_OUTPUT}/buildtools/pmdinfogen \$$OBJF \$$OBJF.mod.c; \
> > > 
> > > Maybe .pmd.c would be more appropriate than .mod.c?
> > fine
> > > What means mod/MODGEN/MODBUILD?
> > GENerate Module information & BUILD module information.
> 
> I think "module" is not appropriate here.
> 
This is starting to feel very much like bikeshedding.  What do you think would
be more appropriate here?

> > > It deserves to be in a shell script, at least to ease testing.
> > What do you mean by "it" and why would it be easier to test in a shell script?
> 
> "it" is mostly this whole patch.
> With a shell script, we can test the behaviour on one file easily.
> Maybe I'm wrong, but I don't like having too much lines in a Makefile rule.
> We probably need more opinions.
> 
That makes no sense to me. Any such script would need to receive two arguments:
1) The path to a C file for a pmd
2) The path to the corresponding object file for that pmd

Running any such script is then usless unles its predecated on first building
all the object files in the pmd.  And if you want to run something by hand on
the object files, it seems pretty straightforward to do so, just run:
build/builttools/pmdinfogen /path/to/pmd/object/file

The rest of that code is really just a test to avoid having to run pmdinfo gen
on any files other than the ones that contain the PMD_REGISTER_DRIVER macro

Neil

Re: [PATCHv4 1/5] pmdinfogen: Add buildtools and pmdinfogen utility

[Neil Horman]

On Wed, May 25, 2016 at 09:39:43PM +0200, Thomas Monjalon wrote:
> 2016-05-25 15:13, Neil Horman:
> > On Wed, May 25, 2016 at 07:39:30PM +0200, Thomas Monjalon wrote:
> > > 2016-05-25 13:22, Neil Horman:
> > > > On Wed, May 25, 2016 at 03:21:19PM +0200, Thomas Monjalon wrote:
> > > > > 2016-05-24 15:41, Neil Horman:
> > > > > > +include $(RTE_SDK)/mk/rte.buildtools.mk
> > > > > 
> > > > > Why a new Makefile? Can you use rte.hostapp.mk?
> > > > > 
> > > > I don't know, maybe.  Nothing else currently uses rte.hostapp.mk, so I missed
> > > > its existance.  I make the argument that, that being the case, we should stick
> > > > with the Makefile I just tested with, and remove the rte.hostapp.mk file
> > > 
> > > No, rte.hostapp.mk has been used and tested in the history of the project.
> > > Please try it.
> > > 
> > It works, but its really ugly (as it means that the buildtools directory gets
> > install to the hostapp directory under the build).  I could move that of course,
> > but at this point, you are asking me to remove a working makefile to replace it
> > with another makefile that, by all rights should have been removed as part of
> > commit efa2084a840fb83fd9be83adca57e5f23d3fa9fe:
> > Author: Thomas Monjalon <thomas.monjalon@6wind.com>
> > Date:   Tue Mar 10 17:55:25 2015 +0100
> > 
> >     scripts: remove useless build tools
> >     
> >     test-framework.sh is an old script to check building of some dependencies.
> >     testhost is an old app used to check HOSTCC.
> >     
> >     Let's clean the scripts directory.
> > 
> > Here you removed the only user of rte.hostapp.mk, but neglected to remove
> > hostapp.mk itself.
> 
> Yes. I didn't really neglect to remove it. I thought it would be used later.
> 
Ok, thats fair.

> > I really fail to see why making me rework my current
> > makefile setup, that matches the purpose of the tool is a superior solution to
> > just getting rid of the unused makefile thats there right now.
> 
> I'm just trying to avoid creating a new makefile for each tool.
> Is it possible to fix the directory in rte.hostapp.mk?
> Every apps use the same makefile rte.app.mk. I think it should be the same
> for host apps.
> 
Yes, I could do that, I could fix up the directory path in rte.hostapp.mk so
that it installs to buildtools rather than hostapp, and that would be fine.  But
then if I were to additionally issue this command:
git mv mk/rte.hostapp.mk mk/rte.buildtools.mk

We would have exactly what I'm proposing anyway.  

I don't disagree that rte.buildtools.mk and rte.hostapp.mk are simmilar, they
are in fact almost identical, and I simply missed the latter because I didn't
see any uses of it.  What I am saying is that, due to their simmilarity, Its
pretty much an equivalent situation to use either makefile, and its less work
for me to remove hostapp.mk and just use what I have.

> > > > > > +++ b/buildtools/pmdinfogen/pmdinfogen.c
> > > > > [...]
> > > > > > +	/*
> > > > > > + 	 * If this returns NULL, then this is a PMD_VDEV, because
> > > > > > + 	 * it has no pci table reference
> > > > > > + 	 */
> > > > > 
> > > > > We can imagine physical PMD not using PCI.
> > > > > I think this comment should be removed.
> > > > We can, but currently its a true statement.  we have two types of PMDs, a PDEV
> > > > and a VDEV, the former is a pci device, and the latter is a virtual device, so
> > > > you can imply the PDEV type from the presence of pci entries, and VDEV from the
> > > > alternative.  If we were to do something, I would recommend adding a macro to
> > > > explicitly ennumerate each pmds type.  I would prefer to wait until that was a
> > > > need however, as it can be done invisibly to the user.
> > > 
> > > We are removing the PMD types in the EAL rework.
> > > So this comment will be outdated. Better to remove now.
> > > 
> > Then, I'm just not going to enumerate the type of driver at all, I'll remove
> > that attribute entirely.
> 
> OK
> 
> > But I really don't like to write code for things that are 'predictive'.
> 
> Not really predictive as it is an older patch.
And how many older patches never get integrated?  Or languish for long periods
of time?  We've debated this before.

Its really not reasonable to expect developers (myself or others) to
go through the mailing list and create an ordinal list of patches to apply
before doing our development work.  If that were the case, then they should just
be applied immediately so the HEAD of the git tree is an accurate representation
of the development state of the tree.  But thats not the case, and patches don't
always get applied in the order that they are posted.  So, if Davids Patch
series goes in ahead of mine, I'll gladly rebase, but I don't want to create
some artificial ordinality just because we touch the same code, especially if
his patch series has to go back for further revision.

> 
> > > > > [...]
> > > > > > +		fprintf(ofd,"\\\"type\\\" : \\\"%s\\\", ", drv->pci_tbl ? "PMD_PDEV" : "PMD_VDEV");
> > > > > 
> > > > > Please forget the naming PDEV/VDEV.
> > > > > 
> > > > I don't know what you mean here, you would rather they be named PCI and Virtual,
> > > > or something else?
> > > 
> > > Yes please.
> > > 
> > No, If you're removing the types, and you're sure of that, I'm just going to
> > remove the description entirely.  If you're unsure about exactly whats going to
> > happen, we should reflect the state of the build now, and make the appropriate
> > change when it lands.
> 
> OK to remove the type description.
> 
Ok, I'll do that.

> > > > > > +++ b/buildtools/pmdinfogen/pmdinfogen.h
> > > > > [...]
> > > > > > +#define Elf_Ehdr    Elf64_Ehdr
> > > > > > +#define Elf_Shdr    Elf64_Shdr
> > > > > > +#define Elf_Sym     Elf64_Sym
> > > > > > +#define Elf_Addr    Elf64_Addr
> > > > > > +#define Elf_Sword   Elf64_Sxword
> > > > > > +#define Elf_Section Elf64_Half
> > > > > > +#define ELF_ST_BIND ELF64_ST_BIND
> > > > > > +#define ELF_ST_TYPE ELF64_ST_TYPE
> > > > > > +
> > > > > > +#define Elf_Rel     Elf64_Rel
> > > > > > +#define Elf_Rela    Elf64_Rela
> > > > > > +#define ELF_R_SYM   ELF64_R_SYM
> > > > > > +#define ELF_R_TYPE  ELF64_R_TYPE
> > > > > 
> > > > > Why these defines are needed?
> > > > > 
> > > > Because I borrowed the code from modpost.c, which allows for both ELF32 and
> > > > ELF64 compilation.  I wanted to keep it in place should DPDK ever target
> > > > different sized architectures.
> > > 
> > > Maybe a comment is needed.
> > > Is ELF32 used on 32-bit archs like i686 or ARMv7?
> > It depends on exactly how its built, but that would be a common use, yes.
> 
> We have such 32-bit archs in DPDK. Is pmdinfogen working for them?
> 
It was a few revisions ago, but I've changed so much now, I should probably
check again.

> > > > > > +struct rte_pci_id {
> > > > > > +	uint16_t vendor_id;           /**< Vendor ID or PCI_ANY_ID. */
> > > > > > +	uint16_t device_id;           /**< Device ID or PCI_ANY_ID. */
> > > > > > +	uint16_t subsystem_vendor_id; /**< Subsystem vendor ID or PCI_ANY_ID. */
> > > > > > +	uint16_t subsystem_device_id; /**< Subsystem device ID or PCI_ANY_ID. */
> > > > > > +};
> > > > > [...]
> > > > > > +struct pmd_driver {
> > > > > > +	Elf_Sym *name_sym;
> > > > > > +	const char *name;
> > > > > > +	struct rte_pci_id *pci_tbl;
> > > > > > +	struct pmd_driver *next;
> > > > > > +
> > > > > > +	const char* opt_vals[PMD_OPT_MAX];
> > > > > > +};
> > > > > 
> > > > > Are you duplicating some structures from EAL?
> > > > > It will be out of sync easily.
> > > > > 
> > > > Only the rte_pci_id, which hasn't changed since the initial public release of
> > > > the DPDK.  We can clean this up later if you like, but I'm really not too
> > > > worried about it.
> > > 
> > > I would prefer an include if possible.
> > > rte_pci_id is changing in 16.07 ;)
> > > 
> > So, we've had this discussion before :).  Its really not fair to ask anyone to
> > write code based on predictive changes.  If theres some patch out there thats
> > planning on making a change, we can't be expected to write with it in mind.  If
> > you want people to use it, then get it merged.  I understand thats not really
> > the issue here, and I'm making the change because you're right, we should avoid
> > duplicating the structures if we can, but please understand that its impossible
> > to write for change thats predicted to come at a later date.
> 
> I understand your point.
> The rte_pci_id change has been reviewed several times already and should be
> applied very soon.
> 
Ok, As I said in my prior note, I'm making this change, because its sane to do
in and of itself, but again, if its not in HEAD, it really doesn't exist yet
from a development standpoint.

> > > > > > +++ b/mk/rte.buildtools.mk
> > > > > 
> > > > > This file must be removed I think.
> > > > > We are going to be sick after digesting so much makefiles ;)
> > > > > 
> > > > See above, given that I just tested this, and rte.hostapp.mk isn't used, I'd
> > > > recommend deleting the latter, rather than deleting this one and moving to the
> > > > old one.
> > > 
> > > See above, I do not agree :)
> > > 
> > Then we're not going to agree about this :).  I'll re-iterate my stance.  Moving to
> > use rte.hotapp.mk, causes alot more work for me, makes the use of the tool
> > somewhat uglier, and by all rights shouldn't be there at all, due to your
> > previously mentioned commit. It just makes more sense to use the buildtools
> > makefile and remove the vesitgial rte.hostapp.mk makefile.
> 
> 

Re: [PATCHv4 4/5] Makefile: Do post processing on objects that register a driver

[Thomas Monjalon]

2016-05-25 15:43, Neil Horman:
> On Wed, May 25, 2016 at 08:56:25PM +0200, Thomas Monjalon wrote:
> > 2016-05-25 13:40, Neil Horman:
> > > On Wed, May 25, 2016 at 07:08:19PM +0200, Thomas Monjalon wrote:
> > > > 2016-05-24 15:41, Neil Horman:
> > > > > +		echo MODGEN $@; \
> > > > > +		OBJF=`readlink -f $@`; \
> > > > > +		${RTE_OUTPUT}/buildtools/pmdinfogen \$$OBJF \$$OBJF.mod.c; \
> > > > 
> > > > Maybe .pmd.c would be more appropriate than .mod.c?
> > > fine
> > > > What means mod/MODGEN/MODBUILD?
> > > GENerate Module information & BUILD module information.
> > 
> > I think "module" is not appropriate here.
> > 
> This is starting to feel very much like bikeshedding.  What do you think would
> be more appropriate here?

pmd/PMDINFO//

> > > > It deserves to be in a shell script, at least to ease testing.
> > > What do you mean by "it" and why would it be easier to test in a shell script?
> > 
> > "it" is mostly this whole patch.
> > With a shell script, we can test the behaviour on one file easily.
> > Maybe I'm wrong, but I don't like having too much lines in a Makefile rule.
> > We probably need more opinions.
> > 
> That makes no sense to me. Any such script would need to receive two arguments:
> 1) The path to a C file for a pmd
> 2) The path to the corresponding object file for that pmd
> 
> Running any such script is then usless unles its predecated on first building
> all the object files in the pmd.  And if you want to run something by hand on
> the object files, it seems pretty straightforward to do so, just run:
> build/builttools/pmdinfogen /path/to/pmd/object/file
> 
> The rest of that code is really just a test to avoid having to run pmdinfo gen
> on any files other than the ones that contain the PMD_REGISTER_DRIVER macro

OK, no strong opinion here.
If you feel comfortable with multi-lines "sh -c" and escaping, up to you.
If I discover something wrong in this part and needs to do some maintenance
work, I'll probably think differently.

Re: [PATCHv4 4/5] Makefile: Do post processing on objects that register a driver

[Neil Horman]

On Wed, May 25, 2016 at 10:04:11PM +0200, Thomas Monjalon wrote:
> 2016-05-25 15:43, Neil Horman:
> > On Wed, May 25, 2016 at 08:56:25PM +0200, Thomas Monjalon wrote:
> > > 2016-05-25 13:40, Neil Horman:
> > > > On Wed, May 25, 2016 at 07:08:19PM +0200, Thomas Monjalon wrote:
> > > > > 2016-05-24 15:41, Neil Horman:
> > > > > > +		echo MODGEN $@; \
> > > > > > +		OBJF=`readlink -f $@`; \
> > > > > > +		${RTE_OUTPUT}/buildtools/pmdinfogen \$$OBJF \$$OBJF.mod.c; \
> > > > > 
> > > > > Maybe .pmd.c would be more appropriate than .mod.c?
> > > > fine
> > > > > What means mod/MODGEN/MODBUILD?
> > > > GENerate Module information & BUILD module information.
> > > 
> > > I think "module" is not appropriate here.
> > > 
> > This is starting to feel very much like bikeshedding.  What do you think would
> > be more appropriate here?
> 
> pmd/PMDINFO//
> 
> > > > > It deserves to be in a shell script, at least to ease testing.
> > > > What do you mean by "it" and why would it be easier to test in a shell script?
> > > 
> > > "it" is mostly this whole patch.
> > > With a shell script, we can test the behaviour on one file easily.
> > > Maybe I'm wrong, but I don't like having too much lines in a Makefile rule.
> > > We probably need more opinions.
> > > 
> > That makes no sense to me. Any such script would need to receive two arguments:
> > 1) The path to a C file for a pmd
> > 2) The path to the corresponding object file for that pmd
> > 
> > Running any such script is then usless unles its predecated on first building
> > all the object files in the pmd.  And if you want to run something by hand on
> > the object files, it seems pretty straightforward to do so, just run:
> > build/builttools/pmdinfogen /path/to/pmd/object/file
> > 
> > The rest of that code is really just a test to avoid having to run pmdinfo gen
> > on any files other than the ones that contain the PMD_REGISTER_DRIVER macro
> 
> OK, no strong opinion here.
> If you feel comfortable with multi-lines "sh -c" and escaping, up to you.
> If I discover something wrong in this part and needs to do some maintenance
> work, I'll probably think differently.
> 
You're welcome to assign the bug to me :)
Neil

[PATCHv5 0/6] Implement pmd hardware support exports

[Neil Horman]

Hey all-
	So heres attempt number 2 at a method for exporting PMD hardware support
information.  As we discussed previously, the consensus seems to be that pmd
information should be:

1) Able to be interrogated on any ELF binary (application binary or individual
DSO)
2) Equally functional on statically linked applications or on DSO's
3) Resilient to symbol stripping
4) Script friendly
5) Show kernel dependencies
6) List driver options
7) Show driver name
8) Offer human readable output
9) Show DPDK version
10) Show driver version
11) Allow for expansion
12) Not place additional build environment dependencies on an application

I added item 12 myself, because I don't think its reasonable to require
applications to use a specific linker script to get hardware support information
(which precludes the use of a special .modinfo section like the kernel uses)

However, we still can use some tricks from the kernel to make this work.  In
this approach, what I've done is the following:

A) Modified the driver registration macro to also define a variable:
this_pmd_name<n>= "name"

Based on the unique name string pointed to by the above variable, we can
query for an arbitrary number of other symbols following the pattern:
<name>_<tag>

Where tag is some well known identifier for information we wish to export

B) Added a utility called pmdinfogen.  This utility is not meant for general use,
but rather is used by the dpdk build environment itself when compiling pmds.
for each object that uses the PMD_REGISTER_DRIVER macro, this utiity is run.  It
searches for the symbols in (A), and using those, extracts the hardware support
info, and module name from the object, using that to produce a new c file
containing a single variable in the following format:

static const char <name>[] __attribute__((used)) = "PMD_DRIVER_INFO=<json>";

The <name> is arbitrary, as its static and not referenced.  The relevant bit is
the string value assigned to it.  The <json> is a json encoded string of the
extracted hardware support information pmdinfo found for the corresponding
object.  This C file is suitable for compilation and relocatable linking back
into the parent object file.  The result of this operation is that the object
string table now contains a string that will not e removed by stripping, whos
leading text (PMD_DRIVER_INFO) can be easily searched for at any time weather
the symbol referring to it is stripped or not.

C) Added a utilty called pmdinfo.py.  This python script, searches the
string table of the .rodata section of any provided ELF file looking for the
PMD_DRIVER_INFO prefix on a string.  When found, it will interpret the remainder
of the string as json, and output the hardware support for that ELF file (if
any).


This approach ticks most of the above boxes:
1) Impervious to stripping
2) Works on static and dynamic binaries
3) Human and script friendly
4) Allows for expansion

Because of (4) the other items should be pretty easy to implement, as its just a
matter of modifying the macros to export the info, pmdinfo to encode it to json,
and pmd_hw_support.py to read it.  I'm not adding them now, as theres alot of
rote work to do to get some of them in place (e.g. DPDK has no current global
VERSION macro, drivers don't have a consistent version scheme, command line
strings have to be built for each driver, etc).  But once this is accepted,
those items can be done as time allows and should be fairly easy to implement.

Change Notes:
v2)
 * Made the export macros a bit easier to expand
 * Added a macro to optionally export command line strings
 * Renamed utilties to be more appropriate
   (pmdinfo -> pmdinfogen, pmd_hw_support.py -> pmdinfo.py)
 * Added search capabilities to pmdinfo.py so that we search for libraries
   linked using DT_NEEDINFO entries.  We search DT_RUNPATH, LD_LIBRARY_PATH,
   /usr/lib and /lib
 * Added serch abilities if the specified binary to pmdinfo isn't found, we
   search LD_LIBRARY_PATH, /usr/lib and /lib
 * Added an option to pmdinfo.py to pretty-print hardware support info using the
   pci.ids database

v3)
 * Made table mode printing the default mode
 * Added a default path to the pci.ids file
 * Modifed pmdinfo to use python rather than python3
 * Cleaned up some changelog entries
 * Added an export for RTE_EAL_PMD_PATH
 * Added a plugin option to pmdinfo to scan for autoloaded DSO's

v4)
 * Modified the operation of the -p option. As much as I don't like implying
that autoloaded pmds are guaranteed to be there at run time, I'm having a hard
time seeing how we can avoid specifying the application file to scan for the
autoload directory.  Without it we can't determine which library the user means
in a multiversion installation
 * Cleaned up the help text
 * Added a rule for an install target for pmdinfo
 * Guarded against some tracebacks in pmdinfo
 * Use DT_NEEDED entries to get versioned libraries in -p mode
 * Fixed traceback that occurs on lack of input arguments
 * Fixed some erroneous macro usage in drivers that aren't in the default build

v5)
 * Added a dpdk- prefix to pmdinfo symlink
 * Renamed build announcement to PMDINFOGEN/BUILD
 * Removed erroneous true statement from makefile
 * Removed duplicate rte.hostapp.mk makefile
 * Fixed some whitespace
 * Whitespace fixups
 * Fixed makefile if; then style
 * Renamed module string C file
 * Removed duplicate rte_pci_id definition
 * Clarified macro names
 * Removed PMD type attribute
 * Fixed tools usage for 32 bit arches
 * Removed some unused code
 * Added a few comments

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>

[PATCHv5 1/6] pmdinfogen: Add buildtools and pmdinfogen utility

[Neil Horman]

pmdinfogen is a tool used to parse object files and build json strings for use in
later determining hardware support in a dso or application binary.  pmdinfo
looks for the non-exported symbol names this_pmd_name<n> and this_pmd_tbl<n>
(where n is a integer counter).  It records the name of each of these tuples,
using the later to find the symbolic name of the pci_table for physical devices
that the object supports.  With this information, it outputs a C file with a
single line of the form:

static char *<pmd_name>_driver_info[] __attribute__((used)) = " \
	PMD_DRIVER_INFO=<json string>";

Where <pmd_name> is the arbitrary name of the pmd, and <json_string> is the json
encoded string that hold relevant pmd information, including the pmd name, type
and optional array of pci device/vendor ids that the driver supports.

This c file is suitable for compiling to object code, then relocatably linking
into the parent file from which the C was generated.  This creates an entry in
the string table of the object that can inform a later tool about hardware
support.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 GNUmakefile                        |   2 +-
 buildtools/Makefile                |  36 ++++
 buildtools/pmdinfogen/Makefile     |  49 +++++
 buildtools/pmdinfogen/pmdinfogen.c | 410 +++++++++++++++++++++++++++++++++++++
 buildtools/pmdinfogen/pmdinfogen.h |  94 +++++++++
 mk/rte.buildtools.mk               | 148 +++++++++++++
 mk/rte.sdkbuild.mk                 |   3 +-
 7 files changed, 740 insertions(+), 2 deletions(-)
 create mode 100644 buildtools/Makefile
 create mode 100644 buildtools/pmdinfogen/Makefile
 create mode 100644 buildtools/pmdinfogen/pmdinfogen.c
 create mode 100644 buildtools/pmdinfogen/pmdinfogen.h
 create mode 100644 mk/rte.buildtools.mk

diff --git a/GNUmakefile b/GNUmakefile
index b59e4b6..00fe0db 100644
--- a/GNUmakefile
+++ b/GNUmakefile
@@ -40,6 +40,6 @@ export RTE_SDK
 # directory list
 #
 
-ROOTDIRS-y := lib drivers app
+ROOTDIRS-y := buildtools lib drivers app
 
 include $(RTE_SDK)/mk/rte.sdkroot.mk
diff --git a/buildtools/Makefile b/buildtools/Makefile
new file mode 100644
index 0000000..eb565eb
--- /dev/null
+++ b/buildtools/Makefile
@@ -0,0 +1,36 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+DIRS-y += pmdinfogen
+
+include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/buildtools/pmdinfogen/Makefile b/buildtools/pmdinfogen/Makefile
new file mode 100644
index 0000000..223f50c
--- /dev/null
+++ b/buildtools/pmdinfogen/Makefile
@@ -0,0 +1,49 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+#
+# library name
+#
+APP = pmdinfogen
+
+#
+# all sources are stored in SRCS-y
+#
+SRCS-y += pmdinfogen.c
+
+CFLAGS += -g
+
+DEPDIRS-y += lib/librte_eal
+
+include $(RTE_SDK)/mk/rte.buildtools.mk
+
diff --git a/buildtools/pmdinfogen/pmdinfogen.c b/buildtools/pmdinfogen/pmdinfogen.c
new file mode 100644
index 0000000..ee2b606
--- /dev/null
+++ b/buildtools/pmdinfogen/pmdinfogen.c
@@ -0,0 +1,410 @@
+/* Postprocess pmd object files to export hw support 
+ *
+ * Copyright 2016 Neil Horman <nhorman@tuxdriver.com>
+ * Based in part on modpost.c from the linux kernel
+ *
+ * This software may be used and distributed according to the terms
+ * of the GNU General Public License V2, incorporated herein by reference.
+ *
+ */
+
+#define _GNU_SOURCE
+#include <stdio.h>
+#include <ctype.h>
+#include <string.h>
+#include <limits.h>
+#include <stdbool.h>
+#include <errno.h>
+#include "pmdinfogen.h"
+
+
+static const char *sym_name(struct elf_info *elf, Elf_Sym *sym)
+{
+	if (sym)
+		return elf->strtab + sym->st_name;
+	else
+		return "(unknown)";
+}
+
+void *grab_file(const char *filename, unsigned long *size)
+{
+	struct stat st;
+	void *map = MAP_FAILED;
+	int fd;
+
+	fd = open(filename, O_RDONLY);
+	if (fd < 0)
+		return NULL;
+	if (fstat(fd, &st))
+		goto failed;
+
+	*size = st.st_size;
+	map = mmap(NULL, *size, PROT_READ|PROT_WRITE, MAP_PRIVATE, fd, 0);
+
+failed:
+	close(fd);
+	if (map == MAP_FAILED)
+		return NULL;
+	return map;
+}
+
+/**
+  * Return a copy of the next line in a mmap'ed file.
+  * spaces in the beginning of the line is trimmed away.
+  * Return a pointer to a static buffer.
+  **/
+void release_file(void *file, unsigned long size)
+{
+	munmap(file, size);
+}
+
+
+static void *get_sym_value(struct elf_info *info, const Elf_Sym *sym)
+{
+	void *ptr = (void *)info->hdr + info->sechdrs[sym->st_shndx].sh_offset;
+
+	return (void *)(ptr + sym->st_value);
+}
+
+static Elf_Sym *find_sym_in_symtab(struct elf_info *info, 
+				   const char *name, Elf_Sym *last)
+{
+	Elf_Sym *idx;
+	if (last)
+		idx = last+1;
+	else
+		idx = info->symtab_start;
+
+	for(; idx < info->symtab_stop; idx++) {
+		const char *n = sym_name(info, idx);
+		if (!strncmp(n, name, strlen(name)))
+			return idx;
+	}
+	return NULL;
+}
+
+static int parse_elf(struct elf_info *info, const char *filename)
+{
+	unsigned int i;
+	Elf_Ehdr *hdr;
+	Elf_Shdr *sechdrs;
+	Elf_Sym  *sym;
+	const char *secstrings;
+	unsigned int symtab_idx = ~0U, symtab_shndx_idx = ~0U;
+
+	hdr = grab_file(filename, &info->size);
+	if (!hdr) {
+		perror(filename);
+		exit(1);
+	}
+	info->hdr = hdr;
+	if (info->size < sizeof(*hdr)) {
+		/* file too small, assume this is an empty .o file */
+		return 0;
+	}
+	/* Is this a valid ELF file? */
+	if ((hdr->e_ident[EI_MAG0] != ELFMAG0) ||
+	    (hdr->e_ident[EI_MAG1] != ELFMAG1) ||
+	    (hdr->e_ident[EI_MAG2] != ELFMAG2) ||
+	    (hdr->e_ident[EI_MAG3] != ELFMAG3)) {
+		/* Not an ELF file - silently ignore it */
+		return 0;
+	}
+	/* Fix endianness in ELF header */
+	hdr->e_type      = TO_NATIVE(hdr->e_type);
+	hdr->e_machine   = TO_NATIVE(hdr->e_machine);
+	hdr->e_version   = TO_NATIVE(hdr->e_version);
+	hdr->e_entry     = TO_NATIVE(hdr->e_entry);
+	hdr->e_phoff     = TO_NATIVE(hdr->e_phoff);
+	hdr->e_shoff     = TO_NATIVE(hdr->e_shoff);
+	hdr->e_flags     = TO_NATIVE(hdr->e_flags);
+	hdr->e_ehsize    = TO_NATIVE(hdr->e_ehsize);
+	hdr->e_phentsize = TO_NATIVE(hdr->e_phentsize);
+	hdr->e_phnum     = TO_NATIVE(hdr->e_phnum);
+	hdr->e_shentsize = TO_NATIVE(hdr->e_shentsize);
+	hdr->e_shnum     = TO_NATIVE(hdr->e_shnum);
+	hdr->e_shstrndx  = TO_NATIVE(hdr->e_shstrndx);
+	sechdrs = (void *)hdr + hdr->e_shoff;
+	info->sechdrs = sechdrs;
+
+	/* Check if file offset is correct */
+	if (hdr->e_shoff > info->size) {
+		fprintf(stderr, "section header offset=%lu in file '%s' is bigger than "
+		      "filesize=%lu\n", (unsigned long)hdr->e_shoff,
+		      filename, info->size);
+		return 0;
+	}
+
+	if (hdr->e_shnum == SHN_UNDEF) {
+		/*
+		 * There are more than 64k sections,
+		 * read count from .sh_size.
+		 */
+		info->num_sections = TO_NATIVE(sechdrs[0].sh_size);
+	}
+	else {
+		info->num_sections = hdr->e_shnum;
+	}
+	if (hdr->e_shstrndx == SHN_XINDEX) {
+		info->secindex_strings = TO_NATIVE(sechdrs[0].sh_link);
+	}
+	else {
+		info->secindex_strings = hdr->e_shstrndx;
+	}
+
+	/* Fix endianness in section headers */
+	for (i = 0; i < info->num_sections; i++) {
+		sechdrs[i].sh_name      = TO_NATIVE(sechdrs[i].sh_name);
+		sechdrs[i].sh_type      = TO_NATIVE(sechdrs[i].sh_type);
+		sechdrs[i].sh_flags     = TO_NATIVE(sechdrs[i].sh_flags);
+		sechdrs[i].sh_addr      = TO_NATIVE(sechdrs[i].sh_addr);
+		sechdrs[i].sh_offset    = TO_NATIVE(sechdrs[i].sh_offset);
+		sechdrs[i].sh_size      = TO_NATIVE(sechdrs[i].sh_size);
+		sechdrs[i].sh_link      = TO_NATIVE(sechdrs[i].sh_link);
+		sechdrs[i].sh_info      = TO_NATIVE(sechdrs[i].sh_info);
+		sechdrs[i].sh_addralign = TO_NATIVE(sechdrs[i].sh_addralign);
+		sechdrs[i].sh_entsize   = TO_NATIVE(sechdrs[i].sh_entsize);
+	}
+	/* Find symbol table. */
+	secstrings = (void *)hdr + sechdrs[info->secindex_strings].sh_offset;
+	for (i = 1; i < info->num_sections; i++) {
+		int nobits = sechdrs[i].sh_type == SHT_NOBITS;
+
+		if (!nobits && sechdrs[i].sh_offset > info->size) {
+			fprintf(stderr, "%s is truncated. sechdrs[i].sh_offset=%lu > "
+			      "sizeof(*hrd)=%zu\n", filename,
+			      (unsigned long)sechdrs[i].sh_offset,
+			      sizeof(*hdr));
+			return 0;
+		}
+
+		if (sechdrs[i].sh_type == SHT_SYMTAB) {
+			unsigned int sh_link_idx;
+			symtab_idx = i;
+			info->symtab_start = (void *)hdr +
+			    sechdrs[i].sh_offset;
+			info->symtab_stop  = (void *)hdr +
+			    sechdrs[i].sh_offset + sechdrs[i].sh_size;
+			sh_link_idx = sechdrs[i].sh_link;
+			info->strtab       = (void *)hdr +
+			    sechdrs[sh_link_idx].sh_offset;
+		}
+
+		/* 32bit section no. table? ("more than 64k sections") */
+		if (sechdrs[i].sh_type == SHT_SYMTAB_SHNDX) {
+			symtab_shndx_idx = i;
+			info->symtab_shndx_start = (void *)hdr +
+			    sechdrs[i].sh_offset;
+			info->symtab_shndx_stop  = (void *)hdr +
+			    sechdrs[i].sh_offset + sechdrs[i].sh_size;
+		}
+	}
+	if (!info->symtab_start)
+		fprintf(stderr, "%s has no symtab?\n", filename);
+
+	/* Fix endianness in symbols */
+	for (sym = info->symtab_start; sym < info->symtab_stop; sym++) {
+		sym->st_shndx = TO_NATIVE(sym->st_shndx);
+		sym->st_name  = TO_NATIVE(sym->st_name);
+		sym->st_value = TO_NATIVE(sym->st_value);
+		sym->st_size  = TO_NATIVE(sym->st_size);
+	}
+
+	if (symtab_shndx_idx != ~0U) {
+		Elf32_Word *p;
+		if (symtab_idx != sechdrs[symtab_shndx_idx].sh_link)
+			fprintf(stderr, "%s: SYMTAB_SHNDX has bad sh_link: %u!=%u\n",
+			      filename, sechdrs[symtab_shndx_idx].sh_link,
+			      symtab_idx);
+		/* Fix endianness */
+		for (p = info->symtab_shndx_start; p < info->symtab_shndx_stop;
+		     p++)
+			*p = TO_NATIVE(*p);
+	}
+
+	return 1;
+}
+
+static void parse_elf_finish(struct elf_info *info)
+{
+	struct pmd_driver *tmp, *idx = info->drivers;
+	release_file(info->hdr, info->size);
+	while (idx) {
+		tmp = idx->next;
+		free(idx);
+		idx = tmp;
+	}
+}
+
+static const char *sec_name(struct elf_info *elf, int secindex)
+{
+	Elf_Shdr *sechdrs = elf->sechdrs;
+	return (void *)elf->hdr +
+		elf->sechdrs[elf->secindex_strings].sh_offset +
+		sechdrs[secindex].sh_name;
+}
+
+static int get_symbol_index(struct elf_info *info, Elf64_Sym *sym)
+{
+	const char *name =  sym_name(info, sym);
+	const char *idx;
+
+	idx = name;
+	while (idx) {
+		if (isdigit(*idx))
+			return atoi(idx);
+		idx++;
+	}
+	return -1;
+}
+
+struct opt_tag {
+	const char* suffix;
+	const char* json_id;
+};
+
+static const struct opt_tag opt_tags[] = {
+	{"_param_string_export", "params"},
+};
+
+static int complete_pmd_entry(struct elf_info *info, struct pmd_driver *drv)
+{
+	const char *tname;
+	int i;
+	char tmpsymname[128];
+	Elf_Sym *tmpsym;
+	
+
+	drv->name = get_sym_value(info, drv->name_sym);
+
+	for (i=0; i<PMD_OPT_MAX; i++) {
+		memset(tmpsymname, 0, 128);
+		sprintf(tmpsymname, "__%s%s", drv->name, opt_tags[i].suffix);
+		tmpsym = find_sym_in_symtab(info, tmpsymname, NULL);
+		if (!tmpsym)
+			continue;
+		drv->opt_vals[i] = get_sym_value(info, tmpsym);
+	}
+
+	memset(tmpsymname, 0, 128);
+	sprintf(tmpsymname, "__%s_pci_tbl_export", drv->name);
+
+	tmpsym = find_sym_in_symtab(info, tmpsymname, NULL);
+
+
+	/*
+ 	 * If this returns NULL, then this is a PMD_VDEV, because
+ 	 * it has no pci table reference
+ 	 */
+	if (!tmpsym) {
+		drv->pci_tbl = NULL;
+		return 0;
+	}
+
+	tname = get_sym_value(info, tmpsym);
+	tmpsym = find_sym_in_symtab(info, tname, NULL);
+	if (!tmpsym)
+		return -ENOENT;
+
+	drv->pci_tbl = (struct rte_pci_id *)get_sym_value(info, tmpsym);
+	if (!drv->pci_tbl)
+		return -ENOENT;
+
+	return 0;
+}
+
+static int locate_pmd_entries(struct elf_info *info)
+{
+	Elf_Sym *last = NULL;
+	struct pmd_driver *new;
+
+	info->drivers = NULL;
+
+	do {
+		new = calloc(sizeof(struct pmd_driver), 1);
+		new->name_sym = find_sym_in_symtab(info, "this_pmd_name", last);
+		last = new->name_sym;
+		if (!new->name_sym)
+			free(new);
+		else {
+			if (complete_pmd_entry(info, new)) {
+				fprintf(stderr, "Failed to complete pmd entry\n");
+				free(new);
+			} else {
+				new->next = info->drivers;
+				info->drivers = new;
+			}
+		}
+	} while (last);
+}
+
+static void output_pmd_info_string(struct elf_info *info, char *outfile)
+{
+	FILE *ofd;
+	struct pmd_driver *drv;
+	struct rte_pci_id *pci_ids;
+	int idx = 0;
+
+	ofd = fopen(outfile, "w+");
+	if (!ofd) {
+		fprintf(stderr, "Unable to open output file\n");
+		return;
+	}
+
+	drv = info->drivers;
+
+	while (drv) {
+		fprintf(ofd, "const char %s_pmd_info[] __attribute__((used)) = \"PMD_INFO_STRING= {",
+			drv->name);
+		fprintf(ofd,"\\\"name\\\" : \\\"%s\\\", ", drv->name);
+
+		for(idx=0; idx<PMD_OPT_MAX; idx++) {
+			if (drv->opt_vals[idx])
+				fprintf(ofd,"\\\"%s\\\" : \\\"%s\\\", ", opt_tags[idx].json_id,
+					drv->opt_vals[idx]);
+		}
+
+		pci_ids = drv->pci_tbl;
+		fprintf(ofd, "\\\"pci_ids\\\" : [");
+
+		while (pci_ids && pci_ids->device_id) {
+			fprintf(ofd, "[%d, %d, %d, %d]",
+				pci_ids->vendor_id, pci_ids->device_id,
+				pci_ids->subsystem_vendor_id,
+				pci_ids->subsystem_device_id);
+			pci_ids++;
+			if (pci_ids->device_id)
+				fprintf(ofd, ",");
+			else
+				fprintf(ofd, " ");
+		}
+		fprintf(ofd, "]}\";");
+		drv = drv->next;
+	}
+
+	fclose(ofd);
+}
+
+int main(int argc, char **argv)
+{
+	struct elf_info info;
+	int rc = 1;
+
+	if (argc < 3) {
+		fprintf(stderr, "usage: pmdinfo <object file> <c output file>\n");
+		exit(127);
+	}
+	parse_elf(&info, argv[1]);
+
+	locate_pmd_entries(&info);
+
+	if (info.drivers) {
+		output_pmd_info_string(&info, argv[2]);
+		rc = 0;
+	} else {
+		fprintf(stderr, "Hmm, Appears to be a driver but no drivers registered\n");
+	}
+
+	parse_elf_finish(&info);
+	exit(rc);
+}
diff --git a/buildtools/pmdinfogen/pmdinfogen.h b/buildtools/pmdinfogen/pmdinfogen.h
new file mode 100644
index 0000000..7b158c9
--- /dev/null
+++ b/buildtools/pmdinfogen/pmdinfogen.h
@@ -0,0 +1,94 @@
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdarg.h>
+#include <string.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/mman.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <elf.h>
+#include <rte_pci.h>
+
+/* On BSD-alike OSes elf.h defines these according to host's word size */
+#undef ELF_ST_BIND
+#undef ELF_ST_TYPE
+#undef ELF_R_SYM
+#undef ELF_R_TYPE
+
+/*
+ * Define ELF64_* to ELF_*, the latter being defined in both 32 and 64 bit
+ * flavors in elf.h.  This makes our code a bit more generic between arches
+ * and allows us to support 32 bit code in the future should we ever want to
+ */
+#if __x86_64__ || __aarch64__
+#define Elf_Ehdr    Elf64_Ehdr
+#define Elf_Shdr    Elf64_Shdr
+#define Elf_Sym     Elf64_Sym
+#define Elf_Addr    Elf64_Addr
+#define Elf_Sword   Elf64_Sxword
+#define Elf_Section Elf64_Half
+#define ELF_ST_BIND ELF64_ST_BIND
+#define ELF_ST_TYPE ELF64_ST_TYPE
+
+#define Elf_Rel     Elf64_Rel
+#define Elf_Rela    Elf64_Rela
+#define ELF_R_SYM   ELF64_R_SYM
+#define ELF_R_TYPE  ELF64_R_TYPE
+#else
+#define Elf_Ehdr    Elf32_Ehdr
+#define Elf_Shdr    Elf32_Shdr
+#define Elf_Sym     Elf32_Sym
+#define Elf_Addr    Elf32_Addr
+#define Elf_Sword   Elf32_Sxword
+#define Elf_Section Elf32_Half
+#define ELF_ST_BIND ELF32_ST_BIND
+#define ELF_ST_TYPE ELF32_ST_TYPE
+
+#define Elf_Rel     Elf32_Rel
+#define Elf_Rela    Elf32_Rela
+#define ELF_R_SYM   ELF32_R_SYM
+#define ELF_R_TYPE  ELF32_R_TYPE
+#endif
+
+#define TO_NATIVE(x) (x)
+
+enum opt_params {
+	PMD_PARAM_STRING = 0,
+	PMD_OPT_MAX
+};
+
+struct pmd_driver {
+	Elf_Sym *name_sym;
+	const char *name;
+	struct rte_pci_id *pci_tbl;
+	struct pmd_driver *next;
+
+	const char* opt_vals[PMD_OPT_MAX];
+};
+
+struct elf_info {
+	unsigned long size;
+	Elf_Ehdr     *hdr;
+	Elf_Shdr     *sechdrs;
+	Elf_Sym      *symtab_start;
+	Elf_Sym      *symtab_stop;
+	Elf_Section  export_sec;
+	Elf_Section  export_unused_sec;
+	Elf_Section  export_gpl_sec;
+	Elf_Section  export_unused_gpl_sec;
+	Elf_Section  export_gpl_future_sec;
+	char         *strtab;
+
+	/* support for 32bit section numbers */
+
+	unsigned int num_sections; /* max_secindex + 1 */
+	unsigned int secindex_strings;
+	/* if Nth symbol table entry has .st_shndx = SHN_XINDEX,
+	 * take shndx from symtab_shndx_start[N] instead */
+	Elf32_Word   *symtab_shndx_start;
+	Elf32_Word   *symtab_shndx_stop;
+
+	struct pmd_driver *drivers;
+};
+
diff --git a/mk/rte.buildtools.mk b/mk/rte.buildtools.mk
new file mode 100644
index 0000000..e8bfcef
--- /dev/null
+++ b/mk/rte.buildtools.mk
@@ -0,0 +1,148 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
+#   Copyright(c) 2014-2015 6WIND S.A.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/internal/rte.compile-pre.mk
+include $(RTE_SDK)/mk/internal/rte.install-pre.mk
+include $(RTE_SDK)/mk/internal/rte.clean-pre.mk
+include $(RTE_SDK)/mk/internal/rte.build-pre.mk
+include $(RTE_SDK)/mk/internal/rte.depdirs-pre.mk
+
+# VPATH contains at least SRCDIR
+VPATH += $(SRCDIR)
+
+_BUILD = $(APP)
+_INSTALL = $(INSTALL-FILES-y) $(SYMLINK-FILES-y)
+_INSTALL += $(RTE_OUTPUT)/buildtools/$(APP) $(RTE_OUTPUT)/buildtools/$(APP).map
+POSTINSTALL += target-appinstall
+_CLEAN = doclean
+POSTCLEAN += target-appclean
+
+.PHONY: all
+all: install
+
+.PHONY: install
+install: build _postinstall
+
+_postinstall: build
+
+.PHONY: build
+build: _postbuild
+
+exe2cmd = $(strip $(call dotfile,$(patsubst %,%.cmd,$(1))))
+
+ifeq ($(LINK_USING_CC),1)
+override EXTRA_LDFLAGS := $(call linkerprefix,$(EXTRA_LDFLAGS))
+O_TO_EXE = $(CC) $(CFLAGS) $(LDFLAGS_$(@)) \
+	-Wl,-Map=$(@).map,--cref -o $@ $(OBJS-y) $(call linkerprefix,$(LDFLAGS)) \
+	$(EXTRA_LDFLAGS) $(call linkerprefix,$(LDLIBS))
+else
+O_TO_EXE = $(LD) $(LDFLAGS) $(LDFLAGS_$(@)) $(EXTRA_LDFLAGS) \
+	-Map=$(@).map --cref -o $@ $(OBJS-y) $(LDLIBS)
+endif
+O_TO_EXE_STR = $(subst ','\'',$(O_TO_EXE)) #'# fix syntax highlight
+O_TO_EXE_DISP = $(if $(V),"$(O_TO_EXE_STR)","  LD $(@)")
+O_TO_EXE_CMD = "cmd_$@ = $(O_TO_EXE_STR)"
+O_TO_EXE_DO = @set -e; \
+	echo $(O_TO_EXE_DISP); \
+	$(O_TO_EXE) && \
+	echo $(O_TO_EXE_CMD) > $(call exe2cmd,$(@))
+
+-include .$(APP).cmd
+
+# path where libraries are retrieved
+LDLIBS_PATH := $(subst -Wl$(comma)-L,,$(filter -Wl$(comma)-L%,$(LDLIBS)))
+LDLIBS_PATH += $(subst -L,,$(filter -L%,$(LDLIBS)))
+
+# list of .a files that are linked to this application
+LDLIBS_NAMES := $(patsubst -l%,lib%.a,$(filter -l%,$(LDLIBS)))
+LDLIBS_NAMES += $(patsubst -Wl$(comma)-l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIBS)))
+
+# list of found libraries files (useful for deps). If not found, the
+# library is silently ignored and dep won't be checked
+LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
+	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
+
+#
+# Compile executable file if needed
+#
+$(APP): $(OBJS-y) $(LDLIBS_FILES) $(DEP_$(APP)) $(LDSCRIPT) FORCE
+	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
+	$(if $(D),\
+		@echo -n "$< -> $@ " ; \
+		echo -n "file_missing=$(call boolean,$(file_missing)) " ; \
+		echo -n "cmdline_changed=$(call boolean,$(call cmdline_changed,$(O_TO_EXE_STR))) " ; \
+		echo -n "depfile_missing=$(call boolean,$(depfile_missing)) " ; \
+		echo "depfile_newer=$(call boolean,$(depfile_newer)) ")
+	$(if $(or \
+		$(file_missing),\
+		$(call cmdline_changed,$(O_TO_EXE_STR)),\
+		$(depfile_missing),\
+		$(depfile_newer)),\
+		$(O_TO_EXE_DO))
+
+#
+# install app in $(RTE_OUTPUT)/app
+#
+$(RTE_OUTPUT)/buildtools/$(APP): $(APP)
+	@echo "  INSTALL-APP $(APP)"
+	@[ -d $(RTE_OUTPUT)/buildtools ] || mkdir -p $(RTE_OUTPUT)/buildtools
+	$(Q)cp -f $(APP) $(RTE_OUTPUT)/buildtools
+
+#
+# install app map file in $(RTE_OUTPUT)/app
+#
+$(RTE_OUTPUT)/buildtools/$(APP).map: $(APP)
+	@echo "  INSTALL-MAP $(APP).map"
+	@[ -d $(RTE_OUTPUT)/buildtools ] || mkdir -p $(RTE_OUTPUT)/buildtools
+	$(Q)cp -f $(APP).map $(RTE_OUTPUT)/buildtools
+
+#
+# Clean all generated files
+#
+.PHONY: clean
+clean: _postclean
+	$(Q)rm -f $(_BUILD_TARGETS) $(_INSTALL_TARGETS) $(_CLEAN_TARGETS)
+
+.PHONY: doclean
+doclean:
+	$(Q)rm -rf $(APP) $(OBJS-all) $(DEPS-all) $(DEPSTMP-all) \
+	  $(CMDS-all) $(INSTALL-FILES-all) .$(APP).cmd
+
+
+include $(RTE_SDK)/mk/internal/rte.compile-post.mk
+include $(RTE_SDK)/mk/internal/rte.install-post.mk
+include $(RTE_SDK)/mk/internal/rte.clean-post.mk
+include $(RTE_SDK)/mk/internal/rte.build-post.mk
+include $(RTE_SDK)/mk/internal/rte.depdirs-post.mk
+
+.PHONY: FORCE
+FORCE:
diff --git a/mk/rte.sdkbuild.mk b/mk/rte.sdkbuild.mk
index eec5241..fb68af2 100644
--- a/mk/rte.sdkbuild.mk
+++ b/mk/rte.sdkbuild.mk
@@ -64,7 +64,8 @@ build: $(ROOTDIRS-y)
 clean: $(CLEANDIRS)
 	@rm -rf $(RTE_OUTPUT)/include $(RTE_OUTPUT)/app \
 		$(RTE_OUTPUT)/hostapp $(RTE_OUTPUT)/lib \
-		$(RTE_OUTPUT)/hostlib $(RTE_OUTPUT)/kmod
+		$(RTE_OUTPUT)/hostlib $(RTE_OUTPUT)/kmod \
+		$(RTE_OUTPUT)/buildtools
 	@[ -d $(RTE_OUTPUT)/include ] || mkdir -p $(RTE_OUTPUT)/include
 	@$(RTE_SDK)/scripts/gen-config-h.sh $(RTE_OUTPUT)/.config \
 		> $(RTE_OUTPUT)/include/rte_config.h
-- 
2.5.5

[PATCHv5 2/6] drivers: Update driver registration macro usage

[Neil Horman]

Modify the PMD_REGISTER_DRIVER macro, adding a name argument to it.  The
addition of a name argument creates a token that can be used for subsequent
macros in the creation of unique symbol names to export additional bits of
information for use by the pmdinfogen tool.  For example:

PMD_REGISTER_DRIVER(ena_driver, ena);

registers the ena_driver struct as it always did, and creates a symbol
const char this_pmd_name0[] __attribute__((used)) = "ena";

which pmdinfogen can search for and extract.  The subsequent macro

DRIVER_REGISTER_PCI_TABLE(ena, ena_pci_id_map);

creates a symbol
const char ena_pci_tbl_export[] __attribute__((used)) = "ena_pci_id_map";

Which allows pmdinfogen to find the pci table of this driver

Using this pattern, we can export arbitrary bits of information.

pmdinfo uses this information to extract hardware support from an object file
and create a json string to make hardware support info discoverable later.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 drivers/Makefile                           |  2 ++
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c   |  4 +++-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c |  4 +++-
 drivers/crypto/null/null_crypto_pmd.c      |  4 +++-
 drivers/crypto/qat/rte_qat_cryptodev.c     |  4 +++-
 drivers/crypto/snow3g/rte_snow3g_pmd.c     |  4 +++-
 drivers/net/af_packet/rte_eth_af_packet.c  |  4 +++-
 drivers/net/bnx2x/bnx2x_ethdev.c           |  6 ++++--
 drivers/net/bonding/rte_eth_bond_pmd.c     |  7 ++++++-
 drivers/net/cxgbe/cxgbe_ethdev.c           |  4 +++-
 drivers/net/e1000/em_ethdev.c              |  3 ++-
 drivers/net/e1000/igb_ethdev.c             |  6 ++++--
 drivers/net/ena/ena_ethdev.c               |  3 ++-
 drivers/net/enic/enic_ethdev.c             |  3 ++-
 drivers/net/fm10k/fm10k_ethdev.c           |  3 ++-
 drivers/net/i40e/i40e_ethdev.c             |  3 ++-
 drivers/net/i40e/i40e_ethdev_vf.c          |  3 ++-
 drivers/net/ixgbe/ixgbe_ethdev.c           |  6 ++++--
 drivers/net/mlx4/mlx4.c                    |  3 ++-
 drivers/net/mlx5/mlx5.c                    |  3 ++-
 drivers/net/mpipe/mpipe_tilegx.c           |  4 ++--
 drivers/net/nfp/nfp_net.c                  |  3 ++-
 drivers/net/null/rte_eth_null.c            |  3 ++-
 drivers/net/pcap/rte_eth_pcap.c            |  4 +++-
 drivers/net/qede/qede_ethdev.c             |  6 ++++--
 drivers/net/ring/rte_eth_ring.c            |  3 ++-
 drivers/net/szedata2/rte_eth_szedata2.c    |  3 ++-
 drivers/net/vhost/rte_eth_vhost.c          |  3 ++-
 drivers/net/virtio/virtio_ethdev.c         |  3 ++-
 drivers/net/vmxnet3/vmxnet3_ethdev.c       |  3 ++-
 drivers/net/xenvirt/rte_eth_xenvirt.c      |  2 +-
 lib/librte_eal/common/include/rte_dev.h    | 27 +++++++++++++++++++++------
 32 files changed, 102 insertions(+), 41 deletions(-)

diff --git a/drivers/Makefile b/drivers/Makefile
index 81c03a8..75a3168 100644
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -34,4 +34,6 @@ include $(RTE_SDK)/mk/rte.vars.mk
 DIRS-y += net
 DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += crypto
 
+DEPDIRS-y += buildtools/pmdinfo
+
 include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
index 2987ef6..593769f 100644
--- a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
+++ b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
@@ -521,4 +521,6 @@ static struct rte_driver aesni_gcm_pmd_drv = {
 	.uninit = aesni_gcm_uninit
 };
 
-PMD_REGISTER_DRIVER(aesni_gcm_pmd_drv);
+PMD_REGISTER_DRIVER(aesni_gcm_pmd_drv, aesni_gcm);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> \
+max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
index 3415ac1..2e30939 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
@@ -716,4 +716,6 @@ static struct rte_driver cryptodev_aesni_mb_pmd_drv = {
 	.uninit = cryptodev_aesni_mb_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drv);
+PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drvi, aesni_mb);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> \
+max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/null/null_crypto_pmd.c b/drivers/crypto/null/null_crypto_pmd.c
index bdaf13c..44c4fba 100644
--- a/drivers/crypto/null/null_crypto_pmd.c
+++ b/drivers/crypto/null/null_crypto_pmd.c
@@ -275,4 +275,6 @@ static struct rte_driver cryptodev_null_pmd_drv = {
 	.uninit = cryptodev_null_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_null_pmd_drv);
+PMD_REGISTER_DRIVER(cryptodev_null_pmd_drv, cryptodev_null_pmd);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> \
+max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index a7912f5..d2de6a6 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -137,4 +137,6 @@ static struct rte_driver pmd_qat_drv = {
 	.init = rte_qat_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(pmd_qat_drv);
+PMD_REGISTER_DRIVER(pmd_qat_drv, qat);
+DRIVER_REGISTER_PCI_TABLE(qat, pci_id_qat_map);
+
diff --git a/drivers/crypto/snow3g/rte_snow3g_pmd.c b/drivers/crypto/snow3g/rte_snow3g_pmd.c
index f3e0e66..eee2544 100644
--- a/drivers/crypto/snow3g/rte_snow3g_pmd.c
+++ b/drivers/crypto/snow3g/rte_snow3g_pmd.c
@@ -548,4 +548,6 @@ static struct rte_driver cryptodev_snow3g_pmd_drv = {
 	.uninit = cryptodev_snow3g_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_snow3g_pmd_drv);
+PMD_REGISTER_DRIVER(cryptodev_snow3g_pmd_drv, snow3g);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> \
+max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index f17bd7e..e960512 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -878,4 +878,6 @@ static struct rte_driver pmd_af_packet_drv = {
 	.uninit = rte_pmd_af_packet_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_af_packet_drv);
+PMD_REGISTER_DRIVER(pmd_af_packet_drv, eth_af_packet);
+DRIVER_REGISTER_PARAM_STRING(eth_af_packet, "iface=<string> \
+qpairs=<int> blocksz=<int> framesz=<int> framecnt=<int>");
diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c b/drivers/net/bnx2x/bnx2x_ethdev.c
index 071b44f..2b88324 100644
--- a/drivers/net/bnx2x/bnx2x_ethdev.c
+++ b/drivers/net/bnx2x/bnx2x_ethdev.c
@@ -550,5 +550,7 @@ static struct rte_driver rte_bnx2xvf_driver = {
 	.init = rte_bnx2xvf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_bnx2x_driver);
-PMD_REGISTER_DRIVER(rte_bnx2xvf_driver);
+PMD_REGISTER_DRIVER(rte_bnx2x_driver, bnx2x);
+DRIVER_REGISTER_PCI_TABLE(bnx2x, pci_id_bnx2x_map);
+PMD_REGISTER_DRIVER(rte_bnx2xvf_driver, bnx2xvf);
+DRIVER_REGISTER_PCI_TABLE(bnx2xvf, pci_id_bnx2xvf_map);
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 129f04b..b4556be 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -2511,4 +2511,9 @@ static struct rte_driver bond_drv = {
 	.uninit = bond_uninit,
 };
 
-PMD_REGISTER_DRIVER(bond_drv);
+PMD_REGISTER_DRIVER(bond_drv, bonding);
+
+DRIVER_REGISTER_PARAM_STRING(bonding, "slave=<ifc> primary=<ifc> mode=[0-4] \
+xmit_policy=[l2 | l23 | l34] socket_id=<int> mac=<mac addr> \
+lsc_poll_period_ms=<int> up_delay=<int> down_delay=<int>");
+
diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index 04eddaf..e84e48b 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -894,4 +894,6 @@ static struct rte_driver rte_cxgbe_driver = {
 	.init = rte_cxgbe_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_cxgbe_driver);
+PMD_REGISTER_DRIVER(rte_cxgbe_driver, cxgb4);
+DRIVER_REGISTER_PCI_TABLE(cxgb4, cxgb4_pci_tbl);
+
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index 653be09..4de5eb2 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -1777,4 +1777,5 @@ struct rte_driver em_pmd_drv = {
 	.init = rte_em_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(em_pmd_drv);
+PMD_REGISTER_DRIVER(em_pmd_drv, em);
+DRIVER_REGISTER_PCI_TABLE(em, pci_id_em_map);
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index f0921ee..f49aae7 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -4973,5 +4973,7 @@ eth_igb_configure_msix_intr(struct rte_eth_dev *dev)
 	E1000_WRITE_FLUSH(hw);
 }
 
-PMD_REGISTER_DRIVER(pmd_igb_drv);
-PMD_REGISTER_DRIVER(pmd_igbvf_drv);
+PMD_REGISTER_DRIVER(pmd_igb_drv, igb);
+DRIVER_REGISTER_PCI_TABLE(igb, pci_id_igb_map);
+PMD_REGISTER_DRIVER(pmd_igbvf_drv, igbvf);
+DRIVER_REGISTER_PCI_TABLE(igbvf, pci_id_igbvf_map);
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index e157587..702289b 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -1450,4 +1450,5 @@ struct rte_driver ena_pmd_drv = {
 	.init = rte_ena_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(ena_pmd_drv);
+PMD_REGISTER_DRIVER(ena_pmd_drv, ena);
+DRIVER_REGISTER_PCI_TABLE(ena, pci_id_ena_map);
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index 6bea940..36ee7ca 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -684,4 +684,5 @@ static struct rte_driver rte_enic_driver = {
 	.init = rte_enic_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_enic_driver);
+PMD_REGISTER_DRIVER(rte_enic_driver, enic);
+DRIVER_REGISTER_PCI_TABLE(enic, pci_id_enic_map);
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index c2d377f..9b2a7ee 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -3055,4 +3055,5 @@ static struct rte_driver rte_fm10k_driver = {
 	.init = rte_pmd_fm10k_init,
 };
 
-PMD_REGISTER_DRIVER(rte_fm10k_driver);
+PMD_REGISTER_DRIVER(rte_fm10k_driver, fm10k);
+DRIVER_REGISTER_PCI_TABLE(fm10k, pci_id_fm10k_map);
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 24777d5..3b6c661 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -695,7 +695,8 @@ static struct rte_driver rte_i40e_driver = {
 	.init = rte_i40e_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_i40e_driver);
+PMD_REGISTER_DRIVER(rte_i40e_driver, i40e);
+DRIVER_REGISTER_PCI_TABLE(i40e, pci_id_i40e_map);
 
 /*
  * Initialize registers for flexible payload, which should be set by NVM.
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 90682ac..e94a459 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1563,7 +1563,8 @@ static struct rte_driver rte_i40evf_driver = {
 	.init = rte_i40evf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_i40evf_driver);
+PMD_REGISTER_DRIVER(rte_i40evf_driver, i40evf);
+DRIVER_REGISTER_PCI_TABLE(i40evf, pci_id_i40evf_map);
 
 static int
 i40evf_dev_configure(struct rte_eth_dev *dev)
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index a2b170b..af1ca56 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -7164,5 +7164,7 @@ static struct rte_driver rte_ixgbevf_driver = {
 	.init = rte_ixgbevf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_ixgbe_driver);
-PMD_REGISTER_DRIVER(rte_ixgbevf_driver);
+PMD_REGISTER_DRIVER(rte_ixgbe_driver, ixgbe);
+DRIVER_REGISTER_PCI_TABLE(ixgbe, pci_id_ixgbe_map);
+PMD_REGISTER_DRIVER(rte_ixgbevf_driver, ixgbevf);
+DRIVER_REGISTER_PCI_TABLE(ixgbevf, pci_id_ixgbevf_map);
diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index 9ed1491..aedf1e9 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -5865,4 +5865,5 @@ static struct rte_driver rte_mlx4_driver = {
 	.init = rte_mlx4_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_mlx4_driver)
+PMD_REGISTER_DRIVER(rte_mlx4_driveri, mlx4)
+DRIVER_REGISTER_PCI_TABLE(mlx4, mlx4_pci_id_map);
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 041cfc3..5f2d6c8 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -672,4 +672,5 @@ static struct rte_driver rte_mlx5_driver = {
 	.init = rte_mlx5_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_mlx5_driver)
+PMD_REGISTER_DRIVER(rte_mlx5_driveri, mlx5)
+DRIVER_REGISTER_PCI_TABLE(mlx5, mlx5_pci_id_map);
diff --git a/drivers/net/mpipe/mpipe_tilegx.c b/drivers/net/mpipe/mpipe_tilegx.c
index 26e1424..4a2a905 100644
--- a/drivers/net/mpipe/mpipe_tilegx.c
+++ b/drivers/net/mpipe/mpipe_tilegx.c
@@ -1635,8 +1635,8 @@ static struct rte_driver pmd_mpipe_gbe_drv = {
 	.init = rte_pmd_mpipe_devinit,
 };
 
-PMD_REGISTER_DRIVER(pmd_mpipe_xgbe_drv);
-PMD_REGISTER_DRIVER(pmd_mpipe_gbe_drv);
+PMD_REGISTER_DRIVER(pmd_mpipe_xgbe_drv, mpipe_xgbe);
+PMD_REGISTER_DRIVER(pmd_mpipe_gbe_drv, mpipe_gbe);
 
 static void __attribute__((constructor, used))
 mpipe_init_contexts(void)
diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
index ea5a2a3..78394a1 100644
--- a/drivers/net/nfp/nfp_net.c
+++ b/drivers/net/nfp/nfp_net.c
@@ -2490,7 +2490,8 @@ static struct rte_driver rte_nfp_net_driver = {
 	.init = nfp_net_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_nfp_net_driver);
+PMD_REGISTER_DRIVER(rte_nfp_net_driver, nfp);
+DRIVER_REGISTER_PCI_TABLE(nfp, pci_id_nfp_net_map);
 
 /*
  * Local variables:
diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index 5e8e203..88eb582 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -689,4 +689,5 @@ static struct rte_driver pmd_null_drv = {
 	.uninit = rte_pmd_null_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_null_drv);
+PMD_REGISTER_DRIVER(pmd_null_drv, eth_null);
+DRIVER_REGISTER_PARAM_STRING(eth_null, "size=<int> copy=<int>");
diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
index c98e234..397f01c 100644
--- a/drivers/net/pcap/rte_eth_pcap.c
+++ b/drivers/net/pcap/rte_eth_pcap.c
@@ -1090,4 +1090,6 @@ static struct rte_driver pmd_pcap_drv = {
 	.uninit = rte_pmd_pcap_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_pcap_drv);
+PMD_REGISTER_DRIVER(pmd_pcap_drv, pcap);
+DRIVER_REGISTER_PARAM_STRING(pcap, "rx_pcap=<string> tx_pcap=<string> \
+rx_iface=<ifc> tx_iface=<ifc> iface=<ifc>");
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index 1273fd3..187ccf1 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -1102,5 +1102,7 @@ static struct rte_driver rte_qede_driver = {
 	.init = rte_qedevf_pmd_init
 };
 
-PMD_REGISTER_DRIVER(rte_qede_driver);
-PMD_REGISTER_DRIVER(rte_qedevf_driver);
+PMD_REGISTER_DRIVER(rte_qede_driver, qede);
+DRIVER_REGISTER_PCI_TABLE(qede, pci_id_qede_map);
+PMD_REGISTER_DRIVER(rte_qedevf_driver, qedevf);
+DRIVER_REGISTER_PCI_TABLE(qedevf, pci_id_qedevf_map);
diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index b1783c3..f2f5217 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -630,4 +630,5 @@ static struct rte_driver pmd_ring_drv = {
 	.uninit = rte_pmd_ring_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_ring_drv);
+PMD_REGISTER_DRIVER(pmd_ring_drv, eth_ring);
+DRIVER_REGISTER_PARAM_STRING(eth_ring, "nodeaction=[attach|detach]");
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 78c43b0..abbc907 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1601,4 +1601,5 @@ static struct rte_driver rte_szedata2_driver = {
 	.uninit = rte_szedata2_uninit,
 };
 
-PMD_REGISTER_DRIVER(rte_szedata2_driver);
+PMD_REGISTER_DRIVER(rte_szedata2_driver, rte_szedata2_pmd);
+DRIVER_REGISTER_PCI_TABLE(rte_szedata2_pmd, rte_szedata2_pci_id_table);
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index 310cbef..d321c3e 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -924,4 +924,5 @@ static struct rte_driver pmd_vhost_drv = {
 	.uninit = rte_pmd_vhost_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_vhost_drv);
+PMD_REGISTER_DRIVER(pmd_vhost_drv, eth_vhost);
+DRIVER_REGISTER_PARAM_STRING(eth_vhost, "iface=<ifc> queues=<int>");
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index c3fb628..24d387f 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -1459,4 +1459,5 @@ static struct rte_driver rte_virtio_driver = {
 	.init = rte_virtio_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_virtio_driver);
+PMD_REGISTER_DRIVER(rte_virtio_driver, virtio_net);
+DRIVER_REGISTER_PCI_TABLE(virtio_net, pci_id_virtio_map);
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index 29b469c..8da4449 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -954,4 +954,5 @@ static struct rte_driver rte_vmxnet3_driver = {
 	.init = rte_vmxnet3_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_vmxnet3_driver);
+PMD_REGISTER_DRIVER(rte_vmxnet3_driver, vmxnet3);
+DRIVER_REGISTER_PCI_TABLE(vmxnet3, pci_id_vmxnet3_map);
diff --git a/drivers/net/xenvirt/rte_eth_xenvirt.c b/drivers/net/xenvirt/rte_eth_xenvirt.c
index 1adeb5b..81f8af9 100644
--- a/drivers/net/xenvirt/rte_eth_xenvirt.c
+++ b/drivers/net/xenvirt/rte_eth_xenvirt.c
@@ -763,4 +763,4 @@ static struct rte_driver pmd_xenvirt_drv = {
 	.uninit = rte_pmd_xenvirt_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_xenvirt_drv);
+PMD_REGISTER_DRIVER(pmd_xenvirt_drv, xenvirt);
diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
index f1b5507..281d20d 100644
--- a/lib/librte_eal/common/include/rte_dev.h
+++ b/lib/librte_eal/common/include/rte_dev.h
@@ -48,7 +48,7 @@ extern "C" {
 
 #include <stdio.h>
 #include <sys/queue.h>
-
+#include <rte_pci.h>
 #include <rte_log.h>
 
 __attribute__((format(printf, 2, 0)))
@@ -178,12 +178,27 @@ int rte_eal_vdev_init(const char *name, const char *args);
  */
 int rte_eal_vdev_uninit(const char *name);
 
-#define PMD_REGISTER_DRIVER(d)\
-void devinitfn_ ##d(void);\
-void __attribute__((constructor, used)) devinitfn_ ##d(void)\
+#define DRIVER_EXPORT_NAME_ARRAY(n, idx) n##idx[] __attribute__((used))
+
+#define DRIVER_EXPORT_NAME(name, idx) \
+static const char DRIVER_EXPORT_NAME_ARRAY(this_pmd_name, idx) = RTE_STR(name);\
+ 
+#define PMD_REGISTER_DRIVER(drv, nm)\
+void devinitfn_ ##drv(void);\
+void __attribute__((constructor, used)) devinitfn_ ##drv(void)\
 {\
-	rte_eal_driver_register(&d);\
-}
+	(drv).name = RTE_STR(nm);\
+        rte_eal_driver_register(&drv);\
+}\
+DRIVER_EXPORT_NAME(nm, __COUNTER__)
+
+#define DRV_EXP_TAG(name, tag) __##name##_##tag
+
+#define DRIVER_REGISTER_PCI_TABLE(name, table) \
+static const char DRV_EXP_TAG(name, pci_tbl_export)[] __attribute__((used)) = RTE_STR(table)
+
+#define DRIVER_REGISTER_PARAM_STRING(name, str) \
+static const char DRV_EXP_TAG(name, param_string_export)[] __attribute__((used)) = str
 
 #ifdef __cplusplus
 }
-- 
2.5.5

[PATCHv5 3/6] eal: Add an export symbol to expose the autoload path to external tools

[Neil Horman]

Export a symbol containing the string:
DPDK_PLUGIN_PATH="$(CONFIG_RTE_EAL_PMD_PATH)"

Where the latter half of the string is set at build time to a location from
which autoloaded DSO's will be found.  This string is used by pmdinfo in
'plugin' mode, whereby a user can specify a dpdk installation directory (or
static binary), and scan the associated path (if found) for pmd DSO's and report
on their hardware support.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 lib/librte_eal/common/eal_common_options.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c
index 3efc90f..1af0fb3 100644
--- a/lib/librte_eal/common/eal_common_options.c
+++ b/lib/librte_eal/common/eal_common_options.c
@@ -115,6 +115,15 @@ TAILQ_HEAD_INITIALIZER(solib_list);
 /* Default path of external loadable drivers */
 static const char *default_solib_dir = RTE_EAL_PMD_PATH;
 
+/*
+ * Stringified version of solib path used by pmdinfo.py 
+ * Note: PLEASE DO NOT ALTER THIS without making a corresponding
+ * change to tools/pmdinfo.py
+ */
+static const char dpdk_solib_path[] __attribute__((used)) = \
+"DPDK_PLUGIN_PATH=" RTE_EAL_PMD_PATH;
+
+
 static int master_lcore_parsed;
 static int mem_parsed;
 
-- 
2.5.5

[PATCHv5 4/6] Makefile: Do post processing on objects that register a driver

[Neil Horman]

Modify the compilation makefile to identify C files that export PMD information,
and use that to trigger execution of the pmdinfo binary.  If the execution of
pmdinfo is successful, compile the output C file to an object, and use the
linker to do relocatable linking on the resultant object file into the parent
object that it came from.  This effectively just adds the json string into the
string table of the object that defines the PMD to the outside world.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 mk/internal/rte.compile-pre.mk | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/mk/internal/rte.compile-pre.mk b/mk/internal/rte.compile-pre.mk
index b9bff4a..92517ad 100644
--- a/mk/internal/rte.compile-pre.mk
+++ b/mk/internal/rte.compile-pre.mk
@@ -88,10 +88,24 @@ C_TO_O_CMD = 'cmd_$@ = $(C_TO_O_STR)'
 C_TO_O_DO = @set -e; \
 	echo $(C_TO_O_DISP); \
 	$(C_TO_O) && \
+	sh -c "grep -q \"PMD_REGISTER_DRIVER(.*)\" $<; \
+	if [ \$$? -eq 0 ]; then \
+		echo \"  PMDINFOGEN\" $@; \
+		OBJF=`readlink -f $@`; \
+		${RTE_OUTPUT}/buildtools/pmdinfogen \$$OBJF \$$OBJF.pmd.c; \
+		if [ \$$? -eq 0 ]; \
+		then \
+			echo \"  PMDINFOBUILD\" $@; \
+			$(CC) $(CFLAGS) -c -o \$$OBJF.pmd.o \$$OBJF.pmd.c; \
+			$(CROSS)ld $(LDFLAGS) -r -o \$$OBJF.o \$$OBJF.pmd.o \$$OBJF; \
+			mv -f \$$OBJF.o \$$OBJF; \
+		fi; \
+	fi;" && \
 	echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \
 	sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \
 	rm -f $(call obj2dep,$(@)).tmp
 
+
 # return an empty string if string are equal
 compare = $(strip $(subst $(1),,$(2)) $(subst $(2),,$(1)))
 
-- 
2.5.5

[PATCHv5 5/6] pmdinfo.py: Add tool to query binaries for hw and other support information

[Neil Horman]

This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
and, if found parses the remainder of the string as a json encoded string,
outputting the results in either a human readable or raw, script parseable
format

Note that, in the case of dynamically linked applications, pmdinfo.py will scan
for implicitly linked PMDs by searching the specified binaries .dynamic section
for DT_NEEDED entries that contain the substring librte_pmd.  The DT_RUNPATH,
LD_LIBRARY_PATH, /usr/lib and /lib are searched for these libraries, in that
order

If a file is specified with no path, it is assumed to be a PMD DSO, and the
LD_LIBRARY_PATH, /usr/lib[64]/ and /lib[64] is searched for it

Currently the tool can output data in 3 formats:

a) raw, suitable for scripting, where the raw JSON strings are dumped out
b) table format (default) where hex pci ids are dumped in a table format
c) pretty, where a user supplied pci.ids file is used to print out vendor and
device strings

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 mk/rte.sdkinstall.mk |   2 +
 tools/pmdinfo.py     | 617 +++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 619 insertions(+)
 create mode 100755 tools/pmdinfo.py

diff --git a/mk/rte.sdkinstall.mk b/mk/rte.sdkinstall.mk
index 68e56b6..dc36df5 100644
--- a/mk/rte.sdkinstall.mk
+++ b/mk/rte.sdkinstall.mk
@@ -126,6 +126,8 @@ install-runtime:
 	$(Q)$(call rte_mkdir,      $(DESTDIR)$(sbindir))
 	$(Q)$(call rte_symlink,    $(DESTDIR)$(datadir)/tools/dpdk_nic_bind.py, \
 	                           $(DESTDIR)$(sbindir)/dpdk_nic_bind)
+	$(Q)$(call rte_symlink,    $(DESTDIR)$(datadir)/tools/pmdinfo.py, \
+				   $(DESTDIR)$(bindir)/dpdk-pmdinfo)
 
 install-kmod:
 ifneq ($(wildcard $O/kmod/*),)
diff --git a/tools/pmdinfo.py b/tools/pmdinfo.py
new file mode 100755
index 0000000..169121a
--- /dev/null
+++ b/tools/pmdinfo.py
@@ -0,0 +1,617 @@
+#!/usr/bin/python
+#-------------------------------------------------------------------------------
+# scripts/pmd_hw_support.py
+#
+# Utility to dump PMD_INFO_STRING support from an object file
+#
+#-------------------------------------------------------------------------------
+import os, sys
+from optparse import OptionParser
+import string
+import json
+
+# For running from development directory. It should take precedence over the
+# installed pyelftools.
+sys.path.insert(0, '.')
+
+
+from elftools import __version__
+from elftools.common.exceptions import ELFError
+from elftools.common.py3compat import (
+        ifilter, byte2int, bytes2str, itervalues, str2bytes)
+from elftools.elf.elffile import ELFFile
+from elftools.elf.dynamic import DynamicSection, DynamicSegment
+from elftools.elf.enums import ENUM_D_TAG
+from elftools.elf.segments import InterpSegment
+from elftools.elf.sections import SymbolTableSection
+from elftools.elf.gnuversions import (
+    GNUVerSymSection, GNUVerDefSection,
+    GNUVerNeedSection,
+    )
+from elftools.elf.relocation import RelocationSection
+from elftools.elf.descriptions import (
+    describe_ei_class, describe_ei_data, describe_ei_version,
+    describe_ei_osabi, describe_e_type, describe_e_machine,
+    describe_e_version_numeric, describe_p_type, describe_p_flags,
+    describe_sh_type, describe_sh_flags,
+    describe_symbol_type, describe_symbol_bind, describe_symbol_visibility,
+    describe_symbol_shndx, describe_reloc_type, describe_dyn_tag,
+    describe_ver_flags,
+    )
+from elftools.elf.constants import E_FLAGS
+from elftools.dwarf.dwarfinfo import DWARFInfo
+from elftools.dwarf.descriptions import (
+    describe_reg_name, describe_attr_value, set_global_machine_arch,
+    describe_CFI_instructions, describe_CFI_register_rule,
+    describe_CFI_CFA_rule,
+    )
+from elftools.dwarf.constants import (
+    DW_LNS_copy, DW_LNS_set_file, DW_LNE_define_file)
+from elftools.dwarf.callframe import CIE, FDE
+
+raw_output = False
+pcidb = None
+
+#===========================================
+
+class Vendor:
+    """
+    Class for vendors. This is the top level class
+    for the devices belong to a specific vendor.
+    self.devices is the device dictionary
+    subdevices are in each device.
+    """
+    def __init__(self, vendorStr):
+        """
+        Class initializes with the raw line from pci.ids
+        Parsing takes place inside __init__
+        """
+        self.ID = vendorStr.split()[0]
+        self.name = vendorStr.replace("%s " % self.ID,"").rstrip()
+        self.devices = {}
+
+    def addDevice(self, deviceStr):
+        """
+        Adds a device to self.devices
+        takes the raw line from pci.ids
+        """
+        s = deviceStr.strip()
+        devID = s.split()[0]
+        if devID in self.devices:
+            pass
+        else:
+            self.devices[devID] = Device(deviceStr)
+
+    def report(self):
+        print self.ID, self.name
+        for id, dev in self.devices.items():
+            dev.report()
+
+    def find_device(self, devid):
+        # convert to a hex string and remove 0x
+        devid = hex(devid)[2:]
+        try:
+            return self.devices[devid]
+        except:
+            return Device("%s  Unknown Device" % devid) 
+
+class Device:
+    def __init__(self, deviceStr):
+        """
+        Class for each device.
+        Each vendor has its own devices dictionary.
+        """
+        s = deviceStr.strip()
+        self.ID = s.split()[0]
+        self.name = s.replace("%s  " % self.ID,"")
+        self.subdevices = {}
+
+    def report(self):
+        print "\t%s\t%s" % (self.ID, self.name)
+        for subID, subdev in self.subdevices.items():
+            subdev.report()
+
+    def addSubDevice(self, subDeviceStr):
+        """
+        Adds a subvendor, subdevice to device.
+        Uses raw line from pci.ids
+        """
+        s = subDeviceStr.strip()
+        spl = s.split()
+        subVendorID  = spl[0]
+        subDeviceID  = spl[1]
+        subDeviceName = s.split("  ")[-1]
+        devID = "%s:%s" % (subVendorID,subDeviceID)
+        self.subdevices[devID] = SubDevice(subVendorID,subDeviceID,subDeviceName)
+
+    def find_subid(self, subven, subdev):
+        subven = hex(subven)[2:]
+        subdev = hex(subdev)[2:]
+        devid ="%s:%s" % (subven, subdev)
+
+        try:
+            return self.subdevices[devid]
+        except:
+            if (subven == "ffff" and subdev == "ffff"):
+                return SubDevice("ffff", "ffff", "(All Subdevices)");
+            else:
+                return SubDevice(subven, subdev, "(Unknown Subdevice)")
+        
+
+class SubDevice:
+    """
+    Class for subdevices.
+    """
+    def __init__(self, vendor, device, name):
+        """
+        Class initializes with vendorid, deviceid and name
+        """
+        self.vendorID = vendor
+        self.deviceID = device
+        self.name = name
+
+    def report(self):
+        print "\t\t%s\t%s\t%s" % (self.vendorID, self.deviceID,self.name)
+
+class PCIIds:
+    """
+    Top class for all pci.ids entries.
+    All queries will be asked to this class.
+    PCIIds.vendors["0e11"].devices["0046"].subdevices["0e11:4091"].name  =  "Smart Array 6i"
+    """
+    def __init__(self, filename):
+        """
+        Prepares the directories.
+        Checks local data file.
+        Tries to load from local, if not found, downloads from web
+        """
+        self.version = ""
+        self.date = ""
+        self.vendors = {}
+        self.contents = None
+        self.readLocal(filename)
+        self.parse()
+
+    def reportVendors(self):
+        """Reports the vendors
+        """
+        for vid, v in self.vendors.items():
+            print v.ID, v.name
+
+    def report(self, vendor = None):
+        """
+        Reports everything for all vendors or a specific vendor
+        PCIIds.report()  reports everything
+        PCIIDs.report("0e11") reports only "Compaq Computer Corporation"
+        """
+        if vendor != None:
+            self.vendors[vendor].report()
+        else:
+            for vID, v in self.vendors.items():
+                v.report()
+
+    def find_vendor(self, vid):
+        # convert vid to a hex string and remove the 0x
+        vid = hex(vid)[2:]
+
+        try:
+            return self.vendors[vid]
+        except: 
+            return Vendor("%s Unknown Vendor" % (vid))
+
+    def findDate(self, content):
+        for l in content:
+            if l.find("Date:") > -1:
+                return l.split()[-2].replace("-", "")
+        return None
+
+    def parse(self):
+        if len(self.contents) < 1:
+            print "data/%s-pci.ids not found" % self.date
+        else:
+            vendorID = ""
+            deviceID = ""
+            for l in self.contents:
+                if l[0] == "#":
+                    continue
+                elif len(l.strip()) == 0:
+                    continue
+                else:
+                    if l.find("\t\t") == 0:
+                        self.vendors[vendorID].devices[deviceID].addSubDevice(l)
+                    elif l.find("\t") == 0:
+                        deviceID = l.strip().split()[0]
+                        self.vendors[vendorID].addDevice(l)
+                    else:
+                        vendorID = l.split()[0]
+                        self.vendors[vendorID] = Vendor(l)
+
+    def readLocal(self, filename):
+        """
+        Reads the local file
+        """
+        self.contents = open(filename).readlines()
+        self.date = self.findDate(self.contents)
+
+    def loadLocal(self):
+        """
+        Loads database from local. If there is no file,
+        it creates a new one from web
+        """
+        self.date = idsfile[0].split("/")[1].split("-")[0]
+        self.readLocal()
+
+
+
+#=======================================
+
+def search_file(filename, search_path):
+    """ Given a search path, find file with requested name """
+    for path in string.split(search_path, ":"):
+        candidate = os.path.join(path, filename)
+        if os.path.exists(candidate): return os.path.abspath(candidate)
+    return None
+
+class ReadElf(object):
+    """ display_* methods are used to emit output into the output stream
+    """
+    def __init__(self, file, output):
+        """ file:
+                stream object with the ELF file to read
+
+            output:
+                output stream to write to
+        """
+        self.elffile = ELFFile(file)
+        self.output = output
+
+        # Lazily initialized if a debug dump is requested
+        self._dwarfinfo = None
+
+        self._versioninfo = None
+
+    def _section_from_spec(self, spec):
+        """ Retrieve a section given a "spec" (either number or name).
+            Return None if no such section exists in the file.
+        """
+        try:
+            num = int(spec)
+            if num < self.elffile.num_sections():
+                return self.elffile.get_section(num)
+            else:
+                return None
+        except ValueError:
+            # Not a number. Must be a name then
+            return self.elffile.get_section_by_name(str2bytes(spec))
+
+    def pretty_print_pmdinfo(self, pmdinfo):
+        global pcidb
+
+        for i in pmdinfo["pci_ids"]:
+            vendor = pcidb.find_vendor(i[0])
+            device = vendor.find_device(i[1])
+            subdev = device.find_subid(i[2], i[3])
+            print("%s (%s) : %s (%s) %s" % (vendor.name, vendor.ID, device.name, device.ID, subdev.name))
+   
+    def parse_pmd_info_string(self, mystring):
+        global raw_output
+        global pcidb
+
+        optional_pmd_info = [{'id': 'params', 'tag' : 'PMD PARAMETERS'}]
+
+        i = mystring.index("=");
+        mystring = mystring[i+2:]
+        pmdinfo = json.loads(mystring)
+
+        if raw_output:
+            print(pmdinfo)
+            return
+
+        print("PMD NAME: " + pmdinfo["name"])
+        for i in optional_pmd_info:
+            try:
+                print("%s: %s" % (i['tag'], pmdinfo[i['id']]))
+            except KeyError as e:
+                continue
+
+        if (len(pmdinfo["pci_ids"]) != 0):
+            print("PMD HW SUPPORT:")
+            if pcidb != None:
+                self.pretty_print_pmdinfo(pmdinfo)
+            else:
+                print("VENDOR\t DEVICE\t SUBVENDOR\t SUBDEVICE")
+                for i in pmdinfo["pci_ids"]:
+                    print("0x%04x\t 0x%04x\t 0x%04x\t\t 0x%04x" % (i[0], i[1], i[2], i[3]))
+
+        print("")
+
+
+    def display_pmd_info_strings(self, section_spec):
+        """ Display a strings dump of a section. section_spec is either a
+            section number or a name.
+        """
+        section = self._section_from_spec(section_spec)
+        if section is None:
+            return
+
+
+        data = section.data()
+        dataptr = 0
+
+        while dataptr < len(data):
+            while ( dataptr < len(data) and
+                    not (32 <= byte2int(data[dataptr]) <= 127)):
+                dataptr += 1
+
+            if dataptr >= len(data):
+                break
+
+            endptr = dataptr
+            while endptr < len(data) and byte2int(data[endptr]) != 0:
+                endptr += 1
+
+            mystring = bytes2str(data[dataptr:endptr])
+            rc = mystring.find("PMD_INFO_STRING")
+            if (rc != -1):
+                self.parse_pmd_info_string(mystring)
+
+            dataptr = endptr
+
+    def find_librte_eal(self, section):
+        for tag in section.iter_tags():
+            if tag.entry.d_tag == 'DT_NEEDED':
+                if "librte_eal" in tag.needed:
+                    return tag.needed
+        return None 
+
+        
+    def search_for_autoload_path(self):
+        scanelf = self
+        scanfile = None
+        library = None
+
+        section = self._section_from_spec(".dynamic")
+        try:
+            eallib = self.find_librte_eal(section)
+            if (eallib != None):
+                ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+                if (ldlibpath == None):
+                    ldlibpath = ""
+                dtr = self.get_dt_runpath(section)
+                library = search_file(eallib,
+                                      dtr + ":" + ldlibpath +
+                                      ":/usr/lib64:/lib64:/usr/lib:/lib")
+                if (library == None):
+                    return (None, None)
+                if (raw_output == False):
+                    print("Scanning for autoload path in %s" % library)
+                scanfile = open(library, 'rb')
+                scanelf = ReadElf(scanfile, sys.stdout)
+        except AttributeError:
+            # Not a dynamic binary
+            pass
+        except ELFError:
+            scanfile.close()
+            return (None, None)
+
+        section = scanelf._section_from_spec(".rodata")
+        if section is None:
+            if (scanfile != None):
+                scanfile.close()
+            return (None, None)
+
+        data = section.data()
+        dataptr = 0
+
+        while dataptr < len(data):
+            while ( dataptr < len(data) and
+                    not (32 <= byte2int(data[dataptr]) <= 127)):
+                dataptr += 1
+
+            if dataptr >= len(data):
+                break
+
+            endptr = dataptr
+            while endptr < len(data) and byte2int(data[endptr]) != 0:
+                endptr += 1
+
+            mystring = bytes2str(data[dataptr:endptr])
+            rc = mystring.find("DPDK_PLUGIN_PATH")
+            if (rc != -1):
+                rc = mystring.find("=")
+                return (mystring[rc+1:], library)
+
+            dataptr = endptr
+        if (scanfile != None):
+            scanfile.close()
+        return (None, None)
+
+    def get_dt_runpath(self, dynsec):
+        for tag in dynsec.iter_tags():
+            if tag.entry.d_tag == 'DT_RUNPATH':
+                return tag.runpath
+        return ""
+
+
+    def process_dt_needed_entries(self):
+        """ Look to see if there are any DT_NEEDED entries in the binary
+            And process those if there are
+        """
+        global raw_output
+        runpath = ""
+        ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+        if (ldlibpath == None):
+            ldlibpath = ""
+
+        dynsec = self._section_from_spec(".dynamic")
+        try:
+            runpath = self.get_dt_runpath(dynsec)
+        except AttributeError:
+            # dynsec is None, just return
+            return
+
+        for tag in dynsec.iter_tags():
+            if tag.entry.d_tag == 'DT_NEEDED':
+                rc = tag.needed.find("librte_pmd")
+                if (rc != -1):
+                    library = search_file(tag.needed,
+                            runpath + ":" + ldlibpath +
+                            ":/usr/lib64:/lib64:/usr/lib:/lib")
+                    if (library != None):
+                        if (raw_output == False):
+                            print("Scanning %s for pmd information" % library)
+                        with open(library, 'rb') as file:
+                            try:
+                                libelf = ReadElf(file, sys.stdout)
+                            except ELFError as e:
+                                print("%s is no an ELF file" % library)
+                                continue
+                            libelf.process_dt_needed_entries()
+                            libelf.display_pmd_info_strings(".rodata")
+                            file.close()
+
+def scan_autoload_path(autoload_path):
+    global raw_output
+
+    if os.path.exists(autoload_path) == False:
+        return
+
+    try:
+        dirs = os.listdir(autoload_path)
+    except OSError as e:
+        # Couldn't read the directory, give up
+        return
+
+    for d in dirs:
+        dpath = os.path.join(autoload_path, d)
+        if os.path.isdir(dpath):
+            scan_autoload_path(dpath)
+        if os.path.isfile(dpath):
+            try:
+                file = open(dpath, 'rb')
+                readelf = ReadElf(file, sys.stdout)
+            except ELFError as e:
+                # this is likely not an elf file, skip it
+               continue
+            except IOError as e:
+                # No permission to read the file, skip it
+                continue
+
+            if (raw_output == False):
+                print("Hw Support for library %s" % d)
+            readelf.display_pmd_info_strings(".rodata")
+            file.close()
+
+
+
+def scan_for_autoload_pmds(dpdk_path):
+    """
+    search the specified application or path for a pmd autoload path
+    then scan said path for pmds and report hw support
+    """
+    global raw_output
+
+    if (os.path.isfile(dpdk_path) == False):
+        if (raw_output == False):
+            print("Must specify a file name")
+        return
+        
+
+    file = open(dpdk_path, 'rb')
+    try:
+        readelf = ReadElf(file, sys.stdout)
+    except ElfError as e:
+        if (raw_output == False):
+            print("Unable to parse %s" % file)
+        return
+
+    (autoload_path, scannedfile) = readelf.search_for_autoload_path()
+    if (autoload_path == None or autoload_path == ""):
+        if (raw_output == False):
+            print("No autoload path configured in %s" % dpdk_path)
+        return
+    if (raw_output == False):
+        if (scannedfile == None):
+            scannedfile = dpdk_path
+        print("Found autoload path %s in %s" % (autoload_path, scannedfile))
+
+    file.close()
+    if (raw_output == False):
+        print("Discovered Autoload HW Support:") 
+    scan_autoload_path(autoload_path) 
+    return
+    
+    
+def main(stream=None):
+    global raw_output
+    global pcidb
+
+    optparser = OptionParser(
+            usage='usage: %prog [-hrtp] [-d <pci id file] <elf-file>',
+            description="Dump pmd hardware support info",
+            add_help_option=True,
+            prog='pmdinfo.py')
+    optparser.add_option('-r', '--raw',
+            action='store_true', dest='raw_output',
+            help='Dump raw json strings')
+    optparser.add_option("-d", "--pcidb", dest="pcifile",
+            help="specify a pci database to get vendor names from",
+            default="/usr/share/hwdata/pci.ids", metavar="FILE")
+    optparser.add_option("-t", "--table", dest="tblout",
+            help="output infomation on hw support as a hex table",
+            action='store_true')
+    optparser.add_option("-p", "--plugindir", dest="pdir",
+            help="scan dpdk for autoload plugins", action='store_true')
+
+    options, args = optparser.parse_args()
+
+    if options.raw_output:
+        raw_output = True
+
+    if options.pcifile:
+        pcidb = PCIIds(options.pcifile)
+        if pcidb == None:
+            print("Pci DB file not found")
+            exit(1)
+
+    if options.tblout:   
+        options.pcifile = None
+        pcidb = None
+
+    if (len(args) == 0):
+        optparser.print_usage()
+        exit(1)
+
+    if options.pdir == True:
+        exit (scan_for_autoload_pmds(args[0]))
+
+    ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+    if (ldlibpath == None):
+        ldlibpath = ""
+
+    if (os.path.exists(args[0]) == True):
+        myelffile = args[0]
+    else:
+        myelffile = search_file(args[0], ldlibpath + ":/usr/lib64:/lib64:/usr/lib:/lib")
+
+    if (myelffile == None):
+        print("File not found")
+        sys.exit(1)
+
+    with open(myelffile, 'rb') as file:
+        try:
+            readelf = ReadElf(file, sys.stdout)
+            readelf.process_dt_needed_entries()
+            readelf.display_pmd_info_strings(".rodata") 
+            sys.exit(0)
+ 
+        except ELFError as ex:
+            sys.stderr.write('ELF error: %s\n' % ex)
+            sys.exit(1)
+
+
+#-------------------------------------------------------------------------------
+if __name__ == '__main__':
+    main()
+
+
-- 
2.5.5

[PATCHv5 6/6] remove rte.hostapp.mk

[Neil Horman]

During the review of this patchset it was discovered that rte.hostapp.mk was
going unused.  All its users had been previously removed, and so this file too
can go.  It and rte.buildtools.mk are effectively duplicates, but given that
pmdinfogen is a tool used to create the dpdk build, I think the naming of the
buildtools variant is more appropriate.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 mk/rte.hostapp.mk | 123 ------------------------------------------------------
 1 file changed, 123 deletions(-)
 delete mode 100644 mk/rte.hostapp.mk

diff --git a/mk/rte.hostapp.mk b/mk/rte.hostapp.mk
deleted file mode 100644
index c44d0f8..0000000
--- a/mk/rte.hostapp.mk
+++ /dev/null
@@ -1,123 +0,0 @@
-#   BSD LICENSE
-#
-#   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
-#   All rights reserved.
-#
-#   Redistribution and use in source and binary forms, with or without
-#   modification, are permitted provided that the following conditions
-#   are met:
-#
-#     * Redistributions of source code must retain the above copyright
-#       notice, this list of conditions and the following disclaimer.
-#     * Redistributions in binary form must reproduce the above copyright
-#       notice, this list of conditions and the following disclaimer in
-#       the documentation and/or other materials provided with the
-#       distribution.
-#     * Neither the name of Intel Corporation nor the names of its
-#       contributors may be used to endorse or promote products derived
-#       from this software without specific prior written permission.
-#
-#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-# tell rte.compile-pre.mk to use HOSTCC instead of CC
-USE_HOST := 1
-include $(RTE_SDK)/mk/internal/rte.compile-pre.mk
-include $(RTE_SDK)/mk/internal/rte.install-pre.mk
-include $(RTE_SDK)/mk/internal/rte.clean-pre.mk
-include $(RTE_SDK)/mk/internal/rte.build-pre.mk
-include $(RTE_SDK)/mk/internal/rte.depdirs-pre.mk
-
-# VPATH contains at least SRCDIR
-VPATH += $(SRCDIR)
-
-_BUILD = $(HOSTAPP)
-_INSTALL = $(INSTALL-FILES-y) $(SYMLINK-FILES-y) $(RTE_OUTPUT)/hostapp/$(HOSTAPP)
-_CLEAN = doclean
-
-.PHONY: all
-all: install
-
-.PHONY: install
-install: build _postinstall
-
-_postinstall: build
-
-.PHONY: build
-build: _postbuild
-
-exe2cmd = $(strip $(call dotfile,$(patsubst %,%.cmd,$(1))))
-
-O_TO_EXE = $(HOSTCC) $(HOST_LDFLAGS) $(LDFLAGS_$(@)) \
-	$(EXTRA_HOST_LDFLAGS) -o $@ $(OBJS-y) $(LDLIBS)
-O_TO_EXE_STR = $(subst ','\'',$(O_TO_EXE)) #'# fix syntax highlight
-O_TO_EXE_DISP = $(if $(V),"$(O_TO_EXE_STR)","  HOSTLD $(@)")
-O_TO_EXE_CMD = "cmd_$@ = $(O_TO_EXE_STR)"
-O_TO_EXE_DO = @set -e; \
-	echo $(O_TO_EXE_DISP); \
-	$(O_TO_EXE) && \
-	echo $(O_TO_EXE_CMD) > $(call exe2cmd,$(@))
-
--include .$(HOSTAPP).cmd
-
-# list of .a files that are linked to this application
-LDLIBS_FILES := $(wildcard \
-	$(addprefix $(RTE_OUTPUT)/lib/, \
-	$(patsubst -l%,lib%.a,$(filter -l%,$(LDLIBS)))))
-
-#
-# Compile executable file if needed
-#
-$(HOSTAPP): $(OBJS-y) $(LDLIBS_FILES) FORCE
-	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
-	$(if $(D),\
-		@echo -n "$@ -> $< " ; \
-		echo -n "file_missing=$(call boolean,$(file_missing)) " ; \
-		echo -n "cmdline_changed=$(call boolean,$(call cmdline_changed,$(O_TO_EXE_STR))) " ; \
-		echo -n "depfile_missing=$(call boolean,$(depfile_missing)) " ; \
-		echo "depfile_newer=$(call boolean,$(depfile_newer)) ")
-	$(if $(or \
-		$(file_missing),\
-		$(call cmdline_changed,$(O_TO_EXE_STR)),\
-		$(depfile_missing),\
-		$(depfile_newer)),\
-		$(O_TO_EXE_DO))
-
-#
-# install app in $(RTE_OUTPUT)/hostapp
-#
-$(RTE_OUTPUT)/hostapp/$(HOSTAPP): $(HOSTAPP)
-	@echo "  INSTALL-HOSTAPP $(HOSTAPP)"
-	@[ -d $(RTE_OUTPUT)/hostapp ] || mkdir -p $(RTE_OUTPUT)/hostapp
-	$(Q)cp -f $(HOSTAPP) $(RTE_OUTPUT)/hostapp
-
-#
-# Clean all generated files
-#
-.PHONY: clean
-clean: _postclean
-	$(Q)rm -f $(_BUILD_TARGETS) $(_INSTALL_TARGETS) $(_CLEAN_TARGETS)
-
-.PHONY: doclean
-doclean:
-	$(Q)rm -rf $(HOSTAPP) $(OBJS-all) $(DEPS-all) $(DEPSTMP-all) \
-	  $(CMDS-all) $(INSTALL-FILES-all) .$(HOSTAPP).cmd
-
-
-include $(RTE_SDK)/mk/internal/rte.compile-post.mk
-include $(RTE_SDK)/mk/internal/rte.install-post.mk
-include $(RTE_SDK)/mk/internal/rte.clean-post.mk
-include $(RTE_SDK)/mk/internal/rte.build-post.mk
-include $(RTE_SDK)/mk/internal/rte.depdirs-post.mk
-
-.PHONY: FORCE
-FORCE:
-- 
2.5.5

Re: [PATCHv4 5/5] pmdinfo.py: Add tool to query binaries for hw and other support information

[Panu Matilainen]

On 05/25/2016 09:58 PM, Thomas Monjalon wrote:
> 2016-05-25 13:47, Neil Horman:
>> On Wed, May 25, 2016 at 07:22:39PM +0200, Thomas Monjalon wrote:
>>> 2016-05-24 15:41, Neil Horman:
>>>> Note that, in the case of dynamically linked applications, pmdinfo.py will scan
>>>> for implicitly linked PMDs by searching the specified binaries .dynamic section
>>>> for DT_NEEDED entries that contain the substring librte_pmd.
>>>
>>> I don't know any DPDK app dynamically linked with a PMD (with DT_NEEDED).
>> I know lots of them, they're all in the dpdk.  everything under app that uses a
>> virutal device links at link time to librte_pmd_bonding and librte_pmd_pipe (and
>> a few others), because they have additional apis that they need to resolve at
>> load time.
>
> Oh yes! you are right.

Also everything linking against the linker script (in its current form) 
such as OVS will pull in absolutely everything, including the pmds:

[pmatilai@sopuli ~]$ ldd /usr/sbin/ovs-vswitchd | awk '/librte_pmd/ 
{print $1}'
librte_pmd_bnx2x.so.1
librte_pmd_virtio.so.1
librte_pmd_cxgbe.so.1
librte_pmd_ring.so.2
librte_pmd_af_packet.so.1
librte_pmd_vhost.so.1
librte_pmd_ixgbe.so.1
librte_pmd_vmxnet3_uio.so.1
librte_pmd_fm10k.so.1
librte_pmd_i40e.so.1
librte_pmd_null.so.1
librte_pmd_pcap.so.1
librte_pmd_null_crypto.so.1
librte_pmd_ena.so.1
librte_pmd_e1000.so.1
librte_pmd_qede.so.1
librte_pmd_bond.so.1
librte_pmd_enic.so.1
[pmatilai@sopuli ~]$

>
>>> However it is a good idea to handle this case.

Yup. But there's also a bit of a gotcha involved with the virtual 
devices with added api. This is how eg testpmd looks when built in 
shared library config:

[pmatilai@sopuli ~]$ pmdinfo /usr/bin/testpmd
Scanning /usr/lib64/librte_pmd_bond.so.1 for pmd information
PMD NAME: bonding
PMD TYPE: PMD_VDEV
PMD PARAMETERS: slave=<ifc> primary=<ifc> mode=[0-4] xmit_policy=[l2 | 
l23 | l34] socket_id=<int> mac=<mac addr> lsc_poll_period_ms=<int> 
up_delay=<int> down_delay=<int>

[pmatilai@sopuli ~]$

Having support for bonding driver but nothing else will seem pretty damn 
confusing unless you happen to be a dpdk developer knowing this fact 
about some pmds also providing API and being linked directly etc.

Might make sense to have extra cli switch to control which kind of pmds 
are shown, and maybe default to skipping vdevs since they dont provide 
any actual hardware support anyway. Showing nothing at all in the above 
case would likely be a little less confusing. Maybe :)

Anyway, this is "refine later if needed once we gain more experience" 
material to me.

	- Panu -

Re: [PATCHv4 5/5] pmdinfo.py: Add tool to query binaries for hw and other support information

[Neil Horman]

On Fri, May 27, 2016 at 12:16:03PM +0300, Panu Matilainen wrote:
> On 05/25/2016 09:58 PM, Thomas Monjalon wrote:
> > 2016-05-25 13:47, Neil Horman:
> > > On Wed, May 25, 2016 at 07:22:39PM +0200, Thomas Monjalon wrote:
> > > > 2016-05-24 15:41, Neil Horman:
> > > > > Note that, in the case of dynamically linked applications, pmdinfo.py will scan
> > > > > for implicitly linked PMDs by searching the specified binaries .dynamic section
> > > > > for DT_NEEDED entries that contain the substring librte_pmd.
> > > > 
> > > > I don't know any DPDK app dynamically linked with a PMD (with DT_NEEDED).
> > > I know lots of them, they're all in the dpdk.  everything under app that uses a
> > > virutal device links at link time to librte_pmd_bonding and librte_pmd_pipe (and
> > > a few others), because they have additional apis that they need to resolve at
> > > load time.
> > 
> > Oh yes! you are right.
> 
> Also everything linking against the linker script (in its current form) such
> as OVS will pull in absolutely everything, including the pmds:
> 
> [pmatilai@sopuli ~]$ ldd /usr/sbin/ovs-vswitchd | awk '/librte_pmd/ {print
> $1}'
> librte_pmd_bnx2x.so.1
> librte_pmd_virtio.so.1
> librte_pmd_cxgbe.so.1
> librte_pmd_ring.so.2
> librte_pmd_af_packet.so.1
> librte_pmd_vhost.so.1
> librte_pmd_ixgbe.so.1
> librte_pmd_vmxnet3_uio.so.1
> librte_pmd_fm10k.so.1
> librte_pmd_i40e.so.1
> librte_pmd_null.so.1
> librte_pmd_pcap.so.1
> librte_pmd_null_crypto.so.1
> librte_pmd_ena.so.1
> librte_pmd_e1000.so.1
> librte_pmd_qede.so.1
> librte_pmd_bond.so.1
> librte_pmd_enic.so.1
> [pmatilai@sopuli ~]$
> 
> > 
> > > > However it is a good idea to handle this case.
> 
> Yup. But there's also a bit of a gotcha involved with the virtual devices
> with added api. This is how eg testpmd looks when built in shared library
> config:
> 
> [pmatilai@sopuli ~]$ pmdinfo /usr/bin/testpmd
> Scanning /usr/lib64/librte_pmd_bond.so.1 for pmd information
> PMD NAME: bonding
> PMD TYPE: PMD_VDEV
> PMD PARAMETERS: slave=<ifc> primary=<ifc> mode=[0-4] xmit_policy=[l2 | l23 |
> l34] socket_id=<int> mac=<mac addr> lsc_poll_period_ms=<int> up_delay=<int>
> down_delay=<int>
> 
> [pmatilai@sopuli ~]$
> 
> Having support for bonding driver but nothing else will seem pretty damn
> confusing unless you happen to be a dpdk developer knowing this fact about
> some pmds also providing API and being linked directly etc.
> 
Possibly, but I think its important to remember that we aren't really dealing
with end users buying this software at best buy.  The target user is more likely
a sysadmin, that I would like to think has the wherewithal to understand that
hardware support might be either (a) built-in, getting loaded at runtime by
virtue of the program itself, or a library that it is explicitly linked to, or
(b) configured-in, getting resolved an loaded at run time by virtue of a
configuration file or other site specific element.  They should be able to
figure out that the latter won't be discoverable by pmdinfo without help.

> Might make sense to have extra cli switch to control which kind of pmds are
> shown, and maybe default to skipping vdevs since they dont provide any
> actual hardware support anyway. Showing nothing at all in the above case
> would likely be a little less confusing. Maybe :)
> 
So, Thomas and I just finished arguing about the VDEV/PDEV issue.  Since he
asserted that the ennumeration for device types was being removed I removed any
notion of differentiation from the tool.  Unless there are plans to keep that
differentiation, I don't think adding a filter for device types is really
advisable

What might be useful is a filter on vendor/device id's.  That is to say only
show pmd information that matches the specified vendor/device tuple, so that a
user can search to see if the application supports the hardware they have.

> Anyway, this is "refine later if needed once we gain more experience"
> material to me.
Agreed, this isn't an addition that needs to happen now, walk before we run.

Neil

> 
> 	- Panu -
> 
> 

Re: [PATCHv4 5/5] pmdinfo.py: Add tool to query binaries for hw and other support information

[Panu Matilainen]

On 05/27/2016 02:35 PM, Neil Horman wrote:
> On Fri, May 27, 2016 at 12:16:03PM +0300, Panu Matilainen wrote:
>
>> Yup. But there's also a bit of a gotcha involved with the virtual devices
>> with added api. This is how eg testpmd looks when built in shared library
>> config:
>>
>> [pmatilai@sopuli ~]$ pmdinfo /usr/bin/testpmd
>> Scanning /usr/lib64/librte_pmd_bond.so.1 for pmd information
>> PMD NAME: bonding
>> PMD TYPE: PMD_VDEV
>> PMD PARAMETERS: slave=<ifc> primary=<ifc> mode=[0-4] xmit_policy=[l2 | l23 |
>> l34] socket_id=<int> mac=<mac addr> lsc_poll_period_ms=<int> up_delay=<int>
>> down_delay=<int>
>>
>> [pmatilai@sopuli ~]$
>>
>> Having support for bonding driver but nothing else will seem pretty damn
>> confusing unless you happen to be a dpdk developer knowing this fact about
>> some pmds also providing API and being linked directly etc.
>>
> Possibly, but I think its important to remember that we aren't really dealing
> with end users buying this software at best buy.  The target user is more likely
> a sysadmin, that I would like to think has the wherewithal to understand that
> hardware support might be either (a) built-in, getting loaded at runtime by
> virtue of the program itself, or a library that it is explicitly linked to, or
> (b) configured-in, getting resolved an loaded at run time by virtue of a
> configuration file or other site specific element.  They should be able to
> figure out that the latter won't be discoverable by pmdinfo without help.

Sure, but ability to figure something out doesn't necessarily mean they 
should have to, much less want to, do so. Somebody setting up DPDK is 
interested in moving network packets, and the details of how a 
particular piece of software was built and linked together couldn't be 
much more further from away from that task.

I do remember the days when you had to manually load device drivers by 
kernel module names during Linux installation and while I certainly 
could do it, I dont really miss that part a single bit :)

I also realize I'm probably way ahead of things with my these usability 
ramblings when you're only trying to get the feature in, and that's 
clearly been causing some unintended friction here. Sorry about that, 
I'm really just excited about the possibilities this feature opens.

>> Might make sense to have extra cli switch to control which kind of pmds are
>> shown, and maybe default to skipping vdevs since they dont provide any
>> actual hardware support anyway. Showing nothing at all in the above case
>> would likely be a little less confusing. Maybe :)
>>
> So, Thomas and I just finished arguing about the VDEV/PDEV issue.  Since he
> asserted that the ennumeration for device types was being removed I removed any
> notion of differentiation from the tool.  Unless there are plans to keep that
> differentiation, I don't think adding a filter for device types is really
> advisable

Ah yes, sorry. I did see the discussion but the actual message 
apparently failed to register :)

>
> What might be useful is a filter on vendor/device id's.  That is to say only
> show pmd information that matches the specified vendor/device tuple, so that a
> user can search to see if the application supports the hardware they have.

Yeah, makes sense.

>> Anyway, this is "refine later if needed once we gain more experience"
>> material to me.
> Agreed, this isn't an addition that needs to happen now, walk before we run.

Nod.

	- Panu -

> Neil
>
>>
>> 	- Panu -
>>
>>

Re: [PATCHv5 5/6] pmdinfo.py: Add tool to query binaries for hw and other support information

[Mcnamara, John]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
> Sent: Thursday, May 26, 2016 6:17 PM
> To: dev@dpdk.org
> Cc: Neil Horman <nhorman@tuxdriver.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; Thomas Monjalon <thomas.monjalon@6wind.com>;
> Stephen Hemminger <stephen@networkplumber.org>; Panu Matilainen
> <pmatilai@redhat.com>
> Subject: [dpdk-dev] [PATCHv5 5/6] pmdinfo.py: Add tool to query binaries
> for hw and other support information

Hi Neil,

Could you make the Python code PEP8 (Style Guide for Python Code) compliant: 

    https://www.python.org/dev/peps/pep-0008/

You can use the pep8 tool to test code for compliance.

We have been using PEP8 recently as the standard for Python based submissions
into DPDK. I'll submit a patch to add it to the DPDK Coding Style guide.

Thanks,

John.

Re: [PATCHv5 5/6] pmdinfo.py: Add tool to query binaries for hw and other support information

[Neil Horman]

On Fri, May 27, 2016 at 02:38:54PM +0000, Mcnamara, John wrote:
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
> > Sent: Thursday, May 26, 2016 6:17 PM
> > To: dev@dpdk.org
> > Cc: Neil Horman <nhorman@tuxdriver.com>; Richardson, Bruce
> > <bruce.richardson@intel.com>; Thomas Monjalon <thomas.monjalon@6wind.com>;
> > Stephen Hemminger <stephen@networkplumber.org>; Panu Matilainen
> > <pmatilai@redhat.com>
> > Subject: [dpdk-dev] [PATCHv5 5/6] pmdinfo.py: Add tool to query binaries
> > for hw and other support information
> 
> Hi Neil,
> 
> Could you make the Python code PEP8 (Style Guide for Python Code) compliant: 
> 
>     https://www.python.org/dev/peps/pep-0008/
> 
> You can use the pep8 tool to test code for compliance.
> 
> We have been using PEP8 recently as the standard for Python based submissions
> into DPDK. I'll submit a patch to add it to the DPDK Coding Style guide.
> 
> Thanks,
> 
> John.
> 

Sure, though only one of the scripts in the tools directly seems to even be
loosely compliant with the standard.

[PATCHv6 0/7] Implement pmd hardware support exports

[Neil Horman]

Hey all-
	So heres attempt number 2 at a method for exporting PMD hardware support
information.  As we discussed previously, the consensus seems to be that pmd
information should be:

1) Able to be interrogated on any ELF binary (application binary or individual
DSO)
2) Equally functional on statically linked applications or on DSO's
3) Resilient to symbol stripping
4) Script friendly
5) Show kernel dependencies
6) List driver options
7) Show driver name
8) Offer human readable output
9) Show DPDK version
10) Show driver version
11) Allow for expansion
12) Not place additional build environment dependencies on an application

I added item 12 myself, because I don't think its reasonable to require
applications to use a specific linker script to get hardware support information
(which precludes the use of a special .modinfo section like the kernel uses)

However, we still can use some tricks from the kernel to make this work.  In
this approach, what I've done is the following:

A) Modified the driver registration macro to also define a variable:
this_pmd_name<n>= "name"

Based on the unique name string pointed to by the above variable, we can
query for an arbitrary number of other symbols following the pattern:
<name>_<tag>

Where tag is some well known identifier for information we wish to export

B) Added a utility called pmdinfogen.  This utility is not meant for general use,
but rather is used by the dpdk build environment itself when compiling pmds.
for each object that uses the PMD_REGISTER_DRIVER macro, this utiity is run.  It
searches for the symbols in (A), and using those, extracts the hardware support
info, and module name from the object, using that to produce a new c file
containing a single variable in the following format:

static const char <name>[] __attribute__((used)) = "PMD_DRIVER_INFO=<json>";

The <name> is arbitrary, as its static and not referenced.  The relevant bit is
the string value assigned to it.  The <json> is a json encoded string of the
extracted hardware support information pmdinfo found for the corresponding
object.  This C file is suitable for compilation and relocatable linking back
into the parent object file.  The result of this operation is that the object
string table now contains a string that will not e removed by stripping, whos
leading text (PMD_DRIVER_INFO) can be easily searched for at any time weather
the symbol referring to it is stripped or not.

C) Added a utilty called pmdinfo.py.  This python script, searches the
string table of the .rodata section of any provided ELF file looking for the
PMD_DRIVER_INFO prefix on a string.  When found, it will interpret the remainder
of the string as json, and output the hardware support for that ELF file (if
any).


This approach ticks most of the above boxes:
1) Impervious to stripping
2) Works on static and dynamic binaries
3) Human and script friendly
4) Allows for expansion

Because of (4) the other items should be pretty easy to implement, as its just a
matter of modifying the macros to export the info, pmdinfo to encode it to json,
and pmd_hw_support.py to read it.  I'm not adding them now, as theres alot of
rote work to do to get some of them in place (e.g. DPDK has no current global
VERSION macro, drivers don't have a consistent version scheme, command line
strings have to be built for each driver, etc).  But once this is accepted,
those items can be done as time allows and should be fairly easy to implement.

Change Notes:
v2)
 * Made the export macros a bit easier to expand
 * Added a macro to optionally export command line strings
 * Renamed utilties to be more appropriate
   (pmdinfo -> pmdinfogen, pmd_hw_support.py -> pmdinfo.py)
 * Added search capabilities to pmdinfo.py so that we search for libraries
   linked using DT_NEEDINFO entries.  We search DT_RUNPATH, LD_LIBRARY_PATH,
   /usr/lib and /lib
 * Added serch abilities if the specified binary to pmdinfo isn't found, we
   search LD_LIBRARY_PATH, /usr/lib and /lib
 * Added an option to pmdinfo.py to pretty-print hardware support info using the
   pci.ids database

v3)
 * Made table mode printing the default mode
 * Added a default path to the pci.ids file
 * Modifed pmdinfo to use python rather than python3
 * Cleaned up some changelog entries
 * Added an export for RTE_EAL_PMD_PATH
 * Added a plugin option to pmdinfo to scan for autoloaded DSO's

v4)
 * Modified the operation of the -p option. As much as I don't like implying
that autoloaded pmds are guaranteed to be there at run time, I'm having a hard
time seeing how we can avoid specifying the application file to scan for the
autoload directory.  Without it we can't determine which library the user means
in a multiversion installation
 * Cleaned up the help text
 * Added a rule for an install target for pmdinfo
 * Guarded against some tracebacks in pmdinfo
 * Use DT_NEEDED entries to get versioned libraries in -p mode
 * Fixed traceback that occurs on lack of input arguments
 * Fixed some erroneous macro usage in drivers that aren't in the default build

v5)
 * Added a dpdk- prefix to pmdinfo symlink
 * Renamed build announcement to PMDINFOGEN/BUILD
 * Removed erroneous true statement from makefile
 * Removed duplicate rte.hostapp.mk makefile
 * Fixed some whitespace
 * Whitespace fixups
 * Fixed makefile if; then style
 * Renamed module string C file
 * Removed duplicate rte_pci_id definition
 * Clarified macro names
 * Removed PMD type attribute
 * Fixed tools usage for 32 bit arches
 * Removed some unused code
 * Added a few comments

v6)
 * Added some programming guide documentation
 * Reformatted python script with pep8

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>

[PATCHv6 1/7] pmdinfogen: Add buildtools and pmdinfogen utility

[Neil Horman]

pmdinfogen is a tool used to parse object files and build json strings for use in
later determining hardware support in a dso or application binary.  pmdinfo
looks for the non-exported symbol names this_pmd_name<n> and this_pmd_tbl<n>
(where n is a integer counter).  It records the name of each of these tuples,
using the later to find the symbolic name of the pci_table for physical devices
that the object supports.  With this information, it outputs a C file with a
single line of the form:

static char *<pmd_name>_driver_info[] __attribute__((used)) = " \
	PMD_DRIVER_INFO=<json string>";

Where <pmd_name> is the arbitrary name of the pmd, and <json_string> is the json
encoded string that hold relevant pmd information, including the pmd name, type
and optional array of pci device/vendor ids that the driver supports.

This c file is suitable for compiling to object code, then relocatably linking
into the parent file from which the C was generated.  This creates an entry in
the string table of the object that can inform a later tool about hardware
support.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 GNUmakefile                        |   2 +-
 buildtools/Makefile                |  36 ++++
 buildtools/pmdinfogen/Makefile     |  49 +++++
 buildtools/pmdinfogen/pmdinfogen.c | 410 +++++++++++++++++++++++++++++++++++++
 buildtools/pmdinfogen/pmdinfogen.h |  94 +++++++++
 mk/rte.buildtools.mk               | 148 +++++++++++++
 mk/rte.sdkbuild.mk                 |   3 +-
 7 files changed, 740 insertions(+), 2 deletions(-)
 create mode 100644 buildtools/Makefile
 create mode 100644 buildtools/pmdinfogen/Makefile
 create mode 100644 buildtools/pmdinfogen/pmdinfogen.c
 create mode 100644 buildtools/pmdinfogen/pmdinfogen.h
 create mode 100644 mk/rte.buildtools.mk

diff --git a/GNUmakefile b/GNUmakefile
index b59e4b6..00fe0db 100644
--- a/GNUmakefile
+++ b/GNUmakefile
@@ -40,6 +40,6 @@ export RTE_SDK
 # directory list
 #
 
-ROOTDIRS-y := lib drivers app
+ROOTDIRS-y := buildtools lib drivers app
 
 include $(RTE_SDK)/mk/rte.sdkroot.mk
diff --git a/buildtools/Makefile b/buildtools/Makefile
new file mode 100644
index 0000000..eb565eb
--- /dev/null
+++ b/buildtools/Makefile
@@ -0,0 +1,36 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+DIRS-y += pmdinfogen
+
+include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/buildtools/pmdinfogen/Makefile b/buildtools/pmdinfogen/Makefile
new file mode 100644
index 0000000..223f50c
--- /dev/null
+++ b/buildtools/pmdinfogen/Makefile
@@ -0,0 +1,49 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+#
+# library name
+#
+APP = pmdinfogen
+
+#
+# all sources are stored in SRCS-y
+#
+SRCS-y += pmdinfogen.c
+
+CFLAGS += -g
+
+DEPDIRS-y += lib/librte_eal
+
+include $(RTE_SDK)/mk/rte.buildtools.mk
+
diff --git a/buildtools/pmdinfogen/pmdinfogen.c b/buildtools/pmdinfogen/pmdinfogen.c
new file mode 100644
index 0000000..ee2b606
--- /dev/null
+++ b/buildtools/pmdinfogen/pmdinfogen.c
@@ -0,0 +1,410 @@
+/* Postprocess pmd object files to export hw support 
+ *
+ * Copyright 2016 Neil Horman <nhorman@tuxdriver.com>
+ * Based in part on modpost.c from the linux kernel
+ *
+ * This software may be used and distributed according to the terms
+ * of the GNU General Public License V2, incorporated herein by reference.
+ *
+ */
+
+#define _GNU_SOURCE
+#include <stdio.h>
+#include <ctype.h>
+#include <string.h>
+#include <limits.h>
+#include <stdbool.h>
+#include <errno.h>
+#include "pmdinfogen.h"
+
+
+static const char *sym_name(struct elf_info *elf, Elf_Sym *sym)
+{
+	if (sym)
+		return elf->strtab + sym->st_name;
+	else
+		return "(unknown)";
+}
+
+void *grab_file(const char *filename, unsigned long *size)
+{
+	struct stat st;
+	void *map = MAP_FAILED;
+	int fd;
+
+	fd = open(filename, O_RDONLY);
+	if (fd < 0)
+		return NULL;
+	if (fstat(fd, &st))
+		goto failed;
+
+	*size = st.st_size;
+	map = mmap(NULL, *size, PROT_READ|PROT_WRITE, MAP_PRIVATE, fd, 0);
+
+failed:
+	close(fd);
+	if (map == MAP_FAILED)
+		return NULL;
+	return map;
+}
+
+/**
+  * Return a copy of the next line in a mmap'ed file.
+  * spaces in the beginning of the line is trimmed away.
+  * Return a pointer to a static buffer.
+  **/
+void release_file(void *file, unsigned long size)
+{
+	munmap(file, size);
+}
+
+
+static void *get_sym_value(struct elf_info *info, const Elf_Sym *sym)
+{
+	void *ptr = (void *)info->hdr + info->sechdrs[sym->st_shndx].sh_offset;
+
+	return (void *)(ptr + sym->st_value);
+}
+
+static Elf_Sym *find_sym_in_symtab(struct elf_info *info, 
+				   const char *name, Elf_Sym *last)
+{
+	Elf_Sym *idx;
+	if (last)
+		idx = last+1;
+	else
+		idx = info->symtab_start;
+
+	for(; idx < info->symtab_stop; idx++) {
+		const char *n = sym_name(info, idx);
+		if (!strncmp(n, name, strlen(name)))
+			return idx;
+	}
+	return NULL;
+}
+
+static int parse_elf(struct elf_info *info, const char *filename)
+{
+	unsigned int i;
+	Elf_Ehdr *hdr;
+	Elf_Shdr *sechdrs;
+	Elf_Sym  *sym;
+	const char *secstrings;
+	unsigned int symtab_idx = ~0U, symtab_shndx_idx = ~0U;
+
+	hdr = grab_file(filename, &info->size);
+	if (!hdr) {
+		perror(filename);
+		exit(1);
+	}
+	info->hdr = hdr;
+	if (info->size < sizeof(*hdr)) {
+		/* file too small, assume this is an empty .o file */
+		return 0;
+	}
+	/* Is this a valid ELF file? */
+	if ((hdr->e_ident[EI_MAG0] != ELFMAG0) ||
+	    (hdr->e_ident[EI_MAG1] != ELFMAG1) ||
+	    (hdr->e_ident[EI_MAG2] != ELFMAG2) ||
+	    (hdr->e_ident[EI_MAG3] != ELFMAG3)) {
+		/* Not an ELF file - silently ignore it */
+		return 0;
+	}
+	/* Fix endianness in ELF header */
+	hdr->e_type      = TO_NATIVE(hdr->e_type);
+	hdr->e_machine   = TO_NATIVE(hdr->e_machine);
+	hdr->e_version   = TO_NATIVE(hdr->e_version);
+	hdr->e_entry     = TO_NATIVE(hdr->e_entry);
+	hdr->e_phoff     = TO_NATIVE(hdr->e_phoff);
+	hdr->e_shoff     = TO_NATIVE(hdr->e_shoff);
+	hdr->e_flags     = TO_NATIVE(hdr->e_flags);
+	hdr->e_ehsize    = TO_NATIVE(hdr->e_ehsize);
+	hdr->e_phentsize = TO_NATIVE(hdr->e_phentsize);
+	hdr->e_phnum     = TO_NATIVE(hdr->e_phnum);
+	hdr->e_shentsize = TO_NATIVE(hdr->e_shentsize);
+	hdr->e_shnum     = TO_NATIVE(hdr->e_shnum);
+	hdr->e_shstrndx  = TO_NATIVE(hdr->e_shstrndx);
+	sechdrs = (void *)hdr + hdr->e_shoff;
+	info->sechdrs = sechdrs;
+
+	/* Check if file offset is correct */
+	if (hdr->e_shoff > info->size) {
+		fprintf(stderr, "section header offset=%lu in file '%s' is bigger than "
+		      "filesize=%lu\n", (unsigned long)hdr->e_shoff,
+		      filename, info->size);
+		return 0;
+	}
+
+	if (hdr->e_shnum == SHN_UNDEF) {
+		/*
+		 * There are more than 64k sections,
+		 * read count from .sh_size.
+		 */
+		info->num_sections = TO_NATIVE(sechdrs[0].sh_size);
+	}
+	else {
+		info->num_sections = hdr->e_shnum;
+	}
+	if (hdr->e_shstrndx == SHN_XINDEX) {
+		info->secindex_strings = TO_NATIVE(sechdrs[0].sh_link);
+	}
+	else {
+		info->secindex_strings = hdr->e_shstrndx;
+	}
+
+	/* Fix endianness in section headers */
+	for (i = 0; i < info->num_sections; i++) {
+		sechdrs[i].sh_name      = TO_NATIVE(sechdrs[i].sh_name);
+		sechdrs[i].sh_type      = TO_NATIVE(sechdrs[i].sh_type);
+		sechdrs[i].sh_flags     = TO_NATIVE(sechdrs[i].sh_flags);
+		sechdrs[i].sh_addr      = TO_NATIVE(sechdrs[i].sh_addr);
+		sechdrs[i].sh_offset    = TO_NATIVE(sechdrs[i].sh_offset);
+		sechdrs[i].sh_size      = TO_NATIVE(sechdrs[i].sh_size);
+		sechdrs[i].sh_link      = TO_NATIVE(sechdrs[i].sh_link);
+		sechdrs[i].sh_info      = TO_NATIVE(sechdrs[i].sh_info);
+		sechdrs[i].sh_addralign = TO_NATIVE(sechdrs[i].sh_addralign);
+		sechdrs[i].sh_entsize   = TO_NATIVE(sechdrs[i].sh_entsize);
+	}
+	/* Find symbol table. */
+	secstrings = (void *)hdr + sechdrs[info->secindex_strings].sh_offset;
+	for (i = 1; i < info->num_sections; i++) {
+		int nobits = sechdrs[i].sh_type == SHT_NOBITS;
+
+		if (!nobits && sechdrs[i].sh_offset > info->size) {
+			fprintf(stderr, "%s is truncated. sechdrs[i].sh_offset=%lu > "
+			      "sizeof(*hrd)=%zu\n", filename,
+			      (unsigned long)sechdrs[i].sh_offset,
+			      sizeof(*hdr));
+			return 0;
+		}
+
+		if (sechdrs[i].sh_type == SHT_SYMTAB) {
+			unsigned int sh_link_idx;
+			symtab_idx = i;
+			info->symtab_start = (void *)hdr +
+			    sechdrs[i].sh_offset;
+			info->symtab_stop  = (void *)hdr +
+			    sechdrs[i].sh_offset + sechdrs[i].sh_size;
+			sh_link_idx = sechdrs[i].sh_link;
+			info->strtab       = (void *)hdr +
+			    sechdrs[sh_link_idx].sh_offset;
+		}
+
+		/* 32bit section no. table? ("more than 64k sections") */
+		if (sechdrs[i].sh_type == SHT_SYMTAB_SHNDX) {
+			symtab_shndx_idx = i;
+			info->symtab_shndx_start = (void *)hdr +
+			    sechdrs[i].sh_offset;
+			info->symtab_shndx_stop  = (void *)hdr +
+			    sechdrs[i].sh_offset + sechdrs[i].sh_size;
+		}
+	}
+	if (!info->symtab_start)
+		fprintf(stderr, "%s has no symtab?\n", filename);
+
+	/* Fix endianness in symbols */
+	for (sym = info->symtab_start; sym < info->symtab_stop; sym++) {
+		sym->st_shndx = TO_NATIVE(sym->st_shndx);
+		sym->st_name  = TO_NATIVE(sym->st_name);
+		sym->st_value = TO_NATIVE(sym->st_value);
+		sym->st_size  = TO_NATIVE(sym->st_size);
+	}
+
+	if (symtab_shndx_idx != ~0U) {
+		Elf32_Word *p;
+		if (symtab_idx != sechdrs[symtab_shndx_idx].sh_link)
+			fprintf(stderr, "%s: SYMTAB_SHNDX has bad sh_link: %u!=%u\n",
+			      filename, sechdrs[symtab_shndx_idx].sh_link,
+			      symtab_idx);
+		/* Fix endianness */
+		for (p = info->symtab_shndx_start; p < info->symtab_shndx_stop;
+		     p++)
+			*p = TO_NATIVE(*p);
+	}
+
+	return 1;
+}
+
+static void parse_elf_finish(struct elf_info *info)
+{
+	struct pmd_driver *tmp, *idx = info->drivers;
+	release_file(info->hdr, info->size);
+	while (idx) {
+		tmp = idx->next;
+		free(idx);
+		idx = tmp;
+	}
+}
+
+static const char *sec_name(struct elf_info *elf, int secindex)
+{
+	Elf_Shdr *sechdrs = elf->sechdrs;
+	return (void *)elf->hdr +
+		elf->sechdrs[elf->secindex_strings].sh_offset +
+		sechdrs[secindex].sh_name;
+}
+
+static int get_symbol_index(struct elf_info *info, Elf64_Sym *sym)
+{
+	const char *name =  sym_name(info, sym);
+	const char *idx;
+
+	idx = name;
+	while (idx) {
+		if (isdigit(*idx))
+			return atoi(idx);
+		idx++;
+	}
+	return -1;
+}
+
+struct opt_tag {
+	const char* suffix;
+	const char* json_id;
+};
+
+static const struct opt_tag opt_tags[] = {
+	{"_param_string_export", "params"},
+};
+
+static int complete_pmd_entry(struct elf_info *info, struct pmd_driver *drv)
+{
+	const char *tname;
+	int i;
+	char tmpsymname[128];
+	Elf_Sym *tmpsym;
+	
+
+	drv->name = get_sym_value(info, drv->name_sym);
+
+	for (i=0; i<PMD_OPT_MAX; i++) {
+		memset(tmpsymname, 0, 128);
+		sprintf(tmpsymname, "__%s%s", drv->name, opt_tags[i].suffix);
+		tmpsym = find_sym_in_symtab(info, tmpsymname, NULL);
+		if (!tmpsym)
+			continue;
+		drv->opt_vals[i] = get_sym_value(info, tmpsym);
+	}
+
+	memset(tmpsymname, 0, 128);
+	sprintf(tmpsymname, "__%s_pci_tbl_export", drv->name);
+
+	tmpsym = find_sym_in_symtab(info, tmpsymname, NULL);
+
+
+	/*
+ 	 * If this returns NULL, then this is a PMD_VDEV, because
+ 	 * it has no pci table reference
+ 	 */
+	if (!tmpsym) {
+		drv->pci_tbl = NULL;
+		return 0;
+	}
+
+	tname = get_sym_value(info, tmpsym);
+	tmpsym = find_sym_in_symtab(info, tname, NULL);
+	if (!tmpsym)
+		return -ENOENT;
+
+	drv->pci_tbl = (struct rte_pci_id *)get_sym_value(info, tmpsym);
+	if (!drv->pci_tbl)
+		return -ENOENT;
+
+	return 0;
+}
+
+static int locate_pmd_entries(struct elf_info *info)
+{
+	Elf_Sym *last = NULL;
+	struct pmd_driver *new;
+
+	info->drivers = NULL;
+
+	do {
+		new = calloc(sizeof(struct pmd_driver), 1);
+		new->name_sym = find_sym_in_symtab(info, "this_pmd_name", last);
+		last = new->name_sym;
+		if (!new->name_sym)
+			free(new);
+		else {
+			if (complete_pmd_entry(info, new)) {
+				fprintf(stderr, "Failed to complete pmd entry\n");
+				free(new);
+			} else {
+				new->next = info->drivers;
+				info->drivers = new;
+			}
+		}
+	} while (last);
+}
+
+static void output_pmd_info_string(struct elf_info *info, char *outfile)
+{
+	FILE *ofd;
+	struct pmd_driver *drv;
+	struct rte_pci_id *pci_ids;
+	int idx = 0;
+
+	ofd = fopen(outfile, "w+");
+	if (!ofd) {
+		fprintf(stderr, "Unable to open output file\n");
+		return;
+	}
+
+	drv = info->drivers;
+
+	while (drv) {
+		fprintf(ofd, "const char %s_pmd_info[] __attribute__((used)) = \"PMD_INFO_STRING= {",
+			drv->name);
+		fprintf(ofd,"\\\"name\\\" : \\\"%s\\\", ", drv->name);
+
+		for(idx=0; idx<PMD_OPT_MAX; idx++) {
+			if (drv->opt_vals[idx])
+				fprintf(ofd,"\\\"%s\\\" : \\\"%s\\\", ", opt_tags[idx].json_id,
+					drv->opt_vals[idx]);
+		}
+
+		pci_ids = drv->pci_tbl;
+		fprintf(ofd, "\\\"pci_ids\\\" : [");
+
+		while (pci_ids && pci_ids->device_id) {
+			fprintf(ofd, "[%d, %d, %d, %d]",
+				pci_ids->vendor_id, pci_ids->device_id,
+				pci_ids->subsystem_vendor_id,
+				pci_ids->subsystem_device_id);
+			pci_ids++;
+			if (pci_ids->device_id)
+				fprintf(ofd, ",");
+			else
+				fprintf(ofd, " ");
+		}
+		fprintf(ofd, "]}\";");
+		drv = drv->next;
+	}
+
+	fclose(ofd);
+}
+
+int main(int argc, char **argv)
+{
+	struct elf_info info;
+	int rc = 1;
+
+	if (argc < 3) {
+		fprintf(stderr, "usage: pmdinfo <object file> <c output file>\n");
+		exit(127);
+	}
+	parse_elf(&info, argv[1]);
+
+	locate_pmd_entries(&info);
+
+	if (info.drivers) {
+		output_pmd_info_string(&info, argv[2]);
+		rc = 0;
+	} else {
+		fprintf(stderr, "Hmm, Appears to be a driver but no drivers registered\n");
+	}
+
+	parse_elf_finish(&info);
+	exit(rc);
+}
diff --git a/buildtools/pmdinfogen/pmdinfogen.h b/buildtools/pmdinfogen/pmdinfogen.h
new file mode 100644
index 0000000..7b158c9
--- /dev/null
+++ b/buildtools/pmdinfogen/pmdinfogen.h
@@ -0,0 +1,94 @@
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdarg.h>
+#include <string.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/mman.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <elf.h>
+#include <rte_pci.h>
+
+/* On BSD-alike OSes elf.h defines these according to host's word size */
+#undef ELF_ST_BIND
+#undef ELF_ST_TYPE
+#undef ELF_R_SYM
+#undef ELF_R_TYPE
+
+/*
+ * Define ELF64_* to ELF_*, the latter being defined in both 32 and 64 bit
+ * flavors in elf.h.  This makes our code a bit more generic between arches
+ * and allows us to support 32 bit code in the future should we ever want to
+ */
+#if __x86_64__ || __aarch64__
+#define Elf_Ehdr    Elf64_Ehdr
+#define Elf_Shdr    Elf64_Shdr
+#define Elf_Sym     Elf64_Sym
+#define Elf_Addr    Elf64_Addr
+#define Elf_Sword   Elf64_Sxword
+#define Elf_Section Elf64_Half
+#define ELF_ST_BIND ELF64_ST_BIND
+#define ELF_ST_TYPE ELF64_ST_TYPE
+
+#define Elf_Rel     Elf64_Rel
+#define Elf_Rela    Elf64_Rela
+#define ELF_R_SYM   ELF64_R_SYM
+#define ELF_R_TYPE  ELF64_R_TYPE
+#else
+#define Elf_Ehdr    Elf32_Ehdr
+#define Elf_Shdr    Elf32_Shdr
+#define Elf_Sym     Elf32_Sym
+#define Elf_Addr    Elf32_Addr
+#define Elf_Sword   Elf32_Sxword
+#define Elf_Section Elf32_Half
+#define ELF_ST_BIND ELF32_ST_BIND
+#define ELF_ST_TYPE ELF32_ST_TYPE
+
+#define Elf_Rel     Elf32_Rel
+#define Elf_Rela    Elf32_Rela
+#define ELF_R_SYM   ELF32_R_SYM
+#define ELF_R_TYPE  ELF32_R_TYPE
+#endif
+
+#define TO_NATIVE(x) (x)
+
+enum opt_params {
+	PMD_PARAM_STRING = 0,
+	PMD_OPT_MAX
+};
+
+struct pmd_driver {
+	Elf_Sym *name_sym;
+	const char *name;
+	struct rte_pci_id *pci_tbl;
+	struct pmd_driver *next;
+
+	const char* opt_vals[PMD_OPT_MAX];
+};
+
+struct elf_info {
+	unsigned long size;
+	Elf_Ehdr     *hdr;
+	Elf_Shdr     *sechdrs;
+	Elf_Sym      *symtab_start;
+	Elf_Sym      *symtab_stop;
+	Elf_Section  export_sec;
+	Elf_Section  export_unused_sec;
+	Elf_Section  export_gpl_sec;
+	Elf_Section  export_unused_gpl_sec;
+	Elf_Section  export_gpl_future_sec;
+	char         *strtab;
+
+	/* support for 32bit section numbers */
+
+	unsigned int num_sections; /* max_secindex + 1 */
+	unsigned int secindex_strings;
+	/* if Nth symbol table entry has .st_shndx = SHN_XINDEX,
+	 * take shndx from symtab_shndx_start[N] instead */
+	Elf32_Word   *symtab_shndx_start;
+	Elf32_Word   *symtab_shndx_stop;
+
+	struct pmd_driver *drivers;
+};
+
diff --git a/mk/rte.buildtools.mk b/mk/rte.buildtools.mk
new file mode 100644
index 0000000..e8bfcef
--- /dev/null
+++ b/mk/rte.buildtools.mk
@@ -0,0 +1,148 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
+#   Copyright(c) 2014-2015 6WIND S.A.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/internal/rte.compile-pre.mk
+include $(RTE_SDK)/mk/internal/rte.install-pre.mk
+include $(RTE_SDK)/mk/internal/rte.clean-pre.mk
+include $(RTE_SDK)/mk/internal/rte.build-pre.mk
+include $(RTE_SDK)/mk/internal/rte.depdirs-pre.mk
+
+# VPATH contains at least SRCDIR
+VPATH += $(SRCDIR)
+
+_BUILD = $(APP)
+_INSTALL = $(INSTALL-FILES-y) $(SYMLINK-FILES-y)
+_INSTALL += $(RTE_OUTPUT)/buildtools/$(APP) $(RTE_OUTPUT)/buildtools/$(APP).map
+POSTINSTALL += target-appinstall
+_CLEAN = doclean
+POSTCLEAN += target-appclean
+
+.PHONY: all
+all: install
+
+.PHONY: install
+install: build _postinstall
+
+_postinstall: build
+
+.PHONY: build
+build: _postbuild
+
+exe2cmd = $(strip $(call dotfile,$(patsubst %,%.cmd,$(1))))
+
+ifeq ($(LINK_USING_CC),1)
+override EXTRA_LDFLAGS := $(call linkerprefix,$(EXTRA_LDFLAGS))
+O_TO_EXE = $(CC) $(CFLAGS) $(LDFLAGS_$(@)) \
+	-Wl,-Map=$(@).map,--cref -o $@ $(OBJS-y) $(call linkerprefix,$(LDFLAGS)) \
+	$(EXTRA_LDFLAGS) $(call linkerprefix,$(LDLIBS))
+else
+O_TO_EXE = $(LD) $(LDFLAGS) $(LDFLAGS_$(@)) $(EXTRA_LDFLAGS) \
+	-Map=$(@).map --cref -o $@ $(OBJS-y) $(LDLIBS)
+endif
+O_TO_EXE_STR = $(subst ','\'',$(O_TO_EXE)) #'# fix syntax highlight
+O_TO_EXE_DISP = $(if $(V),"$(O_TO_EXE_STR)","  LD $(@)")
+O_TO_EXE_CMD = "cmd_$@ = $(O_TO_EXE_STR)"
+O_TO_EXE_DO = @set -e; \
+	echo $(O_TO_EXE_DISP); \
+	$(O_TO_EXE) && \
+	echo $(O_TO_EXE_CMD) > $(call exe2cmd,$(@))
+
+-include .$(APP).cmd
+
+# path where libraries are retrieved
+LDLIBS_PATH := $(subst -Wl$(comma)-L,,$(filter -Wl$(comma)-L%,$(LDLIBS)))
+LDLIBS_PATH += $(subst -L,,$(filter -L%,$(LDLIBS)))
+
+# list of .a files that are linked to this application
+LDLIBS_NAMES := $(patsubst -l%,lib%.a,$(filter -l%,$(LDLIBS)))
+LDLIBS_NAMES += $(patsubst -Wl$(comma)-l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIBS)))
+
+# list of found libraries files (useful for deps). If not found, the
+# library is silently ignored and dep won't be checked
+LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
+	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
+
+#
+# Compile executable file if needed
+#
+$(APP): $(OBJS-y) $(LDLIBS_FILES) $(DEP_$(APP)) $(LDSCRIPT) FORCE
+	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
+	$(if $(D),\
+		@echo -n "$< -> $@ " ; \
+		echo -n "file_missing=$(call boolean,$(file_missing)) " ; \
+		echo -n "cmdline_changed=$(call boolean,$(call cmdline_changed,$(O_TO_EXE_STR))) " ; \
+		echo -n "depfile_missing=$(call boolean,$(depfile_missing)) " ; \
+		echo "depfile_newer=$(call boolean,$(depfile_newer)) ")
+	$(if $(or \
+		$(file_missing),\
+		$(call cmdline_changed,$(O_TO_EXE_STR)),\
+		$(depfile_missing),\
+		$(depfile_newer)),\
+		$(O_TO_EXE_DO))
+
+#
+# install app in $(RTE_OUTPUT)/app
+#
+$(RTE_OUTPUT)/buildtools/$(APP): $(APP)
+	@echo "  INSTALL-APP $(APP)"
+	@[ -d $(RTE_OUTPUT)/buildtools ] || mkdir -p $(RTE_OUTPUT)/buildtools
+	$(Q)cp -f $(APP) $(RTE_OUTPUT)/buildtools
+
+#
+# install app map file in $(RTE_OUTPUT)/app
+#
+$(RTE_OUTPUT)/buildtools/$(APP).map: $(APP)
+	@echo "  INSTALL-MAP $(APP).map"
+	@[ -d $(RTE_OUTPUT)/buildtools ] || mkdir -p $(RTE_OUTPUT)/buildtools
+	$(Q)cp -f $(APP).map $(RTE_OUTPUT)/buildtools
+
+#
+# Clean all generated files
+#
+.PHONY: clean
+clean: _postclean
+	$(Q)rm -f $(_BUILD_TARGETS) $(_INSTALL_TARGETS) $(_CLEAN_TARGETS)
+
+.PHONY: doclean
+doclean:
+	$(Q)rm -rf $(APP) $(OBJS-all) $(DEPS-all) $(DEPSTMP-all) \
+	  $(CMDS-all) $(INSTALL-FILES-all) .$(APP).cmd
+
+
+include $(RTE_SDK)/mk/internal/rte.compile-post.mk
+include $(RTE_SDK)/mk/internal/rte.install-post.mk
+include $(RTE_SDK)/mk/internal/rte.clean-post.mk
+include $(RTE_SDK)/mk/internal/rte.build-post.mk
+include $(RTE_SDK)/mk/internal/rte.depdirs-post.mk
+
+.PHONY: FORCE
+FORCE:
diff --git a/mk/rte.sdkbuild.mk b/mk/rte.sdkbuild.mk
index eec5241..fb68af2 100644
--- a/mk/rte.sdkbuild.mk
+++ b/mk/rte.sdkbuild.mk
@@ -64,7 +64,8 @@ build: $(ROOTDIRS-y)
 clean: $(CLEANDIRS)
 	@rm -rf $(RTE_OUTPUT)/include $(RTE_OUTPUT)/app \
 		$(RTE_OUTPUT)/hostapp $(RTE_OUTPUT)/lib \
-		$(RTE_OUTPUT)/hostlib $(RTE_OUTPUT)/kmod
+		$(RTE_OUTPUT)/hostlib $(RTE_OUTPUT)/kmod \
+		$(RTE_OUTPUT)/buildtools
 	@[ -d $(RTE_OUTPUT)/include ] || mkdir -p $(RTE_OUTPUT)/include
 	@$(RTE_SDK)/scripts/gen-config-h.sh $(RTE_OUTPUT)/.config \
 		> $(RTE_OUTPUT)/include/rte_config.h
-- 
2.5.5

[PATCHv6 2/7] drivers: Update driver registration macro usage

[Neil Horman]

Modify the PMD_REGISTER_DRIVER macro, adding a name argument to it.  The
addition of a name argument creates a token that can be used for subsequent
macros in the creation of unique symbol names to export additional bits of
information for use by the pmdinfogen tool.  For example:

PMD_REGISTER_DRIVER(ena_driver, ena);

registers the ena_driver struct as it always did, and creates a symbol
const char this_pmd_name0[] __attribute__((used)) = "ena";

which pmdinfogen can search for and extract.  The subsequent macro

DRIVER_REGISTER_PCI_TABLE(ena, ena_pci_id_map);

creates a symbol
const char ena_pci_tbl_export[] __attribute__((used)) = "ena_pci_id_map";

Which allows pmdinfogen to find the pci table of this driver

Using this pattern, we can export arbitrary bits of information.

pmdinfo uses this information to extract hardware support from an object file
and create a json string to make hardware support info discoverable later.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 drivers/Makefile                           |  2 ++
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c   |  4 +++-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c |  4 +++-
 drivers/crypto/null/null_crypto_pmd.c      |  4 +++-
 drivers/crypto/qat/rte_qat_cryptodev.c     |  4 +++-
 drivers/crypto/snow3g/rte_snow3g_pmd.c     |  4 +++-
 drivers/net/af_packet/rte_eth_af_packet.c  |  4 +++-
 drivers/net/bnx2x/bnx2x_ethdev.c           |  6 ++++--
 drivers/net/bonding/rte_eth_bond_pmd.c     |  7 ++++++-
 drivers/net/cxgbe/cxgbe_ethdev.c           |  4 +++-
 drivers/net/e1000/em_ethdev.c              |  3 ++-
 drivers/net/e1000/igb_ethdev.c             |  6 ++++--
 drivers/net/ena/ena_ethdev.c               |  3 ++-
 drivers/net/enic/enic_ethdev.c             |  3 ++-
 drivers/net/fm10k/fm10k_ethdev.c           |  3 ++-
 drivers/net/i40e/i40e_ethdev.c             |  3 ++-
 drivers/net/i40e/i40e_ethdev_vf.c          |  3 ++-
 drivers/net/ixgbe/ixgbe_ethdev.c           |  6 ++++--
 drivers/net/mlx4/mlx4.c                    |  3 ++-
 drivers/net/mlx5/mlx5.c                    |  3 ++-
 drivers/net/mpipe/mpipe_tilegx.c           |  4 ++--
 drivers/net/nfp/nfp_net.c                  |  3 ++-
 drivers/net/null/rte_eth_null.c            |  3 ++-
 drivers/net/pcap/rte_eth_pcap.c            |  4 +++-
 drivers/net/qede/qede_ethdev.c             |  6 ++++--
 drivers/net/ring/rte_eth_ring.c            |  3 ++-
 drivers/net/szedata2/rte_eth_szedata2.c    |  3 ++-
 drivers/net/vhost/rte_eth_vhost.c          |  3 ++-
 drivers/net/virtio/virtio_ethdev.c         |  3 ++-
 drivers/net/vmxnet3/vmxnet3_ethdev.c       |  3 ++-
 drivers/net/xenvirt/rte_eth_xenvirt.c      |  2 +-
 lib/librte_eal/common/include/rte_dev.h    | 27 +++++++++++++++++++++------
 32 files changed, 102 insertions(+), 41 deletions(-)

diff --git a/drivers/Makefile b/drivers/Makefile
index 81c03a8..75a3168 100644
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -34,4 +34,6 @@ include $(RTE_SDK)/mk/rte.vars.mk
 DIRS-y += net
 DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += crypto
 
+DEPDIRS-y += buildtools/pmdinfo
+
 include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
index 2987ef6..593769f 100644
--- a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
+++ b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
@@ -521,4 +521,6 @@ static struct rte_driver aesni_gcm_pmd_drv = {
 	.uninit = aesni_gcm_uninit
 };
 
-PMD_REGISTER_DRIVER(aesni_gcm_pmd_drv);
+PMD_REGISTER_DRIVER(aesni_gcm_pmd_drv, aesni_gcm);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> \
+max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
index 3415ac1..2e30939 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
@@ -716,4 +716,6 @@ static struct rte_driver cryptodev_aesni_mb_pmd_drv = {
 	.uninit = cryptodev_aesni_mb_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drv);
+PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drvi, aesni_mb);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> \
+max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/null/null_crypto_pmd.c b/drivers/crypto/null/null_crypto_pmd.c
index bdaf13c..44c4fba 100644
--- a/drivers/crypto/null/null_crypto_pmd.c
+++ b/drivers/crypto/null/null_crypto_pmd.c
@@ -275,4 +275,6 @@ static struct rte_driver cryptodev_null_pmd_drv = {
 	.uninit = cryptodev_null_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_null_pmd_drv);
+PMD_REGISTER_DRIVER(cryptodev_null_pmd_drv, cryptodev_null_pmd);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> \
+max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index a7912f5..d2de6a6 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -137,4 +137,6 @@ static struct rte_driver pmd_qat_drv = {
 	.init = rte_qat_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(pmd_qat_drv);
+PMD_REGISTER_DRIVER(pmd_qat_drv, qat);
+DRIVER_REGISTER_PCI_TABLE(qat, pci_id_qat_map);
+
diff --git a/drivers/crypto/snow3g/rte_snow3g_pmd.c b/drivers/crypto/snow3g/rte_snow3g_pmd.c
index f3e0e66..eee2544 100644
--- a/drivers/crypto/snow3g/rte_snow3g_pmd.c
+++ b/drivers/crypto/snow3g/rte_snow3g_pmd.c
@@ -548,4 +548,6 @@ static struct rte_driver cryptodev_snow3g_pmd_drv = {
 	.uninit = cryptodev_snow3g_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_snow3g_pmd_drv);
+PMD_REGISTER_DRIVER(cryptodev_snow3g_pmd_drv, snow3g);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> \
+max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index f17bd7e..e960512 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -878,4 +878,6 @@ static struct rte_driver pmd_af_packet_drv = {
 	.uninit = rte_pmd_af_packet_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_af_packet_drv);
+PMD_REGISTER_DRIVER(pmd_af_packet_drv, eth_af_packet);
+DRIVER_REGISTER_PARAM_STRING(eth_af_packet, "iface=<string> \
+qpairs=<int> blocksz=<int> framesz=<int> framecnt=<int>");
diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c b/drivers/net/bnx2x/bnx2x_ethdev.c
index 071b44f..2b88324 100644
--- a/drivers/net/bnx2x/bnx2x_ethdev.c
+++ b/drivers/net/bnx2x/bnx2x_ethdev.c
@@ -550,5 +550,7 @@ static struct rte_driver rte_bnx2xvf_driver = {
 	.init = rte_bnx2xvf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_bnx2x_driver);
-PMD_REGISTER_DRIVER(rte_bnx2xvf_driver);
+PMD_REGISTER_DRIVER(rte_bnx2x_driver, bnx2x);
+DRIVER_REGISTER_PCI_TABLE(bnx2x, pci_id_bnx2x_map);
+PMD_REGISTER_DRIVER(rte_bnx2xvf_driver, bnx2xvf);
+DRIVER_REGISTER_PCI_TABLE(bnx2xvf, pci_id_bnx2xvf_map);
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 129f04b..b4556be 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -2511,4 +2511,9 @@ static struct rte_driver bond_drv = {
 	.uninit = bond_uninit,
 };
 
-PMD_REGISTER_DRIVER(bond_drv);
+PMD_REGISTER_DRIVER(bond_drv, bonding);
+
+DRIVER_REGISTER_PARAM_STRING(bonding, "slave=<ifc> primary=<ifc> mode=[0-4] \
+xmit_policy=[l2 | l23 | l34] socket_id=<int> mac=<mac addr> \
+lsc_poll_period_ms=<int> up_delay=<int> down_delay=<int>");
+
diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index 04eddaf..e84e48b 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -894,4 +894,6 @@ static struct rte_driver rte_cxgbe_driver = {
 	.init = rte_cxgbe_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_cxgbe_driver);
+PMD_REGISTER_DRIVER(rte_cxgbe_driver, cxgb4);
+DRIVER_REGISTER_PCI_TABLE(cxgb4, cxgb4_pci_tbl);
+
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index 653be09..4de5eb2 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -1777,4 +1777,5 @@ struct rte_driver em_pmd_drv = {
 	.init = rte_em_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(em_pmd_drv);
+PMD_REGISTER_DRIVER(em_pmd_drv, em);
+DRIVER_REGISTER_PCI_TABLE(em, pci_id_em_map);
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index f0921ee..f49aae7 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -4973,5 +4973,7 @@ eth_igb_configure_msix_intr(struct rte_eth_dev *dev)
 	E1000_WRITE_FLUSH(hw);
 }
 
-PMD_REGISTER_DRIVER(pmd_igb_drv);
-PMD_REGISTER_DRIVER(pmd_igbvf_drv);
+PMD_REGISTER_DRIVER(pmd_igb_drv, igb);
+DRIVER_REGISTER_PCI_TABLE(igb, pci_id_igb_map);
+PMD_REGISTER_DRIVER(pmd_igbvf_drv, igbvf);
+DRIVER_REGISTER_PCI_TABLE(igbvf, pci_id_igbvf_map);
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index e157587..702289b 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -1450,4 +1450,5 @@ struct rte_driver ena_pmd_drv = {
 	.init = rte_ena_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(ena_pmd_drv);
+PMD_REGISTER_DRIVER(ena_pmd_drv, ena);
+DRIVER_REGISTER_PCI_TABLE(ena, pci_id_ena_map);
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index 6bea940..36ee7ca 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -684,4 +684,5 @@ static struct rte_driver rte_enic_driver = {
 	.init = rte_enic_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_enic_driver);
+PMD_REGISTER_DRIVER(rte_enic_driver, enic);
+DRIVER_REGISTER_PCI_TABLE(enic, pci_id_enic_map);
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index c2d377f..9b2a7ee 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -3055,4 +3055,5 @@ static struct rte_driver rte_fm10k_driver = {
 	.init = rte_pmd_fm10k_init,
 };
 
-PMD_REGISTER_DRIVER(rte_fm10k_driver);
+PMD_REGISTER_DRIVER(rte_fm10k_driver, fm10k);
+DRIVER_REGISTER_PCI_TABLE(fm10k, pci_id_fm10k_map);
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 24777d5..3b6c661 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -695,7 +695,8 @@ static struct rte_driver rte_i40e_driver = {
 	.init = rte_i40e_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_i40e_driver);
+PMD_REGISTER_DRIVER(rte_i40e_driver, i40e);
+DRIVER_REGISTER_PCI_TABLE(i40e, pci_id_i40e_map);
 
 /*
  * Initialize registers for flexible payload, which should be set by NVM.
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 90682ac..e94a459 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1563,7 +1563,8 @@ static struct rte_driver rte_i40evf_driver = {
 	.init = rte_i40evf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_i40evf_driver);
+PMD_REGISTER_DRIVER(rte_i40evf_driver, i40evf);
+DRIVER_REGISTER_PCI_TABLE(i40evf, pci_id_i40evf_map);
 
 static int
 i40evf_dev_configure(struct rte_eth_dev *dev)
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index a2b170b..af1ca56 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -7164,5 +7164,7 @@ static struct rte_driver rte_ixgbevf_driver = {
 	.init = rte_ixgbevf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_ixgbe_driver);
-PMD_REGISTER_DRIVER(rte_ixgbevf_driver);
+PMD_REGISTER_DRIVER(rte_ixgbe_driver, ixgbe);
+DRIVER_REGISTER_PCI_TABLE(ixgbe, pci_id_ixgbe_map);
+PMD_REGISTER_DRIVER(rte_ixgbevf_driver, ixgbevf);
+DRIVER_REGISTER_PCI_TABLE(ixgbevf, pci_id_ixgbevf_map);
diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index 9ed1491..aedf1e9 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -5865,4 +5865,5 @@ static struct rte_driver rte_mlx4_driver = {
 	.init = rte_mlx4_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_mlx4_driver)
+PMD_REGISTER_DRIVER(rte_mlx4_driveri, mlx4)
+DRIVER_REGISTER_PCI_TABLE(mlx4, mlx4_pci_id_map);
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 041cfc3..5f2d6c8 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -672,4 +672,5 @@ static struct rte_driver rte_mlx5_driver = {
 	.init = rte_mlx5_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_mlx5_driver)
+PMD_REGISTER_DRIVER(rte_mlx5_driveri, mlx5)
+DRIVER_REGISTER_PCI_TABLE(mlx5, mlx5_pci_id_map);
diff --git a/drivers/net/mpipe/mpipe_tilegx.c b/drivers/net/mpipe/mpipe_tilegx.c
index 26e1424..4a2a905 100644
--- a/drivers/net/mpipe/mpipe_tilegx.c
+++ b/drivers/net/mpipe/mpipe_tilegx.c
@@ -1635,8 +1635,8 @@ static struct rte_driver pmd_mpipe_gbe_drv = {
 	.init = rte_pmd_mpipe_devinit,
 };
 
-PMD_REGISTER_DRIVER(pmd_mpipe_xgbe_drv);
-PMD_REGISTER_DRIVER(pmd_mpipe_gbe_drv);
+PMD_REGISTER_DRIVER(pmd_mpipe_xgbe_drv, mpipe_xgbe);
+PMD_REGISTER_DRIVER(pmd_mpipe_gbe_drv, mpipe_gbe);
 
 static void __attribute__((constructor, used))
 mpipe_init_contexts(void)
diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
index ea5a2a3..78394a1 100644
--- a/drivers/net/nfp/nfp_net.c
+++ b/drivers/net/nfp/nfp_net.c
@@ -2490,7 +2490,8 @@ static struct rte_driver rte_nfp_net_driver = {
 	.init = nfp_net_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_nfp_net_driver);
+PMD_REGISTER_DRIVER(rte_nfp_net_driver, nfp);
+DRIVER_REGISTER_PCI_TABLE(nfp, pci_id_nfp_net_map);
 
 /*
  * Local variables:
diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index 5e8e203..88eb582 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -689,4 +689,5 @@ static struct rte_driver pmd_null_drv = {
 	.uninit = rte_pmd_null_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_null_drv);
+PMD_REGISTER_DRIVER(pmd_null_drv, eth_null);
+DRIVER_REGISTER_PARAM_STRING(eth_null, "size=<int> copy=<int>");
diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
index c98e234..397f01c 100644
--- a/drivers/net/pcap/rte_eth_pcap.c
+++ b/drivers/net/pcap/rte_eth_pcap.c
@@ -1090,4 +1090,6 @@ static struct rte_driver pmd_pcap_drv = {
 	.uninit = rte_pmd_pcap_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_pcap_drv);
+PMD_REGISTER_DRIVER(pmd_pcap_drv, pcap);
+DRIVER_REGISTER_PARAM_STRING(pcap, "rx_pcap=<string> tx_pcap=<string> \
+rx_iface=<ifc> tx_iface=<ifc> iface=<ifc>");
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index 1273fd3..187ccf1 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -1102,5 +1102,7 @@ static struct rte_driver rte_qede_driver = {
 	.init = rte_qedevf_pmd_init
 };
 
-PMD_REGISTER_DRIVER(rte_qede_driver);
-PMD_REGISTER_DRIVER(rte_qedevf_driver);
+PMD_REGISTER_DRIVER(rte_qede_driver, qede);
+DRIVER_REGISTER_PCI_TABLE(qede, pci_id_qede_map);
+PMD_REGISTER_DRIVER(rte_qedevf_driver, qedevf);
+DRIVER_REGISTER_PCI_TABLE(qedevf, pci_id_qedevf_map);
diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index b1783c3..f2f5217 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -630,4 +630,5 @@ static struct rte_driver pmd_ring_drv = {
 	.uninit = rte_pmd_ring_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_ring_drv);
+PMD_REGISTER_DRIVER(pmd_ring_drv, eth_ring);
+DRIVER_REGISTER_PARAM_STRING(eth_ring, "nodeaction=[attach|detach]");
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 78c43b0..abbc907 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1601,4 +1601,5 @@ static struct rte_driver rte_szedata2_driver = {
 	.uninit = rte_szedata2_uninit,
 };
 
-PMD_REGISTER_DRIVER(rte_szedata2_driver);
+PMD_REGISTER_DRIVER(rte_szedata2_driver, rte_szedata2_pmd);
+DRIVER_REGISTER_PCI_TABLE(rte_szedata2_pmd, rte_szedata2_pci_id_table);
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index 310cbef..d321c3e 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -924,4 +924,5 @@ static struct rte_driver pmd_vhost_drv = {
 	.uninit = rte_pmd_vhost_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_vhost_drv);
+PMD_REGISTER_DRIVER(pmd_vhost_drv, eth_vhost);
+DRIVER_REGISTER_PARAM_STRING(eth_vhost, "iface=<ifc> queues=<int>");
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index c3fb628..24d387f 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -1459,4 +1459,5 @@ static struct rte_driver rte_virtio_driver = {
 	.init = rte_virtio_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_virtio_driver);
+PMD_REGISTER_DRIVER(rte_virtio_driver, virtio_net);
+DRIVER_REGISTER_PCI_TABLE(virtio_net, pci_id_virtio_map);
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index 29b469c..8da4449 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -954,4 +954,5 @@ static struct rte_driver rte_vmxnet3_driver = {
 	.init = rte_vmxnet3_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_vmxnet3_driver);
+PMD_REGISTER_DRIVER(rte_vmxnet3_driver, vmxnet3);
+DRIVER_REGISTER_PCI_TABLE(vmxnet3, pci_id_vmxnet3_map);
diff --git a/drivers/net/xenvirt/rte_eth_xenvirt.c b/drivers/net/xenvirt/rte_eth_xenvirt.c
index 1adeb5b..81f8af9 100644
--- a/drivers/net/xenvirt/rte_eth_xenvirt.c
+++ b/drivers/net/xenvirt/rte_eth_xenvirt.c
@@ -763,4 +763,4 @@ static struct rte_driver pmd_xenvirt_drv = {
 	.uninit = rte_pmd_xenvirt_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_xenvirt_drv);
+PMD_REGISTER_DRIVER(pmd_xenvirt_drv, xenvirt);
diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
index f1b5507..281d20d 100644
--- a/lib/librte_eal/common/include/rte_dev.h
+++ b/lib/librte_eal/common/include/rte_dev.h
@@ -48,7 +48,7 @@ extern "C" {
 
 #include <stdio.h>
 #include <sys/queue.h>
-
+#include <rte_pci.h>
 #include <rte_log.h>
 
 __attribute__((format(printf, 2, 0)))
@@ -178,12 +178,27 @@ int rte_eal_vdev_init(const char *name, const char *args);
  */
 int rte_eal_vdev_uninit(const char *name);
 
-#define PMD_REGISTER_DRIVER(d)\
-void devinitfn_ ##d(void);\
-void __attribute__((constructor, used)) devinitfn_ ##d(void)\
+#define DRIVER_EXPORT_NAME_ARRAY(n, idx) n##idx[] __attribute__((used))
+
+#define DRIVER_EXPORT_NAME(name, idx) \
+static const char DRIVER_EXPORT_NAME_ARRAY(this_pmd_name, idx) = RTE_STR(name);\
+ 
+#define PMD_REGISTER_DRIVER(drv, nm)\
+void devinitfn_ ##drv(void);\
+void __attribute__((constructor, used)) devinitfn_ ##drv(void)\
 {\
-	rte_eal_driver_register(&d);\
-}
+	(drv).name = RTE_STR(nm);\
+        rte_eal_driver_register(&drv);\
+}\
+DRIVER_EXPORT_NAME(nm, __COUNTER__)
+
+#define DRV_EXP_TAG(name, tag) __##name##_##tag
+
+#define DRIVER_REGISTER_PCI_TABLE(name, table) \
+static const char DRV_EXP_TAG(name, pci_tbl_export)[] __attribute__((used)) = RTE_STR(table)
+
+#define DRIVER_REGISTER_PARAM_STRING(name, str) \
+static const char DRV_EXP_TAG(name, param_string_export)[] __attribute__((used)) = str
 
 #ifdef __cplusplus
 }
-- 
2.5.5

[PATCHv6 3/7] eal: Add an export symbol to expose the autoload path to external tools

[Neil Horman]

Export a symbol containing the string:
DPDK_PLUGIN_PATH="$(CONFIG_RTE_EAL_PMD_PATH)"

Where the latter half of the string is set at build time to a location from
which autoloaded DSO's will be found.  This string is used by pmdinfo in
'plugin' mode, whereby a user can specify a dpdk installation directory (or
static binary), and scan the associated path (if found) for pmd DSO's and report
on their hardware support.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 lib/librte_eal/common/eal_common_options.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c
index 3efc90f..1af0fb3 100644
--- a/lib/librte_eal/common/eal_common_options.c
+++ b/lib/librte_eal/common/eal_common_options.c
@@ -115,6 +115,15 @@ TAILQ_HEAD_INITIALIZER(solib_list);
 /* Default path of external loadable drivers */
 static const char *default_solib_dir = RTE_EAL_PMD_PATH;
 
+/*
+ * Stringified version of solib path used by pmdinfo.py 
+ * Note: PLEASE DO NOT ALTER THIS without making a corresponding
+ * change to tools/pmdinfo.py
+ */
+static const char dpdk_solib_path[] __attribute__((used)) = \
+"DPDK_PLUGIN_PATH=" RTE_EAL_PMD_PATH;
+
+
 static int master_lcore_parsed;
 static int mem_parsed;
 
-- 
2.5.5

[PATCHv6 4/7] Makefile: Do post processing on objects that register a driver

[Neil Horman]

Modify the compilation makefile to identify C files that export PMD information,
and use that to trigger execution of the pmdinfo binary.  If the execution of
pmdinfo is successful, compile the output C file to an object, and use the
linker to do relocatable linking on the resultant object file into the parent
object that it came from.  This effectively just adds the json string into the
string table of the object that defines the PMD to the outside world.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 mk/internal/rte.compile-pre.mk | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/mk/internal/rte.compile-pre.mk b/mk/internal/rte.compile-pre.mk
index b9bff4a..92517ad 100644
--- a/mk/internal/rte.compile-pre.mk
+++ b/mk/internal/rte.compile-pre.mk
@@ -88,10 +88,24 @@ C_TO_O_CMD = 'cmd_$@ = $(C_TO_O_STR)'
 C_TO_O_DO = @set -e; \
 	echo $(C_TO_O_DISP); \
 	$(C_TO_O) && \
+	sh -c "grep -q \"PMD_REGISTER_DRIVER(.*)\" $<; \
+	if [ \$$? -eq 0 ]; then \
+		echo \"  PMDINFOGEN\" $@; \
+		OBJF=`readlink -f $@`; \
+		${RTE_OUTPUT}/buildtools/pmdinfogen \$$OBJF \$$OBJF.pmd.c; \
+		if [ \$$? -eq 0 ]; \
+		then \
+			echo \"  PMDINFOBUILD\" $@; \
+			$(CC) $(CFLAGS) -c -o \$$OBJF.pmd.o \$$OBJF.pmd.c; \
+			$(CROSS)ld $(LDFLAGS) -r -o \$$OBJF.o \$$OBJF.pmd.o \$$OBJF; \
+			mv -f \$$OBJF.o \$$OBJF; \
+		fi; \
+	fi;" && \
 	echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \
 	sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \
 	rm -f $(call obj2dep,$(@)).tmp
 
+
 # return an empty string if string are equal
 compare = $(strip $(subst $(1),,$(2)) $(subst $(2),,$(1)))
 
-- 
2.5.5

[PATCHv6 5/7] pmdinfo.py: Add tool to query binaries for hw and other support information

[Neil Horman]

This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
and, if found parses the remainder of the string as a json encoded string,
outputting the results in either a human readable or raw, script parseable
format

Note that, in the case of dynamically linked applications, pmdinfo.py will scan
for implicitly linked PMDs by searching the specified binaries .dynamic section
for DT_NEEDED entries that contain the substring librte_pmd.  The DT_RUNPATH,
LD_LIBRARY_PATH, /usr/lib and /lib are searched for these libraries, in that
order

If a file is specified with no path, it is assumed to be a PMD DSO, and the
LD_LIBRARY_PATH, /usr/lib[64]/ and /lib[64] is searched for it

Currently the tool can output data in 3 formats:

a) raw, suitable for scripting, where the raw JSON strings are dumped out
b) table format (default) where hex pci ids are dumped in a table format
c) pretty, where a user supplied pci.ids file is used to print out vendor and
device strings

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 mk/rte.sdkinstall.mk |   2 +
 tools/pmdinfo.py     | 629 +++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 631 insertions(+)
 create mode 100755 tools/pmdinfo.py

diff --git a/mk/rte.sdkinstall.mk b/mk/rte.sdkinstall.mk
index 68e56b6..dc36df5 100644
--- a/mk/rte.sdkinstall.mk
+++ b/mk/rte.sdkinstall.mk
@@ -126,6 +126,8 @@ install-runtime:
 	$(Q)$(call rte_mkdir,      $(DESTDIR)$(sbindir))
 	$(Q)$(call rte_symlink,    $(DESTDIR)$(datadir)/tools/dpdk_nic_bind.py, \
 	                           $(DESTDIR)$(sbindir)/dpdk_nic_bind)
+	$(Q)$(call rte_symlink,    $(DESTDIR)$(datadir)/tools/pmdinfo.py, \
+				   $(DESTDIR)$(bindir)/dpdk-pmdinfo)
 
 install-kmod:
 ifneq ($(wildcard $O/kmod/*),)
diff --git a/tools/pmdinfo.py b/tools/pmdinfo.py
new file mode 100755
index 0000000..c82869d
--- /dev/null
+++ b/tools/pmdinfo.py
@@ -0,0 +1,629 @@
+#!/usr/bin/python
+# -------------------------------------------------------------------------
+# scripts/pmdinfo.py
+#
+# Utility to dump PMD_INFO_STRING support from an object file
+#
+# -------------------------------------------------------------------------
+import os
+import sys
+from optparse import OptionParser
+import string
+import json
+
+# For running from development directory. It should take precedence over the
+# installed pyelftools.
+sys.path.insert(0, '.')
+
+
+from elftools import __version__
+from elftools.common.exceptions import ELFError
+from elftools.common.py3compat import (
+    ifilter, byte2int, bytes2str, itervalues, str2bytes)
+from elftools.elf.elffile import ELFFile
+from elftools.elf.dynamic import DynamicSection, DynamicSegment
+from elftools.elf.enums import ENUM_D_TAG
+from elftools.elf.segments import InterpSegment
+from elftools.elf.sections import SymbolTableSection
+from elftools.elf.gnuversions import (
+    GNUVerSymSection, GNUVerDefSection,
+    GNUVerNeedSection,
+)
+from elftools.elf.relocation import RelocationSection
+from elftools.elf.descriptions import (
+    describe_ei_class, describe_ei_data, describe_ei_version,
+    describe_ei_osabi, describe_e_type, describe_e_machine,
+    describe_e_version_numeric, describe_p_type, describe_p_flags,
+    describe_sh_type, describe_sh_flags,
+    describe_symbol_type, describe_symbol_bind, describe_symbol_visibility,
+    describe_symbol_shndx, describe_reloc_type, describe_dyn_tag,
+    describe_ver_flags,
+)
+from elftools.elf.constants import E_FLAGS
+from elftools.dwarf.dwarfinfo import DWARFInfo
+from elftools.dwarf.descriptions import (
+    describe_reg_name, describe_attr_value, set_global_machine_arch,
+    describe_CFI_instructions, describe_CFI_register_rule,
+    describe_CFI_CFA_rule,
+)
+from elftools.dwarf.constants import (
+    DW_LNS_copy, DW_LNS_set_file, DW_LNE_define_file)
+from elftools.dwarf.callframe import CIE, FDE
+
+raw_output = False
+pcidb = None
+
+# ===========================================
+
+
+class Vendor:
+    """
+    Class for vendors. This is the top level class
+    for the devices belong to a specific vendor.
+    self.devices is the device dictionary
+    subdevices are in each device.
+    """
+
+    def __init__(self, vendorStr):
+        """
+        Class initializes with the raw line from pci.ids
+        Parsing takes place inside __init__
+        """
+        self.ID = vendorStr.split()[0]
+        self.name = vendorStr.replace("%s " % self.ID, "").rstrip()
+        self.devices = {}
+
+    def addDevice(self, deviceStr):
+        """
+        Adds a device to self.devices
+        takes the raw line from pci.ids
+        """
+        s = deviceStr.strip()
+        devID = s.split()[0]
+        if devID in self.devices:
+            pass
+        else:
+            self.devices[devID] = Device(deviceStr)
+
+    def report(self):
+        print self.ID, self.name
+        for id, dev in self.devices.items():
+            dev.report()
+
+    def find_device(self, devid):
+        # convert to a hex string and remove 0x
+        devid = hex(devid)[2:]
+        try:
+            return self.devices[devid]
+        except:
+            return Device("%s  Unknown Device" % devid)
+
+
+class Device:
+
+    def __init__(self, deviceStr):
+        """
+        Class for each device.
+        Each vendor has its own devices dictionary.
+        """
+        s = deviceStr.strip()
+        self.ID = s.split()[0]
+        self.name = s.replace("%s  " % self.ID, "")
+        self.subdevices = {}
+
+    def report(self):
+        print "\t%s\t%s" % (self.ID, self.name)
+        for subID, subdev in self.subdevices.items():
+            subdev.report()
+
+    def addSubDevice(self, subDeviceStr):
+        """
+        Adds a subvendor, subdevice to device.
+        Uses raw line from pci.ids
+        """
+        s = subDeviceStr.strip()
+        spl = s.split()
+        subVendorID = spl[0]
+        subDeviceID = spl[1]
+        subDeviceName = s.split("  ")[-1]
+        devID = "%s:%s" % (subVendorID, subDeviceID)
+        self.subdevices[devID] = SubDevice(
+            subVendorID, subDeviceID, subDeviceName)
+
+    def find_subid(self, subven, subdev):
+        subven = hex(subven)[2:]
+        subdev = hex(subdev)[2:]
+        devid = "%s:%s" % (subven, subdev)
+
+        try:
+            return self.subdevices[devid]
+        except:
+            if (subven == "ffff" and subdev == "ffff"):
+                return SubDevice("ffff", "ffff", "(All Subdevices)")
+            else:
+                return SubDevice(subven, subdev, "(Unknown Subdevice)")
+
+
+class SubDevice:
+    """
+    Class for subdevices.
+    """
+
+    def __init__(self, vendor, device, name):
+        """
+        Class initializes with vendorid, deviceid and name
+        """
+        self.vendorID = vendor
+        self.deviceID = device
+        self.name = name
+
+    def report(self):
+        print "\t\t%s\t%s\t%s" % (self.vendorID, self.deviceID, self.name)
+
+
+class PCIIds:
+    """
+    Top class for all pci.ids entries.
+    All queries will be asked to this class.
+    PCIIds.vendors["0e11"].devices["0046"].\
+    subdevices["0e11:4091"].name  =  "Smart Array 6i"
+    """
+
+    def __init__(self, filename):
+        """
+        Prepares the directories.
+        Checks local data file.
+        Tries to load from local, if not found, downloads from web
+        """
+        self.version = ""
+        self.date = ""
+        self.vendors = {}
+        self.contents = None
+        self.readLocal(filename)
+        self.parse()
+
+    def reportVendors(self):
+        """Reports the vendors
+        """
+        for vid, v in self.vendors.items():
+            print v.ID, v.name
+
+    def report(self, vendor=None):
+        """
+        Reports everything for all vendors or a specific vendor
+        PCIIds.report()  reports everything
+        PCIIDs.report("0e11") reports only "Compaq Computer Corporation"
+        """
+        if vendor is not None:
+            self.vendors[vendor].report()
+        else:
+            for vID, v in self.vendors.items():
+                v.report()
+
+    def find_vendor(self, vid):
+        # convert vid to a hex string and remove the 0x
+        vid = hex(vid)[2:]
+
+        try:
+            return self.vendors[vid]
+        except:
+            return Vendor("%s Unknown Vendor" % (vid))
+
+    def findDate(self, content):
+        for l in content:
+            if l.find("Date:") > -1:
+                return l.split()[-2].replace("-", "")
+        return None
+
+    def parse(self):
+        if len(self.contents) < 1:
+            print "data/%s-pci.ids not found" % self.date
+        else:
+            vendorID = ""
+            deviceID = ""
+            for l in self.contents:
+                if l[0] == "#":
+                    continue
+                elif len(l.strip()) == 0:
+                    continue
+                else:
+                    if l.find("\t\t") == 0:
+                        self.vendors[vendorID].devices[
+                            deviceID].addSubDevice(l)
+                    elif l.find("\t") == 0:
+                        deviceID = l.strip().split()[0]
+                        self.vendors[vendorID].addDevice(l)
+                    else:
+                        vendorID = l.split()[0]
+                        self.vendors[vendorID] = Vendor(l)
+
+    def readLocal(self, filename):
+        """
+        Reads the local file
+        """
+        self.contents = open(filename).readlines()
+        self.date = self.findDate(self.contents)
+
+    def loadLocal(self):
+        """
+        Loads database from local. If there is no file,
+        it creates a new one from web
+        """
+        self.date = idsfile[0].split("/")[1].split("-")[0]
+        self.readLocal()
+
+
+# =======================================
+
+def search_file(filename, search_path):
+    """ Given a search path, find file with requested name """
+    for path in string.split(search_path, ":"):
+        candidate = os.path.join(path, filename)
+        if os.path.exists(candidate):
+            return os.path.abspath(candidate)
+    return None
+
+
+class ReadElf(object):
+    """ display_* methods are used to emit output into the output stream
+    """
+
+    def __init__(self, file, output):
+        """ file:
+                stream object with the ELF file to read
+
+            output:
+                output stream to write to
+        """
+        self.elffile = ELFFile(file)
+        self.output = output
+
+        # Lazily initialized if a debug dump is requested
+        self._dwarfinfo = None
+
+        self._versioninfo = None
+
+    def _section_from_spec(self, spec):
+        """ Retrieve a section given a "spec" (either number or name).
+            Return None if no such section exists in the file.
+        """
+        try:
+            num = int(spec)
+            if num < self.elffile.num_sections():
+                return self.elffile.get_section(num)
+            else:
+                return None
+        except ValueError:
+            # Not a number. Must be a name then
+            return self.elffile.get_section_by_name(str2bytes(spec))
+
+    def pretty_print_pmdinfo(self, pmdinfo):
+        global pcidb
+
+        for i in pmdinfo["pci_ids"]:
+            vendor = pcidb.find_vendor(i[0])
+            device = vendor.find_device(i[1])
+            subdev = device.find_subid(i[2], i[3])
+            print("%s (%s) : %s (%s) %s" %
+                  (vendor.name, vendor.ID, device.name,
+                   device.ID, subdev.name))
+
+    def parse_pmd_info_string(self, mystring):
+        global raw_output
+        global pcidb
+
+        optional_pmd_info = [{'id': 'params', 'tag': 'PMD PARAMETERS'}]
+
+        i = mystring.index("=")
+        mystring = mystring[i + 2:]
+        pmdinfo = json.loads(mystring)
+
+        if raw_output:
+            print(pmdinfo)
+            return
+
+        print("PMD NAME: " + pmdinfo["name"])
+        for i in optional_pmd_info:
+            try:
+                print("%s: %s" % (i['tag'], pmdinfo[i['id']]))
+            except KeyError as e:
+                continue
+
+        if (len(pmdinfo["pci_ids"]) != 0):
+            print("PMD HW SUPPORT:")
+            if pcidb is not None:
+                self.pretty_print_pmdinfo(pmdinfo)
+            else:
+                print("VENDOR\t DEVICE\t SUBVENDOR\t SUBDEVICE")
+                for i in pmdinfo["pci_ids"]:
+                    print("0x%04x\t 0x%04x\t 0x%04x\t\t 0x%04x" %
+                          (i[0], i[1], i[2], i[3]))
+
+        print("")
+
+    def display_pmd_info_strings(self, section_spec):
+        """ Display a strings dump of a section. section_spec is either a
+            section number or a name.
+        """
+        section = self._section_from_spec(section_spec)
+        if section is None:
+            return
+
+        data = section.data()
+        dataptr = 0
+
+        while dataptr < len(data):
+            while (dataptr < len(data) and
+                    not (32 <= byte2int(data[dataptr]) <= 127)):
+                dataptr += 1
+
+            if dataptr >= len(data):
+                break
+
+            endptr = dataptr
+            while endptr < len(data) and byte2int(data[endptr]) != 0:
+                endptr += 1
+
+            mystring = bytes2str(data[dataptr:endptr])
+            rc = mystring.find("PMD_INFO_STRING")
+            if (rc != -1):
+                self.parse_pmd_info_string(mystring)
+
+            dataptr = endptr
+
+    def find_librte_eal(self, section):
+        for tag in section.iter_tags():
+            if tag.entry.d_tag == 'DT_NEEDED':
+                if "librte_eal" in tag.needed:
+                    return tag.needed
+        return None
+
+    def search_for_autoload_path(self):
+        scanelf = self
+        scanfile = None
+        library = None
+
+        section = self._section_from_spec(".dynamic")
+        try:
+            eallib = self.find_librte_eal(section)
+            if eallib is not None:
+                ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+                if ldlibpath is None:
+                    ldlibpath = ""
+                dtr = self.get_dt_runpath(section)
+                library = search_file(eallib,
+                                      dtr + ":" + ldlibpath +
+                                      ":/usr/lib64:/lib64:/usr/lib:/lib")
+                if library is None:
+                    return (None, None)
+                if raw_output is False:
+                    print("Scanning for autoload path in %s" % library)
+                scanfile = open(library, 'rb')
+                scanelf = ReadElf(scanfile, sys.stdout)
+        except AttributeError:
+            # Not a dynamic binary
+            pass
+        except ELFError:
+            scanfile.close()
+            return (None, None)
+
+        section = scanelf._section_from_spec(".rodata")
+        if section is None:
+            if scanfile is not None:
+                scanfile.close()
+            return (None, None)
+
+        data = section.data()
+        dataptr = 0
+
+        while dataptr < len(data):
+            while (dataptr < len(data) and
+                    not (32 <= byte2int(data[dataptr]) <= 127)):
+                dataptr += 1
+
+            if dataptr >= len(data):
+                break
+
+            endptr = dataptr
+            while endptr < len(data) and byte2int(data[endptr]) != 0:
+                endptr += 1
+
+            mystring = bytes2str(data[dataptr:endptr])
+            rc = mystring.find("DPDK_PLUGIN_PATH")
+            if (rc != -1):
+                rc = mystring.find("=")
+                return (mystring[rc + 1:], library)
+
+            dataptr = endptr
+        if scanfile is not None:
+            scanfile.close()
+        return (None, None)
+
+    def get_dt_runpath(self, dynsec):
+        for tag in dynsec.iter_tags():
+            if tag.entry.d_tag == 'DT_RUNPATH':
+                return tag.runpath
+        return ""
+
+    def process_dt_needed_entries(self):
+        """ Look to see if there are any DT_NEEDED entries in the binary
+            And process those if there are
+        """
+        global raw_output
+        runpath = ""
+        ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+        if ldlibpath is None:
+            ldlibpath = ""
+
+        dynsec = self._section_from_spec(".dynamic")
+        try:
+            runpath = self.get_dt_runpath(dynsec)
+        except AttributeError:
+            # dynsec is None, just return
+            return
+
+        for tag in dynsec.iter_tags():
+            if tag.entry.d_tag == 'DT_NEEDED':
+                rc = tag.needed.find("librte_pmd")
+                if (rc != -1):
+                    library = search_file(tag.needed,
+                                          runpath + ":" + ldlibpath +
+                                          ":/usr/lib64:/lib64:/usr/lib:/lib")
+                    if library is not None:
+                        if raw_output is False:
+                            print("Scanning %s for pmd information" % library)
+                        with open(library, 'rb') as file:
+                            try:
+                                libelf = ReadElf(file, sys.stdout)
+                            except ELFError as e:
+                                print("%s is no an ELF file" % library)
+                                continue
+                            libelf.process_dt_needed_entries()
+                            libelf.display_pmd_info_strings(".rodata")
+                            file.close()
+
+
+def scan_autoload_path(autoload_path):
+    global raw_output
+
+    if os.path.exists(autoload_path) is False:
+        return
+
+    try:
+        dirs = os.listdir(autoload_path)
+    except OSError as e:
+        # Couldn't read the directory, give up
+        return
+
+    for d in dirs:
+        dpath = os.path.join(autoload_path, d)
+        if os.path.isdir(dpath):
+            scan_autoload_path(dpath)
+        if os.path.isfile(dpath):
+            try:
+                file = open(dpath, 'rb')
+                readelf = ReadElf(file, sys.stdout)
+            except ELFError as e:
+                # this is likely not an elf file, skip it
+                continue
+            except IOError as e:
+                # No permission to read the file, skip it
+                continue
+
+            if raw_output is False:
+                print("Hw Support for library %s" % d)
+            readelf.display_pmd_info_strings(".rodata")
+            file.close()
+
+
+def scan_for_autoload_pmds(dpdk_path):
+    """
+    search the specified application or path for a pmd autoload path
+    then scan said path for pmds and report hw support
+    """
+    global raw_output
+
+    if (os.path.isfile(dpdk_path) is False):
+        if raw_output is False:
+            print("Must specify a file name")
+        return
+
+    file = open(dpdk_path, 'rb')
+    try:
+        readelf = ReadElf(file, sys.stdout)
+    except ElfError as e:
+        if raw_output is False:
+            print("Unable to parse %s" % file)
+        return
+
+    (autoload_path, scannedfile) = readelf.search_for_autoload_path()
+    if (autoload_path is None or autoload_path is ""):
+        if (raw_output is False):
+            print("No autoload path configured in %s" % dpdk_path)
+        return
+    if (raw_output is False):
+        if (scannedfile is None):
+            scannedfile = dpdk_path
+        print("Found autoload path %s in %s" % (autoload_path, scannedfile))
+
+    file.close()
+    if (raw_output is False):
+        print("Discovered Autoload HW Support:")
+    scan_autoload_path(autoload_path)
+    return
+
+
+def main(stream=None):
+    global raw_output
+    global pcidb
+
+    optparser = OptionParser(
+        usage='usage: %prog [-hrtp] [-d <pci id file] <elf-file>',
+        description="Dump pmd hardware support info",
+        add_help_option=True,
+        prog='pmdinfo.py')
+    optparser.add_option('-r', '--raw',
+                         action='store_true', dest='raw_output',
+                         help='Dump raw json strings')
+    optparser.add_option("-d", "--pcidb", dest="pcifile",
+                         help="specify a pci database "
+                              "to get vendor names from",
+                         default="/usr/share/hwdata/pci.ids", metavar="FILE")
+    optparser.add_option("-t", "--table", dest="tblout",
+                         help="output infomation on hw support as a hex table",
+                         action='store_true')
+    optparser.add_option("-p", "--plugindir", dest="pdir",
+                         help="scan dpdk for autoload plugins",
+                         action='store_true')
+
+    options, args = optparser.parse_args()
+
+    if options.raw_output:
+        raw_output = True
+
+    if options.pcifile:
+        pcidb = PCIIds(options.pcifile)
+        if pcidb is None:
+            print("Pci DB file not found")
+            exit(1)
+
+    if options.tblout:
+        options.pcifile = None
+        pcidb = None
+
+    if (len(args) == 0):
+        optparser.print_usage()
+        exit(1)
+
+    if options.pdir is True:
+        exit(scan_for_autoload_pmds(args[0]))
+
+    ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+    if (ldlibpath is None):
+        ldlibpath = ""
+
+    if (os.path.exists(args[0]) is True):
+        myelffile = args[0]
+    else:
+        myelffile = search_file(
+            args[0], ldlibpath + ":/usr/lib64:/lib64:/usr/lib:/lib")
+
+    if (myelffile is None):
+        print("File not found")
+        sys.exit(1)
+
+    with open(myelffile, 'rb') as file:
+        try:
+            readelf = ReadElf(file, sys.stdout)
+            readelf.process_dt_needed_entries()
+            readelf.display_pmd_info_strings(".rodata")
+            sys.exit(0)
+
+        except ELFError as ex:
+            sys.stderr.write('ELF error: %s\n' % ex)
+            sys.exit(1)
+
+
+# -------------------------------------------------------------------------
+if __name__ == '__main__':
+    main()
-- 
2.5.5

[PATCHv6 6/7] remove rte.hostapp.mk

[Neil Horman]

During the review of this patchset it was discovered that rte.hostapp.mk was
going unused.  All its users had been previously removed, and so this file too
can go.  It and rte.buildtools.mk are effectively duplicates, but given that
pmdinfogen is a tool used to create the dpdk build, I think the naming of the
buildtools variant is more appropriate.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 mk/rte.hostapp.mk | 123 ------------------------------------------------------
 1 file changed, 123 deletions(-)
 delete mode 100644 mk/rte.hostapp.mk

diff --git a/mk/rte.hostapp.mk b/mk/rte.hostapp.mk
deleted file mode 100644
index c44d0f8..0000000
--- a/mk/rte.hostapp.mk
+++ /dev/null
@@ -1,123 +0,0 @@
-#   BSD LICENSE
-#
-#   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
-#   All rights reserved.
-#
-#   Redistribution and use in source and binary forms, with or without
-#   modification, are permitted provided that the following conditions
-#   are met:
-#
-#     * Redistributions of source code must retain the above copyright
-#       notice, this list of conditions and the following disclaimer.
-#     * Redistributions in binary form must reproduce the above copyright
-#       notice, this list of conditions and the following disclaimer in
-#       the documentation and/or other materials provided with the
-#       distribution.
-#     * Neither the name of Intel Corporation nor the names of its
-#       contributors may be used to endorse or promote products derived
-#       from this software without specific prior written permission.
-#
-#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-# tell rte.compile-pre.mk to use HOSTCC instead of CC
-USE_HOST := 1
-include $(RTE_SDK)/mk/internal/rte.compile-pre.mk
-include $(RTE_SDK)/mk/internal/rte.install-pre.mk
-include $(RTE_SDK)/mk/internal/rte.clean-pre.mk
-include $(RTE_SDK)/mk/internal/rte.build-pre.mk
-include $(RTE_SDK)/mk/internal/rte.depdirs-pre.mk
-
-# VPATH contains at least SRCDIR
-VPATH += $(SRCDIR)
-
-_BUILD = $(HOSTAPP)
-_INSTALL = $(INSTALL-FILES-y) $(SYMLINK-FILES-y) $(RTE_OUTPUT)/hostapp/$(HOSTAPP)
-_CLEAN = doclean
-
-.PHONY: all
-all: install
-
-.PHONY: install
-install: build _postinstall
-
-_postinstall: build
-
-.PHONY: build
-build: _postbuild
-
-exe2cmd = $(strip $(call dotfile,$(patsubst %,%.cmd,$(1))))
-
-O_TO_EXE = $(HOSTCC) $(HOST_LDFLAGS) $(LDFLAGS_$(@)) \
-	$(EXTRA_HOST_LDFLAGS) -o $@ $(OBJS-y) $(LDLIBS)
-O_TO_EXE_STR = $(subst ','\'',$(O_TO_EXE)) #'# fix syntax highlight
-O_TO_EXE_DISP = $(if $(V),"$(O_TO_EXE_STR)","  HOSTLD $(@)")
-O_TO_EXE_CMD = "cmd_$@ = $(O_TO_EXE_STR)"
-O_TO_EXE_DO = @set -e; \
-	echo $(O_TO_EXE_DISP); \
-	$(O_TO_EXE) && \
-	echo $(O_TO_EXE_CMD) > $(call exe2cmd,$(@))
-
--include .$(HOSTAPP).cmd
-
-# list of .a files that are linked to this application
-LDLIBS_FILES := $(wildcard \
-	$(addprefix $(RTE_OUTPUT)/lib/, \
-	$(patsubst -l%,lib%.a,$(filter -l%,$(LDLIBS)))))
-
-#
-# Compile executable file if needed
-#
-$(HOSTAPP): $(OBJS-y) $(LDLIBS_FILES) FORCE
-	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
-	$(if $(D),\
-		@echo -n "$@ -> $< " ; \
-		echo -n "file_missing=$(call boolean,$(file_missing)) " ; \
-		echo -n "cmdline_changed=$(call boolean,$(call cmdline_changed,$(O_TO_EXE_STR))) " ; \
-		echo -n "depfile_missing=$(call boolean,$(depfile_missing)) " ; \
-		echo "depfile_newer=$(call boolean,$(depfile_newer)) ")
-	$(if $(or \
-		$(file_missing),\
-		$(call cmdline_changed,$(O_TO_EXE_STR)),\
-		$(depfile_missing),\
-		$(depfile_newer)),\
-		$(O_TO_EXE_DO))
-
-#
-# install app in $(RTE_OUTPUT)/hostapp
-#
-$(RTE_OUTPUT)/hostapp/$(HOSTAPP): $(HOSTAPP)
-	@echo "  INSTALL-HOSTAPP $(HOSTAPP)"
-	@[ -d $(RTE_OUTPUT)/hostapp ] || mkdir -p $(RTE_OUTPUT)/hostapp
-	$(Q)cp -f $(HOSTAPP) $(RTE_OUTPUT)/hostapp
-
-#
-# Clean all generated files
-#
-.PHONY: clean
-clean: _postclean
-	$(Q)rm -f $(_BUILD_TARGETS) $(_INSTALL_TARGETS) $(_CLEAN_TARGETS)
-
-.PHONY: doclean
-doclean:
-	$(Q)rm -rf $(HOSTAPP) $(OBJS-all) $(DEPS-all) $(DEPSTMP-all) \
-	  $(CMDS-all) $(INSTALL-FILES-all) .$(HOSTAPP).cmd
-
-
-include $(RTE_SDK)/mk/internal/rte.compile-post.mk
-include $(RTE_SDK)/mk/internal/rte.install-post.mk
-include $(RTE_SDK)/mk/internal/rte.clean-post.mk
-include $(RTE_SDK)/mk/internal/rte.build-post.mk
-include $(RTE_SDK)/mk/internal/rte.depdirs-post.mk
-
-.PHONY: FORCE
-FORCE:
-- 
2.5.5

[PATCHv6 7/7] doc: Add prog_guide section documenting pmdinfo script

[Neil Horman]

Information on pmdinfogen may be useful to 3rd party driver developers.  Include
documentation on what it does

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 doc/guides/prog_guide/dev_kit_build_system.rst | 43 ++++++++++++++++++++++++--
 1 file changed, 41 insertions(+), 2 deletions(-)

diff --git a/doc/guides/prog_guide/dev_kit_build_system.rst b/doc/guides/prog_guide/dev_kit_build_system.rst
index 3e89eae..e5043dc 100644
--- a/doc/guides/prog_guide/dev_kit_build_system.rst
+++ b/doc/guides/prog_guide/dev_kit_build_system.rst
@@ -70,7 +70,7 @@ Each build directory contains include files, libraries, and applications:
     ...
     ~/DEV/DPDK$ ls i686-native-linuxapp-gcc
 
-    app build hostapp include kmod lib Makefile
+    app build buildtools include kmod lib Makefile
 
 
     ~/DEV/DPDK$ ls i686-native-linuxapp-gcc/app/
@@ -264,7 +264,7 @@ These Makefiles generate a binary application.
 
 *   rte.extapp.mk: External application
 
-*   rte.hostapp.mk: Host application in the development kit framework
+*   rte.buildtools.mk: prerequisite tool to build dpdk 
 
 Library
 ^^^^^^^
@@ -304,6 +304,45 @@ Misc
 
 *   rte.subdir.mk: Build several directories in the development kit framework.
 
+.. _Internally_Generated_Build_Tools
+
+Internally Generated Build Tools
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+buildtools/pmdinfogen
+
+
+pmdinfogen scans an object (.o) file for various well known symbol names.  These
+well know symbol names are defined by various macros and used to export
+important information about hardware support and usage for pmd files.  for
+instance the macro:
+
+.. code-block:: c 
+
+    PMD_REGISTER_DRIVER(drv, name)
+
+
+Creates the following symbol:
+
+.. code-block:: c 
+
+   static char this_pmd_name0[] __attribute__((used)) = "<name>";
+
+
+Which pmdinfogen scans for.  Using this information other relevant bits of data
+can be exported from the object file and used to produce a hardware support
+description, that pmdinfogen then encodes into a json formatted string in the
+following format:
+
+.. code-block:: C
+
+   static char <name_pmd_string>="PMD_INFO_STRING=\"{'name' : '<name>', ...}\"";
+
+
+These strings can then be searched for by external tools to determine the
+hardware support of a given library or application
+
+
 .. _Useful_Variables_Provided_by_the_Build_System:
 
 Useful Variables Provided by the Build System
-- 
2.5.5

Re: [PATCHv6 0/7] Implement pmd hardware support exports

[Neil Horman]

On Tue, May 31, 2016 at 09:57:41AM -0400, Neil Horman wrote:
> Hey all-
> 	So heres attempt number 2 at a method for exporting PMD hardware support
> information.  As we discussed previously, the consensus seems to be that pmd
> information should be:
> 
> 1) Able to be interrogated on any ELF binary (application binary or individual
> DSO)
> 2) Equally functional on statically linked applications or on DSO's
> 3) Resilient to symbol stripping
> 4) Script friendly
> 5) Show kernel dependencies
> 6) List driver options
> 7) Show driver name
> 8) Offer human readable output
> 9) Show DPDK version
> 10) Show driver version
> 11) Allow for expansion
> 12) Not place additional build environment dependencies on an application
> 
> I added item 12 myself, because I don't think its reasonable to require
> applications to use a specific linker script to get hardware support information
> (which precludes the use of a special .modinfo section like the kernel uses)
> 
> However, we still can use some tricks from the kernel to make this work.  In
> this approach, what I've done is the following:
> 
> A) Modified the driver registration macro to also define a variable:
> this_pmd_name<n>= "name"
> 
> Based on the unique name string pointed to by the above variable, we can
> query for an arbitrary number of other symbols following the pattern:
> <name>_<tag>
> 
> Where tag is some well known identifier for information we wish to export
> 
> B) Added a utility called pmdinfogen.  This utility is not meant for general use,
> but rather is used by the dpdk build environment itself when compiling pmds.
> for each object that uses the PMD_REGISTER_DRIVER macro, this utiity is run.  It
> searches for the symbols in (A), and using those, extracts the hardware support
> info, and module name from the object, using that to produce a new c file
> containing a single variable in the following format:
> 
> static const char <name>[] __attribute__((used)) = "PMD_DRIVER_INFO=<json>";
> 
> The <name> is arbitrary, as its static and not referenced.  The relevant bit is
> the string value assigned to it.  The <json> is a json encoded string of the
> extracted hardware support information pmdinfo found for the corresponding
> object.  This C file is suitable for compilation and relocatable linking back
> into the parent object file.  The result of this operation is that the object
> string table now contains a string that will not e removed by stripping, whos
> leading text (PMD_DRIVER_INFO) can be easily searched for at any time weather
> the symbol referring to it is stripped or not.
> 
> C) Added a utilty called pmdinfo.py.  This python script, searches the
> string table of the .rodata section of any provided ELF file looking for the
> PMD_DRIVER_INFO prefix on a string.  When found, it will interpret the remainder
> of the string as json, and output the hardware support for that ELF file (if
> any).
> 
> 
> This approach ticks most of the above boxes:
> 1) Impervious to stripping
> 2) Works on static and dynamic binaries
> 3) Human and script friendly
> 4) Allows for expansion
> 
> Because of (4) the other items should be pretty easy to implement, as its just a
> matter of modifying the macros to export the info, pmdinfo to encode it to json,
> and pmd_hw_support.py to read it.  I'm not adding them now, as theres alot of
> rote work to do to get some of them in place (e.g. DPDK has no current global
> VERSION macro, drivers don't have a consistent version scheme, command line
> strings have to be built for each driver, etc).  But once this is accepted,
> those items can be done as time allows and should be fairly easy to implement.
> 
> Change Notes:
> v2)
>  * Made the export macros a bit easier to expand
>  * Added a macro to optionally export command line strings
>  * Renamed utilties to be more appropriate
>    (pmdinfo -> pmdinfogen, pmd_hw_support.py -> pmdinfo.py)
>  * Added search capabilities to pmdinfo.py so that we search for libraries
>    linked using DT_NEEDINFO entries.  We search DT_RUNPATH, LD_LIBRARY_PATH,
>    /usr/lib and /lib
>  * Added serch abilities if the specified binary to pmdinfo isn't found, we
>    search LD_LIBRARY_PATH, /usr/lib and /lib
>  * Added an option to pmdinfo.py to pretty-print hardware support info using the
>    pci.ids database
> 
> v3)
>  * Made table mode printing the default mode
>  * Added a default path to the pci.ids file
>  * Modifed pmdinfo to use python rather than python3
>  * Cleaned up some changelog entries
>  * Added an export for RTE_EAL_PMD_PATH
>  * Added a plugin option to pmdinfo to scan for autoloaded DSO's
> 
> v4)
>  * Modified the operation of the -p option. As much as I don't like implying
> that autoloaded pmds are guaranteed to be there at run time, I'm having a hard
> time seeing how we can avoid specifying the application file to scan for the
> autoload directory.  Without it we can't determine which library the user means
> in a multiversion installation
>  * Cleaned up the help text
>  * Added a rule for an install target for pmdinfo
>  * Guarded against some tracebacks in pmdinfo
>  * Use DT_NEEDED entries to get versioned libraries in -p mode
>  * Fixed traceback that occurs on lack of input arguments
>  * Fixed some erroneous macro usage in drivers that aren't in the default build
> 
> v5)
>  * Added a dpdk- prefix to pmdinfo symlink
>  * Renamed build announcement to PMDINFOGEN/BUILD
>  * Removed erroneous true statement from makefile
>  * Removed duplicate rte.hostapp.mk makefile
>  * Fixed some whitespace
>  * Whitespace fixups
>  * Fixed makefile if; then style
>  * Renamed module string C file
>  * Removed duplicate rte_pci_id definition
>  * Clarified macro names
>  * Removed PMD type attribute
>  * Fixed tools usage for 32 bit arches
>  * Removed some unused code
>  * Added a few comments
> 
> v6)
>  * Added some programming guide documentation
>  * Reformatted python script with pep8
> 
> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> CC: Bruce Richardson <bruce.richardson@intel.com>
> CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> CC: Stephen Hemminger <stephen@networkplumber.org>
> CC: Panu Matilainen <pmatilai@redhat.com>
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
Ping, feedback here?
Neil

Re: [PATCHv6 0/7] Implement pmd hardware support exports

[Thomas Monjalon]

Hi Neil,

You are creating a new directory buildtools/ which depends on mk/
We also have some scripts used in the build process:
	scripts/auto-config-h.sh
	scripts/depdirs-rule.sh
	scripts/gen-build-mk.sh
	scripts/gen-config-h.sh
	scripts/relpath.sh
I think we should move these scripts somewhere else, probably in mk.
So we would have only 2 directories for the build system and we would
keep the scripts directory only for dev tools.

If we want to make a step further, we can rename scripts/ to devtools/
and tools/ to usertools/.

Opinions?

Re: [PATCHv6 1/7] pmdinfogen: Add buildtools and pmdinfogen utility

[Thomas Monjalon]

2016-05-31 09:57, Neil Horman:
> +++ b/buildtools/Makefile
> @@ -0,0 +1,36 @@
> +#   BSD LICENSE
> +#
> +#   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
> +#   All rights reserved.

I really think it is a strange copyright for a new empty file.

> +#if __x86_64__ || __aarch64__

Better to use CONFIG_RTE_ARCH_64.

> +#define TO_NATIVE(x) (x)

We already have some functions for endianness in
lib/librte_eal/common/include/generic/rte_byteorder.h

> +struct elf_info {
> +	unsigned long size;
> +	Elf_Ehdr     *hdr;
> +	Elf_Shdr     *sechdrs;
> +	Elf_Sym      *symtab_start;
> +	Elf_Sym      *symtab_stop;
> +	Elf_Section  export_sec;
> +	Elf_Section  export_unused_sec;
> +	Elf_Section  export_gpl_sec;
> +	Elf_Section  export_unused_gpl_sec;
> +	Elf_Section  export_gpl_future_sec;
> +	char         *strtab;

The export_* fields are not used.

> --- /dev/null
> +++ b/mk/rte.buildtools.mk

I'm sorry I really do not agree it is a good practice to create a new
makefile type just for a new directory.
My opinion is that you should use and improve rte.hostapp.mk to make
it usable for possible other host apps.

Re: [PATCHv6 1/7] pmdinfogen: Add buildtools and pmdinfogen utility

[Neil Horman]

On Tue, Jun 07, 2016 at 11:57:42AM +0200, Thomas Monjalon wrote:
> 2016-05-31 09:57, Neil Horman:
> > +++ b/buildtools/Makefile
> > @@ -0,0 +1,36 @@
> > +#   BSD LICENSE
> > +#
> > +#   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
> > +#   All rights reserved.
> 
> I really think it is a strange copyright for a new empty file.
> 
Its not empty, It lists the subdirectories to build.  And given that the DPDK is
licensed under multiple licenses (BSD/GPL/LGPL), it introduces confusion to not
call out the license in a specific file, file size is really irrelevant to that.

> > +#if __x86_64__ || __aarch64__
> 
> Better to use CONFIG_RTE_ARCH_64.
> 
I'm not sure why, given that every supported compiler defines the arches I use,
but sure, fine.

> > +#define TO_NATIVE(x) (x)
> 
> We already have some functions for endianness in
> lib/librte_eal/common/include/generic/rte_byteorder.h
> 
Very well, I'll use the rte byteorder macros.

> > +struct elf_info {
> > +	unsigned long size;
> > +	Elf_Ehdr     *hdr;
> > +	Elf_Shdr     *sechdrs;
> > +	Elf_Sym      *symtab_start;
> > +	Elf_Sym      *symtab_stop;
> > +	Elf_Section  export_sec;
> > +	Elf_Section  export_unused_sec;
> > +	Elf_Section  export_gpl_sec;
> > +	Elf_Section  export_unused_gpl_sec;
> > +	Elf_Section  export_gpl_future_sec;
> > +	char         *strtab;
> 
> The export_* fields are not used.
> 
Fine, I'll remove them.

> > --- /dev/null
> > +++ b/mk/rte.buildtools.mk
> 
> I'm sorry I really do not agree it is a good practice to create a new
> makefile type just for a new directory.
> My opinion is that you should use and improve rte.hostapp.mk to make
> it usable for possible other host apps.
> 
I am so exhausted by this argument.

They are the same file Thomas.  I'm not sure how you don't see that.  I've
explained to you that they are, with the exception of whitespace noise,
identical.  buildtools is a better nomenclature because it more closely
describes what is being built at the moment.  The only reason we still have
hostapp is because you didn't remove it when you removed the applications that,
in your own words from the commit log, are "useless".  The argument that we
should keep the build file, and its naming convention on the off chance that
someone might use it in the future really doesn't hold water with me, at least
not to the point that, when we have something that duplicates its function we
should do anything other than take the path of least resistance to make it work.
I'm not sure how you expected anyone to know there is a makefile in place in the
DPDK to build local application, when there are currently no applications in
place, but asking people to use it after the fact is really just the height of
busywork.  If it was already building other utilities, I'd feel differently, but
given that its just sitting there, a vestigual file, makes this all just silly.

But clearly, this isn't going to be done until I do what you want, regardless of
what either of us think of it, So I'll make the change.

Neil
 

Re: [PATCHv6 0/7] Implement pmd hardware support exports

[Neil Horman]

On Tue, Jun 07, 2016 at 11:34:56AM +0200, Thomas Monjalon wrote:
> Hi Neil,
> 
> You are creating a new directory buildtools/ which depends on mk/
I don't know what this means.  Every part of the dpdk that builds code depends
on the infrastructure in mk/ (save for the scripts below)

> We also have some scripts used in the build process:
> 	scripts/auto-config-h.sh
> 	scripts/depdirs-rule.sh
> 	scripts/gen-build-mk.sh
> 	scripts/gen-config-h.sh
> 	scripts/relpath.sh
> I think we should move these scripts somewhere else, probably in mk.
> So we would have only 2 directories for the build system and we would
> keep the scripts directory only for dev tools.
> 
> If we want to make a step further, we can rename scripts/ to devtools/
> and tools/ to usertools/.
> 
> Opinions?
> 
I suppose if you like, sure, though I would caution against moving them to mk/
as no one really expects executable scripts in the infrastructure makefile area.
If you want to move the buildtool scripts elsewhere I would suggest
buildtools/scripts.

Either way though, this movement seems to fall outside of the purview of the
purpose of the changeset associated with this email thread.

Neil

Re: [PATCHv6 0/7] Implement pmd hardware support exports

[Thomas Monjalon]

2016-06-07 08:08, Neil Horman:
> On Tue, Jun 07, 2016 at 11:34:56AM +0200, Thomas Monjalon wrote:
> > Hi Neil,
> > 
> > You are creating a new directory buildtools/ which depends on mk/
> I don't know what this means.  Every part of the dpdk that builds code depends
> on the infrastructure in mk/ (save for the scripts below)

Yes, I'm just doing a summary of the build system.

> > We also have some scripts used in the build process:
> > 	scripts/auto-config-h.sh
> > 	scripts/depdirs-rule.sh
> > 	scripts/gen-build-mk.sh
> > 	scripts/gen-config-h.sh
> > 	scripts/relpath.sh
> > I think we should move these scripts somewhere else, probably in mk.
> > So we would have only 2 directories for the build system and we would
> > keep the scripts directory only for dev tools.
> > 
> > If we want to make a step further, we can rename scripts/ to devtools/
> > and tools/ to usertools/.
> > 
> > Opinions?
> > 
> I suppose if you like, sure, though I would caution against moving them to mk/
> as no one really expects executable scripts in the infrastructure makefile area.
> If you want to move the buildtool scripts elsewhere I would suggest
> buildtools/scripts.

Yes that's what I thought first, but as stated above, buildtools depends on mk,
and mk/ depends on these scripts.

> Either way though, this movement seems to fall outside of the purview of the
> purpose of the changeset associated with this email thread.

Yes, just discussing for possible more changes.

Re: [PATCHv6 1/7] pmdinfogen: Add buildtools and pmdinfogen utility

[Thomas Monjalon]

2016-06-07 08:04, Neil Horman:
> On Tue, Jun 07, 2016 at 11:57:42AM +0200, Thomas Monjalon wrote:
> > 2016-05-31 09:57, Neil Horman:
> > > +++ b/buildtools/Makefile
> > > @@ -0,0 +1,36 @@
> > > +#   BSD LICENSE
> > > +#
> > > +#   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
> > > +#   All rights reserved.
> > 
> > I really think it is a strange copyright for a new empty file.
> > 
> Its not empty, It lists the subdirectories to build.  And given that the DPDK is
> licensed under multiple licenses (BSD/GPL/LGPL), it introduces confusion to not
> call out the license in a specific file, file size is really irrelevant to that.

Neil, please take a drink :)
I'm not talking about license but about copyright.
Don't you think it's strange to put "2010-2014 Intel" copyright on top of
the few lines you wrote?
 
> > > +#if __x86_64__ || __aarch64__
> > 
> > Better to use CONFIG_RTE_ARCH_64.
> > 
> I'm not sure why, given that every supported compiler defines the arches I use,
> but sure, fine.

Because it will work for every 64-bit arch in DPDK.

> > > --- /dev/null
> > > +++ b/mk/rte.buildtools.mk
> > 
> > I'm sorry I really do not agree it is a good practice to create a new
> > makefile type just for a new directory.
> > My opinion is that you should use and improve rte.hostapp.mk to make
> > it usable for possible other host apps.
> > 
> I am so exhausted by this argument.
> 
> They are the same file Thomas.  I'm not sure how you don't see that.  I've
> explained to you that they are, with the exception of whitespace noise,
> identical.  buildtools is a better nomenclature because it more closely
> describes what is being built at the moment.  The only reason we still have
> hostapp is because you didn't remove it when you removed the applications that,
> in your own words from the commit log, are "useless".  The argument that we
> should keep the build file, and its naming convention on the off chance that
> someone might use it in the future really doesn't hold water with me, at least
> not to the point that, when we have something that duplicates its function we
> should do anything other than take the path of least resistance to make it work.
> I'm not sure how you expected anyone to know there is a makefile in place in the
> DPDK to build local application, when there are currently no applications in
> place, but asking people to use it after the fact is really just the height of
> busywork.  If it was already building other utilities, I'd feel differently, but
> given that its just sitting there, a vestigual file, makes this all just silly.
> 
> But clearly, this isn't going to be done until I do what you want, regardless of
> what either of us think of it, So I'll make the change.

You can keep it as is if you find someone else to say that having a makefile
template named and specific to only the buildtools usage is fine.
And no, it is not identical to rte.hostapp.mk.
But I was probably not clear enough:
I do not like rte.hostapp.mk. I just like its explicit name.
I see the same issue in rte.hostapp.mk and rte.buildtools.mk: they should be
build in the app/ subdir like any other app.

So my suggestion is to replace rte.hostapp.mk with your implementation in
a separate patch with the build path changed to app/ instead of hostapp/ or
buildtools/.

Re: [PATCHv6 1/7] pmdinfogen: Add buildtools and pmdinfogen utility

[Neil Horman]

On Tue, Jun 07, 2016 at 02:53:36PM +0200, Thomas Monjalon wrote:
> 2016-06-07 08:04, Neil Horman:
> > On Tue, Jun 07, 2016 at 11:57:42AM +0200, Thomas Monjalon wrote:
> > > 2016-05-31 09:57, Neil Horman:
> > > > +++ b/buildtools/Makefile
> > > > @@ -0,0 +1,36 @@
> > > > +#   BSD LICENSE
> > > > +#
> > > > +#   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
> > > > +#   All rights reserved.
> > > 
> > > I really think it is a strange copyright for a new empty file.
> > > 
> > Its not empty, It lists the subdirectories to build.  And given that the DPDK is
> > licensed under multiple licenses (BSD/GPL/LGPL), it introduces confusion to not
> > call out the license in a specific file, file size is really irrelevant to that.
> 
> Neil, please take a drink :)
> I'm not talking about license but about copyright.
> Don't you think it's strange to put "2010-2014 Intel" copyright on top of
> the few lines you wrote?
>  
Ah, yes, I copied the file, so the copyright years are wrong, so that should be
fixed.

That said, you asked if it was strange to put a copyright on an empty file, and
the answer is no, because its not empty, and it nees a copyright for clarity :)

> > > > +#if __x86_64__ || __aarch64__
> > > 
> > > Better to use CONFIG_RTE_ARCH_64.
> > > 
> > I'm not sure why, given that every supported compiler defines the arches I use,
> > but sure, fine.
> 
> Because it will work for every 64-bit arch in DPDK.
> 
Ok, fair enough.

> > > > --- /dev/null
> > > > +++ b/mk/rte.buildtools.mk
> > > 
> > > I'm sorry I really do not agree it is a good practice to create a new
> > > makefile type just for a new directory.
> > > My opinion is that you should use and improve rte.hostapp.mk to make
> > > it usable for possible other host apps.
> > > 
> > I am so exhausted by this argument.
> > 
> > They are the same file Thomas.  I'm not sure how you don't see that.  I've
> > explained to you that they are, with the exception of whitespace noise,
> > identical.  buildtools is a better nomenclature because it more closely
> > describes what is being built at the moment.  The only reason we still have
> > hostapp is because you didn't remove it when you removed the applications that,
> > in your own words from the commit log, are "useless".  The argument that we
> > should keep the build file, and its naming convention on the off chance that
> > someone might use it in the future really doesn't hold water with me, at least
> > not to the point that, when we have something that duplicates its function we
> > should do anything other than take the path of least resistance to make it work.
> > I'm not sure how you expected anyone to know there is a makefile in place in the
> > DPDK to build local application, when there are currently no applications in
> > place, but asking people to use it after the fact is really just the height of
> > busywork.  If it was already building other utilities, I'd feel differently, but
> > given that its just sitting there, a vestigual file, makes this all just silly.
> > 
> > But clearly, this isn't going to be done until I do what you want, regardless of
> > what either of us think of it, So I'll make the change.
> 
> You can keep it as is if you find someone else to say that having a makefile
> template named and specific to only the buildtools usage is fine.
> And no, it is not identical to rte.hostapp.mk.
> But I was probably not clear enough:
> I do not like rte.hostapp.mk. I just like its explicit name.
> I see the same issue in rte.hostapp.mk and rte.buildtools.mk: they should be
> build in the app/ subdir like any other app.
> 
> So my suggestion is to replace rte.hostapp.mk with your implementation in
> a separate patch with the build path changed to app/ instead of hostapp/ or
> buildtools/.
> 
Soo, I'm confused now.  You don't want rte.buildtools.mk, and you don't really
want rte.hostapp.mk, you want a different makefile, that just builds to the /app
subdirectory?

Neil

Re: [PATCHv6 1/7] pmdinfogen: Add buildtools and pmdinfogen utility

[Thomas Monjalon]

2016-06-07 09:03, Neil Horman:
> On Tue, Jun 07, 2016 at 02:53:36PM +0200, Thomas Monjalon wrote:
> > 2016-06-07 08:04, Neil Horman:
> > > On Tue, Jun 07, 2016 at 11:57:42AM +0200, Thomas Monjalon wrote:
> > > > 2016-05-31 09:57, Neil Horman:
> > > > > +++ b/buildtools/Makefile
> > > > > @@ -0,0 +1,36 @@
> > > > > +#   BSD LICENSE
> > > > > +#
> > > > > +#   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
> > > > > +#   All rights reserved.
> > > > 
> > > > I really think it is a strange copyright for a new empty file.
> > > > 
> > > Its not empty, It lists the subdirectories to build.  And given that the DPDK is
> > > licensed under multiple licenses (BSD/GPL/LGPL), it introduces confusion to not
> > > call out the license in a specific file, file size is really irrelevant to that.
> > 
> > Neil, please take a drink :)
> > I'm not talking about license but about copyright.
> > Don't you think it's strange to put "2010-2014 Intel" copyright on top of
> > the few lines you wrote?
> >  
> Ah, yes, I copied the file, so the copyright years are wrong, so that should be
> fixed.

Not only the years, the copyright holder should be you or your company.

> That said, you asked if it was strange to put a copyright on an empty file, and
> the answer is no, because its not empty, and it nees a copyright for clarity :)

Of course, yes.

> > > > > --- /dev/null
> > > > > +++ b/mk/rte.buildtools.mk
> > > > 
> > > > I'm sorry I really do not agree it is a good practice to create a new
> > > > makefile type just for a new directory.
> > > > My opinion is that you should use and improve rte.hostapp.mk to make
> > > > it usable for possible other host apps.
> > > > 
> > > I am so exhausted by this argument.
> > > 
> > > They are the same file Thomas.  I'm not sure how you don't see that.  I've
> > > explained to you that they are, with the exception of whitespace noise,
> > > identical.  buildtools is a better nomenclature because it more closely
> > > describes what is being built at the moment.  The only reason we still have
> > > hostapp is because you didn't remove it when you removed the applications that,
> > > in your own words from the commit log, are "useless".  The argument that we
> > > should keep the build file, and its naming convention on the off chance that
> > > someone might use it in the future really doesn't hold water with me, at least
> > > not to the point that, when we have something that duplicates its function we
> > > should do anything other than take the path of least resistance to make it work.
> > > I'm not sure how you expected anyone to know there is a makefile in place in the
> > > DPDK to build local application, when there are currently no applications in
> > > place, but asking people to use it after the fact is really just the height of
> > > busywork.  If it was already building other utilities, I'd feel differently, but
> > > given that its just sitting there, a vestigual file, makes this all just silly.
> > > 
> > > But clearly, this isn't going to be done until I do what you want, regardless of
> > > what either of us think of it, So I'll make the change.
> > 
> > You can keep it as is if you find someone else to say that having a makefile
> > template named and specific to only the buildtools usage is fine.
> > And no, it is not identical to rte.hostapp.mk.
> > But I was probably not clear enough:
> > I do not like rte.hostapp.mk. I just like its explicit name.
> > I see the same issue in rte.hostapp.mk and rte.buildtools.mk: they should be
> > build in the app/ subdir like any other app.
> > 
> > So my suggestion is to replace rte.hostapp.mk with your implementation in
> > a separate patch with the build path changed to app/ instead of hostapp/ or
> > buildtools/.
> > 
> Soo, I'm confused now.  You don't want rte.buildtools.mk, and you don't really
> want rte.hostapp.mk, you want a different makefile, that just builds to the /app
> subdirectory?

The apps and examples use rte.app.mk to build a DPDK app.
Here you make a standard app, without DPDK dependency, to run on the host.
So you cannot use rte.app.mk. I think rte.hostapp.mk is not a so bad name
(I have no better one).
About the build directory, the app/ one looks OK, no need to put a reference
to buildtools which is just the user of this makefile.
Except these considerations, the content of your makefile is probably good.

Re: [PATCHv6 1/7] pmdinfogen: Add buildtools and pmdinfogen utility

[Neil Horman]

On Tue, Jun 07, 2016 at 03:24:55PM +0200, Thomas Monjalon wrote:
> 2016-06-07 09:03, Neil Horman:
> > On Tue, Jun 07, 2016 at 02:53:36PM +0200, Thomas Monjalon wrote:
> > > 2016-06-07 08:04, Neil Horman:
> > > > On Tue, Jun 07, 2016 at 11:57:42AM +0200, Thomas Monjalon wrote:
> > > > > 2016-05-31 09:57, Neil Horman:
> > > > > > +++ b/buildtools/Makefile
> > > > > > @@ -0,0 +1,36 @@
> > > > > > +#   BSD LICENSE
> > > > > > +#
> > > > > > +#   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
> > > > > > +#   All rights reserved.
> > > > > 
> > > > > I really think it is a strange copyright for a new empty file.
> > > > > 
> > > > Its not empty, It lists the subdirectories to build.  And given that the DPDK is
> > > > licensed under multiple licenses (BSD/GPL/LGPL), it introduces confusion to not
> > > > call out the license in a specific file, file size is really irrelevant to that.
> > > 
> > > Neil, please take a drink :)
> > > I'm not talking about license but about copyright.
> > > Don't you think it's strange to put "2010-2014 Intel" copyright on top of
> > > the few lines you wrote?
> > >  
> > Ah, yes, I copied the file, so the copyright years are wrong, so that should be
> > fixed.
> 
> Not only the years, the copyright holder should be you or your company.
> 
> > That said, you asked if it was strange to put a copyright on an empty file, and
> > the answer is no, because its not empty, and it nees a copyright for clarity :)
> 
> Of course, yes.
> 
> > > > > > --- /dev/null
> > > > > > +++ b/mk/rte.buildtools.mk
> > > > > 
> > > > > I'm sorry I really do not agree it is a good practice to create a new
> > > > > makefile type just for a new directory.
> > > > > My opinion is that you should use and improve rte.hostapp.mk to make
> > > > > it usable for possible other host apps.
> > > > > 
> > > > I am so exhausted by this argument.
> > > > 
> > > > They are the same file Thomas.  I'm not sure how you don't see that.  I've
> > > > explained to you that they are, with the exception of whitespace noise,
> > > > identical.  buildtools is a better nomenclature because it more closely
> > > > describes what is being built at the moment.  The only reason we still have
> > > > hostapp is because you didn't remove it when you removed the applications that,
> > > > in your own words from the commit log, are "useless".  The argument that we
> > > > should keep the build file, and its naming convention on the off chance that
> > > > someone might use it in the future really doesn't hold water with me, at least
> > > > not to the point that, when we have something that duplicates its function we
> > > > should do anything other than take the path of least resistance to make it work.
> > > > I'm not sure how you expected anyone to know there is a makefile in place in the
> > > > DPDK to build local application, when there are currently no applications in
> > > > place, but asking people to use it after the fact is really just the height of
> > > > busywork.  If it was already building other utilities, I'd feel differently, but
> > > > given that its just sitting there, a vestigual file, makes this all just silly.
> > > > 
> > > > But clearly, this isn't going to be done until I do what you want, regardless of
> > > > what either of us think of it, So I'll make the change.
> > > 
> > > You can keep it as is if you find someone else to say that having a makefile
> > > template named and specific to only the buildtools usage is fine.
> > > And no, it is not identical to rte.hostapp.mk.
> > > But I was probably not clear enough:
> > > I do not like rte.hostapp.mk. I just like its explicit name.
> > > I see the same issue in rte.hostapp.mk and rte.buildtools.mk: they should be
> > > build in the app/ subdir like any other app.
> > > 
> > > So my suggestion is to replace rte.hostapp.mk with your implementation in
> > > a separate patch with the build path changed to app/ instead of hostapp/ or
> > > buildtools/.
> > > 
> > Soo, I'm confused now.  You don't want rte.buildtools.mk, and you don't really
> > want rte.hostapp.mk, you want a different makefile, that just builds to the /app
> > subdirectory?
> 
> The apps and examples use rte.app.mk to build a DPDK app.
> Here you make a standard app, without DPDK dependency, to run on the host.
> So you cannot use rte.app.mk. I think rte.hostapp.mk is not a so bad name
> (I have no better one).
> About the build directory, the app/ one looks OK, no need to put a reference
> to buildtools which is just the user of this makefile.
> Except these considerations, the content of your makefile is probably good.
> 

Sooo....you do actually want to just use the hostapp makefile, because you like
the name, and don't like mine, and you want to just dump the output into the
same app directory that all the dpdk examples get written to, because it looks
ok to you? 

Fine, whatever, I'm tired of arguing.
Neil

Re: [PATCHv6 1/7] pmdinfogen: Add buildtools and pmdinfogen utility

[Thomas Monjalon]

2016-06-07 09:49, Neil Horman:
> On Tue, Jun 07, 2016 at 03:24:55PM +0200, Thomas Monjalon wrote:
> > 2016-06-07 09:03, Neil Horman:
> > > On Tue, Jun 07, 2016 at 02:53:36PM +0200, Thomas Monjalon wrote:
> > > > 2016-06-07 08:04, Neil Horman:
> > > > > On Tue, Jun 07, 2016 at 11:57:42AM +0200, Thomas Monjalon wrote:
> > > > > > 2016-05-31 09:57, Neil Horman:
> > > > > > > +++ b/buildtools/Makefile
> > > > > > > @@ -0,0 +1,36 @@
> > > > > > > +#   BSD LICENSE
> > > > > > > +#
> > > > > > > +#   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
> > > > > > > +#   All rights reserved.
> > > > > > 
> > > > > > I really think it is a strange copyright for a new empty file.
> > > > > > 
> > > > > Its not empty, It lists the subdirectories to build.  And given that the DPDK is
> > > > > licensed under multiple licenses (BSD/GPL/LGPL), it introduces confusion to not
> > > > > call out the license in a specific file, file size is really irrelevant to that.
> > > > 
> > > > Neil, please take a drink :)
> > > > I'm not talking about license but about copyright.
> > > > Don't you think it's strange to put "2010-2014 Intel" copyright on top of
> > > > the few lines you wrote?
> > > >  
> > > Ah, yes, I copied the file, so the copyright years are wrong, so that should be
> > > fixed.
> > 
> > Not only the years, the copyright holder should be you or your company.
> > 
> > > That said, you asked if it was strange to put a copyright on an empty file, and
> > > the answer is no, because its not empty, and it nees a copyright for clarity :)
> > 
> > Of course, yes.
> > 
> > > > > > > --- /dev/null
> > > > > > > +++ b/mk/rte.buildtools.mk
> > > > > > 
> > > > > > I'm sorry I really do not agree it is a good practice to create a new
> > > > > > makefile type just for a new directory.
> > > > > > My opinion is that you should use and improve rte.hostapp.mk to make
> > > > > > it usable for possible other host apps.
> > > > > > 
> > > > > I am so exhausted by this argument.
> > > > > 
> > > > > They are the same file Thomas.  I'm not sure how you don't see that.  I've
> > > > > explained to you that they are, with the exception of whitespace noise,
> > > > > identical.  buildtools is a better nomenclature because it more closely
> > > > > describes what is being built at the moment.  The only reason we still have
> > > > > hostapp is because you didn't remove it when you removed the applications that,
> > > > > in your own words from the commit log, are "useless".  The argument that we
> > > > > should keep the build file, and its naming convention on the off chance that
> > > > > someone might use it in the future really doesn't hold water with me, at least
> > > > > not to the point that, when we have something that duplicates its function we
> > > > > should do anything other than take the path of least resistance to make it work.
> > > > > I'm not sure how you expected anyone to know there is a makefile in place in the
> > > > > DPDK to build local application, when there are currently no applications in
> > > > > place, but asking people to use it after the fact is really just the height of
> > > > > busywork.  If it was already building other utilities, I'd feel differently, but
> > > > > given that its just sitting there, a vestigual file, makes this all just silly.
> > > > > 
> > > > > But clearly, this isn't going to be done until I do what you want, regardless of
> > > > > what either of us think of it, So I'll make the change.
> > > > 
> > > > You can keep it as is if you find someone else to say that having a makefile
> > > > template named and specific to only the buildtools usage is fine.
> > > > And no, it is not identical to rte.hostapp.mk.
> > > > But I was probably not clear enough:
> > > > I do not like rte.hostapp.mk. I just like its explicit name.
> > > > I see the same issue in rte.hostapp.mk and rte.buildtools.mk: they should be
> > > > build in the app/ subdir like any other app.
> > > > 
> > > > So my suggestion is to replace rte.hostapp.mk with your implementation in
> > > > a separate patch with the build path changed to app/ instead of hostapp/ or
> > > > buildtools/.
> > > > 
> > > Soo, I'm confused now.  You don't want rte.buildtools.mk, and you don't really
> > > want rte.hostapp.mk, you want a different makefile, that just builds to the /app
> > > subdirectory?
> > 
> > The apps and examples use rte.app.mk to build a DPDK app.
> > Here you make a standard app, without DPDK dependency, to run on the host.
> > So you cannot use rte.app.mk. I think rte.hostapp.mk is not a so bad name
> > (I have no better one).
> > About the build directory, the app/ one looks OK, no need to put a reference
> > to buildtools which is just the user of this makefile.
> > Except these considerations, the content of your makefile is probably good.
> > 
> 
> Sooo....you do actually want to just use the hostapp makefile, because you like
> the name, and don't like mine, and you want to just dump the output into the
> same app directory that all the dpdk examples get written to, because it looks
> ok to you? 

No.
Your Makefile is fine. I just suggest to rename it to rte.hostapp.mk.
The examples are not built in the app/ directory.
But you can do what you want. I'm just suggesting.

By the way, please use checkpatch.sh.

Re: [PATCHv6 7/7] doc: Add prog_guide section documenting pmdinfo script

[Mcnamara, John]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
> Sent: Tuesday, May 31, 2016 2:58 PM
> To: dev@dpdk.org
> Cc: Neil Horman <nhorman@tuxdriver.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; Thomas Monjalon <thomas.monjalon@6wind.com>;
> Stephen Hemminger <stephen@networkplumber.org>; Panu Matilainen
> <pmatilai@redhat.com>
> Subject: [dpdk-dev] [PATCHv6 7/7] doc: Add prog_guide section documenting
> pmdinfo script
> 
> Information on pmdinfogen may be useful to 3rd party driver developers.
> Include documentation on what it does

Hi,

There are some trailing whitespace warnings on merge. Some other, minor,
comments below.


> 
> +.. _Internally_Generated_Build_Tools

The target needs a colon at the end to make it valid:

    .. _Internally_Generated_Build_Tools:


> +
> +buildtools/pmdinfogen

This might be better with some distinguishing highlighting, either
``buildtools/pmdinfogen`` or **buildtools/pmdinfogen**.


> +pmdinfogen scans an object (.o) file for various well known symbol

Instances of pmdinfogen would be better as a fixed width ``pmdinfogen``.


> +names.  These well know symbol names are defined by various macros and
> +used to export important information about hardware support and usage
> +for pmd files.  for instance the macro:

s/know/known
s/for/For


> +These strings can then be searched for by external tools to determine
> +the hardware support of a given library or application

Missing full stop.

Apart from these minor changes:


Acked-by: John McNamara <john.mcnamara@intel.com>

Re: [PATCHv6 7/7] doc: Add prog_guide section documenting pmdinfo script

[Neil Horman]

On Wed, Jun 08, 2016 at 05:14:05PM +0000, Mcnamara, John wrote:
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
> > Sent: Tuesday, May 31, 2016 2:58 PM
> > To: dev@dpdk.org
> > Cc: Neil Horman <nhorman@tuxdriver.com>; Richardson, Bruce
> > <bruce.richardson@intel.com>; Thomas Monjalon <thomas.monjalon@6wind.com>;
> > Stephen Hemminger <stephen@networkplumber.org>; Panu Matilainen
> > <pmatilai@redhat.com>
> > Subject: [dpdk-dev] [PATCHv6 7/7] doc: Add prog_guide section documenting
> > pmdinfo script
> > 
> > Information on pmdinfogen may be useful to 3rd party driver developers.
> > Include documentation on what it does
> 
> Hi,
> 
> There are some trailing whitespace warnings on merge. Some other, minor,
> comments below.
> 
> 
> > 
> > +.. _Internally_Generated_Build_Tools
> 
> The target needs a colon at the end to make it valid:
> 
>     .. _Internally_Generated_Build_Tools:
> 
> 
> > +
> > +buildtools/pmdinfogen
> 
> This might be better with some distinguishing highlighting, either
> ``buildtools/pmdinfogen`` or **buildtools/pmdinfogen**.
> 
> 
> > +pmdinfogen scans an object (.o) file for various well known symbol
> 
> Instances of pmdinfogen would be better as a fixed width ``pmdinfogen``.
> 
> 
> > +names.  These well know symbol names are defined by various macros and
> > +used to export important information about hardware support and usage
> > +for pmd files.  for instance the macro:
> 
> s/know/known
> s/for/For
> 
> 
> > +These strings can then be searched for by external tools to determine
> > +the hardware support of a given library or application
> 
> Missing full stop.
> 
> Apart from these minor changes:
> 
> 
> Acked-by: John McNamara <john.mcnamara@intel.com>
Thanks, I'm incorporating this with the updates that Thomas and I have been
arguing about.
Neil

> 
> 
> 

[PATCHv7 0/6] Implement pmd hardware support exports

[Neil Horman]

Hey all-
	So heres attempt number 2 at a method for exporting PMD hardware support
information.  As we discussed previously, the consensus seems to be that pmd
information should be:

1) Able to be interrogated on any ELF binary (application binary or individual
DSO)
2) Equally functional on statically linked applications or on DSO's
3) Resilient to symbol stripping
4) Script friendly
5) Show kernel dependencies
6) List driver options
7) Show driver name
8) Offer human readable output
9) Show DPDK version
10) Show driver version
11) Allow for expansion
12) Not place additional build environment dependencies on an application

I added item 12 myself, because I don't think its reasonable to require
applications to use a specific linker script to get hardware support information
(which precludes the use of a special .modinfo section like the kernel uses)

However, we still can use some tricks from the kernel to make this work.  In
this approach, what I've done is the following:

A) Modified the driver registration macro to also define a variable:
this_pmd_name<n>= "name"

Based on the unique name string pointed to by the above variable, we can
query for an arbitrary number of other symbols following the pattern:
<name>_<tag>

Where tag is some well known identifier for information we wish to export

B) Added a utility called pmdinfogen.  This utility is not meant for general use,
but rather is used by the dpdk build environment itself when compiling pmds.
for each object that uses the PMD_REGISTER_DRIVER macro, this utiity is run.  It
searches for the symbols in (A), and using those, extracts the hardware support
info, and module name from the object, using that to produce a new c file
containing a single variable in the following format:

static const char <name>[] __attribute__((used)) = "PMD_DRIVER_INFO=<json>";

The <name> is arbitrary, as its static and not referenced.  The relevant bit is
the string value assigned to it.  The <json> is a json encoded string of the
extracted hardware support information pmdinfo found for the corresponding
object.  This C file is suitable for compilation and relocatable linking back
into the parent object file.  The result of this operation is that the object
string table now contains a string that will not e removed by stripping, whos
leading text (PMD_DRIVER_INFO) can be easily searched for at any time weather
the symbol referring to it is stripped or not.

C) Added a utilty called pmdinfo.py.  This python script, searches the
string table of the .rodata section of any provided ELF file looking for the
PMD_DRIVER_INFO prefix on a string.  When found, it will interpret the remainder
of the string as json, and output the hardware support for that ELF file (if
any).


This approach ticks most of the above boxes:
1) Impervious to stripping
2) Works on static and dynamic binaries
3) Human and script friendly
4) Allows for expansion

Because of (4) the other items should be pretty easy to implement, as its just a
matter of modifying the macros to export the info, pmdinfo to encode it to json,
and pmd_hw_support.py to read it.  I'm not adding them now, as theres alot of
rote work to do to get some of them in place (e.g. DPDK has no current global
VERSION macro, drivers don't have a consistent version scheme, command line
strings have to be built for each driver, etc).  But once this is accepted,
those items can be done as time allows and should be fairly easy to implement.

Change Notes:
v2)
 * Made the export macros a bit easier to expand
 * Added a macro to optionally export command line strings
 * Renamed utilties to be more appropriate
   (pmdinfo -> pmdinfogen, pmd_hw_support.py -> pmdinfo.py)
 * Added search capabilities to pmdinfo.py so that we search for libraries
   linked using DT_NEEDINFO entries.  We search DT_RUNPATH, LD_LIBRARY_PATH,
   /usr/lib and /lib
 * Added serch abilities if the specified binary to pmdinfo isn't found, we
   search LD_LIBRARY_PATH, /usr/lib and /lib
 * Added an option to pmdinfo.py to pretty-print hardware support info using the
   pci.ids database

v3)
 * Made table mode printing the default mode
 * Added a default path to the pci.ids file
 * Modifed pmdinfo to use python rather than python3
 * Cleaned up some changelog entries
 * Added an export for RTE_EAL_PMD_PATH
 * Added a plugin option to pmdinfo to scan for autoloaded DSO's

v4)
 * Modified the operation of the -p option. As much as I don't like implying
that autoloaded pmds are guaranteed to be there at run time, I'm having a hard
time seeing how we can avoid specifying the application file to scan for the
autoload directory.  Without it we can't determine which library the user means
in a multiversion installation
 * Cleaned up the help text
 * Added a rule for an install target for pmdinfo
 * Guarded against some tracebacks in pmdinfo
 * Use DT_NEEDED entries to get versioned libraries in -p mode
 * Fixed traceback that occurs on lack of input arguments
 * Fixed some erroneous macro usage in drivers that aren't in the default build

v5)
 * Added a dpdk- prefix to pmdinfo symlink
 * Renamed build announcement to PMDINFOGEN/BUILD
 * Removed erroneous true statement from makefile
 * Removed duplicate rte.hostapp.mk makefile
 * Fixed some whitespace
 * Whitespace fixups
 * Fixed makefile if; then style
 * Renamed module string C file
 * Removed duplicate rte_pci_id definition
 * Clarified macro names
 * Removed PMD type attribute
 * Fixed tools usage for 32 bit arches
 * Removed some unused code
 * Added a few comments

v6)
 * Added some programming guide documentation
 * Reformatted python script with pep8

v7)
 * Fixed up copyright
 * Switched buildtool makefile to use hostapp.mk
 * Modified arch check to use RTE_ARCH_64
 * Modifed hostapp.mk to drop output in build/app
 * Additional testing on ppc64 to ensure big endian works
 * Modified TO_NATIVE macro to use rte_byteorder inlines 
   based on endianess of target ELF file
 * Ran checkpatch on commits
 * Fixed some typos in doc

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>

[PATCHv7 1/6] pmdinfogen: Add buildtools and pmdinfogen utility

[Neil Horman]

pmdinfogen is a tool used to parse object files and build json strings for
use in later determining hardware support in a dso or application binary.
pmdinfo looks for the non-exported symbol names this_pmd_name<n> and
this_pmd_tbl<n> (where n is a integer counter).  It records the name of
each of these tuples, using the later to find the symbolic name of the
pci_table for physical devices that the object supports.  With this
information, it outputs a C file with a single line of the form:

static char *<pmd_name>_driver_info[] __attribute__((used)) = " \
	PMD_DRIVER_INFO=<json string>";

Where <pmd_name> is the arbitrary name of the pmd, and <json_string> is the
json encoded string that hold relevant pmd information, including the pmd
name, type and optional array of pci device/vendor ids that the driver
supports.

This c file is suitable for compiling to object code, then relocatably
linking into the parent file from which the C was generated.  This creates
an entry in the string table of the object that can inform a later tool
about hardware support.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 GNUmakefile                        |   2 +-
 buildtools/Makefile                |  36 +++
 buildtools/pmdinfogen/Makefile     |  49 +++++
 buildtools/pmdinfogen/pmdinfogen.c | 439 +++++++++++++++++++++++++++++++++++++
 buildtools/pmdinfogen/pmdinfogen.h | 120 ++++++++++
 mk/rte.hostapp.mk                  |   8 +-
 mk/rte.sdkbuild.mk                 |   3 +-
 7 files changed, 651 insertions(+), 6 deletions(-)
 create mode 100644 buildtools/Makefile
 create mode 100644 buildtools/pmdinfogen/Makefile
 create mode 100644 buildtools/pmdinfogen/pmdinfogen.c
 create mode 100644 buildtools/pmdinfogen/pmdinfogen.h

diff --git a/GNUmakefile b/GNUmakefile
index b59e4b6..00fe0db 100644
--- a/GNUmakefile
+++ b/GNUmakefile
@@ -40,6 +40,6 @@ export RTE_SDK
 # directory list
 #
 
-ROOTDIRS-y := lib drivers app
+ROOTDIRS-y := buildtools lib drivers app
 
 include $(RTE_SDK)/mk/rte.sdkroot.mk
diff --git a/buildtools/Makefile b/buildtools/Makefile
new file mode 100644
index 0000000..35a42ff
--- /dev/null
+++ b/buildtools/Makefile
@@ -0,0 +1,36 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2016 Neil Horman. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+DIRS-y += pmdinfogen
+
+include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/buildtools/pmdinfogen/Makefile b/buildtools/pmdinfogen/Makefile
new file mode 100644
index 0000000..125901b
--- /dev/null
+++ b/buildtools/pmdinfogen/Makefile
@@ -0,0 +1,49 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2016 Neil Horman. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+#
+# library name
+#
+HOSTAPP = pmdinfogen
+
+#
+# all sources are stored in SRCS-y
+#
+SRCS-y += pmdinfogen.c
+
+HOST_EXTRA_CFLAGS += -g -I${RTE_OUTPUT}/include
+
+DEPDIRS-y += lib/librte_eal
+
+include $(RTE_SDK)/mk/rte.hostapp.mk
+
diff --git a/buildtools/pmdinfogen/pmdinfogen.c b/buildtools/pmdinfogen/pmdinfogen.c
new file mode 100644
index 0000000..0947dc6
--- /dev/null
+++ b/buildtools/pmdinfogen/pmdinfogen.c
@@ -0,0 +1,439 @@
+/* Postprocess pmd object files to export hw support
+ *
+ * Copyright 2016 Neil Horman <nhorman@tuxdriver.com>
+ * Based in part on modpost.c from the linux kernel
+ *
+ * This software may be used and distributed according to the terms
+ * of the GNU General Public License V2, incorporated herein by reference.
+ *
+ */
+
+#define _GNU_SOURCE
+#include <stdio.h>
+#include <ctype.h>
+#include <string.h>
+#include <limits.h>
+#include <stdbool.h>
+#include <errno.h>
+#include "pmdinfogen.h"
+
+#ifdef RTE_ARCH_64
+#define ADDR_SIZE 64
+#else
+#define ADDR_SIZE 32
+#endif
+
+
+static const char *sym_name(struct elf_info *elf, Elf_Sym *sym)
+{
+	if (sym)
+		return elf->strtab + sym->st_name;
+	else
+		return "(unknown)";
+}
+
+void *grab_file(const char *filename, unsigned long *size)
+{
+	struct stat st;
+	void *map = MAP_FAILED;
+	int fd;
+
+	fd = open(filename, O_RDONLY);
+	if (fd < 0)
+		return NULL;
+	if (fstat(fd, &st))
+		goto failed;
+
+	*size = st.st_size;
+	map = mmap(NULL, *size, PROT_READ|PROT_WRITE, MAP_PRIVATE, fd, 0);
+
+failed:
+	close(fd);
+	if (map == MAP_FAILED)
+		return NULL;
+	return map;
+}
+
+/**
+  * Return a copy of the next line in a mmap'ed file.
+  * spaces in the beginning of the line is trimmed away.
+  * Return a pointer to a static buffer.
+  **/
+void release_file(void *file, unsigned long size)
+{
+	munmap(file, size);
+}
+
+
+static void *get_sym_value(struct elf_info *info, const Elf_Sym *sym)
+{
+	void *ptr = (void *)info->hdr + info->sechdrs[sym->st_shndx].sh_offset;
+
+	return (void *)(ptr + sym->st_value);
+}
+
+static Elf_Sym *find_sym_in_symtab(struct elf_info *info,
+				   const char *name, Elf_Sym *last)
+{
+	Elf_Sym *idx;
+	if (last)
+		idx = last+1;
+	else
+		idx = info->symtab_start;
+
+	for (; idx < info->symtab_stop; idx++) {
+		const char *n = sym_name(info, idx);
+		if (!strncmp(n, name, strlen(name)))
+			return idx;
+	}
+	return NULL;
+}
+
+static int parse_elf(struct elf_info *info, const char *filename)
+{
+	unsigned int i;
+	Elf_Ehdr *hdr;
+	Elf_Shdr *sechdrs;
+	Elf_Sym  *sym;
+	const char *secstrings;
+	int endian;
+	unsigned int symtab_idx = ~0U, symtab_shndx_idx = ~0U;
+
+	hdr = grab_file(filename, &info->size);
+	if (!hdr) {
+		perror(filename);
+		exit(1);
+	}
+	info->hdr = hdr;
+	if (info->size < sizeof(*hdr)) {
+		/* file too small, assume this is an empty .o file */
+		return 0;
+	}
+	/* Is this a valid ELF file? */
+	if ((hdr->e_ident[EI_MAG0] != ELFMAG0) ||
+	    (hdr->e_ident[EI_MAG1] != ELFMAG1) ||
+	    (hdr->e_ident[EI_MAG2] != ELFMAG2) ||
+	    (hdr->e_ident[EI_MAG3] != ELFMAG3)) {
+		/* Not an ELF file - silently ignore it */
+		return 0;
+	}
+
+	if (!hdr->e_ident[EI_DATA]) {
+		/* Unknown endian */
+		return 0;
+	}
+
+	endian = hdr->e_ident[EI_DATA];
+
+	/* Fix endianness in ELF header */
+	hdr->e_type      = TO_NATIVE(endian, 16, hdr->e_type);
+	hdr->e_machine   = TO_NATIVE(endian, 16, hdr->e_machine);
+	hdr->e_version   = TO_NATIVE(endian, 32, hdr->e_version);
+	hdr->e_entry     = TO_NATIVE(endian, ADDR_SIZE, hdr->e_entry);
+	hdr->e_phoff     = TO_NATIVE(endian, ADDR_SIZE, hdr->e_phoff);
+	hdr->e_shoff     = TO_NATIVE(endian, ADDR_SIZE, hdr->e_shoff);
+	hdr->e_flags     = TO_NATIVE(endian, 32, hdr->e_flags);
+	hdr->e_ehsize    = TO_NATIVE(endian, 16, hdr->e_ehsize);
+	hdr->e_phentsize = TO_NATIVE(endian, 16, hdr->e_phentsize);
+	hdr->e_phnum     = TO_NATIVE(endian, 16, hdr->e_phnum);
+	hdr->e_shentsize = TO_NATIVE(endian, 16, hdr->e_shentsize);
+	hdr->e_shnum     = TO_NATIVE(endian, 16, hdr->e_shnum);
+	hdr->e_shstrndx  = TO_NATIVE(endian, 16, hdr->e_shstrndx);
+
+	sechdrs = (void *)hdr + hdr->e_shoff;
+	info->sechdrs = sechdrs;
+
+	/* Check if file offset is correct */
+	if (hdr->e_shoff > info->size) {
+		fprintf(stderr, "section header offset=%lu in file '%s' "
+		      "is bigger than filesize=%lu\n",
+		      (unsigned long)hdr->e_shoff,
+		      filename, info->size);
+		return 0;
+	}
+
+	if (hdr->e_shnum == SHN_UNDEF) {
+		/*
+		 * There are more than 64k sections,
+		 * read count from .sh_size.
+		 */
+		info->num_sections = TO_NATIVE(endian, 32, sechdrs[0].sh_size);
+	} else {
+		info->num_sections = hdr->e_shnum;
+	}
+	if (hdr->e_shstrndx == SHN_XINDEX)
+		info->secindex_strings =
+			TO_NATIVE(endian, 32, sechdrs[0].sh_link);
+	else
+		info->secindex_strings = hdr->e_shstrndx;
+
+	/* Fix endianness in section headers */
+	for (i = 0; i < info->num_sections; i++) {
+		sechdrs[i].sh_name      =
+			TO_NATIVE(endian, 32, sechdrs[i].sh_name);
+		sechdrs[i].sh_type      =
+			TO_NATIVE(endian, 32, sechdrs[i].sh_type);
+		sechdrs[i].sh_flags     =
+			TO_NATIVE(endian, 32, sechdrs[i].sh_flags);
+		sechdrs[i].sh_addr      =
+			TO_NATIVE(endian, ADDR_SIZE, sechdrs[i].sh_addr);
+		sechdrs[i].sh_offset    =
+			TO_NATIVE(endian, ADDR_SIZE, sechdrs[i].sh_offset);
+		sechdrs[i].sh_size      =
+			TO_NATIVE(endian, 32, sechdrs[i].sh_size);
+		sechdrs[i].sh_link      =
+			TO_NATIVE(endian, 32, sechdrs[i].sh_link);
+		sechdrs[i].sh_info      =
+			TO_NATIVE(endian, 32, sechdrs[i].sh_info);
+		sechdrs[i].sh_addralign =
+			TO_NATIVE(endian, ADDR_SIZE, sechdrs[i].sh_addralign);
+		sechdrs[i].sh_entsize   =
+			TO_NATIVE(endian, ADDR_SIZE, sechdrs[i].sh_entsize);
+	}
+	/* Find symbol table. */
+	secstrings = (void *)hdr + sechdrs[info->secindex_strings].sh_offset;
+	for (i = 1; i < info->num_sections; i++) {
+		int nobits = sechdrs[i].sh_type == SHT_NOBITS;
+
+		if (!nobits && sechdrs[i].sh_offset > info->size) {
+			fprintf(stderr, "%s is truncated. "
+			      "sechdrs[i].sh_offset=%lu > sizeof(*hrd)=%zu\n",
+			      filename, (unsigned long)sechdrs[i].sh_offset,
+			      sizeof(*hdr));
+			return 0;
+		}
+
+		if (sechdrs[i].sh_type == SHT_SYMTAB) {
+			unsigned int sh_link_idx;
+			symtab_idx = i;
+			info->symtab_start = (void *)hdr +
+			    sechdrs[i].sh_offset;
+			info->symtab_stop  = (void *)hdr +
+			    sechdrs[i].sh_offset + sechdrs[i].sh_size;
+			sh_link_idx = sechdrs[i].sh_link;
+			info->strtab       = (void *)hdr +
+			    sechdrs[sh_link_idx].sh_offset;
+		}
+
+		/* 32bit section no. table? ("more than 64k sections") */
+		if (sechdrs[i].sh_type == SHT_SYMTAB_SHNDX) {
+			symtab_shndx_idx = i;
+			info->symtab_shndx_start = (void *)hdr +
+			    sechdrs[i].sh_offset;
+			info->symtab_shndx_stop  = (void *)hdr +
+			    sechdrs[i].sh_offset + sechdrs[i].sh_size;
+		}
+	}
+	if (!info->symtab_start)
+		fprintf(stderr, "%s has no symtab?\n", filename);
+
+	/* Fix endianness in symbols */
+	for (sym = info->symtab_start; sym < info->symtab_stop; sym++) {
+		sym->st_shndx = TO_NATIVE(endian, 16, sym->st_shndx);
+		sym->st_name  = TO_NATIVE(endian, 32, sym->st_name);
+		sym->st_value = TO_NATIVE(endian, ADDR_SIZE, sym->st_value);
+		sym->st_size  = TO_NATIVE(endian, ADDR_SIZE, sym->st_size);
+	}
+
+	if (symtab_shndx_idx != ~0U) {
+		Elf32_Word *p;
+		if (symtab_idx != sechdrs[symtab_shndx_idx].sh_link)
+			fprintf(stderr,
+			      "%s: SYMTAB_SHNDX has bad sh_link: %u!=%u\n",
+			      filename, sechdrs[symtab_shndx_idx].sh_link,
+			      symtab_idx);
+		/* Fix endianness */
+		for (p = info->symtab_shndx_start; p < info->symtab_shndx_stop;
+		     p++)
+			*p = TO_NATIVE(endian, 32, *p);
+	}
+
+	return 1;
+}
+
+static void parse_elf_finish(struct elf_info *info)
+{
+	struct pmd_driver *tmp, *idx = info->drivers;
+	release_file(info->hdr, info->size);
+	while (idx) {
+		tmp = idx->next;
+		free(idx);
+		idx = tmp;
+	}
+}
+
+static const char *sec_name(struct elf_info *elf, int secindex)
+{
+	Elf_Shdr *sechdrs = elf->sechdrs;
+	return (void *)elf->hdr +
+		elf->sechdrs[elf->secindex_strings].sh_offset +
+		sechdrs[secindex].sh_name;
+}
+
+static int get_symbol_index(struct elf_info *info, Elf_Sym *sym)
+{
+	const char *name =  sym_name(info, sym);
+	const char *idx;
+
+	idx = name;
+	while (idx) {
+		if (isdigit(*idx))
+			return atoi(idx);
+		idx++;
+	}
+	return -1;
+}
+
+struct opt_tag {
+	const char *suffix;
+	const char *json_id;
+};
+
+static const struct opt_tag opt_tags[] = {
+	{"_param_string_export", "params"},
+};
+
+static int complete_pmd_entry(struct elf_info *info, struct pmd_driver *drv)
+{
+	const char *tname;
+	int i;
+	char tmpsymname[128];
+	Elf_Sym *tmpsym;
+
+	drv->name = get_sym_value(info, drv->name_sym);
+
+	for (i = 0; i < PMD_OPT_MAX; i++) {
+		memset(tmpsymname, 0, 128);
+		sprintf(tmpsymname, "__%s%s", drv->name, opt_tags[i].suffix);
+		tmpsym = find_sym_in_symtab(info, tmpsymname, NULL);
+		if (!tmpsym)
+			continue;
+		drv->opt_vals[i] = get_sym_value(info, tmpsym);
+	}
+
+	memset(tmpsymname, 0, 128);
+	sprintf(tmpsymname, "__%s_pci_tbl_export", drv->name);
+
+	tmpsym = find_sym_in_symtab(info, tmpsymname, NULL);
+
+
+	/*
+	 * If this returns NULL, then this is a PMD_VDEV, because
+	 * it has no pci table reference
+	 */
+	if (!tmpsym) {
+		drv->pci_tbl = NULL;
+		return 0;
+	}
+
+	tname = get_sym_value(info, tmpsym);
+	tmpsym = find_sym_in_symtab(info, tname, NULL);
+	if (!tmpsym)
+		return -ENOENT;
+
+	drv->pci_tbl = (struct rte_pci_id *)get_sym_value(info, tmpsym);
+	if (!drv->pci_tbl)
+		return -ENOENT;
+
+	return 0;
+}
+
+static int locate_pmd_entries(struct elf_info *info)
+{
+	Elf_Sym *last = NULL;
+	struct pmd_driver *new;
+
+	info->drivers = NULL;
+
+	do {
+		new = calloc(sizeof(struct pmd_driver), 1);
+		new->name_sym = find_sym_in_symtab(info, "this_pmd_name", last);
+		last = new->name_sym;
+		if (!new->name_sym)
+			free(new);
+		else {
+			if (complete_pmd_entry(info, new)) {
+				fprintf(stderr,
+					"Failed to complete pmd entry\n");
+				free(new);
+			} else {
+				new->next = info->drivers;
+				info->drivers = new;
+			}
+		}
+	} while (last);
+}
+
+static void output_pmd_info_string(struct elf_info *info, char *outfile)
+{
+	FILE *ofd;
+	struct pmd_driver *drv;
+	struct rte_pci_id *pci_ids;
+	int idx = 0;
+
+	ofd = fopen(outfile, "w+");
+	if (!ofd) {
+		fprintf(stderr, "Unable to open output file\n");
+		return;
+	}
+
+	drv = info->drivers;
+
+	while (drv) {
+		fprintf(ofd, "const char %s_pmd_info[] __attribute__((used)) = "
+			"\"PMD_INFO_STRING= {",
+			drv->name);
+		fprintf(ofd, "\\\"name\\\" : \\\"%s\\\", ", drv->name);
+
+		for (idx = 0; idx < PMD_OPT_MAX; idx++) {
+			if (drv->opt_vals[idx])
+				fprintf(ofd, "\\\"%s\\\" : \\\"%s\\\", ",
+					opt_tags[idx].json_id,
+					drv->opt_vals[idx]);
+		}
+
+		pci_ids = drv->pci_tbl;
+		fprintf(ofd, "\\\"pci_ids\\\" : [");
+
+		while (pci_ids && pci_ids->device_id) {
+			fprintf(ofd, "[%d, %d, %d, %d]",
+				pci_ids->vendor_id, pci_ids->device_id,
+				pci_ids->subsystem_vendor_id,
+				pci_ids->subsystem_device_id);
+			pci_ids++;
+			if (pci_ids->device_id)
+				fprintf(ofd, ",");
+			else
+				fprintf(ofd, " ");
+		}
+		fprintf(ofd, "]}\";");
+		drv = drv->next;
+	}
+
+	fclose(ofd);
+}
+
+int main(int argc, char **argv)
+{
+	struct elf_info info;
+	int rc = 1;
+
+	if (argc < 3) {
+		fprintf(stderr,
+			"usage: pmdinfo <object file> <c output file>\n");
+		exit(127);
+	}
+	parse_elf(&info, argv[1]);
+
+	locate_pmd_entries(&info);
+
+	if (info.drivers) {
+		output_pmd_info_string(&info, argv[2]);
+		rc = 0;
+	} else {
+		fprintf(stderr, "No drivers registered\n");
+	}
+
+	parse_elf_finish(&info);
+	exit(rc);
+}
diff --git a/buildtools/pmdinfogen/pmdinfogen.h b/buildtools/pmdinfogen/pmdinfogen.h
new file mode 100644
index 0000000..1da2966
--- /dev/null
+++ b/buildtools/pmdinfogen/pmdinfogen.h
@@ -0,0 +1,120 @@
+
+/* Postprocess pmd object files to export hw support
+ *
+ * Copyright 2016 Neil Horman <nhorman@tuxdriver.com>
+ * Based in part on modpost.c from the linux kernel
+ *
+ * This software may be used and distributed according to the terms
+ * of the GNU General Public License V2, incorporated herein by reference.
+ *
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdarg.h>
+#include <string.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/mman.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <elf.h>
+#include <rte_config.h>
+#include <rte_pci.h>
+#include <rte_byteorder.h>
+
+/* On BSD-alike OSes elf.h defines these according to host's word size */
+#undef ELF_ST_BIND
+#undef ELF_ST_TYPE
+#undef ELF_R_SYM
+#undef ELF_R_TYPE
+
+/*
+ * Define ELF64_* to ELF_*, the latter being defined in both 32 and 64 bit
+ * flavors in elf.h.  This makes our code a bit more generic between arches
+ * and allows us to support 32 bit code in the future should we ever want to
+ */
+#ifdef RTE_ARCH_64
+#define Elf_Ehdr    Elf64_Ehdr
+#define Elf_Shdr    Elf64_Shdr
+#define Elf_Sym     Elf64_Sym
+#define Elf_Addr    Elf64_Addr
+#define Elf_Sword   Elf64_Sxword
+#define Elf_Section Elf64_Half
+#define ELF_ST_BIND ELF64_ST_BIND
+#define ELF_ST_TYPE ELF64_ST_TYPE
+
+#define Elf_Rel     Elf64_Rel
+#define Elf_Rela    Elf64_Rela
+#define ELF_R_SYM   ELF64_R_SYM
+#define ELF_R_TYPE  ELF64_R_TYPE
+#else
+#define Elf_Ehdr    Elf32_Ehdr
+#define Elf_Shdr    Elf32_Shdr
+#define Elf_Sym     Elf32_Sym
+#define Elf_Addr    Elf32_Addr
+#define Elf_Sword   Elf32_Sxword
+#define Elf_Section Elf32_Half
+#define ELF_ST_BIND ELF32_ST_BIND
+#define ELF_ST_TYPE ELF32_ST_TYPE
+
+#define Elf_Rel     Elf32_Rel
+#define Elf_Rela    Elf32_Rela
+#define ELF_R_SYM   ELF32_R_SYM
+#define ELF_R_TYPE  ELF32_R_TYPE
+#endif
+
+
+/*
+ * Note, it seems odd that we have both a CONVERT_NATIVE and a TO_NATIVE macro
+ * below.  We do this because the values passed to TO_NATIVE may themselves be
+ * macros and need both macros here to get expanded.  Specifically its the width
+ * variable we are concerned with, because it needs to get expanded prior to
+ * string concatenation
+ */
+#define CONVERT_NATIVE(fend, width, x) ({ \
+typeof(x) ___x; \
+if ((fend) == ELFDATA2LSB) \
+	___x = rte_le_to_cpu_##width(x); \
+else \
+	___x = rte_be_to_cpu_##width(x); \
+	___x; \
+})
+
+#define TO_NATIVE(fend, width, x) CONVERT_NATIVE(fend, width, x)
+
+enum opt_params {
+	PMD_PARAM_STRING = 0,
+	PMD_OPT_MAX
+};
+
+struct pmd_driver {
+	Elf_Sym *name_sym;
+	const char *name;
+	struct rte_pci_id *pci_tbl;
+	struct pmd_driver *next;
+
+	const char *opt_vals[PMD_OPT_MAX];
+};
+
+struct elf_info {
+	unsigned long size;
+	Elf_Ehdr     *hdr;
+	Elf_Shdr     *sechdrs;
+	Elf_Sym      *symtab_start;
+	Elf_Sym      *symtab_stop;
+	char         *strtab;
+
+	/* support for 32bit section numbers */
+
+	unsigned int num_sections; /* max_secindex + 1 */
+	unsigned int secindex_strings;
+	/* if Nth symbol table entry has .st_shndx = SHN_XINDEX,
+	 * take shndx from symtab_shndx_start[N] instead
+	 */
+	Elf32_Word   *symtab_shndx_start;
+	Elf32_Word   *symtab_shndx_stop;
+
+	struct pmd_driver *drivers;
+};
+
diff --git a/mk/rte.hostapp.mk b/mk/rte.hostapp.mk
index c44d0f8..07b391c 100644
--- a/mk/rte.hostapp.mk
+++ b/mk/rte.hostapp.mk
@@ -41,7 +41,7 @@ include $(RTE_SDK)/mk/internal/rte.depdirs-pre.mk
 VPATH += $(SRCDIR)
 
 _BUILD = $(HOSTAPP)
-_INSTALL = $(INSTALL-FILES-y) $(SYMLINK-FILES-y) $(RTE_OUTPUT)/hostapp/$(HOSTAPP)
+_INSTALL = $(INSTALL-FILES-y) $(SYMLINK-FILES-y) $(RTE_OUTPUT)/app/$(HOSTAPP)
 _CLEAN = doclean
 
 .PHONY: all
@@ -95,10 +95,10 @@ $(HOSTAPP): $(OBJS-y) $(LDLIBS_FILES) FORCE
 #
 # install app in $(RTE_OUTPUT)/hostapp
 #
-$(RTE_OUTPUT)/hostapp/$(HOSTAPP): $(HOSTAPP)
+$(RTE_OUTPUT)/app/$(HOSTAPP): $(HOSTAPP)
 	@echo "  INSTALL-HOSTAPP $(HOSTAPP)"
-	@[ -d $(RTE_OUTPUT)/hostapp ] || mkdir -p $(RTE_OUTPUT)/hostapp
-	$(Q)cp -f $(HOSTAPP) $(RTE_OUTPUT)/hostapp
+	@[ -d $(RTE_OUTPUT)/app ] || mkdir -p $(RTE_OUTPUT)/app
+	$(Q)cp -f $(HOSTAPP) $(RTE_OUTPUT)/app
 
 #
 # Clean all generated files
diff --git a/mk/rte.sdkbuild.mk b/mk/rte.sdkbuild.mk
index eec5241..fb68af2 100644
--- a/mk/rte.sdkbuild.mk
+++ b/mk/rte.sdkbuild.mk
@@ -64,7 +64,8 @@ build: $(ROOTDIRS-y)
 clean: $(CLEANDIRS)
 	@rm -rf $(RTE_OUTPUT)/include $(RTE_OUTPUT)/app \
 		$(RTE_OUTPUT)/hostapp $(RTE_OUTPUT)/lib \
-		$(RTE_OUTPUT)/hostlib $(RTE_OUTPUT)/kmod
+		$(RTE_OUTPUT)/hostlib $(RTE_OUTPUT)/kmod \
+		$(RTE_OUTPUT)/buildtools
 	@[ -d $(RTE_OUTPUT)/include ] || mkdir -p $(RTE_OUTPUT)/include
 	@$(RTE_SDK)/scripts/gen-config-h.sh $(RTE_OUTPUT)/.config \
 		> $(RTE_OUTPUT)/include/rte_config.h
-- 
2.5.5

[PATCHv7 2/6] drivers: Update driver registration macro usage

[Neil Horman]

Modify the PMD_REGISTER_DRIVER macro, adding a name argument to it.  The
addition of a name argument creates a token that can be used for subsequent
macros in the creation of unique symbol names to export additional bits of
information for use by the pmdinfogen tool.  For example:

PMD_REGISTER_DRIVER(ena_driver, ena);

registers the ena_driver struct as it always did, and creates a symbol
const char this_pmd_name0[] __attribute__((used)) = "ena";

which pmdinfogen can search for and extract.  The subsequent macro

DRIVER_REGISTER_PCI_TABLE(ena, ena_pci_id_map);

creates a symbol const char ena_pci_tbl_export[] __attribute__((used)) =
"ena_pci_id_map";

Which allows pmdinfogen to find the pci table of this driver

Using this pattern, we can export arbitrary bits of information.

pmdinfo uses this information to extract hardware support from an object
file and create a json string to make hardware support info discoverable
later.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 drivers/Makefile                           |  2 ++
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c   |  4 +++-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c |  4 +++-
 drivers/crypto/null/null_crypto_pmd.c      |  4 +++-
 drivers/crypto/qat/rte_qat_cryptodev.c     |  4 +++-
 drivers/crypto/snow3g/rte_snow3g_pmd.c     |  4 +++-
 drivers/net/af_packet/rte_eth_af_packet.c  |  4 +++-
 drivers/net/bnx2x/bnx2x_ethdev.c           |  6 ++++--
 drivers/net/bonding/rte_eth_bond_pmd.c     |  7 ++++++-
 drivers/net/cxgbe/cxgbe_ethdev.c           |  4 +++-
 drivers/net/e1000/em_ethdev.c              |  3 ++-
 drivers/net/e1000/igb_ethdev.c             |  6 ++++--
 drivers/net/ena/ena_ethdev.c               |  3 ++-
 drivers/net/enic/enic_ethdev.c             |  3 ++-
 drivers/net/fm10k/fm10k_ethdev.c           |  3 ++-
 drivers/net/i40e/i40e_ethdev.c             |  3 ++-
 drivers/net/i40e/i40e_ethdev_vf.c          |  3 ++-
 drivers/net/ixgbe/ixgbe_ethdev.c           |  6 ++++--
 drivers/net/mlx4/mlx4.c                    |  3 ++-
 drivers/net/mlx5/mlx5.c                    |  3 ++-
 drivers/net/mpipe/mpipe_tilegx.c           |  4 ++--
 drivers/net/nfp/nfp_net.c                  |  3 ++-
 drivers/net/null/rte_eth_null.c            |  3 ++-
 drivers/net/pcap/rte_eth_pcap.c            |  4 +++-
 drivers/net/qede/qede_ethdev.c             |  6 ++++--
 drivers/net/ring/rte_eth_ring.c            |  3 ++-
 drivers/net/szedata2/rte_eth_szedata2.c    |  3 ++-
 drivers/net/vhost/rte_eth_vhost.c          |  3 ++-
 drivers/net/virtio/virtio_ethdev.c         |  3 ++-
 drivers/net/vmxnet3/vmxnet3_ethdev.c       |  3 ++-
 drivers/net/xenvirt/rte_eth_xenvirt.c      |  2 +-
 lib/librte_eal/common/include/rte_dev.h    | 30 ++++++++++++++++++++++++------
 32 files changed, 105 insertions(+), 41 deletions(-)

diff --git a/drivers/Makefile b/drivers/Makefile
index 81c03a8..75a3168 100644
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -34,4 +34,6 @@ include $(RTE_SDK)/mk/rte.vars.mk
 DIRS-y += net
 DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += crypto
 
+DEPDIRS-y += buildtools/pmdinfo
+
 include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
index 2987ef6..f43e407 100644
--- a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
+++ b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
@@ -521,4 +521,6 @@ static struct rte_driver aesni_gcm_pmd_drv = {
 	.uninit = aesni_gcm_uninit
 };
 
-PMD_REGISTER_DRIVER(aesni_gcm_pmd_drv);
+PMD_REGISTER_DRIVER(aesni_gcm_pmd_drv, aesni_gcm);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+"max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
index 6554fc4..db3e562 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
@@ -721,4 +721,6 @@ static struct rte_driver cryptodev_aesni_mb_pmd_drv = {
 	.uninit = cryptodev_aesni_mb_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drv);
+PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drvi, aesni_mb);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+"max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/null/null_crypto_pmd.c b/drivers/crypto/null/null_crypto_pmd.c
index bdaf13c..0a195ed 100644
--- a/drivers/crypto/null/null_crypto_pmd.c
+++ b/drivers/crypto/null/null_crypto_pmd.c
@@ -275,4 +275,6 @@ static struct rte_driver cryptodev_null_pmd_drv = {
 	.uninit = cryptodev_null_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_null_pmd_drv);
+PMD_REGISTER_DRIVER(cryptodev_null_pmd_drv, cryptodev_null_pmd);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+"max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index a7912f5..d2de6a6 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -137,4 +137,6 @@ static struct rte_driver pmd_qat_drv = {
 	.init = rte_qat_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(pmd_qat_drv);
+PMD_REGISTER_DRIVER(pmd_qat_drv, qat);
+DRIVER_REGISTER_PCI_TABLE(qat, pci_id_qat_map);
+
diff --git a/drivers/crypto/snow3g/rte_snow3g_pmd.c b/drivers/crypto/snow3g/rte_snow3g_pmd.c
index f3e0e66..6934426 100644
--- a/drivers/crypto/snow3g/rte_snow3g_pmd.c
+++ b/drivers/crypto/snow3g/rte_snow3g_pmd.c
@@ -548,4 +548,6 @@ static struct rte_driver cryptodev_snow3g_pmd_drv = {
 	.uninit = cryptodev_snow3g_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_snow3g_pmd_drv);
+PMD_REGISTER_DRIVER(cryptodev_snow3g_pmd_drv, snow3g);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+"max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index f17bd7e..e10eff2 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -878,4 +878,6 @@ static struct rte_driver pmd_af_packet_drv = {
 	.uninit = rte_pmd_af_packet_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_af_packet_drv);
+PMD_REGISTER_DRIVER(pmd_af_packet_drv, eth_af_packet);
+DRIVER_REGISTER_PARAM_STRING(eth_af_packet, "iface=<string> "
+"qpairs=<int> blocksz=<int> framesz=<int> framecnt=<int>");
diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c b/drivers/net/bnx2x/bnx2x_ethdev.c
index 071b44f..2b88324 100644
--- a/drivers/net/bnx2x/bnx2x_ethdev.c
+++ b/drivers/net/bnx2x/bnx2x_ethdev.c
@@ -550,5 +550,7 @@ static struct rte_driver rte_bnx2xvf_driver = {
 	.init = rte_bnx2xvf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_bnx2x_driver);
-PMD_REGISTER_DRIVER(rte_bnx2xvf_driver);
+PMD_REGISTER_DRIVER(rte_bnx2x_driver, bnx2x);
+DRIVER_REGISTER_PCI_TABLE(bnx2x, pci_id_bnx2x_map);
+PMD_REGISTER_DRIVER(rte_bnx2xvf_driver, bnx2xvf);
+DRIVER_REGISTER_PCI_TABLE(bnx2xvf, pci_id_bnx2xvf_map);
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 129f04b..e94ae36 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -2511,4 +2511,9 @@ static struct rte_driver bond_drv = {
 	.uninit = bond_uninit,
 };
 
-PMD_REGISTER_DRIVER(bond_drv);
+PMD_REGISTER_DRIVER(bond_drv, bonding);
+
+DRIVER_REGISTER_PARAM_STRING(bonding, "slave=<ifc> primary=<ifc> mode=[0-4] "
+"xmit_policy=[l2 | l23 | l34] socket_id=<int> mac=<mac addr> "
+"lsc_poll_period_ms=<int> up_delay=<int> down_delay=<int>");
+
diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index 04eddaf..e84e48b 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -894,4 +894,6 @@ static struct rte_driver rte_cxgbe_driver = {
 	.init = rte_cxgbe_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_cxgbe_driver);
+PMD_REGISTER_DRIVER(rte_cxgbe_driver, cxgb4);
+DRIVER_REGISTER_PCI_TABLE(cxgb4, cxgb4_pci_tbl);
+
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index 653be09..4de5eb2 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -1777,4 +1777,5 @@ struct rte_driver em_pmd_drv = {
 	.init = rte_em_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(em_pmd_drv);
+PMD_REGISTER_DRIVER(em_pmd_drv, em);
+DRIVER_REGISTER_PCI_TABLE(em, pci_id_em_map);
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index f0921ee..f49aae7 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -4973,5 +4973,7 @@ eth_igb_configure_msix_intr(struct rte_eth_dev *dev)
 	E1000_WRITE_FLUSH(hw);
 }
 
-PMD_REGISTER_DRIVER(pmd_igb_drv);
-PMD_REGISTER_DRIVER(pmd_igbvf_drv);
+PMD_REGISTER_DRIVER(pmd_igb_drv, igb);
+DRIVER_REGISTER_PCI_TABLE(igb, pci_id_igb_map);
+PMD_REGISTER_DRIVER(pmd_igbvf_drv, igbvf);
+DRIVER_REGISTER_PCI_TABLE(igbvf, pci_id_igbvf_map);
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index e157587..702289b 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -1450,4 +1450,5 @@ struct rte_driver ena_pmd_drv = {
 	.init = rte_ena_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(ena_pmd_drv);
+PMD_REGISTER_DRIVER(ena_pmd_drv, ena);
+DRIVER_REGISTER_PCI_TABLE(ena, pci_id_ena_map);
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index 6bea940..36ee7ca 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -684,4 +684,5 @@ static struct rte_driver rte_enic_driver = {
 	.init = rte_enic_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_enic_driver);
+PMD_REGISTER_DRIVER(rte_enic_driver, enic);
+DRIVER_REGISTER_PCI_TABLE(enic, pci_id_enic_map);
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index c2d377f..9b2a7ee 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -3055,4 +3055,5 @@ static struct rte_driver rte_fm10k_driver = {
 	.init = rte_pmd_fm10k_init,
 };
 
-PMD_REGISTER_DRIVER(rte_fm10k_driver);
+PMD_REGISTER_DRIVER(rte_fm10k_driver, fm10k);
+DRIVER_REGISTER_PCI_TABLE(fm10k, pci_id_fm10k_map);
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 24777d5..3b6c661 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -695,7 +695,8 @@ static struct rte_driver rte_i40e_driver = {
 	.init = rte_i40e_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_i40e_driver);
+PMD_REGISTER_DRIVER(rte_i40e_driver, i40e);
+DRIVER_REGISTER_PCI_TABLE(i40e, pci_id_i40e_map);
 
 /*
  * Initialize registers for flexible payload, which should be set by NVM.
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 90682ac..e94a459 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1563,7 +1563,8 @@ static struct rte_driver rte_i40evf_driver = {
 	.init = rte_i40evf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_i40evf_driver);
+PMD_REGISTER_DRIVER(rte_i40evf_driver, i40evf);
+DRIVER_REGISTER_PCI_TABLE(i40evf, pci_id_i40evf_map);
 
 static int
 i40evf_dev_configure(struct rte_eth_dev *dev)
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index a2b170b..af1ca56 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -7164,5 +7164,7 @@ static struct rte_driver rte_ixgbevf_driver = {
 	.init = rte_ixgbevf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_ixgbe_driver);
-PMD_REGISTER_DRIVER(rte_ixgbevf_driver);
+PMD_REGISTER_DRIVER(rte_ixgbe_driver, ixgbe);
+DRIVER_REGISTER_PCI_TABLE(ixgbe, pci_id_ixgbe_map);
+PMD_REGISTER_DRIVER(rte_ixgbevf_driver, ixgbevf);
+DRIVER_REGISTER_PCI_TABLE(ixgbevf, pci_id_ixgbevf_map);
diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index 9ed1491..aedf1e9 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -5865,4 +5865,5 @@ static struct rte_driver rte_mlx4_driver = {
 	.init = rte_mlx4_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_mlx4_driver)
+PMD_REGISTER_DRIVER(rte_mlx4_driveri, mlx4)
+DRIVER_REGISTER_PCI_TABLE(mlx4, mlx4_pci_id_map);
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 041cfc3..5f2d6c8 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -672,4 +672,5 @@ static struct rte_driver rte_mlx5_driver = {
 	.init = rte_mlx5_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_mlx5_driver)
+PMD_REGISTER_DRIVER(rte_mlx5_driveri, mlx5)
+DRIVER_REGISTER_PCI_TABLE(mlx5, mlx5_pci_id_map);
diff --git a/drivers/net/mpipe/mpipe_tilegx.c b/drivers/net/mpipe/mpipe_tilegx.c
index 26e1424..4a2a905 100644
--- a/drivers/net/mpipe/mpipe_tilegx.c
+++ b/drivers/net/mpipe/mpipe_tilegx.c
@@ -1635,8 +1635,8 @@ static struct rte_driver pmd_mpipe_gbe_drv = {
 	.init = rte_pmd_mpipe_devinit,
 };
 
-PMD_REGISTER_DRIVER(pmd_mpipe_xgbe_drv);
-PMD_REGISTER_DRIVER(pmd_mpipe_gbe_drv);
+PMD_REGISTER_DRIVER(pmd_mpipe_xgbe_drv, mpipe_xgbe);
+PMD_REGISTER_DRIVER(pmd_mpipe_gbe_drv, mpipe_gbe);
 
 static void __attribute__((constructor, used))
 mpipe_init_contexts(void)
diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
index ea5a2a3..78394a1 100644
--- a/drivers/net/nfp/nfp_net.c
+++ b/drivers/net/nfp/nfp_net.c
@@ -2490,7 +2490,8 @@ static struct rte_driver rte_nfp_net_driver = {
 	.init = nfp_net_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_nfp_net_driver);
+PMD_REGISTER_DRIVER(rte_nfp_net_driver, nfp);
+DRIVER_REGISTER_PCI_TABLE(nfp, pci_id_nfp_net_map);
 
 /*
  * Local variables:
diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index 5e8e203..88eb582 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -689,4 +689,5 @@ static struct rte_driver pmd_null_drv = {
 	.uninit = rte_pmd_null_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_null_drv);
+PMD_REGISTER_DRIVER(pmd_null_drv, eth_null);
+DRIVER_REGISTER_PARAM_STRING(eth_null, "size=<int> copy=<int>");
diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
index c98e234..bb72af1 100644
--- a/drivers/net/pcap/rte_eth_pcap.c
+++ b/drivers/net/pcap/rte_eth_pcap.c
@@ -1090,4 +1090,6 @@ static struct rte_driver pmd_pcap_drv = {
 	.uninit = rte_pmd_pcap_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_pcap_drv);
+PMD_REGISTER_DRIVER(pmd_pcap_drv, pcap);
+DRIVER_REGISTER_PARAM_STRING(pcap, "rx_pcap=<string> tx_pcap=<string> "
+"rx_iface=<ifc> tx_iface=<ifc> iface=<ifc>");
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index 1273fd3..187ccf1 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -1102,5 +1102,7 @@ static struct rte_driver rte_qede_driver = {
 	.init = rte_qedevf_pmd_init
 };
 
-PMD_REGISTER_DRIVER(rte_qede_driver);
-PMD_REGISTER_DRIVER(rte_qedevf_driver);
+PMD_REGISTER_DRIVER(rte_qede_driver, qede);
+DRIVER_REGISTER_PCI_TABLE(qede, pci_id_qede_map);
+PMD_REGISTER_DRIVER(rte_qedevf_driver, qedevf);
+DRIVER_REGISTER_PCI_TABLE(qedevf, pci_id_qedevf_map);
diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index b1783c3..f2f5217 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -630,4 +630,5 @@ static struct rte_driver pmd_ring_drv = {
 	.uninit = rte_pmd_ring_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_ring_drv);
+PMD_REGISTER_DRIVER(pmd_ring_drv, eth_ring);
+DRIVER_REGISTER_PARAM_STRING(eth_ring, "nodeaction=[attach|detach]");
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 78c43b0..abbc907 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1601,4 +1601,5 @@ static struct rte_driver rte_szedata2_driver = {
 	.uninit = rte_szedata2_uninit,
 };
 
-PMD_REGISTER_DRIVER(rte_szedata2_driver);
+PMD_REGISTER_DRIVER(rte_szedata2_driver, rte_szedata2_pmd);
+DRIVER_REGISTER_PCI_TABLE(rte_szedata2_pmd, rte_szedata2_pci_id_table);
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index 310cbef..d321c3e 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -924,4 +924,5 @@ static struct rte_driver pmd_vhost_drv = {
 	.uninit = rte_pmd_vhost_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_vhost_drv);
+PMD_REGISTER_DRIVER(pmd_vhost_drv, eth_vhost);
+DRIVER_REGISTER_PARAM_STRING(eth_vhost, "iface=<ifc> queues=<int>");
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index c3fb628..24d387f 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -1459,4 +1459,5 @@ static struct rte_driver rte_virtio_driver = {
 	.init = rte_virtio_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_virtio_driver);
+PMD_REGISTER_DRIVER(rte_virtio_driver, virtio_net);
+DRIVER_REGISTER_PCI_TABLE(virtio_net, pci_id_virtio_map);
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index 29b469c..8da4449 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -954,4 +954,5 @@ static struct rte_driver rte_vmxnet3_driver = {
 	.init = rte_vmxnet3_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_vmxnet3_driver);
+PMD_REGISTER_DRIVER(rte_vmxnet3_driver, vmxnet3);
+DRIVER_REGISTER_PCI_TABLE(vmxnet3, pci_id_vmxnet3_map);
diff --git a/drivers/net/xenvirt/rte_eth_xenvirt.c b/drivers/net/xenvirt/rte_eth_xenvirt.c
index 1adeb5b..81f8af9 100644
--- a/drivers/net/xenvirt/rte_eth_xenvirt.c
+++ b/drivers/net/xenvirt/rte_eth_xenvirt.c
@@ -763,4 +763,4 @@ static struct rte_driver pmd_xenvirt_drv = {
 	.uninit = rte_pmd_xenvirt_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_xenvirt_drv);
+PMD_REGISTER_DRIVER(pmd_xenvirt_drv, xenvirt);
diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
index f1b5507..e6f0d4c 100644
--- a/lib/librte_eal/common/include/rte_dev.h
+++ b/lib/librte_eal/common/include/rte_dev.h
@@ -48,7 +48,7 @@ extern "C" {
 
 #include <stdio.h>
 #include <sys/queue.h>
-
+#include <rte_pci.h>
 #include <rte_log.h>
 
 __attribute__((format(printf, 2, 0)))
@@ -178,12 +178,30 @@ int rte_eal_vdev_init(const char *name, const char *args);
  */
 int rte_eal_vdev_uninit(const char *name);
 
-#define PMD_REGISTER_DRIVER(d)\
-void devinitfn_ ##d(void);\
-void __attribute__((constructor, used)) devinitfn_ ##d(void)\
+#define DRIVER_EXPORT_NAME_ARRAY(n, idx) n##idx[]
+
+#define DRIVER_EXPORT_NAME(name, idx) \
+static const char DRIVER_EXPORT_NAME_ARRAY(this_pmd_name, idx) \
+__attribute__((used)) = RTE_STR(name)
+
+#define PMD_REGISTER_DRIVER(drv, nm)\
+void devinitfn_ ##drv(void);\
+void __attribute__((constructor, used)) devinitfn_ ##drv(void)\
 {\
-	rte_eal_driver_register(&d);\
-}
+	(drv).name = RTE_STR(nm);\
+	rte_eal_driver_register(&drv);\
+} \
+DRIVER_EXPORT_NAME(nm, __COUNTER__)
+
+#define DRV_EXP_TAG(name, tag) __##name##_##tag
+
+#define DRIVER_REGISTER_PCI_TABLE(name, table) \
+static const char DRV_EXP_TAG(name, pci_tbl_export)[] __attribute__((used)) = \
+RTE_STR(table)
+
+#define DRIVER_REGISTER_PARAM_STRING(name, str) \
+static const char DRV_EXP_TAG(name, param_string_export)[] \
+__attribute__((used)) = str
 
 #ifdef __cplusplus
 }
-- 
2.5.5

[PATCHv7 3/6] eal: Add an export symbol to expose the autoload path to external tools

[Neil Horman]

Export a symbol containing the string:
DPDK_PLUGIN_PATH="$(CONFIG_RTE_EAL_PMD_PATH)"

Where the latter half of the string is set at build time to a location from
which autoloaded DSO's will be found.  This string is used by pmdinfo in
'plugin' mode, whereby a user can specify a dpdk installation directory (or
static binary), and scan the associated path (if found) for pmd DSO's and
report on their hardware support.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 lib/librte_eal/common/eal_common_options.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c
index 3efc90f..0a594d7 100644
--- a/lib/librte_eal/common/eal_common_options.c
+++ b/lib/librte_eal/common/eal_common_options.c
@@ -115,6 +115,15 @@ TAILQ_HEAD_INITIALIZER(solib_list);
 /* Default path of external loadable drivers */
 static const char *default_solib_dir = RTE_EAL_PMD_PATH;
 
+/*
+ * Stringified version of solib path used by pmdinfo.py
+ * Note: PLEASE DO NOT ALTER THIS without making a corresponding
+ * change to tools/pmdinfo.py
+ */
+static const char dpdk_solib_path[] __attribute__((used)) =
+"DPDK_PLUGIN_PATH=" RTE_EAL_PMD_PATH;
+
+
 static int master_lcore_parsed;
 static int mem_parsed;
 
-- 
2.5.5

[PATCHv7 4/6] Makefile: Do post processing on objects that register a driver

[Neil Horman]

Modify the compilation makefile to identify C files that export PMD
information, and use that to trigger execution of the pmdinfo binary.  If
the execution of pmdinfo is successful, compile the output C file to an
object, and use the linker to do relocatable linking on the resultant
object file into the parent object that it came from.  This effectively
just adds the json string into the string table of the object that defines
the PMD to the outside world.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 mk/internal/rte.compile-pre.mk | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/mk/internal/rte.compile-pre.mk b/mk/internal/rte.compile-pre.mk
index b9bff4a..5632d6e 100644
--- a/mk/internal/rte.compile-pre.mk
+++ b/mk/internal/rte.compile-pre.mk
@@ -88,10 +88,24 @@ C_TO_O_CMD = 'cmd_$@ = $(C_TO_O_STR)'
 C_TO_O_DO = @set -e; \
 	echo $(C_TO_O_DISP); \
 	$(C_TO_O) && \
+	sh -c "grep -q \"PMD_REGISTER_DRIVER(.*)\" $<; \
+	if [ \$$? -eq 0 ]; then \
+		echo \"  PMDINFOGEN\" $@; \
+		OBJF=`readlink -f $@`; \
+		${RTE_OUTPUT}/app/pmdinfogen \$$OBJF \$$OBJF.pmd.c; \
+		if [ \$$? -eq 0 ]; \
+		then \
+			echo \"  PMDINFOBUILD\" $@; \
+			$(CC) $(CFLAGS) -c -o \$$OBJF.pmd.o \$$OBJF.pmd.c; \
+			$(CROSS)ld $(LDFLAGS) -r -o \$$OBJF.o \$$OBJF.pmd.o \$$OBJF; \
+			mv -f \$$OBJF.o \$$OBJF; \
+		fi; \
+	fi;" && \
 	echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \
 	sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \
 	rm -f $(call obj2dep,$(@)).tmp
 
+
 # return an empty string if string are equal
 compare = $(strip $(subst $(1),,$(2)) $(subst $(2),,$(1)))
 
-- 
2.5.5

[PATCHv7 5/6] pmdinfo.py: Add tool to query binaries for hw and other support information

[Neil Horman]

This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
and, if found parses the remainder of the string as a json encoded string,
outputting the results in either a human readable or raw, script parseable
format

Note that, in the case of dynamically linked applications, pmdinfo.py will
scan for implicitly linked PMDs by searching the specified binaries
.dynamic section for DT_NEEDED entries that contain the substring
librte_pmd.  The DT_RUNPATH, LD_LIBRARY_PATH, /usr/lib and /lib are
searched for these libraries, in that order

If a file is specified with no path, it is assumed to be a PMD DSO, and the
LD_LIBRARY_PATH, /usr/lib[64]/ and /lib[64] is searched for it

Currently the tool can output data in 3 formats:

a) raw, suitable for scripting, where the raw JSON strings are dumped out
b) table format (default) where hex pci ids are dumped in a table format
c) pretty, where a user supplied pci.ids file is used to print out vendor
and device strings

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 mk/rte.sdkinstall.mk |   2 +
 tools/pmdinfo.py     | 629 +++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 631 insertions(+)
 create mode 100755 tools/pmdinfo.py

diff --git a/mk/rte.sdkinstall.mk b/mk/rte.sdkinstall.mk
index 68e56b6..dc36df5 100644
--- a/mk/rte.sdkinstall.mk
+++ b/mk/rte.sdkinstall.mk
@@ -126,6 +126,8 @@ install-runtime:
 	$(Q)$(call rte_mkdir,      $(DESTDIR)$(sbindir))
 	$(Q)$(call rte_symlink,    $(DESTDIR)$(datadir)/tools/dpdk_nic_bind.py, \
 	                           $(DESTDIR)$(sbindir)/dpdk_nic_bind)
+	$(Q)$(call rte_symlink,    $(DESTDIR)$(datadir)/tools/pmdinfo.py, \
+				   $(DESTDIR)$(bindir)/dpdk-pmdinfo)
 
 install-kmod:
 ifneq ($(wildcard $O/kmod/*),)
diff --git a/tools/pmdinfo.py b/tools/pmdinfo.py
new file mode 100755
index 0000000..e531154
--- /dev/null
+++ b/tools/pmdinfo.py
@@ -0,0 +1,629 @@
+#!/usr/bin/python
+# -------------------------------------------------------------------------
+# scripts/pmdinfo.py
+#
+# Utility to dump PMD_INFO_STRING support from an object file
+#
+# -------------------------------------------------------------------------
+import os
+import sys
+from optparse import OptionParser
+import string
+import json
+
+# For running from development directory. It should take precedence over the
+# installed pyelftools.
+sys.path.insert(0, '.')
+
+
+from elftools import __version__
+from elftools.common.exceptions import ELFError
+from elftools.common.py3compat import (
+    ifilter, byte2int, bytes2str, itervalues, str2bytes)
+from elftools.elf.elffile import ELFFile
+from elftools.elf.dynamic import DynamicSection, DynamicSegment
+from elftools.elf.enums import ENUM_D_TAG
+from elftools.elf.segments import InterpSegment
+from elftools.elf.sections import SymbolTableSection
+from elftools.elf.gnuversions import (
+    GNUVerSymSection, GNUVerDefSection,
+    GNUVerNeedSection,
+)
+from elftools.elf.relocation import RelocationSection
+from elftools.elf.descriptions import (
+    describe_ei_class, describe_ei_data, describe_ei_version,
+    describe_ei_osabi, describe_e_type, describe_e_machine,
+    describe_e_version_numeric, describe_p_type, describe_p_flags,
+    describe_sh_type, describe_sh_flags,
+    describe_symbol_type, describe_symbol_bind, describe_symbol_visibility,
+    describe_symbol_shndx, describe_reloc_type, describe_dyn_tag,
+    describe_ver_flags,
+)
+from elftools.elf.constants import E_FLAGS
+from elftools.dwarf.dwarfinfo import DWARFInfo
+from elftools.dwarf.descriptions import (
+    describe_reg_name, describe_attr_value, set_global_machine_arch,
+    describe_CFI_instructions, describe_CFI_register_rule,
+    describe_CFI_CFA_rule,
+)
+from elftools.dwarf.constants import (
+    DW_LNS_copy, DW_LNS_set_file, DW_LNE_define_file)
+from elftools.dwarf.callframe import CIE, FDE
+
+raw_output = False
+pcidb = None
+
+# ===========================================
+
+
+class Vendor:
+    """
+    Class for vendors. This is the top level class
+    for the devices belong to a specific vendor.
+    self.devices is the device dictionary
+    subdevices are in each device.
+    """
+
+    def __init__(self, vendorStr):
+        """
+        Class initializes with the raw line from pci.ids
+        Parsing takes place inside __init__
+        """
+        self.ID = vendorStr.split()[0]
+        self.name = vendorStr.replace("%s " % self.ID, "").rstrip()
+        self.devices = {}
+
+    def addDevice(self, deviceStr):
+        """
+        Adds a device to self.devices
+        takes the raw line from pci.ids
+        """
+        s = deviceStr.strip()
+        devID = s.split()[0]
+        if devID in self.devices:
+            pass
+        else:
+            self.devices[devID] = Device(deviceStr)
+
+    def report(self):
+        print self.ID, self.name
+        for id, dev in self.devices.items():
+            dev.report()
+
+    def find_device(self, devid):
+        # convert to a hex string and remove 0x
+        devid = hex(devid)[2:]
+        try:
+            return self.devices[devid]
+        except:
+            return Device("%s  Unknown Device" % devid)
+
+
+class Device:
+
+    def __init__(self, deviceStr):
+        """
+        Class for each device.
+        Each vendor has its own devices dictionary.
+        """
+        s = deviceStr.strip()
+        self.ID = s.split()[0]
+        self.name = s.replace("%s  " % self.ID, "")
+        self.subdevices = {}
+
+    def report(self):
+        print "\t%s\t%s" % (self.ID, self.name)
+        for subID, subdev in self.subdevices.items():
+            subdev.report()
+
+    def addSubDevice(self, subDeviceStr):
+        """
+        Adds a subvendor, subdevice to device.
+        Uses raw line from pci.ids
+        """
+        s = subDeviceStr.strip()
+        spl = s.split()
+        subVendorID = spl[0]
+        subDeviceID = spl[1]
+        subDeviceName = s.split("  ")[-1]
+        devID = "%s:%s" % (subVendorID, subDeviceID)
+        self.subdevices[devID] = SubDevice(
+            subVendorID, subDeviceID, subDeviceName)
+
+    def find_subid(self, subven, subdev):
+        subven = hex(subven)[2:]
+        subdev = hex(subdev)[2:]
+        devid = "%s:%s" % (subven, subdev)
+
+        try:
+            return self.subdevices[devid]
+        except:
+            if (subven == "ffff" and subdev == "ffff"):
+                return SubDevice("ffff", "ffff", "(All Subdevices)")
+            else:
+                return SubDevice(subven, subdev, "(Unknown Subdevice)")
+
+
+class SubDevice:
+    """
+    Class for subdevices.
+    """
+
+    def __init__(self, vendor, device, name):
+        """
+        Class initializes with vendorid, deviceid and name
+        """
+        self.vendorID = vendor
+        self.deviceID = device
+        self.name = name
+
+    def report(self):
+        print "\t\t%s\t%s\t%s" % (self.vendorID, self.deviceID, self.name)
+
+
+class PCIIds:
+    """
+    Top class for all pci.ids entries.
+    All queries will be asked to this class.
+    PCIIds.vendors["0e11"].devices["0046"].\
+    subdevices["0e11:4091"].name  =  "Smart Array 6i"
+    """
+
+    def __init__(self, filename):
+        """
+        Prepares the directories.
+        Checks local data file.
+        Tries to load from local, if not found, downloads from web
+        """
+        self.version = ""
+        self.date = ""
+        self.vendors = {}
+        self.contents = None
+        self.readLocal(filename)
+        self.parse()
+
+    def reportVendors(self):
+        """Reports the vendors
+        """
+        for vid, v in self.vendors.items():
+            print v.ID, v.name
+
+    def report(self, vendor=None):
+        """
+        Reports everything for all vendors or a specific vendor
+        PCIIds.report()  reports everything
+        PCIIDs.report("0e11") reports only "Compaq Computer Corporation"
+        """
+        if vendor is not None:
+            self.vendors[vendor].report()
+        else:
+            for vID, v in self.vendors.items():
+                v.report()
+
+    def find_vendor(self, vid):
+        # convert vid to a hex string and remove the 0x
+        vid = hex(vid)[2:]
+
+        try:
+            return self.vendors[vid]
+        except:
+            return Vendor("%s Unknown Vendor" % (vid))
+
+    def findDate(self, content):
+        for l in content:
+            if l.find("Date:") > -1:
+                return l.split()[-2].replace("-", "")
+        return None
+
+    def parse(self):
+        if len(self.contents) < 1:
+            print "data/%s-pci.ids not found" % self.date
+        else:
+            vendorID = ""
+            deviceID = ""
+            for l in self.contents:
+                if l[0] == "#":
+                    continue
+                elif len(l.strip()) == 0:
+                    continue
+                else:
+                    if l.find("\t\t") == 0:
+                        self.vendors[vendorID].devices[
+                            deviceID].addSubDevice(l)
+                    elif l.find("\t") == 0:
+                        deviceID = l.strip().split()[0]
+                        self.vendors[vendorID].addDevice(l)
+                    else:
+                        vendorID = l.split()[0]
+                        self.vendors[vendorID] = Vendor(l)
+
+    def readLocal(self, filename):
+        """
+        Reads the local file
+        """
+        self.contents = open(filename).readlines()
+        self.date = self.findDate(self.contents)
+
+    def loadLocal(self):
+        """
+        Loads database from local. If there is no file,
+        it creates a new one from web
+        """
+        self.date = idsfile[0].split("/")[1].split("-")[0]
+        self.readLocal()
+
+
+# =======================================
+
+def search_file(filename, search_path):
+    """ Given a search path, find file with requested name """
+    for path in string.split(search_path, ":"):
+        candidate = os.path.join(path, filename)
+        if os.path.exists(candidate):
+            return os.path.abspath(candidate)
+    return None
+
+
+class ReadElf(object):
+    """ display_* methods are used to emit output into the output stream
+    """
+
+    def __init__(self, file, output):
+        """ file:
+                stream object with the ELF file to read
+
+            output:
+                output stream to write to
+        """
+        self.elffile = ELFFile(file)
+        self.output = output
+
+        # Lazily initialized if a debug dump is requested
+        self._dwarfinfo = None
+
+        self._versioninfo = None
+
+    def _section_from_spec(self, spec):
+        """ Retrieve a section given a "spec" (either number or name).
+            Return None if no such section exists in the file.
+        """
+        try:
+            num = int(spec)
+            if num < self.elffile.num_sections():
+                return self.elffile.get_section(num)
+            else:
+                return None
+        except ValueError:
+            # Not a number. Must be a name then
+            return self.elffile.get_section_by_name(str2bytes(spec))
+
+    def pretty_print_pmdinfo(self, pmdinfo):
+        global pcidb
+
+        for i in pmdinfo["pci_ids"]:
+            vendor = pcidb.find_vendor(i[0])
+            device = vendor.find_device(i[1])
+            subdev = device.find_subid(i[2], i[3])
+            print("%s (%s) : %s (%s) %s" %
+                  (vendor.name, vendor.ID, device.name,
+                   device.ID, subdev.name))
+
+    def parse_pmd_info_string(self, mystring):
+        global raw_output
+        global pcidb
+
+        optional_pmd_info = [{'id': 'params', 'tag': 'PMD PARAMETERS'}]
+
+        i = mystring.index("=")
+        mystring = mystring[i + 2:]
+        pmdinfo = json.loads(mystring)
+
+        if raw_output:
+            print(pmdinfo)
+            return
+
+        print("PMD NAME: " + pmdinfo["name"])
+        for i in optional_pmd_info:
+            try:
+                print("%s: %s" % (i['tag'], pmdinfo[i['id']]))
+            except KeyError as e:
+                continue
+
+        if (len(pmdinfo["pci_ids"]) != 0):
+            print("PMD HW SUPPORT:")
+            if pcidb is not None:
+                self.pretty_print_pmdinfo(pmdinfo)
+            else:
+                print("VENDOR\t DEVICE\t SUBVENDOR\t SUBDEVICE")
+                for i in pmdinfo["pci_ids"]:
+                    print("0x%04x\t 0x%04x\t 0x%04x\t\t 0x%04x" %
+                          (i[0], i[1], i[2], i[3]))
+
+        print("")
+
+    def display_pmd_info_strings(self, section_spec):
+        """ Display a strings dump of a section. section_spec is either a
+            section number or a name.
+        """
+        section = self._section_from_spec(section_spec)
+        if section is None:
+            return
+
+        data = section.data()
+        dataptr = 0
+
+        while dataptr < len(data):
+            while (dataptr < len(data) and
+                    not (32 <= byte2int(data[dataptr]) <= 127)):
+                dataptr += 1
+
+            if dataptr >= len(data):
+                break
+
+            endptr = dataptr
+            while endptr < len(data) and byte2int(data[endptr]) != 0:
+                endptr += 1
+
+            mystring = bytes2str(data[dataptr:endptr])
+            rc = mystring.find("PMD_INFO_STRING")
+            if (rc != -1):
+                self.parse_pmd_info_string(mystring)
+
+            dataptr = endptr
+
+    def find_librte_eal(self, section):
+        for tag in section.iter_tags():
+            if tag.entry.d_tag == 'DT_NEEDED':
+                if "librte_eal" in tag.needed:
+                    return tag.needed
+        return None
+
+    def search_for_autoload_path(self):
+        scanelf = self
+        scanfile = None
+        library = None
+
+        section = self._section_from_spec(".dynamic")
+        try:
+            eallib = self.find_librte_eal(section)
+            if eallib is not None:
+                ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+                if ldlibpath is None:
+                    ldlibpath = ""
+                dtr = self.get_dt_runpath(section)
+                library = search_file(eallib,
+                                      dtr + ":" + ldlibpath +
+                                      ":/usr/lib64:/lib64:/usr/lib:/lib")
+                if library is None:
+                    return (None, None)
+                if raw_output is False:
+                    print("Scanning for autoload path in %s" % library)
+                scanfile = open(library, 'rb')
+                scanelf = ReadElf(scanfile, sys.stdout)
+        except AttributeError:
+            # Not a dynamic binary
+            pass
+        except ELFError:
+            scanfile.close()
+            return (None, None)
+
+        section = scanelf._section_from_spec(".rodata")
+        if section is None:
+            if scanfile is not None:
+                scanfile.close()
+            return (None, None)
+
+        data = section.data()
+        dataptr = 0
+
+        while dataptr < len(data):
+            while (dataptr < len(data) and
+                    not (32 <= byte2int(data[dataptr]) <= 127)):
+                dataptr += 1
+
+            if dataptr >= len(data):
+                break
+
+            endptr = dataptr
+            while endptr < len(data) and byte2int(data[endptr]) != 0:
+                endptr += 1
+
+            mystring = bytes2str(data[dataptr:endptr])
+            rc = mystring.find("DPDK_PLUGIN_PATH")
+            if (rc != -1):
+                rc = mystring.find("=")
+                return (mystring[rc + 1:], library)
+
+            dataptr = endptr
+        if scanfile is not None:
+            scanfile.close()
+        return (None, None)
+
+    def get_dt_runpath(self, dynsec):
+        for tag in dynsec.iter_tags():
+            if tag.entry.d_tag == 'DT_RUNPATH':
+                return tag.runpath
+        return ""
+
+    def process_dt_needed_entries(self):
+        """ Look to see if there are any DT_NEEDED entries in the binary
+            And process those if there are
+        """
+        global raw_output
+        runpath = ""
+        ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+        if ldlibpath is None:
+            ldlibpath = ""
+
+        dynsec = self._section_from_spec(".dynamic")
+        try:
+            runpath = self.get_dt_runpath(dynsec)
+        except AttributeError:
+            # dynsec is None, just return
+            return
+
+        for tag in dynsec.iter_tags():
+            if tag.entry.d_tag == 'DT_NEEDED':
+                rc = tag.needed.find("librte_pmd")
+                if (rc != -1):
+                    library = search_file(tag.needed,
+                                          runpath + ":" + ldlibpath +
+                                          ":/usr/lib64:/lib64:/usr/lib:/lib")
+                    if library is not None:
+                        if raw_output is False:
+                            print("Scanning %s for pmd information" % library)
+                        with open(library, 'rb') as file:
+                            try:
+                                libelf = ReadElf(file, sys.stdout)
+                            except ELFError as e:
+                                print("%s is no an ELF file" % library)
+                                continue
+                            libelf.process_dt_needed_entries()
+                            libelf.display_pmd_info_strings(".rodata")
+                            file.close()
+
+
+def scan_autoload_path(autoload_path):
+    global raw_output
+
+    if os.path.exists(autoload_path) is False:
+        return
+
+    try:
+        dirs = os.listdir(autoload_path)
+    except OSError as e:
+        # Couldn't read the directory, give up
+        return
+
+    for d in dirs:
+        dpath = os.path.join(autoload_path, d)
+        if os.path.isdir(dpath):
+            scan_autoload_path(dpath)
+        if os.path.isfile(dpath):
+            try:
+                file = open(dpath, 'rb')
+                readelf = ReadElf(file, sys.stdout)
+            except ELFError as e:
+                # this is likely not an elf file, skip it
+                continue
+            except IOError as e:
+                # No permission to read the file, skip it
+                continue
+
+            if raw_output is False:
+                print("Hw Support for library %s" % d)
+            readelf.display_pmd_info_strings(".rodata")
+            file.close()
+
+
+def scan_for_autoload_pmds(dpdk_path):
+    """
+    search the specified application or path for a pmd autoload path
+    then scan said path for pmds and report hw support
+    """
+    global raw_output
+
+    if (os.path.isfile(dpdk_path) is False):
+        if raw_output is False:
+            print("Must specify a file name")
+        return
+
+    file = open(dpdk_path, 'rb')
+    try:
+        readelf = ReadElf(file, sys.stdout)
+    except ElfError as e:
+        if raw_output is False:
+            print("Unable to parse %s" % file)
+        return
+
+    (autoload_path, scannedfile) = readelf.search_for_autoload_path()
+    if (autoload_path is None or autoload_path is ""):
+        if (raw_output is False):
+            print("No autoload path configured in %s" % dpdk_path)
+        return
+    if (raw_output is False):
+        if (scannedfile is None):
+            scannedfile = dpdk_path
+        print("Found autoload path %s in %s" % (autoload_path, scannedfile))
+
+    file.close()
+    if (raw_output is False):
+        print("Discovered Autoload HW Support:")
+    scan_autoload_path(autoload_path)
+    return
+
+
+def main(stream=None):
+    global raw_output
+    global pcidb
+
+    optparser = OptionParser(
+        usage='usage: %prog [-hrtp] [-d <pci id file] <elf-file>',
+        description="Dump pmd hardware support info",
+        add_help_option=True,
+        prog='pmdinfo.py')
+    optparser.add_option('-r', '--raw',
+                         action='store_true', dest='raw_output',
+                         help='Dump raw json strings')
+    optparser.add_option("-d", "--pcidb", dest="pcifile",
+                         help="specify a pci database "
+                              "to get vendor names from",
+                         default="/usr/share/hwdata/pci.ids", metavar="FILE")
+    optparser.add_option("-t", "--table", dest="tblout",
+                         help="output information on hw support as a hex table",
+                         action='store_true')
+    optparser.add_option("-p", "--plugindir", dest="pdir",
+                         help="scan dpdk for autoload plugins",
+                         action='store_true')
+
+    options, args = optparser.parse_args()
+
+    if options.raw_output:
+        raw_output = True
+
+    if options.pcifile:
+        pcidb = PCIIds(options.pcifile)
+        if pcidb is None:
+            print("Pci DB file not found")
+            exit(1)
+
+    if options.tblout:
+        options.pcifile = None
+        pcidb = None
+
+    if (len(args) == 0):
+        optparser.print_usage()
+        exit(1)
+
+    if options.pdir is True:
+        exit(scan_for_autoload_pmds(args[0]))
+
+    ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+    if (ldlibpath is None):
+        ldlibpath = ""
+
+    if (os.path.exists(args[0]) is True):
+        myelffile = args[0]
+    else:
+        myelffile = search_file(
+            args[0], ldlibpath + ":/usr/lib64:/lib64:/usr/lib:/lib")
+
+    if (myelffile is None):
+        print("File not found")
+        sys.exit(1)
+
+    with open(myelffile, 'rb') as file:
+        try:
+            readelf = ReadElf(file, sys.stdout)
+            readelf.process_dt_needed_entries()
+            readelf.display_pmd_info_strings(".rodata")
+            sys.exit(0)
+
+        except ELFError as ex:
+            sys.stderr.write('ELF error: %s\n' % ex)
+            sys.exit(1)
+
+
+# -------------------------------------------------------------------------
+if __name__ == '__main__':
+    main()
-- 
2.5.5

[PATCHv7 6/6] doc: Add prog_guide section documenting pmdinfo script

[Neil Horman]

Information on pmdinfogen may be useful to 3rd party driver developers.
Include documentation on what it does

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 doc/guides/prog_guide/dev_kit_build_system.rst | 43 ++++++++++++++++++++++++--
 1 file changed, 41 insertions(+), 2 deletions(-)

diff --git a/doc/guides/prog_guide/dev_kit_build_system.rst b/doc/guides/prog_guide/dev_kit_build_system.rst
index 3e89eae..1dc1388 100644
--- a/doc/guides/prog_guide/dev_kit_build_system.rst
+++ b/doc/guides/prog_guide/dev_kit_build_system.rst
@@ -70,7 +70,7 @@ Each build directory contains include files, libraries, and applications:
     ...
     ~/DEV/DPDK$ ls i686-native-linuxapp-gcc
 
-    app build hostapp include kmod lib Makefile
+    app build buildtools include kmod lib Makefile
 
 
     ~/DEV/DPDK$ ls i686-native-linuxapp-gcc/app/
@@ -264,7 +264,7 @@ These Makefiles generate a binary application.
 
 *   rte.extapp.mk: External application
 
-*   rte.hostapp.mk: Host application in the development kit framework
+*   rte.hostapp.mk: prerequisite tool to build dpdk
 
 Library
 ^^^^^^^
@@ -304,6 +304,45 @@ Misc
 
 *   rte.subdir.mk: Build several directories in the development kit framework.
 
+.. _Internally_Generated_Build_Tools:
+
+Internally Generated Build Tools
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+``app/pmdinfogen``
+
+
+``pmdinfogen`` scans an object (.o) file for various well known symbol names.  These
+well known symbol names are defined by various macros and used to export
+important information about hardware support and usage for pmd files.  For
+instance the macro:
+
+.. code-block:: c
+
+    PMD_REGISTER_DRIVER(drv, name)
+
+
+Creates the following symbol:
+
+.. code-block:: c
+
+   static char this_pmd_name0[] __attribute__((used)) = "<name>";
+
+
+Which pmdinfogen scans for.  Using this information other relevant bits of data
+can be exported from the object file and used to produce a hardware support
+description, that pmdinfogen then encodes into a json formatted string in the
+following format:
+
+.. code-block:: C
+
+   static char <name_pmd_string>="PMD_INFO_STRING=\"{'name' : '<name>', ...}\"";
+
+
+These strings can then be searched for by external tools to determine the
+hardware support of a given library or application.
+
+
 .. _Useful_Variables_Provided_by_the_Build_System:
 
 Useful Variables Provided by the Build System
-- 
2.5.5

Re: [PATCHv7 6/6] doc: Add prog_guide section documenting pmdinfo script

[Mcnamara, John]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
> Sent: Thursday, June 9, 2016 6:47 PM
> To: dev@dpdk.org
> Cc: Neil Horman <nhorman@tuxdriver.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; Thomas Monjalon <thomas.monjalon@6wind.com>;
> Stephen Hemminger <stephen@networkplumber.org>; Panu Matilainen
> <pmatilai@redhat.com>
> Subject: [dpdk-dev] [PATCHv7 6/6] doc: Add prog_guide section documenting
> pmdinfo script
> 
> Information on pmdinfogen may be useful to 3rd party driver developers.
> Include documentation on what it does
> 
> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> CC: Bruce Richardson <bruce.richardson@intel.com>
> CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> CC: Stephen Hemminger <stephen@networkplumber.org>
> CC: Panu Matilainen <pmatilai@redhat.com>

Acked-by: John McNamara <john.mcnamara@intel.com>

Re: [PATCHv7 1/6] pmdinfogen: Add buildtools and pmdinfogen utility

[Panu Matilainen]

On 06/09/2016 08:46 PM, Neil Horman wrote:
> pmdinfogen is a tool used to parse object files and build json strings for
> use in later determining hardware support in a dso or application binary.
> pmdinfo looks for the non-exported symbol names this_pmd_name<n> and
> this_pmd_tbl<n> (where n is a integer counter).  It records the name of
> each of these tuples, using the later to find the symbolic name of the
> pci_table for physical devices that the object supports.  With this
> information, it outputs a C file with a single line of the form:
>
> static char *<pmd_name>_driver_info[] __attribute__((used)) = " \
> 	PMD_DRIVER_INFO=<json string>";
>
> Where <pmd_name> is the arbitrary name of the pmd, and <json_string> is the
> json encoded string that hold relevant pmd information, including the pmd
> name, type and optional array of pci device/vendor ids that the driver
> supports.
>
> This c file is suitable for compiling to object code, then relocatably
> linking into the parent file from which the C was generated.  This creates
> an entry in the string table of the object that can inform a later tool
> about hardware support.
>
> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> CC: Bruce Richardson <bruce.richardson@intel.com>
> CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> CC: Stephen Hemminger <stephen@networkplumber.org>
> CC: Panu Matilainen <pmatilai@redhat.com>
> ---

Unlike earlier versions, pmdinfogen ends up installed in bindir during 
"make install". Is that intentional, or just a side-effect from using 
rte.hostapp.mk? If its intentional it probably should be prefixed with 
dpdk_ like the other tools.

	- Panu -

Re: [PATCHv7 5/6] pmdinfo.py: Add tool to query binaries for hw and other support information

[Panu Matilainen]

On 06/09/2016 08:47 PM, Neil Horman wrote:
> This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
> and, if found parses the remainder of the string as a json encoded string,
> outputting the results in either a human readable or raw, script parseable
> format
>
> Note that, in the case of dynamically linked applications, pmdinfo.py will
> scan for implicitly linked PMDs by searching the specified binaries
> .dynamic section for DT_NEEDED entries that contain the substring
> librte_pmd.  The DT_RUNPATH, LD_LIBRARY_PATH, /usr/lib and /lib are
> searched for these libraries, in that order
>
> If a file is specified with no path, it is assumed to be a PMD DSO, and the
> LD_LIBRARY_PATH, /usr/lib[64]/ and /lib[64] is searched for it
>
> Currently the tool can output data in 3 formats:
>
> a) raw, suitable for scripting, where the raw JSON strings are dumped out
> b) table format (default) where hex pci ids are dumped in a table format
> c) pretty, where a user supplied pci.ids file is used to print out vendor
> and device strings
>
> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> CC: Bruce Richardson <bruce.richardson@intel.com>
> CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> CC: Stephen Hemminger <stephen@networkplumber.org>
> CC: Panu Matilainen <pmatilai@redhat.com>
> ---
>  mk/rte.sdkinstall.mk |   2 +
>  tools/pmdinfo.py     | 629 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 631 insertions(+)
>  create mode 100755 tools/pmdinfo.py
>
> diff --git a/mk/rte.sdkinstall.mk b/mk/rte.sdkinstall.mk
> index 68e56b6..dc36df5 100644
> --- a/mk/rte.sdkinstall.mk
> +++ b/mk/rte.sdkinstall.mk
> @@ -126,6 +126,8 @@ install-runtime:
>  	$(Q)$(call rte_mkdir,      $(DESTDIR)$(sbindir))
>  	$(Q)$(call rte_symlink,    $(DESTDIR)$(datadir)/tools/dpdk_nic_bind.py, \
>  	                           $(DESTDIR)$(sbindir)/dpdk_nic_bind)
> +	$(Q)$(call rte_symlink,    $(DESTDIR)$(datadir)/tools/pmdinfo.py, \
> +				   $(DESTDIR)$(bindir)/dpdk-pmdinfo)

The symlink should be with underscore instead of dash for consistency 
with all the other tools, ie dpdk_pmdinfo.

Neil, I already gave you an ack on the series as per the functionality, 
feel free to include that in any future versions of the patch series. 
Minor nits like these are ... well, minor nits from my POV at least.

	- Panu -

Re: [PATCHv7 1/6] pmdinfogen: Add buildtools and pmdinfogen utility

[Neil Horman]

On Thu, Jun 16, 2016 at 03:29:57PM +0300, Panu Matilainen wrote:
> On 06/09/2016 08:46 PM, Neil Horman wrote:
> > pmdinfogen is a tool used to parse object files and build json strings for
> > use in later determining hardware support in a dso or application binary.
> > pmdinfo looks for the non-exported symbol names this_pmd_name<n> and
> > this_pmd_tbl<n> (where n is a integer counter).  It records the name of
> > each of these tuples, using the later to find the symbolic name of the
> > pci_table for physical devices that the object supports.  With this
> > information, it outputs a C file with a single line of the form:
> > 
> > static char *<pmd_name>_driver_info[] __attribute__((used)) = " \
> > 	PMD_DRIVER_INFO=<json string>";
> > 
> > Where <pmd_name> is the arbitrary name of the pmd, and <json_string> is the
> > json encoded string that hold relevant pmd information, including the pmd
> > name, type and optional array of pci device/vendor ids that the driver
> > supports.
> > 
> > This c file is suitable for compiling to object code, then relocatably
> > linking into the parent file from which the C was generated.  This creates
> > an entry in the string table of the object that can inform a later tool
> > about hardware support.
> > 
> > Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> > CC: Bruce Richardson <bruce.richardson@intel.com>
> > CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> > CC: Stephen Hemminger <stephen@networkplumber.org>
> > CC: Panu Matilainen <pmatilai@redhat.com>
> > ---
> 
> Unlike earlier versions, pmdinfogen ends up installed in bindir during "make
> install". Is that intentional, or just a side-effect from using
> rte.hostapp.mk? If its intentional it probably should be prefixed with dpdk_
> like the other tools.
> 
Im not sure what the answer is here.  As you can see, Thomas and I argued at
length over which makefile to use, and I gave up, so I suppose you can call it
intentional.  Being in bindir makes a reasonable amount of sense I suppose, as
3rd party developers can use it during their independent driver development.
I'm not sure I agree with prefixing it though.  Given that the hostapp.mk file
installs everything there, and nothing that previously used that make file had a
dpdk_ prefix that I can tell, I'm not sure why this would.  pmdinfogen seems
like a pretty unique name, and I know of no other project that uses the term pmd
to describe anything.

Neil

> 	- Panu -
> 
> 

Re: [PATCHv7 1/6] pmdinfogen: Add buildtools and pmdinfogen utility

[Panu Matilainen]

On 06/16/2016 04:33 PM, Neil Horman wrote:
> On Thu, Jun 16, 2016 at 03:29:57PM +0300, Panu Matilainen wrote:
>> On 06/09/2016 08:46 PM, Neil Horman wrote:
>>> pmdinfogen is a tool used to parse object files and build json strings for
>>> use in later determining hardware support in a dso or application binary.
>>> pmdinfo looks for the non-exported symbol names this_pmd_name<n> and
>>> this_pmd_tbl<n> (where n is a integer counter).  It records the name of
>>> each of these tuples, using the later to find the symbolic name of the
>>> pci_table for physical devices that the object supports.  With this
>>> information, it outputs a C file with a single line of the form:
>>>
>>> static char *<pmd_name>_driver_info[] __attribute__((used)) = " \
>>> 	PMD_DRIVER_INFO=<json string>";
>>>
>>> Where <pmd_name> is the arbitrary name of the pmd, and <json_string> is the
>>> json encoded string that hold relevant pmd information, including the pmd
>>> name, type and optional array of pci device/vendor ids that the driver
>>> supports.
>>>
>>> This c file is suitable for compiling to object code, then relocatably
>>> linking into the parent file from which the C was generated.  This creates
>>> an entry in the string table of the object that can inform a later tool
>>> about hardware support.
>>>
>>> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
>>> CC: Bruce Richardson <bruce.richardson@intel.com>
>>> CC: Thomas Monjalon <thomas.monjalon@6wind.com>
>>> CC: Stephen Hemminger <stephen@networkplumber.org>
>>> CC: Panu Matilainen <pmatilai@redhat.com>
>>> ---
>>
>> Unlike earlier versions, pmdinfogen ends up installed in bindir during "make
>> install". Is that intentional, or just a side-effect from using
>> rte.hostapp.mk? If its intentional it probably should be prefixed with dpdk_
>> like the other tools.
>>
> Im not sure what the answer is here.  As you can see, Thomas and I argued at
> length over which makefile to use, and I gave up, so I suppose you can call it
> intentional.  Being in bindir makes a reasonable amount of sense I suppose, as
> 3rd party developers can use it during their independent driver development.

Right, it'd be useful for 3rd party driver developer, so lets consider 
it intentional :)

> I'm not sure I agree with prefixing it though.  Given that the hostapp.mk file
> installs everything there, and nothing that previously used that make file had a
> dpdk_ prefix that I can tell, I'm not sure why this would.  pmdinfogen seems
> like a pretty unique name, and I know of no other project that uses the term pmd
> to describe anything.

I agree about "pmd" being fairly unique as is, but if pmdinfo is dpdk_ 
prefixed then this should be too, or neither should be prefixed. I dont 
personally care which way, but it should be consistent.

	- Panu -

>
> Neil
>
>> 	- Panu -
>>
>>

Re: [PATCHv7 1/6] pmdinfogen: Add buildtools and pmdinfogen utility

[Neil Horman]

On Thu, Jun 16, 2016 at 05:06:49PM +0300, Panu Matilainen wrote:
> On 06/16/2016 04:33 PM, Neil Horman wrote:
> > On Thu, Jun 16, 2016 at 03:29:57PM +0300, Panu Matilainen wrote:
> > > On 06/09/2016 08:46 PM, Neil Horman wrote:
> > > > pmdinfogen is a tool used to parse object files and build json strings for
> > > > use in later determining hardware support in a dso or application binary.
> > > > pmdinfo looks for the non-exported symbol names this_pmd_name<n> and
> > > > this_pmd_tbl<n> (where n is a integer counter).  It records the name of
> > > > each of these tuples, using the later to find the symbolic name of the
> > > > pci_table for physical devices that the object supports.  With this
> > > > information, it outputs a C file with a single line of the form:
> > > > 
> > > > static char *<pmd_name>_driver_info[] __attribute__((used)) = " \
> > > > 	PMD_DRIVER_INFO=<json string>";
> > > > 
> > > > Where <pmd_name> is the arbitrary name of the pmd, and <json_string> is the
> > > > json encoded string that hold relevant pmd information, including the pmd
> > > > name, type and optional array of pci device/vendor ids that the driver
> > > > supports.
> > > > 
> > > > This c file is suitable for compiling to object code, then relocatably
> > > > linking into the parent file from which the C was generated.  This creates
> > > > an entry in the string table of the object that can inform a later tool
> > > > about hardware support.
> > > > 
> > > > Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> > > > CC: Bruce Richardson <bruce.richardson@intel.com>
> > > > CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> > > > CC: Stephen Hemminger <stephen@networkplumber.org>
> > > > CC: Panu Matilainen <pmatilai@redhat.com>
> > > > ---
> > > 
> > > Unlike earlier versions, pmdinfogen ends up installed in bindir during "make
> > > install". Is that intentional, or just a side-effect from using
> > > rte.hostapp.mk? If its intentional it probably should be prefixed with dpdk_
> > > like the other tools.
> > > 
> > Im not sure what the answer is here.  As you can see, Thomas and I argued at
> > length over which makefile to use, and I gave up, so I suppose you can call it
> > intentional.  Being in bindir makes a reasonable amount of sense I suppose, as
> > 3rd party developers can use it during their independent driver development.
> 
> Right, it'd be useful for 3rd party driver developer, so lets consider it
> intentional :)
> 
> > I'm not sure I agree with prefixing it though.  Given that the hostapp.mk file
> > installs everything there, and nothing that previously used that make file had a
> > dpdk_ prefix that I can tell, I'm not sure why this would.  pmdinfogen seems
> > like a pretty unique name, and I know of no other project that uses the term pmd
> > to describe anything.
> 
> I agree about "pmd" being fairly unique as is, but if pmdinfo is dpdk_
> prefixed then this should be too, or neither should be prefixed. I dont
> personally care which way, but it should be consistent.
> 
I caved on the pmdinfo issue, I guess I'll cave here too.

[PATCHv8 0/6] Implement pmd hardware support exports

[Neil Horman]

Hey all-
	So heres attempt number 2 at a method for exporting PMD hardware support
information.  As we discussed previously, the consensus seems to be that pmd
information should be:

1) Able to be interrogated on any ELF binary (application binary or individual
DSO)
2) Equally functional on statically linked applications or on DSO's
3) Resilient to symbol stripping
4) Script friendly
5) Show kernel dependencies
6) List driver options
7) Show driver name
8) Offer human readable output
9) Show DPDK version
10) Show driver version
11) Allow for expansion
12) Not place additional build environment dependencies on an application

I added item 12 myself, because I don't think its reasonable to require
applications to use a specific linker script to get hardware support information
(which precludes the use of a special .modinfo section like the kernel uses)

However, we still can use some tricks from the kernel to make this work.  In
this approach, what I've done is the following:

A) Modified the driver registration macro to also define a variable:
this_pmd_name<n>= "name"

Based on the unique name string pointed to by the above variable, we can
query for an arbitrary number of other symbols following the pattern:
<name>_<tag>

Where tag is some well known identifier for information we wish to export

B) Added a utility called pmdinfogen.  This utility is not meant for general use,
but rather is used by the dpdk build environment itself when compiling pmds.
for each object that uses the PMD_REGISTER_DRIVER macro, this utiity is run.  It
searches for the symbols in (A), and using those, extracts the hardware support
info, and module name from the object, using that to produce a new c file
containing a single variable in the following format:

static const char <name>[] __attribute__((used)) = "PMD_DRIVER_INFO=<json>";

The <name> is arbitrary, as its static and not referenced.  The relevant bit is
the string value assigned to it.  The <json> is a json encoded string of the
extracted hardware support information pmdinfo found for the corresponding
object.  This C file is suitable for compilation and relocatable linking back
into the parent object file.  The result of this operation is that the object
string table now contains a string that will not e removed by stripping, whos
leading text (PMD_DRIVER_INFO) can be easily searched for at any time weather
the symbol referring to it is stripped or not.

C) Added a utilty called pmdinfo.py.  This python script, searches the
string table of the .rodata section of any provided ELF file looking for the
PMD_DRIVER_INFO prefix on a string.  When found, it will interpret the remainder
of the string as json, and output the hardware support for that ELF file (if
any).


This approach ticks most of the above boxes:
1) Impervious to stripping
2) Works on static and dynamic binaries
3) Human and script friendly
4) Allows for expansion

Because of (4) the other items should be pretty easy to implement, as its just a
matter of modifying the macros to export the info, pmdinfo to encode it to json,
and pmd_hw_support.py to read it.  I'm not adding them now, as theres alot of
rote work to do to get some of them in place (e.g. DPDK has no current global
VERSION macro, drivers don't have a consistent version scheme, command line
strings have to be built for each driver, etc).  But once this is accepted,
those items can be done as time allows and should be fairly easy to implement.

Change Notes:
v2)
 * Made the export macros a bit easier to expand
 * Added a macro to optionally export command line strings
 * Renamed utilties to be more appropriate
   (pmdinfo -> pmdinfogen, pmd_hw_support.py -> pmdinfo.py)
 * Added search capabilities to pmdinfo.py so that we search for libraries
   linked using DT_NEEDINFO entries.  We search DT_RUNPATH, LD_LIBRARY_PATH,
   /usr/lib and /lib
 * Added serch abilities if the specified binary to pmdinfo isn't found, we
   search LD_LIBRARY_PATH, /usr/lib and /lib
 * Added an option to pmdinfo.py to pretty-print hardware support info using the
   pci.ids database

v3)
 * Made table mode printing the default mode
 * Added a default path to the pci.ids file
 * Modifed pmdinfo to use python rather than python3
 * Cleaned up some changelog entries
 * Added an export for RTE_EAL_PMD_PATH
 * Added a plugin option to pmdinfo to scan for autoloaded DSO's

v4)
 * Modified the operation of the -p option. As much as I don't like implying
that autoloaded pmds are guaranteed to be there at run time, I'm having a hard
time seeing how we can avoid specifying the application file to scan for the
autoload directory.  Without it we can't determine which library the user means
in a multiversion installation
 * Cleaned up the help text
 * Added a rule for an install target for pmdinfo
 * Guarded against some tracebacks in pmdinfo
 * Use DT_NEEDED entries to get versioned libraries in -p mode
 * Fixed traceback that occurs on lack of input arguments
 * Fixed some erroneous macro usage in drivers that aren't in the default build

v5)
 * Added a dpdk- prefix to pmdinfo symlink
 * Renamed build announcement to PMDINFOGEN/BUILD
 * Removed erroneous true statement from makefile
 * Removed duplicate rte.hostapp.mk makefile
 * Fixed some whitespace
 * Whitespace fixups
 * Fixed makefile if; then style
 * Renamed module string C file
 * Removed duplicate rte_pci_id definition
 * Clarified macro names
 * Removed PMD type attribute
 * Fixed tools usage for 32 bit arches
 * Removed some unused code
 * Added a few comments

v6)
 * Added some programming guide documentation
 * Reformatted python script with pep8

v7)
 * Fixed up copyright
 * Switched buildtool makefile to use hostapp.mk
 * Modified arch check to use RTE_ARCH_64
 * Modifed hostapp.mk to drop output in build/app
 * Additional testing on ppc64 to ensure big endian works
 * Modified TO_NATIVE macro to use rte_byteorder inlines 
   based on endianess of target ELF file
 * Ran checkpatch on commits
 * Fixed some typos in doc

v8)
 * Modified symlink for pmdinfo to use dpdk_
 * Added dpdk_pmdinfogen for pmdinfogen binary

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Panu Matilainen <pmatilai@redhat.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>

[PATCHv8 1/6] pmdinfogen: Add buildtools and pmdinfogen utility

[Neil Horman]

pmdinfogen is a tool used to parse object files and build json strings for
use in later determining hardware support in a dso or application binary.
pmdinfo looks for the non-exported symbol names this_pmd_name<n> and
this_pmd_tbl<n> (where n is a integer counter).  It records the name of
each of these tuples, using the later to find the symbolic name of the
pci_table for physical devices that the object supports.  With this
information, it outputs a C file with a single line of the form:

static char *<pmd_name>_driver_info[] __attribute__((used)) = " \
	PMD_DRIVER_INFO=<json string>";

Where <pmd_name> is the arbitrary name of the pmd, and <json_string> is the
json encoded string that hold relevant pmd information, including the pmd
name, type and optional array of pci device/vendor ids that the driver
supports.

This c file is suitable for compiling to object code, then relocatably
linking into the parent file from which the C was generated.  This creates
an entry in the string table of the object that can inform a later tool
about hardware support.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
Acked-by: Panu Matilainen <pmatilai@redhat.com>
---
 GNUmakefile                        |   2 +-
 buildtools/Makefile                |  36 +++
 buildtools/pmdinfogen/Makefile     |  49 +++++
 buildtools/pmdinfogen/pmdinfogen.c | 439 +++++++++++++++++++++++++++++++++++++
 buildtools/pmdinfogen/pmdinfogen.h | 120 ++++++++++
 mk/rte.hostapp.mk                  |   8 +-
 mk/rte.sdkbuild.mk                 |   3 +-
 mk/rte.sdkinstall.mk               |   2 +
 8 files changed, 653 insertions(+), 6 deletions(-)
 create mode 100644 buildtools/Makefile
 create mode 100644 buildtools/pmdinfogen/Makefile
 create mode 100644 buildtools/pmdinfogen/pmdinfogen.c
 create mode 100644 buildtools/pmdinfogen/pmdinfogen.h

diff --git a/GNUmakefile b/GNUmakefile
index b59e4b6..00fe0db 100644
--- a/GNUmakefile
+++ b/GNUmakefile
@@ -40,6 +40,6 @@ export RTE_SDK
 # directory list
 #
 
-ROOTDIRS-y := lib drivers app
+ROOTDIRS-y := buildtools lib drivers app
 
 include $(RTE_SDK)/mk/rte.sdkroot.mk
diff --git a/buildtools/Makefile b/buildtools/Makefile
new file mode 100644
index 0000000..35a42ff
--- /dev/null
+++ b/buildtools/Makefile
@@ -0,0 +1,36 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2016 Neil Horman. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+DIRS-y += pmdinfogen
+
+include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/buildtools/pmdinfogen/Makefile b/buildtools/pmdinfogen/Makefile
new file mode 100644
index 0000000..125901b
--- /dev/null
+++ b/buildtools/pmdinfogen/Makefile
@@ -0,0 +1,49 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2016 Neil Horman. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+#
+# library name
+#
+HOSTAPP = pmdinfogen
+
+#
+# all sources are stored in SRCS-y
+#
+SRCS-y += pmdinfogen.c
+
+HOST_EXTRA_CFLAGS += -g -I${RTE_OUTPUT}/include
+
+DEPDIRS-y += lib/librte_eal
+
+include $(RTE_SDK)/mk/rte.hostapp.mk
+
diff --git a/buildtools/pmdinfogen/pmdinfogen.c b/buildtools/pmdinfogen/pmdinfogen.c
new file mode 100644
index 0000000..0947dc6
--- /dev/null
+++ b/buildtools/pmdinfogen/pmdinfogen.c
@@ -0,0 +1,439 @@
+/* Postprocess pmd object files to export hw support
+ *
+ * Copyright 2016 Neil Horman <nhorman@tuxdriver.com>
+ * Based in part on modpost.c from the linux kernel
+ *
+ * This software may be used and distributed according to the terms
+ * of the GNU General Public License V2, incorporated herein by reference.
+ *
+ */
+
+#define _GNU_SOURCE
+#include <stdio.h>
+#include <ctype.h>
+#include <string.h>
+#include <limits.h>
+#include <stdbool.h>
+#include <errno.h>
+#include "pmdinfogen.h"
+
+#ifdef RTE_ARCH_64
+#define ADDR_SIZE 64
+#else
+#define ADDR_SIZE 32
+#endif
+
+
+static const char *sym_name(struct elf_info *elf, Elf_Sym *sym)
+{
+	if (sym)
+		return elf->strtab + sym->st_name;
+	else
+		return "(unknown)";
+}
+
+void *grab_file(const char *filename, unsigned long *size)
+{
+	struct stat st;
+	void *map = MAP_FAILED;
+	int fd;
+
+	fd = open(filename, O_RDONLY);
+	if (fd < 0)
+		return NULL;
+	if (fstat(fd, &st))
+		goto failed;
+
+	*size = st.st_size;
+	map = mmap(NULL, *size, PROT_READ|PROT_WRITE, MAP_PRIVATE, fd, 0);
+
+failed:
+	close(fd);
+	if (map == MAP_FAILED)
+		return NULL;
+	return map;
+}
+
+/**
+  * Return a copy of the next line in a mmap'ed file.
+  * spaces in the beginning of the line is trimmed away.
+  * Return a pointer to a static buffer.
+  **/
+void release_file(void *file, unsigned long size)
+{
+	munmap(file, size);
+}
+
+
+static void *get_sym_value(struct elf_info *info, const Elf_Sym *sym)
+{
+	void *ptr = (void *)info->hdr + info->sechdrs[sym->st_shndx].sh_offset;
+
+	return (void *)(ptr + sym->st_value);
+}
+
+static Elf_Sym *find_sym_in_symtab(struct elf_info *info,
+				   const char *name, Elf_Sym *last)
+{
+	Elf_Sym *idx;
+	if (last)
+		idx = last+1;
+	else
+		idx = info->symtab_start;
+
+	for (; idx < info->symtab_stop; idx++) {
+		const char *n = sym_name(info, idx);
+		if (!strncmp(n, name, strlen(name)))
+			return idx;
+	}
+	return NULL;
+}
+
+static int parse_elf(struct elf_info *info, const char *filename)
+{
+	unsigned int i;
+	Elf_Ehdr *hdr;
+	Elf_Shdr *sechdrs;
+	Elf_Sym  *sym;
+	const char *secstrings;
+	int endian;
+	unsigned int symtab_idx = ~0U, symtab_shndx_idx = ~0U;
+
+	hdr = grab_file(filename, &info->size);
+	if (!hdr) {
+		perror(filename);
+		exit(1);
+	}
+	info->hdr = hdr;
+	if (info->size < sizeof(*hdr)) {
+		/* file too small, assume this is an empty .o file */
+		return 0;
+	}
+	/* Is this a valid ELF file? */
+	if ((hdr->e_ident[EI_MAG0] != ELFMAG0) ||
+	    (hdr->e_ident[EI_MAG1] != ELFMAG1) ||
+	    (hdr->e_ident[EI_MAG2] != ELFMAG2) ||
+	    (hdr->e_ident[EI_MAG3] != ELFMAG3)) {
+		/* Not an ELF file - silently ignore it */
+		return 0;
+	}
+
+	if (!hdr->e_ident[EI_DATA]) {
+		/* Unknown endian */
+		return 0;
+	}
+
+	endian = hdr->e_ident[EI_DATA];
+
+	/* Fix endianness in ELF header */
+	hdr->e_type      = TO_NATIVE(endian, 16, hdr->e_type);
+	hdr->e_machine   = TO_NATIVE(endian, 16, hdr->e_machine);
+	hdr->e_version   = TO_NATIVE(endian, 32, hdr->e_version);
+	hdr->e_entry     = TO_NATIVE(endian, ADDR_SIZE, hdr->e_entry);
+	hdr->e_phoff     = TO_NATIVE(endian, ADDR_SIZE, hdr->e_phoff);
+	hdr->e_shoff     = TO_NATIVE(endian, ADDR_SIZE, hdr->e_shoff);
+	hdr->e_flags     = TO_NATIVE(endian, 32, hdr->e_flags);
+	hdr->e_ehsize    = TO_NATIVE(endian, 16, hdr->e_ehsize);
+	hdr->e_phentsize = TO_NATIVE(endian, 16, hdr->e_phentsize);
+	hdr->e_phnum     = TO_NATIVE(endian, 16, hdr->e_phnum);
+	hdr->e_shentsize = TO_NATIVE(endian, 16, hdr->e_shentsize);
+	hdr->e_shnum     = TO_NATIVE(endian, 16, hdr->e_shnum);
+	hdr->e_shstrndx  = TO_NATIVE(endian, 16, hdr->e_shstrndx);
+
+	sechdrs = (void *)hdr + hdr->e_shoff;
+	info->sechdrs = sechdrs;
+
+	/* Check if file offset is correct */
+	if (hdr->e_shoff > info->size) {
+		fprintf(stderr, "section header offset=%lu in file '%s' "
+		      "is bigger than filesize=%lu\n",
+		      (unsigned long)hdr->e_shoff,
+		      filename, info->size);
+		return 0;
+	}
+
+	if (hdr->e_shnum == SHN_UNDEF) {
+		/*
+		 * There are more than 64k sections,
+		 * read count from .sh_size.
+		 */
+		info->num_sections = TO_NATIVE(endian, 32, sechdrs[0].sh_size);
+	} else {
+		info->num_sections = hdr->e_shnum;
+	}
+	if (hdr->e_shstrndx == SHN_XINDEX)
+		info->secindex_strings =
+			TO_NATIVE(endian, 32, sechdrs[0].sh_link);
+	else
+		info->secindex_strings = hdr->e_shstrndx;
+
+	/* Fix endianness in section headers */
+	for (i = 0; i < info->num_sections; i++) {
+		sechdrs[i].sh_name      =
+			TO_NATIVE(endian, 32, sechdrs[i].sh_name);
+		sechdrs[i].sh_type      =
+			TO_NATIVE(endian, 32, sechdrs[i].sh_type);
+		sechdrs[i].sh_flags     =
+			TO_NATIVE(endian, 32, sechdrs[i].sh_flags);
+		sechdrs[i].sh_addr      =
+			TO_NATIVE(endian, ADDR_SIZE, sechdrs[i].sh_addr);
+		sechdrs[i].sh_offset    =
+			TO_NATIVE(endian, ADDR_SIZE, sechdrs[i].sh_offset);
+		sechdrs[i].sh_size      =
+			TO_NATIVE(endian, 32, sechdrs[i].sh_size);
+		sechdrs[i].sh_link      =
+			TO_NATIVE(endian, 32, sechdrs[i].sh_link);
+		sechdrs[i].sh_info      =
+			TO_NATIVE(endian, 32, sechdrs[i].sh_info);
+		sechdrs[i].sh_addralign =
+			TO_NATIVE(endian, ADDR_SIZE, sechdrs[i].sh_addralign);
+		sechdrs[i].sh_entsize   =
+			TO_NATIVE(endian, ADDR_SIZE, sechdrs[i].sh_entsize);
+	}
+	/* Find symbol table. */
+	secstrings = (void *)hdr + sechdrs[info->secindex_strings].sh_offset;
+	for (i = 1; i < info->num_sections; i++) {
+		int nobits = sechdrs[i].sh_type == SHT_NOBITS;
+
+		if (!nobits && sechdrs[i].sh_offset > info->size) {
+			fprintf(stderr, "%s is truncated. "
+			      "sechdrs[i].sh_offset=%lu > sizeof(*hrd)=%zu\n",
+			      filename, (unsigned long)sechdrs[i].sh_offset,
+			      sizeof(*hdr));
+			return 0;
+		}
+
+		if (sechdrs[i].sh_type == SHT_SYMTAB) {
+			unsigned int sh_link_idx;
+			symtab_idx = i;
+			info->symtab_start = (void *)hdr +
+			    sechdrs[i].sh_offset;
+			info->symtab_stop  = (void *)hdr +
+			    sechdrs[i].sh_offset + sechdrs[i].sh_size;
+			sh_link_idx = sechdrs[i].sh_link;
+			info->strtab       = (void *)hdr +
+			    sechdrs[sh_link_idx].sh_offset;
+		}
+
+		/* 32bit section no. table? ("more than 64k sections") */
+		if (sechdrs[i].sh_type == SHT_SYMTAB_SHNDX) {
+			symtab_shndx_idx = i;
+			info->symtab_shndx_start = (void *)hdr +
+			    sechdrs[i].sh_offset;
+			info->symtab_shndx_stop  = (void *)hdr +
+			    sechdrs[i].sh_offset + sechdrs[i].sh_size;
+		}
+	}
+	if (!info->symtab_start)
+		fprintf(stderr, "%s has no symtab?\n", filename);
+
+	/* Fix endianness in symbols */
+	for (sym = info->symtab_start; sym < info->symtab_stop; sym++) {
+		sym->st_shndx = TO_NATIVE(endian, 16, sym->st_shndx);
+		sym->st_name  = TO_NATIVE(endian, 32, sym->st_name);
+		sym->st_value = TO_NATIVE(endian, ADDR_SIZE, sym->st_value);
+		sym->st_size  = TO_NATIVE(endian, ADDR_SIZE, sym->st_size);
+	}
+
+	if (symtab_shndx_idx != ~0U) {
+		Elf32_Word *p;
+		if (symtab_idx != sechdrs[symtab_shndx_idx].sh_link)
+			fprintf(stderr,
+			      "%s: SYMTAB_SHNDX has bad sh_link: %u!=%u\n",
+			      filename, sechdrs[symtab_shndx_idx].sh_link,
+			      symtab_idx);
+		/* Fix endianness */
+		for (p = info->symtab_shndx_start; p < info->symtab_shndx_stop;
+		     p++)
+			*p = TO_NATIVE(endian, 32, *p);
+	}
+
+	return 1;
+}
+
+static void parse_elf_finish(struct elf_info *info)
+{
+	struct pmd_driver *tmp, *idx = info->drivers;
+	release_file(info->hdr, info->size);
+	while (idx) {
+		tmp = idx->next;
+		free(idx);
+		idx = tmp;
+	}
+}
+
+static const char *sec_name(struct elf_info *elf, int secindex)
+{
+	Elf_Shdr *sechdrs = elf->sechdrs;
+	return (void *)elf->hdr +
+		elf->sechdrs[elf->secindex_strings].sh_offset +
+		sechdrs[secindex].sh_name;
+}
+
+static int get_symbol_index(struct elf_info *info, Elf_Sym *sym)
+{
+	const char *name =  sym_name(info, sym);
+	const char *idx;
+
+	idx = name;
+	while (idx) {
+		if (isdigit(*idx))
+			return atoi(idx);
+		idx++;
+	}
+	return -1;
+}
+
+struct opt_tag {
+	const char *suffix;
+	const char *json_id;
+};
+
+static const struct opt_tag opt_tags[] = {
+	{"_param_string_export", "params"},
+};
+
+static int complete_pmd_entry(struct elf_info *info, struct pmd_driver *drv)
+{
+	const char *tname;
+	int i;
+	char tmpsymname[128];
+	Elf_Sym *tmpsym;
+
+	drv->name = get_sym_value(info, drv->name_sym);
+
+	for (i = 0; i < PMD_OPT_MAX; i++) {
+		memset(tmpsymname, 0, 128);
+		sprintf(tmpsymname, "__%s%s", drv->name, opt_tags[i].suffix);
+		tmpsym = find_sym_in_symtab(info, tmpsymname, NULL);
+		if (!tmpsym)
+			continue;
+		drv->opt_vals[i] = get_sym_value(info, tmpsym);
+	}
+
+	memset(tmpsymname, 0, 128);
+	sprintf(tmpsymname, "__%s_pci_tbl_export", drv->name);
+
+	tmpsym = find_sym_in_symtab(info, tmpsymname, NULL);
+
+
+	/*
+	 * If this returns NULL, then this is a PMD_VDEV, because
+	 * it has no pci table reference
+	 */
+	if (!tmpsym) {
+		drv->pci_tbl = NULL;
+		return 0;
+	}
+
+	tname = get_sym_value(info, tmpsym);
+	tmpsym = find_sym_in_symtab(info, tname, NULL);
+	if (!tmpsym)
+		return -ENOENT;
+
+	drv->pci_tbl = (struct rte_pci_id *)get_sym_value(info, tmpsym);
+	if (!drv->pci_tbl)
+		return -ENOENT;
+
+	return 0;
+}
+
+static int locate_pmd_entries(struct elf_info *info)
+{
+	Elf_Sym *last = NULL;
+	struct pmd_driver *new;
+
+	info->drivers = NULL;
+
+	do {
+		new = calloc(sizeof(struct pmd_driver), 1);
+		new->name_sym = find_sym_in_symtab(info, "this_pmd_name", last);
+		last = new->name_sym;
+		if (!new->name_sym)
+			free(new);
+		else {
+			if (complete_pmd_entry(info, new)) {
+				fprintf(stderr,
+					"Failed to complete pmd entry\n");
+				free(new);
+			} else {
+				new->next = info->drivers;
+				info->drivers = new;
+			}
+		}
+	} while (last);
+}
+
+static void output_pmd_info_string(struct elf_info *info, char *outfile)
+{
+	FILE *ofd;
+	struct pmd_driver *drv;
+	struct rte_pci_id *pci_ids;
+	int idx = 0;
+
+	ofd = fopen(outfile, "w+");
+	if (!ofd) {
+		fprintf(stderr, "Unable to open output file\n");
+		return;
+	}
+
+	drv = info->drivers;
+
+	while (drv) {
+		fprintf(ofd, "const char %s_pmd_info[] __attribute__((used)) = "
+			"\"PMD_INFO_STRING= {",
+			drv->name);
+		fprintf(ofd, "\\\"name\\\" : \\\"%s\\\", ", drv->name);
+
+		for (idx = 0; idx < PMD_OPT_MAX; idx++) {
+			if (drv->opt_vals[idx])
+				fprintf(ofd, "\\\"%s\\\" : \\\"%s\\\", ",
+					opt_tags[idx].json_id,
+					drv->opt_vals[idx]);
+		}
+
+		pci_ids = drv->pci_tbl;
+		fprintf(ofd, "\\\"pci_ids\\\" : [");
+
+		while (pci_ids && pci_ids->device_id) {
+			fprintf(ofd, "[%d, %d, %d, %d]",
+				pci_ids->vendor_id, pci_ids->device_id,
+				pci_ids->subsystem_vendor_id,
+				pci_ids->subsystem_device_id);
+			pci_ids++;
+			if (pci_ids->device_id)
+				fprintf(ofd, ",");
+			else
+				fprintf(ofd, " ");
+		}
+		fprintf(ofd, "]}\";");
+		drv = drv->next;
+	}
+
+	fclose(ofd);
+}
+
+int main(int argc, char **argv)
+{
+	struct elf_info info;
+	int rc = 1;
+
+	if (argc < 3) {
+		fprintf(stderr,
+			"usage: pmdinfo <object file> <c output file>\n");
+		exit(127);
+	}
+	parse_elf(&info, argv[1]);
+
+	locate_pmd_entries(&info);
+
+	if (info.drivers) {
+		output_pmd_info_string(&info, argv[2]);
+		rc = 0;
+	} else {
+		fprintf(stderr, "No drivers registered\n");
+	}
+
+	parse_elf_finish(&info);
+	exit(rc);
+}
diff --git a/buildtools/pmdinfogen/pmdinfogen.h b/buildtools/pmdinfogen/pmdinfogen.h
new file mode 100644
index 0000000..1da2966
--- /dev/null
+++ b/buildtools/pmdinfogen/pmdinfogen.h
@@ -0,0 +1,120 @@
+
+/* Postprocess pmd object files to export hw support
+ *
+ * Copyright 2016 Neil Horman <nhorman@tuxdriver.com>
+ * Based in part on modpost.c from the linux kernel
+ *
+ * This software may be used and distributed according to the terms
+ * of the GNU General Public License V2, incorporated herein by reference.
+ *
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdarg.h>
+#include <string.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/mman.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <elf.h>
+#include <rte_config.h>
+#include <rte_pci.h>
+#include <rte_byteorder.h>
+
+/* On BSD-alike OSes elf.h defines these according to host's word size */
+#undef ELF_ST_BIND
+#undef ELF_ST_TYPE
+#undef ELF_R_SYM
+#undef ELF_R_TYPE
+
+/*
+ * Define ELF64_* to ELF_*, the latter being defined in both 32 and 64 bit
+ * flavors in elf.h.  This makes our code a bit more generic between arches
+ * and allows us to support 32 bit code in the future should we ever want to
+ */
+#ifdef RTE_ARCH_64
+#define Elf_Ehdr    Elf64_Ehdr
+#define Elf_Shdr    Elf64_Shdr
+#define Elf_Sym     Elf64_Sym
+#define Elf_Addr    Elf64_Addr
+#define Elf_Sword   Elf64_Sxword
+#define Elf_Section Elf64_Half
+#define ELF_ST_BIND ELF64_ST_BIND
+#define ELF_ST_TYPE ELF64_ST_TYPE
+
+#define Elf_Rel     Elf64_Rel
+#define Elf_Rela    Elf64_Rela
+#define ELF_R_SYM   ELF64_R_SYM
+#define ELF_R_TYPE  ELF64_R_TYPE
+#else
+#define Elf_Ehdr    Elf32_Ehdr
+#define Elf_Shdr    Elf32_Shdr
+#define Elf_Sym     Elf32_Sym
+#define Elf_Addr    Elf32_Addr
+#define Elf_Sword   Elf32_Sxword
+#define Elf_Section Elf32_Half
+#define ELF_ST_BIND ELF32_ST_BIND
+#define ELF_ST_TYPE ELF32_ST_TYPE
+
+#define Elf_Rel     Elf32_Rel
+#define Elf_Rela    Elf32_Rela
+#define ELF_R_SYM   ELF32_R_SYM
+#define ELF_R_TYPE  ELF32_R_TYPE
+#endif
+
+
+/*
+ * Note, it seems odd that we have both a CONVERT_NATIVE and a TO_NATIVE macro
+ * below.  We do this because the values passed to TO_NATIVE may themselves be
+ * macros and need both macros here to get expanded.  Specifically its the width
+ * variable we are concerned with, because it needs to get expanded prior to
+ * string concatenation
+ */
+#define CONVERT_NATIVE(fend, width, x) ({ \
+typeof(x) ___x; \
+if ((fend) == ELFDATA2LSB) \
+	___x = rte_le_to_cpu_##width(x); \
+else \
+	___x = rte_be_to_cpu_##width(x); \
+	___x; \
+})
+
+#define TO_NATIVE(fend, width, x) CONVERT_NATIVE(fend, width, x)
+
+enum opt_params {
+	PMD_PARAM_STRING = 0,
+	PMD_OPT_MAX
+};
+
+struct pmd_driver {
+	Elf_Sym *name_sym;
+	const char *name;
+	struct rte_pci_id *pci_tbl;
+	struct pmd_driver *next;
+
+	const char *opt_vals[PMD_OPT_MAX];
+};
+
+struct elf_info {
+	unsigned long size;
+	Elf_Ehdr     *hdr;
+	Elf_Shdr     *sechdrs;
+	Elf_Sym      *symtab_start;
+	Elf_Sym      *symtab_stop;
+	char         *strtab;
+
+	/* support for 32bit section numbers */
+
+	unsigned int num_sections; /* max_secindex + 1 */
+	unsigned int secindex_strings;
+	/* if Nth symbol table entry has .st_shndx = SHN_XINDEX,
+	 * take shndx from symtab_shndx_start[N] instead
+	 */
+	Elf32_Word   *symtab_shndx_start;
+	Elf32_Word   *symtab_shndx_stop;
+
+	struct pmd_driver *drivers;
+};
+
diff --git a/mk/rte.hostapp.mk b/mk/rte.hostapp.mk
index c44d0f8..07b391c 100644
--- a/mk/rte.hostapp.mk
+++ b/mk/rte.hostapp.mk
@@ -41,7 +41,7 @@ include $(RTE_SDK)/mk/internal/rte.depdirs-pre.mk
 VPATH += $(SRCDIR)
 
 _BUILD = $(HOSTAPP)
-_INSTALL = $(INSTALL-FILES-y) $(SYMLINK-FILES-y) $(RTE_OUTPUT)/hostapp/$(HOSTAPP)
+_INSTALL = $(INSTALL-FILES-y) $(SYMLINK-FILES-y) $(RTE_OUTPUT)/app/$(HOSTAPP)
 _CLEAN = doclean
 
 .PHONY: all
@@ -95,10 +95,10 @@ $(HOSTAPP): $(OBJS-y) $(LDLIBS_FILES) FORCE
 #
 # install app in $(RTE_OUTPUT)/hostapp
 #
-$(RTE_OUTPUT)/hostapp/$(HOSTAPP): $(HOSTAPP)
+$(RTE_OUTPUT)/app/$(HOSTAPP): $(HOSTAPP)
 	@echo "  INSTALL-HOSTAPP $(HOSTAPP)"
-	@[ -d $(RTE_OUTPUT)/hostapp ] || mkdir -p $(RTE_OUTPUT)/hostapp
-	$(Q)cp -f $(HOSTAPP) $(RTE_OUTPUT)/hostapp
+	@[ -d $(RTE_OUTPUT)/app ] || mkdir -p $(RTE_OUTPUT)/app
+	$(Q)cp -f $(HOSTAPP) $(RTE_OUTPUT)/app
 
 #
 # Clean all generated files
diff --git a/mk/rte.sdkbuild.mk b/mk/rte.sdkbuild.mk
index eec5241..fb68af2 100644
--- a/mk/rte.sdkbuild.mk
+++ b/mk/rte.sdkbuild.mk
@@ -64,7 +64,8 @@ build: $(ROOTDIRS-y)
 clean: $(CLEANDIRS)
 	@rm -rf $(RTE_OUTPUT)/include $(RTE_OUTPUT)/app \
 		$(RTE_OUTPUT)/hostapp $(RTE_OUTPUT)/lib \
-		$(RTE_OUTPUT)/hostlib $(RTE_OUTPUT)/kmod
+		$(RTE_OUTPUT)/hostlib $(RTE_OUTPUT)/kmod \
+		$(RTE_OUTPUT)/buildtools
 	@[ -d $(RTE_OUTPUT)/include ] || mkdir -p $(RTE_OUTPUT)/include
 	@$(RTE_SDK)/scripts/gen-config-h.sh $(RTE_OUTPUT)/.config \
 		> $(RTE_OUTPUT)/include/rte_config.h
diff --git a/mk/rte.sdkinstall.mk b/mk/rte.sdkinstall.mk
index abdab0f..1f3b59c 100644
--- a/mk/rte.sdkinstall.mk
+++ b/mk/rte.sdkinstall.mk
@@ -126,6 +126,8 @@ install-runtime:
 	$(Q)$(call rte_mkdir,      $(DESTDIR)$(sbindir))
 	$(Q)$(call rte_symlink,    $(DESTDIR)$(datadir)/tools/dpdk_nic_bind.py, \
 	                           $(DESTDIR)$(sbindir)/dpdk_nic_bind)
+	$(Q)$(call rte_symlink,    $(DESTDIR)$(bindir)/pmdinfogen, \
+				   $(DESTDIR)$(bindir)/dpdk_pmdinfogen)
 
 install-kmod:
 ifneq ($(wildcard $O/kmod/*),)
-- 
2.5.5

[PATCHv8 2/6] drivers: Update driver registration macro usage

[Neil Horman]

Modify the PMD_REGISTER_DRIVER macro, adding a name argument to it.  The
addition of a name argument creates a token that can be used for subsequent
macros in the creation of unique symbol names to export additional bits of
information for use by the pmdinfogen tool.  For example:

PMD_REGISTER_DRIVER(ena_driver, ena);

registers the ena_driver struct as it always did, and creates a symbol
const char this_pmd_name0[] __attribute__((used)) = "ena";

which pmdinfogen can search for and extract.  The subsequent macro

DRIVER_REGISTER_PCI_TABLE(ena, ena_pci_id_map);

creates a symbol const char ena_pci_tbl_export[] __attribute__((used)) =
"ena_pci_id_map";

Which allows pmdinfogen to find the pci table of this driver

Using this pattern, we can export arbitrary bits of information.

pmdinfo uses this information to extract hardware support from an object
file and create a json string to make hardware support info discoverable
later.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Panu Matilainen <pmatilai@redhat.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 drivers/Makefile                           |  2 ++
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c   |  4 +++-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c |  4 +++-
 drivers/crypto/null/null_crypto_pmd.c      |  4 +++-
 drivers/crypto/qat/rte_qat_cryptodev.c     |  4 +++-
 drivers/crypto/snow3g/rte_snow3g_pmd.c     |  4 +++-
 drivers/net/af_packet/rte_eth_af_packet.c  |  4 +++-
 drivers/net/bnx2x/bnx2x_ethdev.c           |  6 ++++--
 drivers/net/bonding/rte_eth_bond_pmd.c     |  7 ++++++-
 drivers/net/cxgbe/cxgbe_ethdev.c           |  4 +++-
 drivers/net/e1000/em_ethdev.c              |  3 ++-
 drivers/net/e1000/igb_ethdev.c             |  6 ++++--
 drivers/net/ena/ena_ethdev.c               |  3 ++-
 drivers/net/enic/enic_ethdev.c             |  3 ++-
 drivers/net/fm10k/fm10k_ethdev.c           |  3 ++-
 drivers/net/i40e/i40e_ethdev.c             |  3 ++-
 drivers/net/i40e/i40e_ethdev_vf.c          |  3 ++-
 drivers/net/ixgbe/ixgbe_ethdev.c           |  6 ++++--
 drivers/net/mlx4/mlx4.c                    |  3 ++-
 drivers/net/mlx5/mlx5.c                    |  3 ++-
 drivers/net/mpipe/mpipe_tilegx.c           |  4 ++--
 drivers/net/nfp/nfp_net.c                  |  3 ++-
 drivers/net/null/rte_eth_null.c            |  3 ++-
 drivers/net/pcap/rte_eth_pcap.c            |  4 +++-
 drivers/net/qede/qede_ethdev.c             |  6 ++++--
 drivers/net/ring/rte_eth_ring.c            |  3 ++-
 drivers/net/szedata2/rte_eth_szedata2.c    |  3 ++-
 drivers/net/vhost/rte_eth_vhost.c          |  3 ++-
 drivers/net/virtio/virtio_ethdev.c         |  3 ++-
 drivers/net/vmxnet3/vmxnet3_ethdev.c       |  3 ++-
 drivers/net/xenvirt/rte_eth_xenvirt.c      |  2 +-
 lib/librte_eal/common/include/rte_dev.h    | 30 ++++++++++++++++++++++++------
 32 files changed, 105 insertions(+), 41 deletions(-)

diff --git a/drivers/Makefile b/drivers/Makefile
index 81c03a8..75a3168 100644
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -34,4 +34,6 @@ include $(RTE_SDK)/mk/rte.vars.mk
 DIRS-y += net
 DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += crypto
 
+DEPDIRS-y += buildtools/pmdinfo
+
 include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
index 2987ef6..f43e407 100644
--- a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
+++ b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
@@ -521,4 +521,6 @@ static struct rte_driver aesni_gcm_pmd_drv = {
 	.uninit = aesni_gcm_uninit
 };
 
-PMD_REGISTER_DRIVER(aesni_gcm_pmd_drv);
+PMD_REGISTER_DRIVER(aesni_gcm_pmd_drv, aesni_gcm);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+"max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
index 6554fc4..db3e562 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
@@ -721,4 +721,6 @@ static struct rte_driver cryptodev_aesni_mb_pmd_drv = {
 	.uninit = cryptodev_aesni_mb_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drv);
+PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drvi, aesni_mb);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+"max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/null/null_crypto_pmd.c b/drivers/crypto/null/null_crypto_pmd.c
index bdaf13c..0a195ed 100644
--- a/drivers/crypto/null/null_crypto_pmd.c
+++ b/drivers/crypto/null/null_crypto_pmd.c
@@ -275,4 +275,6 @@ static struct rte_driver cryptodev_null_pmd_drv = {
 	.uninit = cryptodev_null_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_null_pmd_drv);
+PMD_REGISTER_DRIVER(cryptodev_null_pmd_drv, cryptodev_null_pmd);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+"max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index a7912f5..d2de6a6 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -137,4 +137,6 @@ static struct rte_driver pmd_qat_drv = {
 	.init = rte_qat_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(pmd_qat_drv);
+PMD_REGISTER_DRIVER(pmd_qat_drv, qat);
+DRIVER_REGISTER_PCI_TABLE(qat, pci_id_qat_map);
+
diff --git a/drivers/crypto/snow3g/rte_snow3g_pmd.c b/drivers/crypto/snow3g/rte_snow3g_pmd.c
index f3e0e66..6934426 100644
--- a/drivers/crypto/snow3g/rte_snow3g_pmd.c
+++ b/drivers/crypto/snow3g/rte_snow3g_pmd.c
@@ -548,4 +548,6 @@ static struct rte_driver cryptodev_snow3g_pmd_drv = {
 	.uninit = cryptodev_snow3g_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_snow3g_pmd_drv);
+PMD_REGISTER_DRIVER(cryptodev_snow3g_pmd_drv, snow3g);
+DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+"max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index f17bd7e..e10eff2 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -878,4 +878,6 @@ static struct rte_driver pmd_af_packet_drv = {
 	.uninit = rte_pmd_af_packet_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_af_packet_drv);
+PMD_REGISTER_DRIVER(pmd_af_packet_drv, eth_af_packet);
+DRIVER_REGISTER_PARAM_STRING(eth_af_packet, "iface=<string> "
+"qpairs=<int> blocksz=<int> framesz=<int> framecnt=<int>");
diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c b/drivers/net/bnx2x/bnx2x_ethdev.c
index 071b44f..2b88324 100644
--- a/drivers/net/bnx2x/bnx2x_ethdev.c
+++ b/drivers/net/bnx2x/bnx2x_ethdev.c
@@ -550,5 +550,7 @@ static struct rte_driver rte_bnx2xvf_driver = {
 	.init = rte_bnx2xvf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_bnx2x_driver);
-PMD_REGISTER_DRIVER(rte_bnx2xvf_driver);
+PMD_REGISTER_DRIVER(rte_bnx2x_driver, bnx2x);
+DRIVER_REGISTER_PCI_TABLE(bnx2x, pci_id_bnx2x_map);
+PMD_REGISTER_DRIVER(rte_bnx2xvf_driver, bnx2xvf);
+DRIVER_REGISTER_PCI_TABLE(bnx2xvf, pci_id_bnx2xvf_map);
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 129f04b..e94ae36 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -2511,4 +2511,9 @@ static struct rte_driver bond_drv = {
 	.uninit = bond_uninit,
 };
 
-PMD_REGISTER_DRIVER(bond_drv);
+PMD_REGISTER_DRIVER(bond_drv, bonding);
+
+DRIVER_REGISTER_PARAM_STRING(bonding, "slave=<ifc> primary=<ifc> mode=[0-4] "
+"xmit_policy=[l2 | l23 | l34] socket_id=<int> mac=<mac addr> "
+"lsc_poll_period_ms=<int> up_delay=<int> down_delay=<int>");
+
diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index 04eddaf..e84e48b 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -894,4 +894,6 @@ static struct rte_driver rte_cxgbe_driver = {
 	.init = rte_cxgbe_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_cxgbe_driver);
+PMD_REGISTER_DRIVER(rte_cxgbe_driver, cxgb4);
+DRIVER_REGISTER_PCI_TABLE(cxgb4, cxgb4_pci_tbl);
+
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index 653be09..4de5eb2 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -1777,4 +1777,5 @@ struct rte_driver em_pmd_drv = {
 	.init = rte_em_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(em_pmd_drv);
+PMD_REGISTER_DRIVER(em_pmd_drv, em);
+DRIVER_REGISTER_PCI_TABLE(em, pci_id_em_map);
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index b822992..f6eea43 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -5015,5 +5015,7 @@ eth_igb_configure_msix_intr(struct rte_eth_dev *dev)
 	E1000_WRITE_FLUSH(hw);
 }
 
-PMD_REGISTER_DRIVER(pmd_igb_drv);
-PMD_REGISTER_DRIVER(pmd_igbvf_drv);
+PMD_REGISTER_DRIVER(pmd_igb_drv, igb);
+DRIVER_REGISTER_PCI_TABLE(igb, pci_id_igb_map);
+PMD_REGISTER_DRIVER(pmd_igbvf_drv, igbvf);
+DRIVER_REGISTER_PCI_TABLE(igbvf, pci_id_igbvf_map);
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index e157587..702289b 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -1450,4 +1450,5 @@ struct rte_driver ena_pmd_drv = {
 	.init = rte_ena_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(ena_pmd_drv);
+PMD_REGISTER_DRIVER(ena_pmd_drv, ena);
+DRIVER_REGISTER_PCI_TABLE(ena, pci_id_ena_map);
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index 6bea940..36ee7ca 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -684,4 +684,5 @@ static struct rte_driver rte_enic_driver = {
 	.init = rte_enic_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_enic_driver);
+PMD_REGISTER_DRIVER(rte_enic_driver, enic);
+DRIVER_REGISTER_PCI_TABLE(enic, pci_id_enic_map);
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index ce053b0..23ea718 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -3089,4 +3089,5 @@ static struct rte_driver rte_fm10k_driver = {
 	.init = rte_pmd_fm10k_init,
 };
 
-PMD_REGISTER_DRIVER(rte_fm10k_driver);
+PMD_REGISTER_DRIVER(rte_fm10k_driver, fm10k);
+DRIVER_REGISTER_PCI_TABLE(fm10k, pci_id_fm10k_map);
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index f94ad87..047e731 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -699,7 +699,8 @@ static struct rte_driver rte_i40e_driver = {
 	.init = rte_i40e_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_i40e_driver);
+PMD_REGISTER_DRIVER(rte_i40e_driver, i40e);
+DRIVER_REGISTER_PCI_TABLE(i40e, pci_id_i40e_map);
 
 /*
  * Initialize registers for flexible payload, which should be set by NVM.
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 37af399..ec6c850 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1582,7 +1582,8 @@ static struct rte_driver rte_i40evf_driver = {
 	.init = rte_i40evf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_i40evf_driver);
+PMD_REGISTER_DRIVER(rte_i40evf_driver, i40evf);
+DRIVER_REGISTER_PCI_TABLE(i40evf, pci_id_i40evf_map);
 
 static int
 i40evf_dev_configure(struct rte_eth_dev *dev)
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index e11a431..e0cc8f8 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -7243,5 +7243,7 @@ static struct rte_driver rte_ixgbevf_driver = {
 	.init = rte_ixgbevf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_ixgbe_driver);
-PMD_REGISTER_DRIVER(rte_ixgbevf_driver);
+PMD_REGISTER_DRIVER(rte_ixgbe_driver, ixgbe);
+DRIVER_REGISTER_PCI_TABLE(ixgbe, pci_id_ixgbe_map);
+PMD_REGISTER_DRIVER(rte_ixgbevf_driver, ixgbevf);
+DRIVER_REGISTER_PCI_TABLE(ixgbevf, pci_id_ixgbevf_map);
diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index 9ed1491..aedf1e9 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -5865,4 +5865,5 @@ static struct rte_driver rte_mlx4_driver = {
 	.init = rte_mlx4_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_mlx4_driver)
+PMD_REGISTER_DRIVER(rte_mlx4_driveri, mlx4)
+DRIVER_REGISTER_PCI_TABLE(mlx4, mlx4_pci_id_map);
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 041cfc3..5f2d6c8 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -672,4 +672,5 @@ static struct rte_driver rte_mlx5_driver = {
 	.init = rte_mlx5_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_mlx5_driver)
+PMD_REGISTER_DRIVER(rte_mlx5_driveri, mlx5)
+DRIVER_REGISTER_PCI_TABLE(mlx5, mlx5_pci_id_map);
diff --git a/drivers/net/mpipe/mpipe_tilegx.c b/drivers/net/mpipe/mpipe_tilegx.c
index 26e1424..4a2a905 100644
--- a/drivers/net/mpipe/mpipe_tilegx.c
+++ b/drivers/net/mpipe/mpipe_tilegx.c
@@ -1635,8 +1635,8 @@ static struct rte_driver pmd_mpipe_gbe_drv = {
 	.init = rte_pmd_mpipe_devinit,
 };
 
-PMD_REGISTER_DRIVER(pmd_mpipe_xgbe_drv);
-PMD_REGISTER_DRIVER(pmd_mpipe_gbe_drv);
+PMD_REGISTER_DRIVER(pmd_mpipe_xgbe_drv, mpipe_xgbe);
+PMD_REGISTER_DRIVER(pmd_mpipe_gbe_drv, mpipe_gbe);
 
 static void __attribute__((constructor, used))
 mpipe_init_contexts(void)
diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
index 5c9f350..091640d 100644
--- a/drivers/net/nfp/nfp_net.c
+++ b/drivers/net/nfp/nfp_net.c
@@ -2490,7 +2490,8 @@ static struct rte_driver rte_nfp_net_driver = {
 	.init = nfp_net_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_nfp_net_driver);
+PMD_REGISTER_DRIVER(rte_nfp_net_driver, nfp);
+DRIVER_REGISTER_PCI_TABLE(nfp, pci_id_nfp_net_map);
 
 /*
  * Local variables:
diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index 5e8e203..88eb582 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -689,4 +689,5 @@ static struct rte_driver pmd_null_drv = {
 	.uninit = rte_pmd_null_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_null_drv);
+PMD_REGISTER_DRIVER(pmd_null_drv, eth_null);
+DRIVER_REGISTER_PARAM_STRING(eth_null, "size=<int> copy=<int>");
diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
index c98e234..bb72af1 100644
--- a/drivers/net/pcap/rte_eth_pcap.c
+++ b/drivers/net/pcap/rte_eth_pcap.c
@@ -1090,4 +1090,6 @@ static struct rte_driver pmd_pcap_drv = {
 	.uninit = rte_pmd_pcap_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_pcap_drv);
+PMD_REGISTER_DRIVER(pmd_pcap_drv, pcap);
+DRIVER_REGISTER_PARAM_STRING(pcap, "rx_pcap=<string> tx_pcap=<string> "
+"rx_iface=<ifc> tx_iface=<ifc> iface=<ifc>");
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index 1273fd3..187ccf1 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -1102,5 +1102,7 @@ static struct rte_driver rte_qede_driver = {
 	.init = rte_qedevf_pmd_init
 };
 
-PMD_REGISTER_DRIVER(rte_qede_driver);
-PMD_REGISTER_DRIVER(rte_qedevf_driver);
+PMD_REGISTER_DRIVER(rte_qede_driver, qede);
+DRIVER_REGISTER_PCI_TABLE(qede, pci_id_qede_map);
+PMD_REGISTER_DRIVER(rte_qedevf_driver, qedevf);
+DRIVER_REGISTER_PCI_TABLE(qedevf, pci_id_qedevf_map);
diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index b1783c3..f2f5217 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -630,4 +630,5 @@ static struct rte_driver pmd_ring_drv = {
 	.uninit = rte_pmd_ring_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_ring_drv);
+PMD_REGISTER_DRIVER(pmd_ring_drv, eth_ring);
+DRIVER_REGISTER_PARAM_STRING(eth_ring, "nodeaction=[attach|detach]");
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 985a8d6..e71bce5 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1601,4 +1601,5 @@ static struct rte_driver rte_szedata2_driver = {
 	.uninit = rte_szedata2_uninit,
 };
 
-PMD_REGISTER_DRIVER(rte_szedata2_driver);
+PMD_REGISTER_DRIVER(rte_szedata2_driver, rte_szedata2_pmd);
+DRIVER_REGISTER_PCI_TABLE(rte_szedata2_pmd, rte_szedata2_pci_id_table);
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index 310cbef..d321c3e 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -924,4 +924,5 @@ static struct rte_driver pmd_vhost_drv = {
 	.uninit = rte_pmd_vhost_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_vhost_drv);
+PMD_REGISTER_DRIVER(pmd_vhost_drv, eth_vhost);
+DRIVER_REGISTER_PARAM_STRING(eth_vhost, "iface=<ifc> queues=<int>");
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index a833740..f428447 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -1505,4 +1505,5 @@ static struct rte_driver rte_virtio_driver = {
 	.init = rte_virtio_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_virtio_driver);
+PMD_REGISTER_DRIVER(rte_virtio_driver, virtio_net);
+DRIVER_REGISTER_PCI_TABLE(virtio_net, pci_id_virtio_map);
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index 29b469c..8da4449 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -954,4 +954,5 @@ static struct rte_driver rte_vmxnet3_driver = {
 	.init = rte_vmxnet3_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_vmxnet3_driver);
+PMD_REGISTER_DRIVER(rte_vmxnet3_driver, vmxnet3);
+DRIVER_REGISTER_PCI_TABLE(vmxnet3, pci_id_vmxnet3_map);
diff --git a/drivers/net/xenvirt/rte_eth_xenvirt.c b/drivers/net/xenvirt/rte_eth_xenvirt.c
index 1adeb5b..81f8af9 100644
--- a/drivers/net/xenvirt/rte_eth_xenvirt.c
+++ b/drivers/net/xenvirt/rte_eth_xenvirt.c
@@ -763,4 +763,4 @@ static struct rte_driver pmd_xenvirt_drv = {
 	.uninit = rte_pmd_xenvirt_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_xenvirt_drv);
+PMD_REGISTER_DRIVER(pmd_xenvirt_drv, xenvirt);
diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
index f1b5507..e6f0d4c 100644
--- a/lib/librte_eal/common/include/rte_dev.h
+++ b/lib/librte_eal/common/include/rte_dev.h
@@ -48,7 +48,7 @@ extern "C" {
 
 #include <stdio.h>
 #include <sys/queue.h>
-
+#include <rte_pci.h>
 #include <rte_log.h>
 
 __attribute__((format(printf, 2, 0)))
@@ -178,12 +178,30 @@ int rte_eal_vdev_init(const char *name, const char *args);
  */
 int rte_eal_vdev_uninit(const char *name);
 
-#define PMD_REGISTER_DRIVER(d)\
-void devinitfn_ ##d(void);\
-void __attribute__((constructor, used)) devinitfn_ ##d(void)\
+#define DRIVER_EXPORT_NAME_ARRAY(n, idx) n##idx[]
+
+#define DRIVER_EXPORT_NAME(name, idx) \
+static const char DRIVER_EXPORT_NAME_ARRAY(this_pmd_name, idx) \
+__attribute__((used)) = RTE_STR(name)
+
+#define PMD_REGISTER_DRIVER(drv, nm)\
+void devinitfn_ ##drv(void);\
+void __attribute__((constructor, used)) devinitfn_ ##drv(void)\
 {\
-	rte_eal_driver_register(&d);\
-}
+	(drv).name = RTE_STR(nm);\
+	rte_eal_driver_register(&drv);\
+} \
+DRIVER_EXPORT_NAME(nm, __COUNTER__)
+
+#define DRV_EXP_TAG(name, tag) __##name##_##tag
+
+#define DRIVER_REGISTER_PCI_TABLE(name, table) \
+static const char DRV_EXP_TAG(name, pci_tbl_export)[] __attribute__((used)) = \
+RTE_STR(table)
+
+#define DRIVER_REGISTER_PARAM_STRING(name, str) \
+static const char DRV_EXP_TAG(name, param_string_export)[] \
+__attribute__((used)) = str
 
 #ifdef __cplusplus
 }
-- 
2.5.5

[PATCHv8 3/6] eal: Add an export symbol to expose the autoload path to external tools

[Neil Horman]

Export a symbol containing the string:
DPDK_PLUGIN_PATH="$(CONFIG_RTE_EAL_PMD_PATH)"

Where the latter half of the string is set at build time to a location from
which autoloaded DSO's will be found.  This string is used by pmdinfo in
'plugin' mode, whereby a user can specify a dpdk installation directory (or
static binary), and scan the associated path (if found) for pmd DSO's and
report on their hardware support.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Panu Matilainen <pmatilai@redhat.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 lib/librte_eal/common/eal_common_options.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c
index 3efc90f..0a594d7 100644
--- a/lib/librte_eal/common/eal_common_options.c
+++ b/lib/librte_eal/common/eal_common_options.c
@@ -115,6 +115,15 @@ TAILQ_HEAD_INITIALIZER(solib_list);
 /* Default path of external loadable drivers */
 static const char *default_solib_dir = RTE_EAL_PMD_PATH;
 
+/*
+ * Stringified version of solib path used by pmdinfo.py
+ * Note: PLEASE DO NOT ALTER THIS without making a corresponding
+ * change to tools/pmdinfo.py
+ */
+static const char dpdk_solib_path[] __attribute__((used)) =
+"DPDK_PLUGIN_PATH=" RTE_EAL_PMD_PATH;
+
+
 static int master_lcore_parsed;
 static int mem_parsed;
 
-- 
2.5.5

[PATCHv8 4/6] Makefile: Do post processing on objects that register a driver

[Neil Horman]

Modify the compilation makefile to identify C files that export PMD
information, and use that to trigger execution of the pmdinfo binary.  If
the execution of pmdinfo is successful, compile the output C file to an
object, and use the linker to do relocatable linking on the resultant
object file into the parent object that it came from.  This effectively
just adds the json string into the string table of the object that defines
the PMD to the outside world.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Panu Matilainen <pmatilai@redhat.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 mk/internal/rte.compile-pre.mk | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/mk/internal/rte.compile-pre.mk b/mk/internal/rte.compile-pre.mk
index b9bff4a..5632d6e 100644
--- a/mk/internal/rte.compile-pre.mk
+++ b/mk/internal/rte.compile-pre.mk
@@ -88,10 +88,24 @@ C_TO_O_CMD = 'cmd_$@ = $(C_TO_O_STR)'
 C_TO_O_DO = @set -e; \
 	echo $(C_TO_O_DISP); \
 	$(C_TO_O) && \
+	sh -c "grep -q \"PMD_REGISTER_DRIVER(.*)\" $<; \
+	if [ \$$? -eq 0 ]; then \
+		echo \"  PMDINFOGEN\" $@; \
+		OBJF=`readlink -f $@`; \
+		${RTE_OUTPUT}/app/pmdinfogen \$$OBJF \$$OBJF.pmd.c; \
+		if [ \$$? -eq 0 ]; \
+		then \
+			echo \"  PMDINFOBUILD\" $@; \
+			$(CC) $(CFLAGS) -c -o \$$OBJF.pmd.o \$$OBJF.pmd.c; \
+			$(CROSS)ld $(LDFLAGS) -r -o \$$OBJF.o \$$OBJF.pmd.o \$$OBJF; \
+			mv -f \$$OBJF.o \$$OBJF; \
+		fi; \
+	fi;" && \
 	echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \
 	sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \
 	rm -f $(call obj2dep,$(@)).tmp
 
+
 # return an empty string if string are equal
 compare = $(strip $(subst $(1),,$(2)) $(subst $(2),,$(1)))
 
-- 
2.5.5

[PATCHv8 5/6] pmdinfo.py: Add tool to query binaries for hw and other support information

[Neil Horman]

This tool searches for the primer sting PMD_DRIVER_INFO= in any ELF binary,
and, if found parses the remainder of the string as a json encoded string,
outputting the results in either a human readable or raw, script parseable
format

Note that, in the case of dynamically linked applications, pmdinfo.py will
scan for implicitly linked PMDs by searching the specified binaries
.dynamic section for DT_NEEDED entries that contain the substring
librte_pmd.  The DT_RUNPATH, LD_LIBRARY_PATH, /usr/lib and /lib are
searched for these libraries, in that order

If a file is specified with no path, it is assumed to be a PMD DSO, and the
LD_LIBRARY_PATH, /usr/lib[64]/ and /lib[64] is searched for it

Currently the tool can output data in 3 formats:

a) raw, suitable for scripting, where the raw JSON strings are dumped out
b) table format (default) where hex pci ids are dumped in a table format
c) pretty, where a user supplied pci.ids file is used to print out vendor
and device strings

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Panu Matilainen <pmatilai@redhat.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 mk/rte.sdkinstall.mk |   2 +
 tools/pmdinfo.py     | 629 +++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 631 insertions(+)
 create mode 100755 tools/pmdinfo.py

diff --git a/mk/rte.sdkinstall.mk b/mk/rte.sdkinstall.mk
index 1f3b59c..b0d985c 100644
--- a/mk/rte.sdkinstall.mk
+++ b/mk/rte.sdkinstall.mk
@@ -128,6 +128,8 @@ install-runtime:
 	                           $(DESTDIR)$(sbindir)/dpdk_nic_bind)
 	$(Q)$(call rte_symlink,    $(DESTDIR)$(bindir)/pmdinfogen, \
 				   $(DESTDIR)$(bindir)/dpdk_pmdinfogen)
+	$(Q)$(call rte_symlink,    $(DESTDIR)$(datadir)/tools/pmdinfo.py, \
+				   $(DESTDIR)$(bindir)/dpdk_pmdinfo)
 
 install-kmod:
 ifneq ($(wildcard $O/kmod/*),)
diff --git a/tools/pmdinfo.py b/tools/pmdinfo.py
new file mode 100755
index 0000000..e531154
--- /dev/null
+++ b/tools/pmdinfo.py
@@ -0,0 +1,629 @@
+#!/usr/bin/python
+# -------------------------------------------------------------------------
+# scripts/pmdinfo.py
+#
+# Utility to dump PMD_INFO_STRING support from an object file
+#
+# -------------------------------------------------------------------------
+import os
+import sys
+from optparse import OptionParser
+import string
+import json
+
+# For running from development directory. It should take precedence over the
+# installed pyelftools.
+sys.path.insert(0, '.')
+
+
+from elftools import __version__
+from elftools.common.exceptions import ELFError
+from elftools.common.py3compat import (
+    ifilter, byte2int, bytes2str, itervalues, str2bytes)
+from elftools.elf.elffile import ELFFile
+from elftools.elf.dynamic import DynamicSection, DynamicSegment
+from elftools.elf.enums import ENUM_D_TAG
+from elftools.elf.segments import InterpSegment
+from elftools.elf.sections import SymbolTableSection
+from elftools.elf.gnuversions import (
+    GNUVerSymSection, GNUVerDefSection,
+    GNUVerNeedSection,
+)
+from elftools.elf.relocation import RelocationSection
+from elftools.elf.descriptions import (
+    describe_ei_class, describe_ei_data, describe_ei_version,
+    describe_ei_osabi, describe_e_type, describe_e_machine,
+    describe_e_version_numeric, describe_p_type, describe_p_flags,
+    describe_sh_type, describe_sh_flags,
+    describe_symbol_type, describe_symbol_bind, describe_symbol_visibility,
+    describe_symbol_shndx, describe_reloc_type, describe_dyn_tag,
+    describe_ver_flags,
+)
+from elftools.elf.constants import E_FLAGS
+from elftools.dwarf.dwarfinfo import DWARFInfo
+from elftools.dwarf.descriptions import (
+    describe_reg_name, describe_attr_value, set_global_machine_arch,
+    describe_CFI_instructions, describe_CFI_register_rule,
+    describe_CFI_CFA_rule,
+)
+from elftools.dwarf.constants import (
+    DW_LNS_copy, DW_LNS_set_file, DW_LNE_define_file)
+from elftools.dwarf.callframe import CIE, FDE
+
+raw_output = False
+pcidb = None
+
+# ===========================================
+
+
+class Vendor:
+    """
+    Class for vendors. This is the top level class
+    for the devices belong to a specific vendor.
+    self.devices is the device dictionary
+    subdevices are in each device.
+    """
+
+    def __init__(self, vendorStr):
+        """
+        Class initializes with the raw line from pci.ids
+        Parsing takes place inside __init__
+        """
+        self.ID = vendorStr.split()[0]
+        self.name = vendorStr.replace("%s " % self.ID, "").rstrip()
+        self.devices = {}
+
+    def addDevice(self, deviceStr):
+        """
+        Adds a device to self.devices
+        takes the raw line from pci.ids
+        """
+        s = deviceStr.strip()
+        devID = s.split()[0]
+        if devID in self.devices:
+            pass
+        else:
+            self.devices[devID] = Device(deviceStr)
+
+    def report(self):
+        print self.ID, self.name
+        for id, dev in self.devices.items():
+            dev.report()
+
+    def find_device(self, devid):
+        # convert to a hex string and remove 0x
+        devid = hex(devid)[2:]
+        try:
+            return self.devices[devid]
+        except:
+            return Device("%s  Unknown Device" % devid)
+
+
+class Device:
+
+    def __init__(self, deviceStr):
+        """
+        Class for each device.
+        Each vendor has its own devices dictionary.
+        """
+        s = deviceStr.strip()
+        self.ID = s.split()[0]
+        self.name = s.replace("%s  " % self.ID, "")
+        self.subdevices = {}
+
+    def report(self):
+        print "\t%s\t%s" % (self.ID, self.name)
+        for subID, subdev in self.subdevices.items():
+            subdev.report()
+
+    def addSubDevice(self, subDeviceStr):
+        """
+        Adds a subvendor, subdevice to device.
+        Uses raw line from pci.ids
+        """
+        s = subDeviceStr.strip()
+        spl = s.split()
+        subVendorID = spl[0]
+        subDeviceID = spl[1]
+        subDeviceName = s.split("  ")[-1]
+        devID = "%s:%s" % (subVendorID, subDeviceID)
+        self.subdevices[devID] = SubDevice(
+            subVendorID, subDeviceID, subDeviceName)
+
+    def find_subid(self, subven, subdev):
+        subven = hex(subven)[2:]
+        subdev = hex(subdev)[2:]
+        devid = "%s:%s" % (subven, subdev)
+
+        try:
+            return self.subdevices[devid]
+        except:
+            if (subven == "ffff" and subdev == "ffff"):
+                return SubDevice("ffff", "ffff", "(All Subdevices)")
+            else:
+                return SubDevice(subven, subdev, "(Unknown Subdevice)")
+
+
+class SubDevice:
+    """
+    Class for subdevices.
+    """
+
+    def __init__(self, vendor, device, name):
+        """
+        Class initializes with vendorid, deviceid and name
+        """
+        self.vendorID = vendor
+        self.deviceID = device
+        self.name = name
+
+    def report(self):
+        print "\t\t%s\t%s\t%s" % (self.vendorID, self.deviceID, self.name)
+
+
+class PCIIds:
+    """
+    Top class for all pci.ids entries.
+    All queries will be asked to this class.
+    PCIIds.vendors["0e11"].devices["0046"].\
+    subdevices["0e11:4091"].name  =  "Smart Array 6i"
+    """
+
+    def __init__(self, filename):
+        """
+        Prepares the directories.
+        Checks local data file.
+        Tries to load from local, if not found, downloads from web
+        """
+        self.version = ""
+        self.date = ""
+        self.vendors = {}
+        self.contents = None
+        self.readLocal(filename)
+        self.parse()
+
+    def reportVendors(self):
+        """Reports the vendors
+        """
+        for vid, v in self.vendors.items():
+            print v.ID, v.name
+
+    def report(self, vendor=None):
+        """
+        Reports everything for all vendors or a specific vendor
+        PCIIds.report()  reports everything
+        PCIIDs.report("0e11") reports only "Compaq Computer Corporation"
+        """
+        if vendor is not None:
+            self.vendors[vendor].report()
+        else:
+            for vID, v in self.vendors.items():
+                v.report()
+
+    def find_vendor(self, vid):
+        # convert vid to a hex string and remove the 0x
+        vid = hex(vid)[2:]
+
+        try:
+            return self.vendors[vid]
+        except:
+            return Vendor("%s Unknown Vendor" % (vid))
+
+    def findDate(self, content):
+        for l in content:
+            if l.find("Date:") > -1:
+                return l.split()[-2].replace("-", "")
+        return None
+
+    def parse(self):
+        if len(self.contents) < 1:
+            print "data/%s-pci.ids not found" % self.date
+        else:
+            vendorID = ""
+            deviceID = ""
+            for l in self.contents:
+                if l[0] == "#":
+                    continue
+                elif len(l.strip()) == 0:
+                    continue
+                else:
+                    if l.find("\t\t") == 0:
+                        self.vendors[vendorID].devices[
+                            deviceID].addSubDevice(l)
+                    elif l.find("\t") == 0:
+                        deviceID = l.strip().split()[0]
+                        self.vendors[vendorID].addDevice(l)
+                    else:
+                        vendorID = l.split()[0]
+                        self.vendors[vendorID] = Vendor(l)
+
+    def readLocal(self, filename):
+        """
+        Reads the local file
+        """
+        self.contents = open(filename).readlines()
+        self.date = self.findDate(self.contents)
+
+    def loadLocal(self):
+        """
+        Loads database from local. If there is no file,
+        it creates a new one from web
+        """
+        self.date = idsfile[0].split("/")[1].split("-")[0]
+        self.readLocal()
+
+
+# =======================================
+
+def search_file(filename, search_path):
+    """ Given a search path, find file with requested name """
+    for path in string.split(search_path, ":"):
+        candidate = os.path.join(path, filename)
+        if os.path.exists(candidate):
+            return os.path.abspath(candidate)
+    return None
+
+
+class ReadElf(object):
+    """ display_* methods are used to emit output into the output stream
+    """
+
+    def __init__(self, file, output):
+        """ file:
+                stream object with the ELF file to read
+
+            output:
+                output stream to write to
+        """
+        self.elffile = ELFFile(file)
+        self.output = output
+
+        # Lazily initialized if a debug dump is requested
+        self._dwarfinfo = None
+
+        self._versioninfo = None
+
+    def _section_from_spec(self, spec):
+        """ Retrieve a section given a "spec" (either number or name).
+            Return None if no such section exists in the file.
+        """
+        try:
+            num = int(spec)
+            if num < self.elffile.num_sections():
+                return self.elffile.get_section(num)
+            else:
+                return None
+        except ValueError:
+            # Not a number. Must be a name then
+            return self.elffile.get_section_by_name(str2bytes(spec))
+
+    def pretty_print_pmdinfo(self, pmdinfo):
+        global pcidb
+
+        for i in pmdinfo["pci_ids"]:
+            vendor = pcidb.find_vendor(i[0])
+            device = vendor.find_device(i[1])
+            subdev = device.find_subid(i[2], i[3])
+            print("%s (%s) : %s (%s) %s" %
+                  (vendor.name, vendor.ID, device.name,
+                   device.ID, subdev.name))
+
+    def parse_pmd_info_string(self, mystring):
+        global raw_output
+        global pcidb
+
+        optional_pmd_info = [{'id': 'params', 'tag': 'PMD PARAMETERS'}]
+
+        i = mystring.index("=")
+        mystring = mystring[i + 2:]
+        pmdinfo = json.loads(mystring)
+
+        if raw_output:
+            print(pmdinfo)
+            return
+
+        print("PMD NAME: " + pmdinfo["name"])
+        for i in optional_pmd_info:
+            try:
+                print("%s: %s" % (i['tag'], pmdinfo[i['id']]))
+            except KeyError as e:
+                continue
+
+        if (len(pmdinfo["pci_ids"]) != 0):
+            print("PMD HW SUPPORT:")
+            if pcidb is not None:
+                self.pretty_print_pmdinfo(pmdinfo)
+            else:
+                print("VENDOR\t DEVICE\t SUBVENDOR\t SUBDEVICE")
+                for i in pmdinfo["pci_ids"]:
+                    print("0x%04x\t 0x%04x\t 0x%04x\t\t 0x%04x" %
+                          (i[0], i[1], i[2], i[3]))
+
+        print("")
+
+    def display_pmd_info_strings(self, section_spec):
+        """ Display a strings dump of a section. section_spec is either a
+            section number or a name.
+        """
+        section = self._section_from_spec(section_spec)
+        if section is None:
+            return
+
+        data = section.data()
+        dataptr = 0
+
+        while dataptr < len(data):
+            while (dataptr < len(data) and
+                    not (32 <= byte2int(data[dataptr]) <= 127)):
+                dataptr += 1
+
+            if dataptr >= len(data):
+                break
+
+            endptr = dataptr
+            while endptr < len(data) and byte2int(data[endptr]) != 0:
+                endptr += 1
+
+            mystring = bytes2str(data[dataptr:endptr])
+            rc = mystring.find("PMD_INFO_STRING")
+            if (rc != -1):
+                self.parse_pmd_info_string(mystring)
+
+            dataptr = endptr
+
+    def find_librte_eal(self, section):
+        for tag in section.iter_tags():
+            if tag.entry.d_tag == 'DT_NEEDED':
+                if "librte_eal" in tag.needed:
+                    return tag.needed
+        return None
+
+    def search_for_autoload_path(self):
+        scanelf = self
+        scanfile = None
+        library = None
+
+        section = self._section_from_spec(".dynamic")
+        try:
+            eallib = self.find_librte_eal(section)
+            if eallib is not None:
+                ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+                if ldlibpath is None:
+                    ldlibpath = ""
+                dtr = self.get_dt_runpath(section)
+                library = search_file(eallib,
+                                      dtr + ":" + ldlibpath +
+                                      ":/usr/lib64:/lib64:/usr/lib:/lib")
+                if library is None:
+                    return (None, None)
+                if raw_output is False:
+                    print("Scanning for autoload path in %s" % library)
+                scanfile = open(library, 'rb')
+                scanelf = ReadElf(scanfile, sys.stdout)
+        except AttributeError:
+            # Not a dynamic binary
+            pass
+        except ELFError:
+            scanfile.close()
+            return (None, None)
+
+        section = scanelf._section_from_spec(".rodata")
+        if section is None:
+            if scanfile is not None:
+                scanfile.close()
+            return (None, None)
+
+        data = section.data()
+        dataptr = 0
+
+        while dataptr < len(data):
+            while (dataptr < len(data) and
+                    not (32 <= byte2int(data[dataptr]) <= 127)):
+                dataptr += 1
+
+            if dataptr >= len(data):
+                break
+
+            endptr = dataptr
+            while endptr < len(data) and byte2int(data[endptr]) != 0:
+                endptr += 1
+
+            mystring = bytes2str(data[dataptr:endptr])
+            rc = mystring.find("DPDK_PLUGIN_PATH")
+            if (rc != -1):
+                rc = mystring.find("=")
+                return (mystring[rc + 1:], library)
+
+            dataptr = endptr
+        if scanfile is not None:
+            scanfile.close()
+        return (None, None)
+
+    def get_dt_runpath(self, dynsec):
+        for tag in dynsec.iter_tags():
+            if tag.entry.d_tag == 'DT_RUNPATH':
+                return tag.runpath
+        return ""
+
+    def process_dt_needed_entries(self):
+        """ Look to see if there are any DT_NEEDED entries in the binary
+            And process those if there are
+        """
+        global raw_output
+        runpath = ""
+        ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+        if ldlibpath is None:
+            ldlibpath = ""
+
+        dynsec = self._section_from_spec(".dynamic")
+        try:
+            runpath = self.get_dt_runpath(dynsec)
+        except AttributeError:
+            # dynsec is None, just return
+            return
+
+        for tag in dynsec.iter_tags():
+            if tag.entry.d_tag == 'DT_NEEDED':
+                rc = tag.needed.find("librte_pmd")
+                if (rc != -1):
+                    library = search_file(tag.needed,
+                                          runpath + ":" + ldlibpath +
+                                          ":/usr/lib64:/lib64:/usr/lib:/lib")
+                    if library is not None:
+                        if raw_output is False:
+                            print("Scanning %s for pmd information" % library)
+                        with open(library, 'rb') as file:
+                            try:
+                                libelf = ReadElf(file, sys.stdout)
+                            except ELFError as e:
+                                print("%s is no an ELF file" % library)
+                                continue
+                            libelf.process_dt_needed_entries()
+                            libelf.display_pmd_info_strings(".rodata")
+                            file.close()
+
+
+def scan_autoload_path(autoload_path):
+    global raw_output
+
+    if os.path.exists(autoload_path) is False:
+        return
+
+    try:
+        dirs = os.listdir(autoload_path)
+    except OSError as e:
+        # Couldn't read the directory, give up
+        return
+
+    for d in dirs:
+        dpath = os.path.join(autoload_path, d)
+        if os.path.isdir(dpath):
+            scan_autoload_path(dpath)
+        if os.path.isfile(dpath):
+            try:
+                file = open(dpath, 'rb')
+                readelf = ReadElf(file, sys.stdout)
+            except ELFError as e:
+                # this is likely not an elf file, skip it
+                continue
+            except IOError as e:
+                # No permission to read the file, skip it
+                continue
+
+            if raw_output is False:
+                print("Hw Support for library %s" % d)
+            readelf.display_pmd_info_strings(".rodata")
+            file.close()
+
+
+def scan_for_autoload_pmds(dpdk_path):
+    """
+    search the specified application or path for a pmd autoload path
+    then scan said path for pmds and report hw support
+    """
+    global raw_output
+
+    if (os.path.isfile(dpdk_path) is False):
+        if raw_output is False:
+            print("Must specify a file name")
+        return
+
+    file = open(dpdk_path, 'rb')
+    try:
+        readelf = ReadElf(file, sys.stdout)
+    except ElfError as e:
+        if raw_output is False:
+            print("Unable to parse %s" % file)
+        return
+
+    (autoload_path, scannedfile) = readelf.search_for_autoload_path()
+    if (autoload_path is None or autoload_path is ""):
+        if (raw_output is False):
+            print("No autoload path configured in %s" % dpdk_path)
+        return
+    if (raw_output is False):
+        if (scannedfile is None):
+            scannedfile = dpdk_path
+        print("Found autoload path %s in %s" % (autoload_path, scannedfile))
+
+    file.close()
+    if (raw_output is False):
+        print("Discovered Autoload HW Support:")
+    scan_autoload_path(autoload_path)
+    return
+
+
+def main(stream=None):
+    global raw_output
+    global pcidb
+
+    optparser = OptionParser(
+        usage='usage: %prog [-hrtp] [-d <pci id file] <elf-file>',
+        description="Dump pmd hardware support info",
+        add_help_option=True,
+        prog='pmdinfo.py')
+    optparser.add_option('-r', '--raw',
+                         action='store_true', dest='raw_output',
+                         help='Dump raw json strings')
+    optparser.add_option("-d", "--pcidb", dest="pcifile",
+                         help="specify a pci database "
+                              "to get vendor names from",
+                         default="/usr/share/hwdata/pci.ids", metavar="FILE")
+    optparser.add_option("-t", "--table", dest="tblout",
+                         help="output information on hw support as a hex table",
+                         action='store_true')
+    optparser.add_option("-p", "--plugindir", dest="pdir",
+                         help="scan dpdk for autoload plugins",
+                         action='store_true')
+
+    options, args = optparser.parse_args()
+
+    if options.raw_output:
+        raw_output = True
+
+    if options.pcifile:
+        pcidb = PCIIds(options.pcifile)
+        if pcidb is None:
+            print("Pci DB file not found")
+            exit(1)
+
+    if options.tblout:
+        options.pcifile = None
+        pcidb = None
+
+    if (len(args) == 0):
+        optparser.print_usage()
+        exit(1)
+
+    if options.pdir is True:
+        exit(scan_for_autoload_pmds(args[0]))
+
+    ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+    if (ldlibpath is None):
+        ldlibpath = ""
+
+    if (os.path.exists(args[0]) is True):
+        myelffile = args[0]
+    else:
+        myelffile = search_file(
+            args[0], ldlibpath + ":/usr/lib64:/lib64:/usr/lib:/lib")
+
+    if (myelffile is None):
+        print("File not found")
+        sys.exit(1)
+
+    with open(myelffile, 'rb') as file:
+        try:
+            readelf = ReadElf(file, sys.stdout)
+            readelf.process_dt_needed_entries()
+            readelf.display_pmd_info_strings(".rodata")
+            sys.exit(0)
+
+        except ELFError as ex:
+            sys.stderr.write('ELF error: %s\n' % ex)
+            sys.exit(1)
+
+
+# -------------------------------------------------------------------------
+if __name__ == '__main__':
+    main()
-- 
2.5.5

[PATCHv8 6/6] doc: Add prog_guide section documenting pmdinfo script

[Neil Horman]

Information on pmdinfogen may be useful to 3rd party driver developers.
Include documentation on what it does

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Panu Matilainen <pmatilai@redhat.com>
Acked-by: John McNamara <john.mcnamara@intel.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 doc/guides/prog_guide/dev_kit_build_system.rst | 43 ++++++++++++++++++++++++--
 1 file changed, 41 insertions(+), 2 deletions(-)

diff --git a/doc/guides/prog_guide/dev_kit_build_system.rst b/doc/guides/prog_guide/dev_kit_build_system.rst
index 3e89eae..1dc1388 100644
--- a/doc/guides/prog_guide/dev_kit_build_system.rst
+++ b/doc/guides/prog_guide/dev_kit_build_system.rst
@@ -70,7 +70,7 @@ Each build directory contains include files, libraries, and applications:
     ...
     ~/DEV/DPDK$ ls i686-native-linuxapp-gcc
 
-    app build hostapp include kmod lib Makefile
+    app build buildtools include kmod lib Makefile
 
 
     ~/DEV/DPDK$ ls i686-native-linuxapp-gcc/app/
@@ -264,7 +264,7 @@ These Makefiles generate a binary application.
 
 *   rte.extapp.mk: External application
 
-*   rte.hostapp.mk: Host application in the development kit framework
+*   rte.hostapp.mk: prerequisite tool to build dpdk
 
 Library
 ^^^^^^^
@@ -304,6 +304,45 @@ Misc
 
 *   rte.subdir.mk: Build several directories in the development kit framework.
 
+.. _Internally_Generated_Build_Tools:
+
+Internally Generated Build Tools
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+``app/pmdinfogen``
+
+
+``pmdinfogen`` scans an object (.o) file for various well known symbol names.  These
+well known symbol names are defined by various macros and used to export
+important information about hardware support and usage for pmd files.  For
+instance the macro:
+
+.. code-block:: c
+
+    PMD_REGISTER_DRIVER(drv, name)
+
+
+Creates the following symbol:
+
+.. code-block:: c
+
+   static char this_pmd_name0[] __attribute__((used)) = "<name>";
+
+
+Which pmdinfogen scans for.  Using this information other relevant bits of data
+can be exported from the object file and used to produce a hardware support
+description, that pmdinfogen then encodes into a json formatted string in the
+following format:
+
+.. code-block:: C
+
+   static char <name_pmd_string>="PMD_INFO_STRING=\"{'name' : '<name>', ...}\"";
+
+
+These strings can then be searched for by external tools to determine the
+hardware support of a given library or application.
+
+
 .. _Useful_Variables_Provided_by_the_Build_System:
 
 Useful Variables Provided by the Build System
-- 
2.5.5

Re: [PATCHv8 0/6] Implement pmd hardware support exports

[Remy Horton]

Morning,

Compile error with virtio_user_ethdev.c - looks like a new addition 
(commit ce2eabdd43ec) that also needs to be converted.

Regards,

..Remy


On 17/06/2016 19:46, Neil Horman wrote:
> Hey all-
> 	So heres attempt number 2 at a method for exporting PMD hardware support
> information.  As we discussed previously, the consensus seems to be that pmd
> information should be:
[..]
> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> Acked-by: Panu Matilainen <pmatilai@redhat.com>
> CC: Bruce Richardson <bruce.richardson@intel.com>
> CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> CC: Stephen Hemminger <stephen@networkplumber.org>
> CC: Panu Matilainen <pmatilai@redhat.com>

Re: [PATCHv8 0/6] Implement pmd hardware support exports

[Neil Horman]

On Wed, Jun 29, 2016 at 10:18:26AM +0100, Remy Horton wrote:
> Morning,
> 
> Compile error with virtio_user_ethdev.c - looks like a new addition (commit
> ce2eabdd43ec) that also needs to be converted.
> 
> Regards,
> 
> ..Remy
> 


diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index 5ab2471..aba9ebe 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -437,4 +437,4 @@ static struct rte_driver virtio_user_driver = {
 	.uninit = virtio_user_pmd_devuninit,
 };
 
-PMD_REGISTER_DRIVER(virtio_user_driver);
+PMD_REGISTER_DRIVER(virtio_user_driver, vio_user);

Re: [PATCHv8 5/6] pmdinfo.py: Add tool to query binaries for hw and other support information

[Remy Horton]

'noon,

The tool does not work for static PMD libraries (e.g. librte_pmd_i40e.a) 
- is this an intended limitation?

DPDK doesn't to my knowledge have any coding guidelines for Python, so 
the comments below should be considered advisory rather than 
merge-blocking issues.


On 17/06/2016 19:46, Neil Horman wrote:
[..]
> +++ b/tools/pmdinfo.py
> @@ -0,0 +1,629 @@
> +#!/usr/bin/python
> +# -------------------------------------------------------------------------
> +# scripts/pmdinfo.py
> +#
> +# Utility to dump PMD_INFO_STRING support from an object file
> +#

No licence..?


> +# -------------------------------------------------------------------------
> +import os
> +import sys
> +from optparse import OptionParser
> +import string
> +import json
> +
> +# For running from development directory. It should take precedence over the
> +# installed pyelftools.
> +sys.path.insert(0, '.')

Aside from causing all the subsequent imports to have PEP8 errors, this 
does not looks like a good way of pulling in project-specific Python 
library installs. Usual method is either using virtualenv or the 
PYTHONPATH enviornment variable.


> +from elftools import __version__
> +from elftools.common.exceptions import ELFError
[..]
> +from elftools.dwarf.constants import (
> +    DW_LNS_copy, DW_LNS_set_file, DW_LNE_define_file)
> +from elftools.dwarf.callframe import CIE, FDE

According to PyLint, most of these imports are unused.


> +
> +
> +class Vendor:

Old style class definition. Using modern notation it is:

class Vendor(object):


> +    def report(self):
> +        print "\t%s\t%s" % (self.ID, self.name)
> +        for subID, subdev in self.subdevices.items():
> +            subdev.report()

subID unused. An underscore can be used as a placeholder for these types 
of loops.


> +    optparser.add_option("-t", "--table", dest="tblout",
> +                         help="output information on hw support as a hex table",

PEP8: pmdinfo.py:573:80: E501 line too long (80 > 79 characters)

Re: [PATCHv8 5/6] pmdinfo.py: Add tool to query binaries for hw and other support information

[Neil Horman]

On Wed, Jun 29, 2016 at 04:12:21PM +0100, Remy Horton wrote:
> 'noon,
> 
> The tool does not work for static PMD libraries (e.g. librte_pmd_i40e.a) -
> is this an intended limitation?
> 
Yes, because .a files are archives, not ELF files.  If you want to run pmdinfo
on an archive, you have to unarchive it first (ar x).  It would be the same if
you had bound them together with tar or zip.

> DPDK doesn't to my knowledge have any coding guidelines for Python, so the
> comments below should be considered advisory rather than merge-blocking
> issues.
> 
> 
> On 17/06/2016 19:46, Neil Horman wrote:
> [..]
> > +++ b/tools/pmdinfo.py
> > @@ -0,0 +1,629 @@
> > +#!/usr/bin/python
> > +# -------------------------------------------------------------------------
> > +# scripts/pmdinfo.py
> > +#
> > +# Utility to dump PMD_INFO_STRING support from an object file
> > +#
> 
> No licence..?
> 
Its BSD, same as the others, I let the README cover that.  We can include
it if we must, though we have lots of examples where we haven't bothered


> 
> > +# -------------------------------------------------------------------------
> > +import os
> > +import sys
> > +from optparse import OptionParser
> > +import string
> > +import json
> > +
> > +# For running from development directory. It should take precedence over the
> > +# installed pyelftools.
> > +sys.path.insert(0, '.')
> 
> Aside from causing all the subsequent imports to have PEP8 errors, this does
> not looks like a good way of pulling in project-specific Python library
> installs. Usual method is either using virtualenv or the PYTHONPATH
> enviornment variable.
> 
Nope, pep8 doesn't complain about this at all:
[nhorman@hmsreliant tools]$ pep8 ./pmdinfo.py 
pmdinfo.py:573:80: E501 line too long (80 > 79 characters)
[nhorman@hmsreliant tools]$ 

The line too long snuck in there recently on the last round of formatting errors


> 
> > +from elftools import __version__
> > +from elftools.common.exceptions import ELFError
> [..]
> > +from elftools.dwarf.constants import (
> > +    DW_LNS_copy, DW_LNS_set_file, DW_LNE_define_file)
> > +from elftools.dwarf.callframe import CIE, FDE
> 
> According to PyLint, most of these imports are unused.
Some of them are, the code was borrowed from some examples.

> 
> 
> > +
> > +
> > +class Vendor:
> 
> Old style class definition. Using modern notation it is:
> 
> class Vendor(object):
> 
> 
> > +    def report(self):
> > +        print "\t%s\t%s" % (self.ID, self.name)
> > +        for subID, subdev in self.subdevices.items():
> > +            subdev.report()
> 
> subID unused. An underscore can be used as a placeholder for these types of
> loops.
> 
> 
> > +    optparser.add_option("-t", "--table", dest="tblout",
> > +                         help="output information on hw support as a hex table",
> 
> PEP8: pmdinfo.py:573:80: E501 line too long (80 > 79 characters)
> 
As you note, none of these are catastrophic.  I'm willing to fix them up, but
given, the number of iterations I've gone through for minior nits, I would
prefer to see it incorporated before I post this series again.

Neil

> 
> 

Re: [PATCHv8 5/6] pmdinfo.py: Add tool to query binaries for hw and other support information

[Remy Horton]

On 29/06/2016 17:18, Neil Horman wrote:
> On Wed, Jun 29, 2016 at 04:12:21PM +0100, Remy Horton wrote:
>> 'noon,
[..]
>> No licence..?
>>
> Its BSD, same as the others, I let the README cover that.  We can include
> it if we must, though we have lots of examples where we haven't bothered

Ok. The DPDK repo has a mix of BSD and GPL licenced files, so probably 
bit more critical than in other projects.


> Nope, pep8 doesn't complain about this at all:
> [nhorman@hmsreliant tools]$ pep8 ./pmdinfo.py
> pmdinfo.py:573:80: E501 line too long (80 > 79 characters)
> [nhorman@hmsreliant tools]$

Hmm..

[remy@VM tools]$ pep8 pmdinfo.py
pmdinfo.py:19:1: E402 module level import not at top of file
pmdinfo.py:20:1: E402 module level import not at top of file
pmdinfo.py:21:1: E402 module level import not at top of file
pmdinfo.py:23:1: E402 module level import not at top of file
pmdinfo.py:24:1: E402 module level import not at top of file
pmdinfo.py:25:1: E402 module level import not at top of file
pmdinfo.py:26:1: E402 module level import not at top of file
pmdinfo.py:27:1: E402 module level import not at top of file
pmdinfo.py:28:1: E402 module level import not at top of file
pmdinfo.py:32:1: E402 module level import not at top of file
pmdinfo.py:33:1: E402 module level import not at top of file
pmdinfo.py:42:1: E402 module level import not at top of file
pmdinfo.py:43:1: E402 module level import not at top of file
pmdinfo.py:44:1: E402 module level import not at top of file
pmdinfo.py:49:1: E402 module level import not at top of file
pmdinfo.py:51:1: E402 module level import not at top of file
pmdinfo.py:573:80: E501 line too long (80 > 79 characters)
[remy@VM tools]$ pep8 --version
1.7.0

To be fair the Python lot seem to add bucketloads of extra checks to 
PEP8/PyLint with every release.


> As you note, none of these are catastrophic.  I'm willing to fix them up, but
> given, the number of iterations I've gone through for minior nits, I would
> prefer to see it incorporated before I post this series again.

I agree. Merge first and ask questions later.. :)

..Remy

Re: [PATCHv8 0/6] Implement pmd hardware support exports

[Remy Horton]

On 17/06/2016 19:46, Neil Horman wrote:
> Hey all-
> 	So heres attempt number 2 at a method for exporting PMD hardware support
> information.  As we discussed previously, the consensus seems to be that pmd
> information should be:
[..]
> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> Acked-by: Panu Matilainen <pmatilai@redhat.com>
> CC: Bruce Richardson <bruce.richardson@intel.com>
> CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> CC: Stephen Hemminger <stephen@networkplumber.org>
> CC: Panu Matilainen <pmatilai@redhat.com>

Acked-by: Remy Horton <remy.horton@intel.com>

[PATCH v9 0/7] export PMD infos

[Thomas Monjalon]

This is a respin of the series from Neil.
It was planned to be integrated in 16.07-rc1 but the discovered issues
make a new revision needed.
There are still few things which could be improved but it is not mandatory
to fix them for an integration in 16.07-rc2:
  - fix make clean after pmdinfogen
  - build installable pmdinfogen for target
  - convert pmdinfo.py to Python 3
  - document dependency pyelftools

Changes done in this v9:
  - fix build dependency of drivers on pmdinfogen
  - fix build of mlx4, mlx5, aesni
  - fix new drivers bnxt, thunderx, kasumi
  - fix MAINTAINERS file
  - fix coding style in pmdinfogen
  - add compiler checks for pmdinfogen
  - remove useless functions in pmdinfogen
  - fail build if pmdinfogen fails (set -e)
  - fix verbose pmdinfogen run
  - build pmdinfogen in buildtools directory (was app)
  - install pmdinfogen in sdk package (was runtime)
  - fix CamelCase in pmdinfo.py
  - prefix executables with dpdk-
  - rename PMD_REGISTER_DRIVER -> RTE_REGISTER_DRIVER
  - separate commit for hostapp.mk refresh
  - remove useless hostlib.mk
  - spread doc in appropriate patches

Please review carefully.

Older changes:

v8)
 * Modified symlink for pmdinfo to use dpdk_
 * Added dpdk_pmdinfogen for pmdinfogen binary

v7)
 * Fixed up copyright
 * Switched buildtool makefile to use hostapp.mk
 * Modified arch check to use RTE_ARCH_64
 * Modifed hostapp.mk to drop output in build/app
 * Additional testing on ppc64 to ensure big endian works
 * Modified TO_NATIVE macro to use rte_byteorder inlines 
   based on endianess of target ELF file
 * Ran checkpatch on commits
 * Fixed some typos in doc

v6)
 * Added some programming guide documentation
 * Reformatted python script with pep8

v5)
 * Added a dpdk- prefix to pmdinfo symlink
 * Renamed build announcement to PMDINFOGEN/BUILD
 * Removed erroneous true statement from makefile
 * Removed duplicate rte.hostapp.mk makefile
 * Fixed some whitespace
 * Whitespace fixups
 * Fixed makefile if; then style
 * Renamed module string C file
 * Removed duplicate rte_pci_id definition
 * Clarified macro names
 * Removed PMD type attribute
 * Fixed tools usage for 32 bit arches
 * Removed some unused code
 * Added a few comments

v4)
 * Modified the operation of the -p option. As much as I don't like implying
that autoloaded pmds are guaranteed to be there at run time, I'm having a hard
time seeing how we can avoid specifying the application file to scan for the
autoload directory.  Without it we can't determine which library the user means
in a multiversion installation
 * Cleaned up the help text
 * Added a rule for an install target for pmdinfo
 * Guarded against some tracebacks in pmdinfo
 * Use DT_NEEDED entries to get versioned libraries in -p mode
 * Fixed traceback that occurs on lack of input arguments
 * Fixed some erroneous macro usage in drivers that aren't in the default build

v3)
 * Made table mode printing the default mode
 * Added a default path to the pci.ids file
 * Modifed pmdinfo to use python rather than python3
 * Cleaned up some changelog entries
 * Added an export for RTE_EAL_PMD_PATH
 * Added a plugin option to pmdinfo to scan for autoloaded DSO's

v2)
 * Made the export macros a bit easier to expand
 * Added a macro to optionally export command line strings
 * Renamed utilties to be more appropriate
   (pmdinfo -> pmdinfogen, pmd_hw_support.py -> pmdinfo.py)
 * Added search capabilities to pmdinfo.py so that we search for libraries
   linked using DT_NEEDINFO entries.  We search DT_RUNPATH, LD_LIBRARY_PATH,
   /usr/lib and /lib
 * Added serch abilities if the specified binary to pmdinfo isn't found, we
   search LD_LIBRARY_PATH, /usr/lib and /lib
 * Added an option to pmdinfo.py to pretty-print hardware support info using the
   pci.ids database

----------------
Original summary
----------------

        So heres attempt number 2 at a method for exporting PMD hardware support
information.  As we discussed previously, the consensus seems to be that pmd
information should be:

1) Able to be interrogated on any ELF binary (application binary or individual
DSO)
2) Equally functional on statically linked applications or on DSO's
3) Resilient to symbol stripping
4) Script friendly
5) Show kernel dependencies
6) List driver options
7) Show driver name
8) Offer human readable output
9) Show DPDK version
10) Show driver version
11) Allow for expansion
12) Not place additional build environment dependencies on an application

I added item 12 myself, because I don't think its reasonable to require
applications to use a specific linker script to get hardware support information
(which precludes the use of a special .modinfo section like the kernel uses)

However, we still can use some tricks from the kernel to make this work.  In
this approach, what I've done is the following:

A) Modified the driver registration macro to also define a variable:
this_pmd_name<n>= "name"

Based on the unique name string pointed to by the above variable, we can
query for an arbitrary number of other symbols following the pattern:
<name>_<tag>

Where tag is some well known identifier for information we wish to export

B) Added a utility called pmdinfogen.  This utility is not meant for general use,
but rather is used by the dpdk build environment itself when compiling pmds.
for each object that uses the PMD_REGISTER_DRIVER macro, this utiity is run.  It
searches for the symbols in (A), and using those, extracts the hardware support
info, and module name from the object, using that to produce a new c file
containing a single variable in the following format:

static const char <name>[] __attribute__((used)) = "PMD_DRIVER_INFO=<json>";

The <name> is arbitrary, as its static and not referenced.  The relevant bit is
the string value assigned to it.  The <json> is a json encoded string of the
extracted hardware support information pmdinfo found for the corresponding
object.  This C file is suitable for compilation and relocatable linking back
into the parent object file.  The result of this operation is that the object
string table now contains a string that will not e removed by stripping, whos
leading text (PMD_DRIVER_INFO) can be easily searched for at any time weather
the symbol referring to it is stripped or not.

C) Added a utilty called pmdinfo.py.  This python script, searches the
string table of the .rodata section of any provided ELF file looking for the
PMD_DRIVER_INFO prefix on a string.  When found, it will interpret the remainder
of the string as json, and output the hardware support for that ELF file (if
any).


This approach ticks most of the above boxes:
1) Impervious to stripping
2) Works on static and dynamic binaries
3) Human and script friendly
4) Allows for expansion

Because of (4) the other items should be pretty easy to implement, as its just a
matter of modifying the macros to export the info, pmdinfo to encode it to json,
and pmd_hw_support.py to read it.  I'm not adding them now, as theres alot of
rote work to do to get some of them in place (e.g. DPDK has no current global
VERSION macro, drivers don't have a consistent version scheme, command line
strings have to be built for each driver, etc).  But once this is accepted,
those items can be done as time allows and should be fairly easy to implement.

----------------

Neil Horman (4):
  drivers: export infos as string symbols
  pmdinfogen: parse driver to generate code to export
  eal: export default plugin path to external tools
  tools: query binaries for support information

Thomas Monjalon (3):
  mk: remove recipe for tool library
  mk: refresh recipe for any host application
  mk: link infos generated by pmdinfogen

 GNUmakefile                                    |   2 +-
 MAINTAINERS                                    |   5 +
 GNUmakefile => buildtools/Makefile             |  17 +-
 GNUmakefile => buildtools/pmdinfogen/Makefile  |  21 +-
 buildtools/pmdinfogen/pmdinfogen.c             | 451 ++++++++++++++++++
 buildtools/pmdinfogen/pmdinfogen.h             |  99 ++++
 doc/guides/freebsd_gsg/build_dpdk.rst          |   2 +-
 doc/guides/linux_gsg/build_dpdk.rst            |   2 +-
 doc/guides/prog_guide/dev_kit_build_system.rst |  38 +-
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c       |   4 +-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c     |   4 +-
 drivers/crypto/kasumi/rte_kasumi_pmd.c         |   4 +-
 drivers/crypto/null/null_crypto_pmd.c          |   4 +-
 drivers/crypto/qat/rte_qat_cryptodev.c         |   3 +-
 drivers/crypto/snow3g/rte_snow3g_pmd.c         |   4 +-
 drivers/net/af_packet/rte_eth_af_packet.c      |   4 +-
 drivers/net/bnx2x/bnx2x_ethdev.c               |   6 +-
 drivers/net/bnxt/bnxt_ethdev.c                 |   3 +-
 drivers/net/bonding/rte_eth_bond_pmd.c         |   5 +-
 drivers/net/cxgbe/cxgbe_ethdev.c               |   3 +-
 drivers/net/e1000/em_ethdev.c                  |   3 +-
 drivers/net/e1000/igb_ethdev.c                 |   6 +-
 drivers/net/ena/ena_ethdev.c                   |   3 +-
 drivers/net/enic/enic_ethdev.c                 |   3 +-
 drivers/net/fm10k/fm10k_ethdev.c               |   3 +-
 drivers/net/i40e/i40e_ethdev.c                 |   3 +-
 drivers/net/i40e/i40e_ethdev_vf.c              |   3 +-
 drivers/net/ixgbe/ixgbe_ethdev.c               |   6 +-
 drivers/net/mlx4/mlx4.c                        |   3 +-
 drivers/net/mlx5/mlx5.c                        |   3 +-
 drivers/net/mpipe/mpipe_tilegx.c               |   4 +-
 drivers/net/nfp/nfp_net.c                      |   3 +-
 drivers/net/null/rte_eth_null.c                |   3 +-
 drivers/net/pcap/rte_eth_pcap.c                |   4 +-
 drivers/net/qede/qede_ethdev.c                 |   6 +-
 drivers/net/ring/rte_eth_ring.c                |   3 +-
 drivers/net/szedata2/rte_eth_szedata2.c        |   3 +-
 drivers/net/thunderx/nicvf_ethdev.c            |   3 +-
 drivers/net/vhost/rte_eth_vhost.c              |   3 +-
 drivers/net/virtio/virtio_ethdev.c             |   3 +-
 drivers/net/virtio/virtio_user_ethdev.c        |   2 +-
 drivers/net/vmxnet3/vmxnet3_ethdev.c           |   3 +-
 drivers/net/xenvirt/rte_eth_xenvirt.c          |   2 +-
 lib/librte_eal/common/eal_common_dev.c         |   2 +-
 lib/librte_eal/common/eal_common_options.c     |   4 +
 lib/librte_eal/common/include/rte_dev.h        |  30 +-
 mk/internal/rte.compile-pre.mk                 |  12 +
 mk/rte.hostapp.mk                              |  23 +-
 mk/rte.hostlib.mk                              | 116 -----
 mk/rte.sdkbuild.mk                             |   4 +-
 mk/rte.sdkconfig.mk                            |   3 +-
 mk/rte.sdkinstall.mk                           |   5 +
 tools/dpdk-pmdinfo.py                          | 628 +++++++++++++++++++++++++
 53 files changed, 1371 insertions(+), 215 deletions(-)
 copy GNUmakefile => buildtools/Makefile (87%)
 copy GNUmakefile => buildtools/pmdinfogen/Makefile (84%)
 create mode 100644 buildtools/pmdinfogen/pmdinfogen.c
 create mode 100644 buildtools/pmdinfogen/pmdinfogen.h
 delete mode 100644 mk/rte.hostlib.mk
 create mode 100755 tools/dpdk-pmdinfo.py

-- 
2.7.0

[PATCH v9 1/7] drivers: export infos as string symbols

[Thomas Monjalon]

From: Neil Horman <nhorman@tuxdriver.com>

Modify the PMD_REGISTER_DRIVER macro, adding a name argument to it.  The
addition of a name argument creates a token that can be used for subsequent
macros in the creation of unique symbol names to export additional bits of
information for use by the dpdk-pmdinfogen tool.  For example:

    RTE_REGISTER_DRIVER(ena, ena_driver);
registers the ena_driver struct as PMD_REGISTER_DRIVER did, and creates
a symbol
    const char rte_pmd_name0[] __attribute__((used)) = "ena";
which dpdk-pmdinfogen can search for and extract.  The subsequent macro
    RTE_EXPORT_PCI_TABLE(ena, ena_pci_id_map);
creates a symbol
    const char ena_pci_table_export[] __attribute__((used)) = "ena_pci_id_map";
Which allows dpdk-pmdinfogen to find the pci table of this driver.

Using this pattern, we can export arbitrary bits of information.

dpdk-pmdinfo uses this information to extract hardware support from an object
file and create a json string to make hardware support info discoverable
later.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Panu Matilainen <pmatilai@redhat.com>
Acked-by: Remy Horton <remy.horton@intel.com>
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
---
 doc/guides/prog_guide/dev_kit_build_system.rst | 19 ++++++++++++++++
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c       |  4 +++-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c     |  4 +++-
 drivers/crypto/kasumi/rte_kasumi_pmd.c         |  4 +++-
 drivers/crypto/null/null_crypto_pmd.c          |  4 +++-
 drivers/crypto/qat/rte_qat_cryptodev.c         |  3 ++-
 drivers/crypto/snow3g/rte_snow3g_pmd.c         |  4 +++-
 drivers/net/af_packet/rte_eth_af_packet.c      |  4 +++-
 drivers/net/bnx2x/bnx2x_ethdev.c               |  6 ++++--
 drivers/net/bnxt/bnxt_ethdev.c                 |  3 ++-
 drivers/net/bonding/rte_eth_bond_pmd.c         |  5 ++++-
 drivers/net/cxgbe/cxgbe_ethdev.c               |  3 ++-
 drivers/net/e1000/em_ethdev.c                  |  3 ++-
 drivers/net/e1000/igb_ethdev.c                 |  6 ++++--
 drivers/net/ena/ena_ethdev.c                   |  3 ++-
 drivers/net/enic/enic_ethdev.c                 |  3 ++-
 drivers/net/fm10k/fm10k_ethdev.c               |  3 ++-
 drivers/net/i40e/i40e_ethdev.c                 |  3 ++-
 drivers/net/i40e/i40e_ethdev_vf.c              |  3 ++-
 drivers/net/ixgbe/ixgbe_ethdev.c               |  6 ++++--
 drivers/net/mlx4/mlx4.c                        |  3 ++-
 drivers/net/mlx5/mlx5.c                        |  3 ++-
 drivers/net/mpipe/mpipe_tilegx.c               |  4 ++--
 drivers/net/nfp/nfp_net.c                      |  3 ++-
 drivers/net/null/rte_eth_null.c                |  3 ++-
 drivers/net/pcap/rte_eth_pcap.c                |  4 +++-
 drivers/net/qede/qede_ethdev.c                 |  6 ++++--
 drivers/net/ring/rte_eth_ring.c                |  3 ++-
 drivers/net/szedata2/rte_eth_szedata2.c        |  3 ++-
 drivers/net/thunderx/nicvf_ethdev.c            |  3 ++-
 drivers/net/vhost/rte_eth_vhost.c              |  3 ++-
 drivers/net/virtio/virtio_ethdev.c             |  3 ++-
 drivers/net/virtio/virtio_user_ethdev.c        |  2 +-
 drivers/net/vmxnet3/vmxnet3_ethdev.c           |  3 ++-
 drivers/net/xenvirt/rte_eth_xenvirt.c          |  2 +-
 lib/librte_eal/common/eal_common_dev.c         |  2 +-
 lib/librte_eal/common/include/rte_dev.h        | 30 ++++++++++++++++++++------
 37 files changed, 127 insertions(+), 46 deletions(-)

diff --git a/doc/guides/prog_guide/dev_kit_build_system.rst b/doc/guides/prog_guide/dev_kit_build_system.rst
index 3e89eae..f3a4bff 100644
--- a/doc/guides/prog_guide/dev_kit_build_system.rst
+++ b/doc/guides/prog_guide/dev_kit_build_system.rst
@@ -304,6 +304,25 @@ Misc
 
 *   rte.subdir.mk: Build several directories in the development kit framework.
 
+.. _Internally_Generated_Build_Tools:
+
+Internally Generated Build Tools
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+These well known symbol names are defined by various macros and used to export
+important information about hardware support and usage for PMD files.  For
+instance the macro:
+
+.. code-block:: c
+
+   RTE_REGISTER_DRIVER(drv, name)
+
+Creates the following symbol:
+
+.. code-block:: c
+
+   static char rte_pmd_name0[] __attribute__((used)) = "<name>";
+
 .. _Useful_Variables_Provided_by_the_Build_System:
 
 Useful Variables Provided by the Build System
diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
index 2987ef6..3e13eae 100644
--- a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
+++ b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
@@ -521,4 +521,6 @@ static struct rte_driver aesni_gcm_pmd_drv = {
 	.uninit = aesni_gcm_uninit
 };
 
-PMD_REGISTER_DRIVER(aesni_gcm_pmd_drv);
+RTE_REGISTER_DRIVER(aesni_gcm, aesni_gcm_pmd_drv);
+RTE_EXPORT_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+"max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
index 6554fc4..2afc2bb 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
@@ -721,4 +721,6 @@ static struct rte_driver cryptodev_aesni_mb_pmd_drv = {
 	.uninit = cryptodev_aesni_mb_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drv);
+RTE_REGISTER_DRIVER(aesni_mb, cryptodev_aesni_mb_pmd_drv);
+RTE_EXPORT_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+"max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/kasumi/rte_kasumi_pmd.c b/drivers/crypto/kasumi/rte_kasumi_pmd.c
index 5f8c7a2..3dda257 100644
--- a/drivers/crypto/kasumi/rte_kasumi_pmd.c
+++ b/drivers/crypto/kasumi/rte_kasumi_pmd.c
@@ -654,4 +654,6 @@ static struct rte_driver cryptodev_kasumi_pmd_drv = {
 	.uninit = cryptodev_kasumi_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_kasumi_pmd_drv);
+RTE_REGISTER_DRIVER(kasumi, cryptodev_kasumi_pmd_drv);
+RTE_EXPORT_PARAM_STRING(kasumi, "max_nb_queue_pairs=<int> "
+"max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/null/null_crypto_pmd.c b/drivers/crypto/null/null_crypto_pmd.c
index bdaf13c..003dce7 100644
--- a/drivers/crypto/null/null_crypto_pmd.c
+++ b/drivers/crypto/null/null_crypto_pmd.c
@@ -275,4 +275,6 @@ static struct rte_driver cryptodev_null_pmd_drv = {
 	.uninit = cryptodev_null_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_null_pmd_drv);
+RTE_REGISTER_DRIVER(cryptodev_null_pmd, cryptodev_null_pmd_drv);
+RTE_EXPORT_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+"max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index f46ec85..d9cd10a 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -134,4 +134,5 @@ static struct rte_driver pmd_qat_drv = {
 	.init = rte_qat_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(pmd_qat_drv);
+RTE_REGISTER_DRIVER(qat, pmd_qat_drv);
+RTE_EXPORT_PCI_TABLE(qat, pci_id_qat_map);
diff --git a/drivers/crypto/snow3g/rte_snow3g_pmd.c b/drivers/crypto/snow3g/rte_snow3g_pmd.c
index dc8de6b..e531408 100644
--- a/drivers/crypto/snow3g/rte_snow3g_pmd.c
+++ b/drivers/crypto/snow3g/rte_snow3g_pmd.c
@@ -645,4 +645,6 @@ static struct rte_driver cryptodev_snow3g_pmd_drv = {
 	.uninit = cryptodev_snow3g_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_snow3g_pmd_drv);
+RTE_REGISTER_DRIVER(snow3g, cryptodev_snow3g_pmd_drv);
+RTE_EXPORT_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+"max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index 2d7f344..9a9547c 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -896,4 +896,6 @@ static struct rte_driver pmd_af_packet_drv = {
 	.uninit = rte_pmd_af_packet_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_af_packet_drv);
+RTE_REGISTER_DRIVER(eth_af_packet, pmd_af_packet_drv);
+RTE_EXPORT_PARAM_STRING(eth_af_packet, "iface=<string> "
+"qpairs=<int> blocksz=<int> framesz=<int> framecnt=<int>");
diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c b/drivers/net/bnx2x/bnx2x_ethdev.c
index 3ff57c4..ec0d88d 100644
--- a/drivers/net/bnx2x/bnx2x_ethdev.c
+++ b/drivers/net/bnx2x/bnx2x_ethdev.c
@@ -566,5 +566,7 @@ static struct rte_driver rte_bnx2xvf_driver = {
 	.init = rte_bnx2xvf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_bnx2x_driver);
-PMD_REGISTER_DRIVER(rte_bnx2xvf_driver);
+RTE_REGISTER_DRIVER(bnx2x,rte_bnx2x_driver);
+RTE_EXPORT_PCI_TABLE(bnx2x, pci_id_bnx2x_map);
+RTE_REGISTER_DRIVER(bnx2xvf, rte_bnx2xvf_driver);
+RTE_EXPORT_PCI_TABLE(bnx2xvf, pci_id_bnx2xvf_map);
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 406e38a..2e0801f 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -1046,4 +1046,5 @@ static struct rte_driver bnxt_pmd_drv = {
 	.init = bnxt_rte_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(bnxt_pmd_drv);
+RTE_REGISTER_DRIVER(bnxt, bnxt_pmd_drv);
+RTE_EXPORT_PCI_TABLE(bnxt, bnxt_pci_id_map);
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 9a2518f..579e67e 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -2515,4 +2515,7 @@ static struct rte_driver bond_drv = {
 	.uninit = bond_uninit,
 };
 
-PMD_REGISTER_DRIVER(bond_drv);
+RTE_REGISTER_DRIVER(bonding, bond_drv);
+RTE_EXPORT_PARAM_STRING(bonding, "slave=<ifc> primary=<ifc> mode=[0-4] "
+"xmit_policy=[l2 | l23 | l34] socket_id=<int> mac=<mac addr> "
+"lsc_poll_period_ms=<int> up_delay=<int> down_delay=<int>");
diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index 6c130ed..7e12617 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -1061,4 +1061,5 @@ static struct rte_driver rte_cxgbe_driver = {
 	.init = rte_cxgbe_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_cxgbe_driver);
+RTE_REGISTER_DRIVER(cxgb4, rte_cxgbe_driver);
+RTE_EXPORT_PCI_TABLE(cxgb4, cxgb4_pci_tbl);
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index 653be09..d0a104e 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -1777,4 +1777,5 @@ struct rte_driver em_pmd_drv = {
 	.init = rte_em_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(em_pmd_drv);
+RTE_REGISTER_DRIVER(em, em_pmd_drv);
+RTE_EXPORT_PCI_TABLE(em, pci_id_em_map);
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index 5067d20..164aae5 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -5210,5 +5210,7 @@ eth_igb_configure_msix_intr(struct rte_eth_dev *dev)
 	E1000_WRITE_FLUSH(hw);
 }
 
-PMD_REGISTER_DRIVER(pmd_igb_drv);
-PMD_REGISTER_DRIVER(pmd_igbvf_drv);
+RTE_REGISTER_DRIVER(igb, pmd_igb_drv);
+RTE_EXPORT_PCI_TABLE(igb, pci_id_igb_map);
+RTE_REGISTER_DRIVER(igbvf, pmd_igbvf_drv);
+RTE_EXPORT_PCI_TABLE(igbvf, pci_id_igbvf_map);
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index e157587..82ade83 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -1450,4 +1450,5 @@ struct rte_driver ena_pmd_drv = {
 	.init = rte_ena_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(ena_pmd_drv);
+RTE_REGISTER_DRIVER(ena, ena_pmd_drv);
+RTE_EXPORT_PCI_TABLE(ena, pci_id_ena_map);
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index a7ce064..4e62282 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -636,4 +636,5 @@ static struct rte_driver rte_enic_driver = {
 	.init = rte_enic_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_enic_driver);
+RTE_REGISTER_DRIVER(enic, rte_enic_driver);
+RTE_EXPORT_PCI_TABLE(enic, pci_id_enic_map);
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index eb77705..62f7d41 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -3086,4 +3086,5 @@ static struct rte_driver rte_fm10k_driver = {
 	.init = rte_pmd_fm10k_init,
 };
 
-PMD_REGISTER_DRIVER(rte_fm10k_driver);
+RTE_REGISTER_DRIVER(fm10k, rte_fm10k_driver);
+RTE_EXPORT_PCI_TABLE(fm10k, pci_id_fm10k_map);
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index f414d93..8fc0852 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -705,7 +705,8 @@ static struct rte_driver rte_i40e_driver = {
 	.init = rte_i40e_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_i40e_driver);
+RTE_REGISTER_DRIVER(i40e, rte_i40e_driver);
+RTE_EXPORT_PCI_TABLE(i40e, pci_id_i40e_map);
 
 /*
  * Initialize registers for flexible payload, which should be set by NVM.
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 7b6df1d..27b7b57 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1581,7 +1581,8 @@ static struct rte_driver rte_i40evf_driver = {
 	.init = rte_i40evf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_i40evf_driver);
+RTE_REGISTER_DRIVER(i40evf, rte_i40evf_driver);
+RTE_EXPORT_PCI_TABLE(i40evf, pci_id_i40evf_map);
 
 static int
 i40evf_dev_configure(struct rte_eth_dev *dev)
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 0629b42..014b972 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -7352,5 +7352,7 @@ static struct rte_driver rte_ixgbevf_driver = {
 	.init = rte_ixgbevf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_ixgbe_driver);
-PMD_REGISTER_DRIVER(rte_ixgbevf_driver);
+RTE_REGISTER_DRIVER(ixgbe, rte_ixgbe_driver);
+RTE_EXPORT_PCI_TABLE(ixgbe, pci_id_ixgbe_map);
+RTE_REGISTER_DRIVER(ixgbevf, rte_ixgbevf_driver);
+RTE_EXPORT_PCI_TABLE(ixgbevf, pci_id_ixgbevf_map);
diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index f8ed42b..1681fc3 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -5861,4 +5861,5 @@ static struct rte_driver rte_mlx4_driver = {
 	.init = rte_mlx4_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_mlx4_driver)
+RTE_REGISTER_DRIVER(mlx4, rte_mlx4_driver);
+RTE_EXPORT_PCI_TABLE(mlx4, mlx4_pci_id_map);
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 5aa4adc..5f4186a 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -762,4 +762,5 @@ static struct rte_driver rte_mlx5_driver = {
 	.init = rte_mlx5_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_mlx5_driver)
+RTE_REGISTER_DRIVER(mlx5, rte_mlx5_driver);
+RTE_EXPORT_PCI_TABLE(mlx5, mlx5_pci_id_map);
diff --git a/drivers/net/mpipe/mpipe_tilegx.c b/drivers/net/mpipe/mpipe_tilegx.c
index 26e1424..94f71ad 100644
--- a/drivers/net/mpipe/mpipe_tilegx.c
+++ b/drivers/net/mpipe/mpipe_tilegx.c
@@ -1635,8 +1635,8 @@ static struct rte_driver pmd_mpipe_gbe_drv = {
 	.init = rte_pmd_mpipe_devinit,
 };
 
-PMD_REGISTER_DRIVER(pmd_mpipe_xgbe_drv);
-PMD_REGISTER_DRIVER(pmd_mpipe_gbe_drv);
+RTE_REGISTER_DRIVER(mpipe_xgbe, pmd_mpipe_xgbe_drv);
+RTE_REGISTER_DRIVER(mpipe_gbe, pmd_mpipe_gbe_drv);
 
 static void __attribute__((constructor, used))
 mpipe_init_contexts(void)
diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
index 6afd49b..de0f961 100644
--- a/drivers/net/nfp/nfp_net.c
+++ b/drivers/net/nfp/nfp_net.c
@@ -2486,7 +2486,8 @@ static struct rte_driver rte_nfp_net_driver = {
 	.init = nfp_net_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_nfp_net_driver);
+RTE_REGISTER_DRIVER(nfp, rte_nfp_net_driver);
+RTE_EXPORT_PCI_TABLE(nfp, pci_id_nfp_net_map);
 
 /*
  * Local variables:
diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index ab440f3..e22b5c3 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -693,4 +693,5 @@ static struct rte_driver pmd_null_drv = {
 	.uninit = rte_pmd_null_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_null_drv);
+RTE_REGISTER_DRIVER(eth_null, pmd_null_drv);
+RTE_EXPORT_PARAM_STRING(eth_null, "size=<int> copy=<int>");
diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c
index c86f17b..2dd536e 100644
--- a/drivers/net/pcap/rte_eth_pcap.c
+++ b/drivers/net/pcap/rte_eth_pcap.c
@@ -1090,4 +1090,6 @@ static struct rte_driver pmd_pcap_drv = {
 	.uninit = rte_pmd_pcap_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_pcap_drv);
+RTE_REGISTER_DRIVER(pcap, pmd_pcap_drv);
+RTE_EXPORT_PARAM_STRING(pcap, "rx_pcap=<string> tx_pcap=<string> "
+"rx_iface=<ifc> tx_iface=<ifc> iface=<ifc>");
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index bb531be..bc07b99 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -1340,5 +1340,7 @@ static struct rte_driver rte_qede_driver = {
 	.init = rte_qedevf_pmd_init
 };
 
-PMD_REGISTER_DRIVER(rte_qede_driver);
-PMD_REGISTER_DRIVER(rte_qedevf_driver);
+RTE_REGISTER_DRIVER(qede, rte_qede_driver);
+RTE_EXPORT_PCI_TABLE(qede, pci_id_qede_map);
+RTE_REGISTER_DRIVER(qedevf, rte_qedevf_driver);
+RTE_EXPORT_PCI_TABLE(qedevf, pci_id_qedevf_map);
diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index b1783c3..7a11d78 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -630,4 +630,5 @@ static struct rte_driver pmd_ring_drv = {
 	.uninit = rte_pmd_ring_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_ring_drv);
+RTE_REGISTER_DRIVER(eth_ring, pmd_ring_drv);
+RTE_EXPORT_PARAM_STRING(eth_ring, "nodeaction=[attach|detach]");
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 985a8d6..d8564d9 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1601,4 +1601,5 @@ static struct rte_driver rte_szedata2_driver = {
 	.uninit = rte_szedata2_uninit,
 };
 
-PMD_REGISTER_DRIVER(rte_szedata2_driver);
+RTE_REGISTER_DRIVER(rte_szedata2_pmd, rte_szedata2_driver);
+RTE_EXPORT_PCI_TABLE(rte_szedata2_pmd, rte_szedata2_pci_id_table);
diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c
index 48ed381..4593251 100644
--- a/drivers/net/thunderx/nicvf_ethdev.c
+++ b/drivers/net/thunderx/nicvf_ethdev.c
@@ -1788,4 +1788,5 @@ static struct rte_driver rte_nicvf_driver = {
 	.init = rte_nicvf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_nicvf_driver);
+RTE_REGISTER_DRIVER(thunderx_nicvf, rte_nicvf_driver);
+RTE_EXPORT_PCI_TABLE(thunderx_nicvf, pci_id_nicvf_map);
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index c5bbb87..ea3a248 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -921,4 +921,5 @@ static struct rte_driver pmd_vhost_drv = {
 	.uninit = rte_pmd_vhost_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_vhost_drv);
+RTE_REGISTER_DRIVER(eth_vhost, pmd_vhost_drv);
+RTE_EXPORT_PARAM_STRING(eth_vhost, "iface=<ifc> queues=<int>");
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 480daa3..a884d77 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -1571,4 +1571,5 @@ static struct rte_driver rte_virtio_driver = {
 	.init = rte_virtio_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_virtio_driver);
+RTE_REGISTER_DRIVER(virtio_net, rte_virtio_driver);
+RTE_EXPORT_PCI_TABLE(virtio_net, pci_id_virtio_map);
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index 5ab2471..587de75 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -437,4 +437,4 @@ static struct rte_driver virtio_user_driver = {
 	.uninit = virtio_user_pmd_devuninit,
 };
 
-PMD_REGISTER_DRIVER(virtio_user_driver);
+RTE_REGISTER_DRIVER(virtio_user, virtio_user_driver);
diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index 29b469c..be4c404 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -954,4 +954,5 @@ static struct rte_driver rte_vmxnet3_driver = {
 	.init = rte_vmxnet3_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_vmxnet3_driver);
+RTE_REGISTER_DRIVER(vmxnet3, rte_vmxnet3_driver);
+RTE_EXPORT_PCI_TABLE(vmxnet3, pci_id_vmxnet3_map);
diff --git a/drivers/net/xenvirt/rte_eth_xenvirt.c b/drivers/net/xenvirt/rte_eth_xenvirt.c
index 3e45808..f28f5f0 100644
--- a/drivers/net/xenvirt/rte_eth_xenvirt.c
+++ b/drivers/net/xenvirt/rte_eth_xenvirt.c
@@ -766,4 +766,4 @@ static struct rte_driver pmd_xenvirt_drv = {
 	.uninit = rte_pmd_xenvirt_devuninit,
 };
 
-PMD_REGISTER_DRIVER(pmd_xenvirt_drv);
+RTE_REGISTER_DRIVER(xenvirt, pmd_xenvirt_drv);
diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c
index a8a4146..707e875 100644
--- a/lib/librte_eal/common/eal_common_dev.c
+++ b/lib/librte_eal/common/eal_common_dev.c
@@ -98,7 +98,7 @@ rte_eal_dev_init(void)
 	/*
 	 * Note that the dev_driver_list is populated here
 	 * from calls made to rte_eal_driver_register from constructor functions
-	 * embedded into PMD modules via the PMD_REGISTER_DRIVER macro
+	 * embedded into PMD modules via the RTE_REGISTER_DRIVER macro
 	 */
 
 	/* call the init function for each virtual device */
diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h
index f1b5507..1aa7656 100644
--- a/lib/librte_eal/common/include/rte_dev.h
+++ b/lib/librte_eal/common/include/rte_dev.h
@@ -48,7 +48,7 @@ extern "C" {
 
 #include <stdio.h>
 #include <sys/queue.h>
-
+#include <rte_pci.h>
 #include <rte_log.h>
 
 __attribute__((format(printf, 2, 0)))
@@ -178,12 +178,30 @@ int rte_eal_vdev_init(const char *name, const char *args);
  */
 int rte_eal_vdev_uninit(const char *name);
 
-#define PMD_REGISTER_DRIVER(d)\
-void devinitfn_ ##d(void);\
-void __attribute__((constructor, used)) devinitfn_ ##d(void)\
+#define _RTE_EXPORT_DRIVER_NAME(name, index) name##index[]
+
+#define RTE_EXPORT_DRIVER_NAME(name, index) \
+static const char _RTE_EXPORT_DRIVER_NAME(rte_pmd_name, index) \
+__attribute__((used)) = RTE_STR(name)
+
+#define RTE_REGISTER_DRIVER(_name, driver)\
+void devinitfn_ ##driver(void);\
+void __attribute__((constructor, used)) devinitfn_ ##driver(void)\
 {\
-	rte_eal_driver_register(&d);\
-}
+	(driver).name = RTE_STR(_name);\
+	rte_eal_driver_register(&driver);\
+} \
+RTE_EXPORT_DRIVER_NAME(_name, __COUNTER__)
+
+#define _RTE_EXPORT_TAG(name, tag) __##name##_##tag
+
+#define RTE_EXPORT_PCI_TABLE(name, table) \
+static const char _RTE_EXPORT_TAG(name, pci_table_export)[] \
+__attribute__((used)) = RTE_STR(table)
+
+#define RTE_EXPORT_PARAM_STRING(name, str) \
+static const char _RTE_EXPORT_TAG(name, param_string_export)[] \
+__attribute__((used)) = str
 
 #ifdef __cplusplus
 }
-- 
2.7.0

[PATCH v9 2/7] mk: remove recipe for tool library

[Thomas Monjalon]

It is difficult to imagine why it could be needed to build a
library for the build environment.
If building a tool, rte.hostapp.mk should be sufficient.

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
---
 doc/guides/prog_guide/dev_kit_build_system.rst |   2 -
 mk/rte.hostlib.mk                              | 116 -------------------------
 mk/rte.sdkbuild.mk                             |   2 +-
 mk/rte.sdkconfig.mk                            |   2 +-
 4 files changed, 2 insertions(+), 120 deletions(-)
 delete mode 100644 mk/rte.hostlib.mk

diff --git a/doc/guides/prog_guide/dev_kit_build_system.rst b/doc/guides/prog_guide/dev_kit_build_system.rst
index f3a4bff..9b0de83 100644
--- a/doc/guides/prog_guide/dev_kit_build_system.rst
+++ b/doc/guides/prog_guide/dev_kit_build_system.rst
@@ -275,8 +275,6 @@ Generate a .a library.
 
 *   rte.extlib.mk: external library
 
-*   rte.hostlib.mk: host library in the development kit framework
-
 Install
 ^^^^^^^
 
diff --git a/mk/rte.hostlib.mk b/mk/rte.hostlib.mk
deleted file mode 100644
index fe24049..0000000
--- a/mk/rte.hostlib.mk
+++ /dev/null
@@ -1,116 +0,0 @@
-#   BSD LICENSE
-#
-#   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
-#   All rights reserved.
-#
-#   Redistribution and use in source and binary forms, with or without
-#   modification, are permitted provided that the following conditions
-#   are met:
-#
-#     * Redistributions of source code must retain the above copyright
-#       notice, this list of conditions and the following disclaimer.
-#     * Redistributions in binary form must reproduce the above copyright
-#       notice, this list of conditions and the following disclaimer in
-#       the documentation and/or other materials provided with the
-#       distribution.
-#     * Neither the name of Intel Corporation nor the names of its
-#       contributors may be used to endorse or promote products derived
-#       from this software without specific prior written permission.
-#
-#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-# tell rte.compile-pre.mk to use HOSTCC instead of CC
-USE_HOST := 1
-include $(RTE_SDK)/mk/internal/rte.compile-pre.mk
-include $(RTE_SDK)/mk/internal/rte.install-pre.mk
-include $(RTE_SDK)/mk/internal/rte.clean-pre.mk
-include $(RTE_SDK)/mk/internal/rte.build-pre.mk
-include $(RTE_SDK)/mk/internal/rte.depdirs-pre.mk
-
-# VPATH contains at least SRCDIR
-VPATH += $(SRCDIR)
-
-_BUILD = $(HOSTLIB)
-_INSTALL = $(INSTALL-FILES-y) $(SYMLINK-FILES-y) $(RTE_OUTPUT)/hostlib/$(HOSTLIB)
-_CLEAN = doclean
-
-.PHONY: all
-all: install
-
-.PHONY: install
-install: build _postinstall
-
-_postinstall: build
-
-.PHONY: build
-build: _postbuild
-
-exe2cmd = $(strip $(call dotfile,$(patsubst %,%.cmd,$(1))))
-
-O_TO_A = $(AR) crus $(HOSTLIB) $(OBJS-y)
-O_TO_A_STR = $(subst ','\'',$(O_TO_A)) #'# fix syntax highlight
-O_TO_A_DISP = $(if $(V),"$(O_TO_A_STR)","  HOSTAR $(@)")
-O_TO_A_CMD = "cmd_$@ = $(O_TO_A_STR)"
-O_TO_A_DO = @set -e; \
-	echo $(O_TO_A_DISP); \
-	$(O_TO_A) && \
-	echo $(O_TO_A_CMD) > $(call exe2cmd,$(@))
-
--include .$(HOSTLIB).cmd
-
-#
-# Archive objects in .a file if needed
-#
-$(HOSTLIB): $(OBJS-y) FORCE
-	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
-	$(if $(D),\
-		@echo -n "$@ -> $< " ; \
-		echo -n "file_missing=$(call boolean,$(file_missing)) " ; \
-		echo -n "cmdline_changed=$(call boolean,$(call cmdline_changed,$(O_TO_A_STR))) " ; \
-		echo -n "depfile_missing=$(call boolean,$(depfile_missing)) " ; \
-		echo "depfile_newer=$(call boolean,$(depfile_newer)) ")
-	$(if $(or \
-		$(file_missing),\
-		$(call cmdline_changed,$(O_TO_A_STR)),\
-		$(depfile_missing),\
-		$(depfile_newer)),\
-		$(O_TO_A_DO))
-
-#
-# install lib in $(RTE_OUTPUT)/hostlib
-#
-$(RTE_OUTPUT)/hostlib/$(HOSTLIB): $(HOSTLIB)
-	@echo "  INSTALL-HOSTLIB $(HOSTLIB)"
-	@[ -d $(RTE_OUTPUT)/hostlib ] || mkdir -p $(RTE_OUTPUT)/hostlib
-	$(Q)cp -f $(HOSTLIB) $(RTE_OUTPUT)/hostlib
-
-#
-# Clean all generated files
-#
-.PHONY: clean
-clean: _postclean
-
-.PHONY: doclean
-doclean:
-	$(Q)rm -rf $(HOSTLIB) $(OBJS-all) $(DEPS-all) $(DEPSTMP-all) \
-	  $(CMDS-all) $(INSTALL-FILES-all)
-	$(Q)rm -f $(_BUILD_TARGETS) $(_INSTALL_TARGETS) $(_CLEAN_TARGETS)
-
-include $(RTE_SDK)/mk/internal/rte.compile-post.mk
-include $(RTE_SDK)/mk/internal/rte.install-post.mk
-include $(RTE_SDK)/mk/internal/rte.clean-post.mk
-include $(RTE_SDK)/mk/internal/rte.build-post.mk
-include $(RTE_SDK)/mk/internal/rte.depdirs-post.mk
-
-.PHONY: FORCE
-FORCE:
diff --git a/mk/rte.sdkbuild.mk b/mk/rte.sdkbuild.mk
index eec5241..6dbdb5d 100644
--- a/mk/rte.sdkbuild.mk
+++ b/mk/rte.sdkbuild.mk
@@ -64,7 +64,7 @@ build: $(ROOTDIRS-y)
 clean: $(CLEANDIRS)
 	@rm -rf $(RTE_OUTPUT)/include $(RTE_OUTPUT)/app \
 		$(RTE_OUTPUT)/hostapp $(RTE_OUTPUT)/lib \
-		$(RTE_OUTPUT)/hostlib $(RTE_OUTPUT)/kmod
+		$(RTE_OUTPUT)/kmod
 	@[ -d $(RTE_OUTPUT)/include ] || mkdir -p $(RTE_OUTPUT)/include
 	@$(RTE_SDK)/scripts/gen-config-h.sh $(RTE_OUTPUT)/.config \
 		> $(RTE_OUTPUT)/include/rte_config.h
diff --git a/mk/rte.sdkconfig.mk b/mk/rte.sdkconfig.mk
index a3acfe6..98789af 100644
--- a/mk/rte.sdkconfig.mk
+++ b/mk/rte.sdkconfig.mk
@@ -109,7 +109,7 @@ $(RTE_OUTPUT)/Makefile: | $(RTE_OUTPUT)
 $(RTE_OUTPUT)/include/rte_config.h: $(RTE_OUTPUT)/.config
 	$(Q)rm -rf $(RTE_OUTPUT)/include $(RTE_OUTPUT)/app \
 		$(RTE_OUTPUT)/hostapp $(RTE_OUTPUT)/lib \
-		$(RTE_OUTPUT)/hostlib $(RTE_OUTPUT)/kmod $(RTE_OUTPUT)/build
+		$(RTE_OUTPUT)/kmod $(RTE_OUTPUT)/build
 	$(Q)mkdir -p $(RTE_OUTPUT)/include
 	$(Q)$(RTE_SDK)/scripts/gen-config-h.sh $(RTE_OUTPUT)/.config \
 		> $(RTE_OUTPUT)/include/rte_config.h
-- 
2.7.0

[PATCH v9 3/7] mk: refresh recipe for any host application

[Thomas Monjalon]

Make the recipe simpler to use and build in configurable directory.
HOSTAPP_DIR must be set before including rte.hostapp.mk.

Remove LDLIBS_FILES as libraries should not be used in an hostapp.
Remove the "INSTALL-HOSTAPP" and build directly in the right directory.

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
---
 doc/guides/freebsd_gsg/build_dpdk.rst          |  2 +-
 doc/guides/linux_gsg/build_dpdk.rst            |  2 +-
 doc/guides/prog_guide/dev_kit_build_system.rst |  2 +-
 mk/rte.hostapp.mk                              | 23 +++++------------------
 mk/rte.sdkbuild.mk                             |  3 +--
 mk/rte.sdkconfig.mk                            |  3 +--
 6 files changed, 10 insertions(+), 25 deletions(-)

diff --git a/doc/guides/freebsd_gsg/build_dpdk.rst b/doc/guides/freebsd_gsg/build_dpdk.rst
index 1d92c08..93c4366 100644
--- a/doc/guides/freebsd_gsg/build_dpdk.rst
+++ b/doc/guides/freebsd_gsg/build_dpdk.rst
@@ -183,7 +183,7 @@ contains the kernel modules to install:
 
     ls x86_64-native-bsdapp-gcc
 
-    app build hostapp include kmod lib Makefile
+    app build include kmod lib Makefile
 
 
 .. _loading_contigmem:
diff --git a/doc/guides/linux_gsg/build_dpdk.rst b/doc/guides/linux_gsg/build_dpdk.rst
index 198c0b6..fb2c481 100644
--- a/doc/guides/linux_gsg/build_dpdk.rst
+++ b/doc/guides/linux_gsg/build_dpdk.rst
@@ -152,7 +152,7 @@ A kmod  directory is also present that contains kernel modules which may be load
 
     ls x86_64-native-linuxapp-gcc
 
-    app build hostapp include kmod lib Makefile
+    app build include kmod lib Makefile
 
 Loading Modules to Enable Userspace IO for DPDK
 -----------------------------------------------
diff --git a/doc/guides/prog_guide/dev_kit_build_system.rst b/doc/guides/prog_guide/dev_kit_build_system.rst
index 9b0de83..dedd18a 100644
--- a/doc/guides/prog_guide/dev_kit_build_system.rst
+++ b/doc/guides/prog_guide/dev_kit_build_system.rst
@@ -264,7 +264,7 @@ These Makefiles generate a binary application.
 
 *   rte.extapp.mk: External application
 
-*   rte.hostapp.mk: Host application in the development kit framework
+*   rte.hostapp.mk: Prerequisite tool to build DPDK
 
 Library
 ^^^^^^^
diff --git a/mk/rte.hostapp.mk b/mk/rte.hostapp.mk
index c44d0f8..05bbd26 100644
--- a/mk/rte.hostapp.mk
+++ b/mk/rte.hostapp.mk
@@ -40,8 +40,8 @@ include $(RTE_SDK)/mk/internal/rte.depdirs-pre.mk
 # VPATH contains at least SRCDIR
 VPATH += $(SRCDIR)
 
-_BUILD = $(HOSTAPP)
-_INSTALL = $(INSTALL-FILES-y) $(SYMLINK-FILES-y) $(RTE_OUTPUT)/hostapp/$(HOSTAPP)
+_BUILD = $(RTE_OUTPUT)/$(HOSTAPP_DIR)/$(HOSTAPP)
+_INSTALL = $(INSTALL-FILES-y) $(SYMLINK-FILES-y) $(RTE_OUTPUT)/$(HOSTAPP_DIR)/$(HOSTAPP)
 _CLEAN = doclean
 
 .PHONY: all
@@ -60,7 +60,7 @@ exe2cmd = $(strip $(call dotfile,$(patsubst %,%.cmd,$(1))))
 O_TO_EXE = $(HOSTCC) $(HOST_LDFLAGS) $(LDFLAGS_$(@)) \
 	$(EXTRA_HOST_LDFLAGS) -o $@ $(OBJS-y) $(LDLIBS)
 O_TO_EXE_STR = $(subst ','\'',$(O_TO_EXE)) #'# fix syntax highlight
-O_TO_EXE_DISP = $(if $(V),"$(O_TO_EXE_STR)","  HOSTLD $(@)")
+O_TO_EXE_DISP = $(if $(V),"$(O_TO_EXE_STR)","  HOSTLD $(@F)")
 O_TO_EXE_CMD = "cmd_$@ = $(O_TO_EXE_STR)"
 O_TO_EXE_DO = @set -e; \
 	echo $(O_TO_EXE_DISP); \
@@ -69,15 +69,10 @@ O_TO_EXE_DO = @set -e; \
 
 -include .$(HOSTAPP).cmd
 
-# list of .a files that are linked to this application
-LDLIBS_FILES := $(wildcard \
-	$(addprefix $(RTE_OUTPUT)/lib/, \
-	$(patsubst -l%,lib%.a,$(filter -l%,$(LDLIBS)))))
-
 #
 # Compile executable file if needed
 #
-$(HOSTAPP): $(OBJS-y) $(LDLIBS_FILES) FORCE
+$(RTE_OUTPUT)/$(HOSTAPP_DIR)/$(HOSTAPP): $(OBJS-y) FORCE
 	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
 	$(if $(D),\
 		@echo -n "$@ -> $< " ; \
@@ -93,14 +88,6 @@ $(HOSTAPP): $(OBJS-y) $(LDLIBS_FILES) FORCE
 		$(O_TO_EXE_DO))
 
 #
-# install app in $(RTE_OUTPUT)/hostapp
-#
-$(RTE_OUTPUT)/hostapp/$(HOSTAPP): $(HOSTAPP)
-	@echo "  INSTALL-HOSTAPP $(HOSTAPP)"
-	@[ -d $(RTE_OUTPUT)/hostapp ] || mkdir -p $(RTE_OUTPUT)/hostapp
-	$(Q)cp -f $(HOSTAPP) $(RTE_OUTPUT)/hostapp
-
-#
 # Clean all generated files
 #
 .PHONY: clean
@@ -109,7 +96,7 @@ clean: _postclean
 
 .PHONY: doclean
 doclean:
-	$(Q)rm -rf $(HOSTAPP) $(OBJS-all) $(DEPS-all) $(DEPSTMP-all) \
+	$(Q)rm -rf $(OBJS-all) $(DEPS-all) $(DEPSTMP-all) \
 	  $(CMDS-all) $(INSTALL-FILES-all) .$(HOSTAPP).cmd
 
 
diff --git a/mk/rte.sdkbuild.mk b/mk/rte.sdkbuild.mk
index 6dbdb5d..f1a163a 100644
--- a/mk/rte.sdkbuild.mk
+++ b/mk/rte.sdkbuild.mk
@@ -63,8 +63,7 @@ build: $(ROOTDIRS-y)
 .PHONY: clean
 clean: $(CLEANDIRS)
 	@rm -rf $(RTE_OUTPUT)/include $(RTE_OUTPUT)/app \
-		$(RTE_OUTPUT)/hostapp $(RTE_OUTPUT)/lib \
-		$(RTE_OUTPUT)/kmod
+		$(RTE_OUTPUT)/lib $(RTE_OUTPUT)/kmod
 	@[ -d $(RTE_OUTPUT)/include ] || mkdir -p $(RTE_OUTPUT)/include
 	@$(RTE_SDK)/scripts/gen-config-h.sh $(RTE_OUTPUT)/.config \
 		> $(RTE_OUTPUT)/include/rte_config.h
diff --git a/mk/rte.sdkconfig.mk b/mk/rte.sdkconfig.mk
index 98789af..44522ac 100644
--- a/mk/rte.sdkconfig.mk
+++ b/mk/rte.sdkconfig.mk
@@ -108,8 +108,7 @@ $(RTE_OUTPUT)/Makefile: | $(RTE_OUTPUT)
 # if NODOTCONF variable is defined, don't try to rebuild .config
 $(RTE_OUTPUT)/include/rte_config.h: $(RTE_OUTPUT)/.config
 	$(Q)rm -rf $(RTE_OUTPUT)/include $(RTE_OUTPUT)/app \
-		$(RTE_OUTPUT)/hostapp $(RTE_OUTPUT)/lib \
-		$(RTE_OUTPUT)/kmod $(RTE_OUTPUT)/build
+		$(RTE_OUTPUT)/lib $(RTE_OUTPUT)/kmod $(RTE_OUTPUT)/build
 	$(Q)mkdir -p $(RTE_OUTPUT)/include
 	$(Q)$(RTE_SDK)/scripts/gen-config-h.sh $(RTE_OUTPUT)/.config \
 		> $(RTE_OUTPUT)/include/rte_config.h
-- 
2.7.0

[PATCH v9 4/7] pmdinfogen: parse driver to generate code to export

[Thomas Monjalon]

From: Neil Horman <nhorman@tuxdriver.com>

dpdk-pmdinfogen is a tool used to parse object files and build JSON
strings for use in later determining hardware support in a DSO or
application binary.
dpdk-pmdinfogen looks for the non-exported symbol names rte_pmd_name<n>
(where n is a integer counter) and <name>_pci_table_export.
It records the name of each of these tuples, using the later to find
the symbolic name of the PCI table for physical devices that the object
supports.  With this information, it outputs a C file with a single line
of the form:

static char *<name>_pmd_info[] __attribute__((used)) = " \
	PMD_INFO_STRING=<json_string>";

Where <name> is the arbitrary name of the PMD, and <json_string> is the
JSON encoded string that hold relevant PMD information, including the PMD
name, type and optional array of PCI device/vendor IDs that the driver
supports.

This C file is suitable for compiling to object code, then relocatably
linking into the parent file from which the C was generated.  This creates
an entry in the string table of the object that can inform a later tool
about hardware support.

Note 1: When installed as part of a SDK package, dpdk-pmdinfogen should
        be built for the SDK target. It is not handled currently.
Note 2: Some generated files are not cleaned by "make clean".

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Panu Matilainen <pmatilai@redhat.com>
Acked-by: Remy Horton <remy.horton@intel.com>
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
---
 GNUmakefile                                    |   2 +-
 MAINTAINERS                                    |   4 +
 GNUmakefile => buildtools/Makefile             |  17 +-
 GNUmakefile => buildtools/pmdinfogen/Makefile  |  21 +-
 buildtools/pmdinfogen/pmdinfogen.c             | 451 +++++++++++++++++++++++++
 buildtools/pmdinfogen/pmdinfogen.h             |  99 ++++++
 doc/guides/prog_guide/dev_kit_build_system.rst |  15 +-
 mk/rte.sdkbuild.mk                             |   2 +-
 mk/rte.sdkinstall.mk                           |   3 +
 9 files changed, 587 insertions(+), 27 deletions(-)
 copy GNUmakefile => buildtools/Makefile (87%)
 copy GNUmakefile => buildtools/pmdinfogen/Makefile (84%)
 create mode 100644 buildtools/pmdinfogen/pmdinfogen.c
 create mode 100644 buildtools/pmdinfogen/pmdinfogen.h

diff --git a/GNUmakefile b/GNUmakefile
index b59e4b6..00fe0db 100644
--- a/GNUmakefile
+++ b/GNUmakefile
@@ -40,6 +40,6 @@ export RTE_SDK
 # directory list
 #
 
-ROOTDIRS-y := lib drivers app
+ROOTDIRS-y := buildtools lib drivers app
 
 include $(RTE_SDK)/mk/rte.sdkroot.mk
diff --git a/MAINTAINERS b/MAINTAINERS
index a59191e..1a8a3b7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -68,6 +68,10 @@ F: lib/librte_compat/
 F: doc/guides/rel_notes/deprecation.rst
 F: scripts/validate-abi.sh
 
+Driver information
+M: Neil Horman <nhorman@tuxdriver.com>
+F: buildtools/pmdinfogen/
+
 
 Environment Abstraction Layer
 -----------------------------
diff --git a/GNUmakefile b/buildtools/Makefile
similarity index 87%
copy from GNUmakefile
copy to buildtools/Makefile
index b59e4b6..35a42ff 100644
--- a/GNUmakefile
+++ b/buildtools/Makefile
@@ -1,6 +1,6 @@
 #   BSD LICENSE
 #
-#   Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+#   Copyright(c) 2016 Neil Horman. All rights reserved.
 #   All rights reserved.
 #
 #   Redistribution and use in source and binary forms, with or without
@@ -29,17 +29,8 @@
 #   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 #   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
-#
-# Head Makefile for compiling rte SDK
-#
-
-RTE_SDK := $(CURDIR)
-export RTE_SDK
-
-#
-# directory list
-#
+include $(RTE_SDK)/mk/rte.vars.mk
 
-ROOTDIRS-y := lib drivers app
+DIRS-y += pmdinfogen
 
-include $(RTE_SDK)/mk/rte.sdkroot.mk
+include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/GNUmakefile b/buildtools/pmdinfogen/Makefile
similarity index 84%
copy from GNUmakefile
copy to buildtools/pmdinfogen/Makefile
index b59e4b6..327927e 100644
--- a/GNUmakefile
+++ b/buildtools/pmdinfogen/Makefile
@@ -1,6 +1,6 @@
 #   BSD LICENSE
 #
-#   Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+#   Copyright(c) 2016 Neil Horman. All rights reserved.
 #   All rights reserved.
 #
 #   Redistribution and use in source and binary forms, with or without
@@ -29,17 +29,16 @@
 #   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 #   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
-#
-# Head Makefile for compiling rte SDK
-#
+include $(RTE_SDK)/mk/rte.vars.mk
 
-RTE_SDK := $(CURDIR)
-export RTE_SDK
+HOSTAPP_DIR = buildtools
+HOSTAPP = dpdk-pmdinfogen
 
-#
-# directory list
-#
+SRCS-y += pmdinfogen.c
+
+HOST_CFLAGS += $(WERROR_FLAGS) -g
+HOST_CFLAGS += -I$(RTE_OUTPUT)/include
 
-ROOTDIRS-y := lib drivers app
+DEPDIRS-y += lib/librte_eal
 
-include $(RTE_SDK)/mk/rte.sdkroot.mk
+include $(RTE_SDK)/mk/rte.hostapp.mk
diff --git a/buildtools/pmdinfogen/pmdinfogen.c b/buildtools/pmdinfogen/pmdinfogen.c
new file mode 100644
index 0000000..101bce1
--- /dev/null
+++ b/buildtools/pmdinfogen/pmdinfogen.c
@@ -0,0 +1,451 @@
+/* Postprocess pmd object files to export hw support
+ *
+ * Copyright 2016 Neil Horman <nhorman@tuxdriver.com>
+ * Based in part on modpost.c from the linux kernel
+ *
+ * This software may be used and distributed according to the terms
+ * of the GNU General Public License V2, incorporated herein by reference.
+ */
+
+#define _GNU_SOURCE
+#include <stdio.h>
+#include <ctype.h>
+#include <string.h>
+#include <limits.h>
+#include <stdbool.h>
+#include <errno.h>
+#include <rte_common.h>
+#include "pmdinfogen.h"
+
+#ifdef RTE_ARCH_64
+#define ADDR_SIZE 64
+#else
+#define ADDR_SIZE 32
+#endif
+
+
+static void *
+grab_file(const char *filename, unsigned long *size)
+{
+	struct stat st;
+	void *map = MAP_FAILED;
+	int fd;
+
+	fd = open(filename, O_RDONLY);
+	if (fd < 0)
+		return NULL;
+	if (fstat(fd, &st))
+		goto failed;
+
+	*size = st.st_size;
+	map = mmap(NULL, *size, PROT_READ|PROT_WRITE, MAP_PRIVATE, fd, 0);
+
+failed:
+	close(fd);
+	if (map == MAP_FAILED)
+		return NULL;
+	return map;
+}
+
+/*
+ * Return a copy of the next line in a mmap'ed file.
+ * spaces in the beginning of the line is trimmed away.
+ * Return a pointer to a static buffer.
+ */
+static void
+release_file(void *file, unsigned long size)
+{
+	munmap(file, size);
+}
+
+/*
+ * Note, it seems odd that we have both a CONVERT_NATIVE and a TO_NATIVE macro
+ * below.  We do this because the values passed to TO_NATIVE may themselves be
+ * macros and need both macros here to get expanded.  Specifically its the width
+ * variable we are concerned with, because it needs to get expanded prior to
+ * string concatenation
+ */
+#define CONVERT_NATIVE(fend, width, x) ({ \
+typeof(x) ___x; \
+if ((fend) == ELFDATA2LSB) \
+	___x = rte_le_to_cpu_##width(x); \
+else \
+	___x = rte_be_to_cpu_##width(x); \
+	___x; \
+})
+
+#define TO_NATIVE(fend, width, x) CONVERT_NATIVE(fend, width, x)
+
+static int
+parse_elf(struct elf_info *info, const char *filename)
+{
+	unsigned int i;
+	Elf_Ehdr *hdr;
+	Elf_Shdr *sechdrs;
+	Elf_Sym  *sym;
+	int endian;
+	unsigned int symtab_idx = ~0U, symtab_shndx_idx = ~0U;
+
+	hdr = grab_file(filename, &info->size);
+	if (!hdr) {
+		perror(filename);
+		return -ENOENT;
+	}
+	info->hdr = hdr;
+	if (info->size < sizeof(*hdr)) {
+		/* file too small, assume this is an empty .o file */
+		return 0;
+	}
+	/* Is this a valid ELF file? */
+	if ((hdr->e_ident[EI_MAG0] != ELFMAG0) ||
+	    (hdr->e_ident[EI_MAG1] != ELFMAG1) ||
+	    (hdr->e_ident[EI_MAG2] != ELFMAG2) ||
+	    (hdr->e_ident[EI_MAG3] != ELFMAG3)) {
+		/* Not an ELF file - silently ignore it */
+		return 0;
+	}
+
+	if (!hdr->e_ident[EI_DATA]) {
+		/* Unknown endian */
+		return 0;
+	}
+
+	endian = hdr->e_ident[EI_DATA];
+
+	/* Fix endianness in ELF header */
+	hdr->e_type      = TO_NATIVE(endian, 16, hdr->e_type);
+	hdr->e_machine   = TO_NATIVE(endian, 16, hdr->e_machine);
+	hdr->e_version   = TO_NATIVE(endian, 32, hdr->e_version);
+	hdr->e_entry     = TO_NATIVE(endian, ADDR_SIZE, hdr->e_entry);
+	hdr->e_phoff     = TO_NATIVE(endian, ADDR_SIZE, hdr->e_phoff);
+	hdr->e_shoff     = TO_NATIVE(endian, ADDR_SIZE, hdr->e_shoff);
+	hdr->e_flags     = TO_NATIVE(endian, 32, hdr->e_flags);
+	hdr->e_ehsize    = TO_NATIVE(endian, 16, hdr->e_ehsize);
+	hdr->e_phentsize = TO_NATIVE(endian, 16, hdr->e_phentsize);
+	hdr->e_phnum     = TO_NATIVE(endian, 16, hdr->e_phnum);
+	hdr->e_shentsize = TO_NATIVE(endian, 16, hdr->e_shentsize);
+	hdr->e_shnum     = TO_NATIVE(endian, 16, hdr->e_shnum);
+	hdr->e_shstrndx  = TO_NATIVE(endian, 16, hdr->e_shstrndx);
+
+	sechdrs = RTE_PTR_ADD(hdr, hdr->e_shoff);
+	info->sechdrs = sechdrs;
+
+	/* Check if file offset is correct */
+	if (hdr->e_shoff > info->size) {
+		fprintf(stderr, "section header offset=%lu in file '%s' "
+			"is bigger than filesize=%lu\n",
+			(unsigned long)hdr->e_shoff,
+			filename, info->size);
+		return 0;
+	}
+
+	if (hdr->e_shnum == SHN_UNDEF) {
+		/*
+		 * There are more than 64k sections,
+		 * read count from .sh_size.
+		 */
+		info->num_sections = TO_NATIVE(endian, 32, sechdrs[0].sh_size);
+	} else {
+		info->num_sections = hdr->e_shnum;
+	}
+	if (hdr->e_shstrndx == SHN_XINDEX)
+		info->secindex_strings =
+			TO_NATIVE(endian, 32, sechdrs[0].sh_link);
+	else
+		info->secindex_strings = hdr->e_shstrndx;
+
+	/* Fix endianness in section headers */
+	for (i = 0; i < info->num_sections; i++) {
+		sechdrs[i].sh_name      =
+			TO_NATIVE(endian, 32, sechdrs[i].sh_name);
+		sechdrs[i].sh_type      =
+			TO_NATIVE(endian, 32, sechdrs[i].sh_type);
+		sechdrs[i].sh_flags     =
+			TO_NATIVE(endian, 32, sechdrs[i].sh_flags);
+		sechdrs[i].sh_addr      =
+			TO_NATIVE(endian, ADDR_SIZE, sechdrs[i].sh_addr);
+		sechdrs[i].sh_offset    =
+			TO_NATIVE(endian, ADDR_SIZE, sechdrs[i].sh_offset);
+		sechdrs[i].sh_size      =
+			TO_NATIVE(endian, 32, sechdrs[i].sh_size);
+		sechdrs[i].sh_link      =
+			TO_NATIVE(endian, 32, sechdrs[i].sh_link);
+		sechdrs[i].sh_info      =
+			TO_NATIVE(endian, 32, sechdrs[i].sh_info);
+		sechdrs[i].sh_addralign =
+			TO_NATIVE(endian, ADDR_SIZE, sechdrs[i].sh_addralign);
+		sechdrs[i].sh_entsize   =
+			TO_NATIVE(endian, ADDR_SIZE, sechdrs[i].sh_entsize);
+	}
+	/* Find symbol table. */
+	for (i = 1; i < info->num_sections; i++) {
+		int nobits = sechdrs[i].sh_type == SHT_NOBITS;
+
+		if (!nobits && sechdrs[i].sh_offset > info->size) {
+			fprintf(stderr, "%s is truncated. "
+				"sechdrs[i].sh_offset=%lu > sizeof(*hrd)=%zu\n",
+				filename, (unsigned long)sechdrs[i].sh_offset,
+				sizeof(*hdr));
+			return 0;
+		}
+
+		if (sechdrs[i].sh_type == SHT_SYMTAB) {
+			unsigned int sh_link_idx;
+			symtab_idx = i;
+			info->symtab_start = RTE_PTR_ADD(hdr,
+				sechdrs[i].sh_offset);
+			info->symtab_stop  = RTE_PTR_ADD(hdr,
+				sechdrs[i].sh_offset + sechdrs[i].sh_size);
+			sh_link_idx = sechdrs[i].sh_link;
+			info->strtab       = RTE_PTR_ADD(hdr,
+				sechdrs[sh_link_idx].sh_offset);
+		}
+
+		/* 32bit section no. table? ("more than 64k sections") */
+		if (sechdrs[i].sh_type == SHT_SYMTAB_SHNDX) {
+			symtab_shndx_idx = i;
+			info->symtab_shndx_start = RTE_PTR_ADD(hdr,
+				sechdrs[i].sh_offset);
+			info->symtab_shndx_stop  = RTE_PTR_ADD(hdr,
+				sechdrs[i].sh_offset + sechdrs[i].sh_size);
+		}
+	}
+	if (!info->symtab_start)
+		fprintf(stderr, "%s has no symtab?\n", filename);
+
+	/* Fix endianness in symbols */
+	for (sym = info->symtab_start; sym < info->symtab_stop; sym++) {
+		sym->st_shndx = TO_NATIVE(endian, 16, sym->st_shndx);
+		sym->st_name  = TO_NATIVE(endian, 32, sym->st_name);
+		sym->st_value = TO_NATIVE(endian, ADDR_SIZE, sym->st_value);
+		sym->st_size  = TO_NATIVE(endian, ADDR_SIZE, sym->st_size);
+	}
+
+	if (symtab_shndx_idx != ~0U) {
+		Elf32_Word *p;
+		if (symtab_idx != sechdrs[symtab_shndx_idx].sh_link)
+			fprintf(stderr,
+				"%s: SYMTAB_SHNDX has bad sh_link: %u!=%u\n",
+				filename, sechdrs[symtab_shndx_idx].sh_link,
+				symtab_idx);
+		/* Fix endianness */
+		for (p = info->symtab_shndx_start; p < info->symtab_shndx_stop; p++)
+			*p = TO_NATIVE(endian, 32, *p);
+	}
+
+	return 1;
+}
+
+static void
+parse_elf_finish(struct elf_info *info)
+{
+	struct pmd_driver *tmp, *idx = info->drivers;
+	release_file(info->hdr, info->size);
+	while (idx) {
+		tmp = idx->next;
+		free(idx);
+		idx = tmp;
+	}
+}
+
+static const char *
+get_sym_name(struct elf_info *elf, Elf_Sym *sym)
+{
+	if (sym)
+		return elf->strtab + sym->st_name;
+	else
+		return "(unknown)";
+}
+
+static void *
+get_sym_value(struct elf_info *info, const Elf_Sym *sym)
+{
+	return RTE_PTR_ADD(info->hdr,
+		info->sechdrs[sym->st_shndx].sh_offset + sym->st_value);
+}
+
+static Elf_Sym *
+find_sym_in_symtab(struct elf_info *info, const char *name, Elf_Sym *last)
+{
+	Elf_Sym *idx;
+	if (last)
+		idx = last+1;
+	else
+		idx = info->symtab_start;
+
+	for (; idx < info->symtab_stop; idx++) {
+		const char *n = get_sym_name(info, idx);
+		if (!strncmp(n, name, strlen(name)))
+			return idx;
+	}
+	return NULL;
+}
+
+struct opt_tag {
+	const char *suffix;
+	const char *json_id;
+};
+
+static const struct opt_tag opt_tags[] = {
+	{"_param_string_export", "params"},
+};
+
+static int
+complete_pmd_entry(struct elf_info *info, struct pmd_driver *drv)
+{
+	const char *tname;
+	int i;
+	char tmpsymname[128];
+	Elf_Sym *tmpsym;
+
+	drv->name = get_sym_value(info, drv->name_sym);
+
+	for (i = 0; i < PMD_OPT_MAX; i++) {
+		memset(tmpsymname, 0, 128);
+		sprintf(tmpsymname, "__%s%s", drv->name, opt_tags[i].suffix);
+		tmpsym = find_sym_in_symtab(info, tmpsymname, NULL);
+		if (!tmpsym)
+			continue;
+		drv->opt_vals[i] = get_sym_value(info, tmpsym);
+	}
+
+	memset(tmpsymname, 0, 128);
+	sprintf(tmpsymname, "__%s_pci_table_export", drv->name);
+
+	tmpsym = find_sym_in_symtab(info, tmpsymname, NULL);
+
+	/*
+	 * If this returns NULL, then this is a PMD_VDEV, because
+	 * it has no pci table reference
+	 */
+	if (!tmpsym) {
+		drv->pci_tbl = NULL;
+		return 0;
+	}
+
+	tname = get_sym_value(info, tmpsym);
+	tmpsym = find_sym_in_symtab(info, tname, NULL);
+	if (!tmpsym) {
+		fprintf(stderr, "No symbol %s\n", tname);
+		return -ENOENT;
+	}
+
+	drv->pci_tbl = (struct rte_pci_id *)get_sym_value(info, tmpsym);
+	if (!drv->pci_tbl) {
+		fprintf(stderr, "Failed to get PCI table %s\n", tname);
+		return -ENOENT;
+	}
+
+	return 0;
+}
+
+static int
+locate_pmd_entries(struct elf_info *info)
+{
+	Elf_Sym *last = NULL;
+	struct pmd_driver *new;
+
+	info->drivers = NULL;
+
+	do {
+		new = calloc(sizeof(struct pmd_driver), 1);
+		new->name_sym = find_sym_in_symtab(info, "rte_pmd_name", last);
+		last = new->name_sym;
+		if (!new->name_sym)
+			free(new);
+		else {
+			if (complete_pmd_entry(info, new)) {
+				fprintf(stderr, "Failed to complete pmd entry\n");
+				free(new);
+				return -ENOENT;
+			} else {
+				new->next = info->drivers;
+				info->drivers = new;
+			}
+		}
+	} while (last);
+
+	return 0;
+}
+
+static void
+output_pmd_info_string(struct elf_info *info, char *outfile)
+{
+	FILE *ofd;
+	struct pmd_driver *drv;
+	struct rte_pci_id *pci_ids;
+	int idx = 0;
+
+	ofd = fopen(outfile, "w+");
+	if (!ofd) {
+		fprintf(stderr, "Unable to open output file\n");
+		return;
+	}
+
+	drv = info->drivers;
+
+	while (drv) {
+		fprintf(ofd, "const char %s_pmd_info[] __attribute__((used)) = "
+			"\"PMD_INFO_STRING= {",
+			drv->name);
+		fprintf(ofd, "\\\"name\\\" : \\\"%s\\\", ", drv->name);
+
+		for (idx = 0; idx < PMD_OPT_MAX; idx++) {
+			if (drv->opt_vals[idx])
+				fprintf(ofd, "\\\"%s\\\" : \\\"%s\\\", ",
+					opt_tags[idx].json_id,
+					drv->opt_vals[idx]);
+		}
+
+		pci_ids = drv->pci_tbl;
+		fprintf(ofd, "\\\"pci_ids\\\" : [");
+
+		while (pci_ids && pci_ids->device_id) {
+			fprintf(ofd, "[%d, %d, %d, %d]",
+				pci_ids->vendor_id, pci_ids->device_id,
+				pci_ids->subsystem_vendor_id,
+				pci_ids->subsystem_device_id);
+			pci_ids++;
+			if (pci_ids->device_id)
+				fprintf(ofd, ",");
+			else
+				fprintf(ofd, " ");
+		}
+		fprintf(ofd, "]}\";");
+		drv = drv->next;
+	}
+
+	fclose(ofd);
+}
+
+int main(int argc, char **argv)
+{
+	struct elf_info info;
+	int rc;
+
+	if (argc < 3) {
+		fprintf(stderr,
+			"usage: dpdk-pmdinfogen <object file> <c output file>\n");
+		exit(127);
+	}
+
+	rc = parse_elf(&info, argv[1]);
+	if (rc < 0)
+		exit(-rc);
+
+	rc = locate_pmd_entries(&info);
+	if (rc < 0)
+		goto error;
+
+	if (info.drivers) {
+		output_pmd_info_string(&info, argv[2]);
+		rc = 0;
+	} else {
+		rc = -1;
+		fprintf(stderr, "No drivers registered\n");
+	}
+
+error:
+	parse_elf_finish(&info);
+	exit(-rc);
+}
diff --git a/buildtools/pmdinfogen/pmdinfogen.h b/buildtools/pmdinfogen/pmdinfogen.h
new file mode 100644
index 0000000..7e57702
--- /dev/null
+++ b/buildtools/pmdinfogen/pmdinfogen.h
@@ -0,0 +1,99 @@
+/* Postprocess pmd object files to export hw support
+ *
+ * Copyright 2016 Neil Horman <nhorman@tuxdriver.com>
+ * Based in part on modpost.c from the linux kernel
+ *
+ * This software may be used and distributed according to the terms
+ * of the GNU General Public License V2, incorporated herein by reference.
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdarg.h>
+#include <string.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/mman.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <elf.h>
+#include <rte_config.h>
+#include <rte_pci.h>
+#include <rte_byteorder.h>
+
+/* On BSD-alike OSes elf.h defines these according to host's word size */
+#undef ELF_ST_BIND
+#undef ELF_ST_TYPE
+#undef ELF_R_SYM
+#undef ELF_R_TYPE
+
+/*
+ * Define ELF64_* to ELF_*, the latter being defined in both 32 and 64 bit
+ * flavors in elf.h.  This makes our code a bit more generic between arches
+ * and allows us to support 32 bit code in the future should we ever want to
+ */
+#ifdef RTE_ARCH_64
+#define Elf_Ehdr    Elf64_Ehdr
+#define Elf_Shdr    Elf64_Shdr
+#define Elf_Sym     Elf64_Sym
+#define Elf_Addr    Elf64_Addr
+#define Elf_Sword   Elf64_Sxword
+#define Elf_Section Elf64_Half
+#define ELF_ST_BIND ELF64_ST_BIND
+#define ELF_ST_TYPE ELF64_ST_TYPE
+
+#define Elf_Rel     Elf64_Rel
+#define Elf_Rela    Elf64_Rela
+#define ELF_R_SYM   ELF64_R_SYM
+#define ELF_R_TYPE  ELF64_R_TYPE
+#else
+#define Elf_Ehdr    Elf32_Ehdr
+#define Elf_Shdr    Elf32_Shdr
+#define Elf_Sym     Elf32_Sym
+#define Elf_Addr    Elf32_Addr
+#define Elf_Sword   Elf32_Sxword
+#define Elf_Section Elf32_Half
+#define ELF_ST_BIND ELF32_ST_BIND
+#define ELF_ST_TYPE ELF32_ST_TYPE
+
+#define Elf_Rel     Elf32_Rel
+#define Elf_Rela    Elf32_Rela
+#define ELF_R_SYM   ELF32_R_SYM
+#define ELF_R_TYPE  ELF32_R_TYPE
+#endif
+
+
+enum opt_params {
+	PMD_PARAM_STRING = 0,
+	PMD_OPT_MAX
+};
+
+struct pmd_driver {
+	Elf_Sym *name_sym;
+	const char *name;
+	struct rte_pci_id *pci_tbl;
+	struct pmd_driver *next;
+
+	const char *opt_vals[PMD_OPT_MAX];
+};
+
+struct elf_info {
+	unsigned long size;
+	Elf_Ehdr     *hdr;
+	Elf_Shdr     *sechdrs;
+	Elf_Sym      *symtab_start;
+	Elf_Sym      *symtab_stop;
+	char         *strtab;
+
+	/* support for 32bit section numbers */
+
+	unsigned int num_sections; /* max_secindex + 1 */
+	unsigned int secindex_strings;
+	/* if Nth symbol table entry has .st_shndx = SHN_XINDEX,
+	 * take shndx from symtab_shndx_start[N] instead
+	 */
+	Elf32_Word   *symtab_shndx_start;
+	Elf32_Word   *symtab_shndx_stop;
+
+	struct pmd_driver *drivers;
+};
diff --git a/doc/guides/prog_guide/dev_kit_build_system.rst b/doc/guides/prog_guide/dev_kit_build_system.rst
index dedd18a..fa34fe0 100644
--- a/doc/guides/prog_guide/dev_kit_build_system.rst
+++ b/doc/guides/prog_guide/dev_kit_build_system.rst
@@ -70,7 +70,7 @@ Each build directory contains include files, libraries, and applications:
     ...
     ~/DEV/DPDK$ ls i686-native-linuxapp-gcc
 
-    app build hostapp include kmod lib Makefile
+    app build buildtools include kmod lib Makefile
 
 
     ~/DEV/DPDK$ ls i686-native-linuxapp-gcc/app/
@@ -307,6 +307,7 @@ Misc
 Internally Generated Build Tools
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
+``dpdk-pmdinfogen`` scans an object (.o) file for various well known symbol names.
 These well known symbol names are defined by various macros and used to export
 important information about hardware support and usage for PMD files.  For
 instance the macro:
@@ -321,6 +322,18 @@ Creates the following symbol:
 
    static char rte_pmd_name0[] __attribute__((used)) = "<name>";
 
+Which dpdk-pmdinfogen scans for.  Using this information other relevant bits of
+data can be exported from the object file and used to produce a hardware support
+description, that dpdk-pmdinfogen then encodes into a json formatted string in
+the following format:
+
+.. code-block:: C
+
+   static char <name_pmd_string>="PMD_INFO_STRING=\"{'name' : '<name>', ...}\"";
+
+These strings can then be searched for by external tools to determine the
+hardware support of a given library or application.
+
 .. _Useful_Variables_Provided_by_the_Build_System:
 
 Useful Variables Provided by the Build System
diff --git a/mk/rte.sdkbuild.mk b/mk/rte.sdkbuild.mk
index f1a163a..5edbf50 100644
--- a/mk/rte.sdkbuild.mk
+++ b/mk/rte.sdkbuild.mk
@@ -63,7 +63,7 @@ build: $(ROOTDIRS-y)
 .PHONY: clean
 clean: $(CLEANDIRS)
 	@rm -rf $(RTE_OUTPUT)/include $(RTE_OUTPUT)/app \
-		$(RTE_OUTPUT)/lib $(RTE_OUTPUT)/kmod
+		$(RTE_OUTPUT)/lib $(RTE_OUTPUT)/kmod $(RTE_OUTPUT)/buildtools
 	@[ -d $(RTE_OUTPUT)/include ] || mkdir -p $(RTE_OUTPUT)/include
 	@$(RTE_SDK)/scripts/gen-config-h.sh $(RTE_OUTPUT)/.config \
 		> $(RTE_OUTPUT)/include/rte_config.h
diff --git a/mk/rte.sdkinstall.mk b/mk/rte.sdkinstall.mk
index abdab0f..2b92157 100644
--- a/mk/rte.sdkinstall.mk
+++ b/mk/rte.sdkinstall.mk
@@ -141,10 +141,13 @@ install-sdk:
 	$(Q)$(call rte_mkdir,                            $(DESTDIR)$(sdkdir))
 	$(Q)cp -a               $(RTE_SDK)/mk            $(DESTDIR)$(sdkdir)
 	$(Q)cp -a               $(RTE_SDK)/scripts       $(DESTDIR)$(sdkdir)
+	$(Q)cp -a               $O/buildtools            $(DESTDIR)$(sdkdir)
 	$(Q)$(call rte_mkdir,                            $(DESTDIR)$(targetdir))
 	$(Q)cp -a               $O/.config               $(DESTDIR)$(targetdir)
 	$(Q)$(call rte_symlink, $(DESTDIR)$(includedir), $(DESTDIR)$(targetdir)/include)
 	$(Q)$(call rte_symlink, $(DESTDIR)$(libdir),     $(DESTDIR)$(targetdir)/lib)
+	$(Q)$(call rte_symlink, $(DESTDIR)$(sdkdir)/buildtools, \
+	                        $(DESTDIR)$(targetdir)/buildtools)
 
 install-doc:
 ifneq ($(wildcard $O/doc),)
-- 
2.7.0

[PATCH v9 5/7] mk: link infos generated by pmdinfogen

[Thomas Monjalon]

Generate informations to export from objects which register a driver.
The C code generated by dpdk-pmdinfogen is compiled and linked into the
original object file.
This effectively just adds the JSON string into the string table of the
object that defines the PMD to the outside world.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Panu Matilainen <pmatilai@redhat.com>
Acked-by: Remy Horton <remy.horton@intel.com>
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
---
 mk/internal/rte.compile-pre.mk | 12 ++++++++++++
 mk/rte.sdkbuild.mk             |  1 +
 2 files changed, 13 insertions(+)

diff --git a/mk/internal/rte.compile-pre.mk b/mk/internal/rte.compile-pre.mk
index b9bff4a..3967390 100644
--- a/mk/internal/rte.compile-pre.mk
+++ b/mk/internal/rte.compile-pre.mk
@@ -84,10 +84,22 @@ C_TO_O = $(CC) -Wp,-MD,$(call obj2dep,$(@)).tmp $(CFLAGS) \
 C_TO_O_STR = $(subst ','\'',$(C_TO_O)) #'# fix syntax highlight
 C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)","  CC $(@)")
 endif
+PMDINFO_GEN = $(RTE_SDK_BIN)/buildtools/dpdk-pmdinfogen $@ $@.pmd.c
+PMDINFO_CC = $(CC) $(CFLAGS) -c -o $@.pmd.o $@.pmd.c
+PMDINFO_LD = $(CROSS)ld $(LDFLAGS) -r -o $@.o $@.pmd.o $@
+PMDINFO_TO_O = if grep -q 'RTE_REGISTER_DRIVER(.*)' $<; then \
+	echo "$(if $V,$(PMDINFO_GEN),  PMDINFO $@.pmd.c)" && \
+	$(PMDINFO_GEN) && \
+	echo "$(if $V,$(PMDINFO_CC),  CC $@.pmd.o)" && \
+	$(PMDINFO_CC) && \
+	echo "$(if $V,$(PMDINFO_LD),  LD $@)" && \
+	$(PMDINFO_LD) && \
+	mv -f $@.o $@; fi
 C_TO_O_CMD = 'cmd_$@ = $(C_TO_O_STR)'
 C_TO_O_DO = @set -e; \
 	echo $(C_TO_O_DISP); \
 	$(C_TO_O) && \
+	$(PMDINFO_TO_O) && \
 	echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \
 	sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \
 	rm -f $(call obj2dep,$(@)).tmp
diff --git a/mk/rte.sdkbuild.mk b/mk/rte.sdkbuild.mk
index 5edbf50..b6b66bf 100644
--- a/mk/rte.sdkbuild.mk
+++ b/mk/rte.sdkbuild.mk
@@ -49,6 +49,7 @@ $(1): $(sort $(LOCAL_DEPDIRS-$(1)))
 endef
 
 $(foreach d,$(ROOTDIRS-y),$(eval $(call depdirs_rule,$(d))))
+drivers: | buildtools
 
 #
 # build and clean targets
-- 
2.7.0

[PATCH v9 6/7] eal: export default plugin path to external tools

[Thomas Monjalon]

From: Neil Horman <nhorman@tuxdriver.com>

Export a symbol containing the string:
DPDK_PLUGIN_PATH="$(CONFIG_RTE_EAL_PMD_PATH)"

Where the latter half of the string is set at build time to a location from
which autoloaded DSO's will be found.  This string is used by dpdk-pmdinfo in
'plugin' mode, whereby a user can specify a DPDK installation directory (or
static binary), and scan the associated path (if found) for pmd DSO's and
report on their hardware support.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Panu Matilainen <pmatilai@redhat.com>
Acked-by: Remy Horton <remy.horton@intel.com>
---
 lib/librte_eal/common/eal_common_options.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c
index 3efc90f..7e9f7b8 100644
--- a/lib/librte_eal/common/eal_common_options.c
+++ b/lib/librte_eal/common/eal_common_options.c
@@ -115,6 +115,10 @@ TAILQ_HEAD_INITIALIZER(solib_list);
 /* Default path of external loadable drivers */
 static const char *default_solib_dir = RTE_EAL_PMD_PATH;
 
+/* Stringified version of default solib path */
+static const char dpdk_solib_path[] __attribute__((used)) =
+"DPDK_PLUGIN_PATH=" RTE_EAL_PMD_PATH;
+
 static int master_lcore_parsed;
 static int mem_parsed;
 
-- 
2.7.0

[PATCH v9 7/7] tools: query binaries for support information

[Thomas Monjalon]

From: Neil Horman <nhorman@tuxdriver.com>

This tool searches for the primer string PMD_INFO_STRING= in any ELF binary,
and, if found parses the remainder of the string as a json encoded string,
outputting the results in either a human readable or raw, script parseable
format.

Note that, in the case of dynamically linked applications, pmdinfo.py will
scan for implicitly linked PMDs by searching the specified binaries
.dynamic section for DT_NEEDED entries that contain the substring
librte_pmd.  The DT_RUNPATH, LD_LIBRARY_PATH, /usr/lib and /lib are
searched for these libraries, in that order.

If a file is specified with no path, it is assumed to be a PMD DSO, and the
LD_LIBRARY_PATH, /usr/lib[64]/ and /lib[64] is searched for it.

Currently the tool can output data in 3 formats:
a) raw, suitable for scripting, where the raw JSON strings are dumped out
b) table format (default) where hex pci ids are dumped in a table format
c) pretty, where a user supplied pci.ids file is used to print out vendor
and device strings

There is a dependency on pyelftools.
The script is not yet compatible with Python 3.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Panu Matilainen <pmatilai@redhat.com>
Acked-by: Remy Horton <remy.horton@intel.com>
---
 MAINTAINERS                                |   1 +
 lib/librte_eal/common/eal_common_options.c |   2 +-
 mk/rte.sdkinstall.mk                       |   2 +
 tools/dpdk-pmdinfo.py                      | 628 +++++++++++++++++++++++++++++
 4 files changed, 632 insertions(+), 1 deletion(-)
 create mode 100755 tools/dpdk-pmdinfo.py

diff --git a/MAINTAINERS b/MAINTAINERS
index 1a8a3b7..1e972f0 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -71,6 +71,7 @@ F: scripts/validate-abi.sh
 Driver information
 M: Neil Horman <nhorman@tuxdriver.com>
 F: buildtools/pmdinfogen/
+F: tools/dpdk-pmdinfo.py
 
 
 Environment Abstraction Layer
diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c
index 7e9f7b8..b562c8a 100644
--- a/lib/librte_eal/common/eal_common_options.c
+++ b/lib/librte_eal/common/eal_common_options.c
@@ -115,7 +115,7 @@ TAILQ_HEAD_INITIALIZER(solib_list);
 /* Default path of external loadable drivers */
 static const char *default_solib_dir = RTE_EAL_PMD_PATH;
 
-/* Stringified version of default solib path */
+/* Stringified version of default solib path used by dpdk-pmdinfo.py */
 static const char dpdk_solib_path[] __attribute__((used)) =
 "DPDK_PLUGIN_PATH=" RTE_EAL_PMD_PATH;
 
diff --git a/mk/rte.sdkinstall.mk b/mk/rte.sdkinstall.mk
index 2b92157..76be308 100644
--- a/mk/rte.sdkinstall.mk
+++ b/mk/rte.sdkinstall.mk
@@ -126,6 +126,8 @@ install-runtime:
 	$(Q)$(call rte_mkdir,      $(DESTDIR)$(sbindir))
 	$(Q)$(call rte_symlink,    $(DESTDIR)$(datadir)/tools/dpdk_nic_bind.py, \
 	                           $(DESTDIR)$(sbindir)/dpdk_nic_bind)
+	$(Q)$(call rte_symlink,    $(DESTDIR)$(datadir)/tools/dpdk-pmdinfo.py, \
+	                           $(DESTDIR)$(bindir)/dpdk-pmdinfo)
 
 install-kmod:
 ifneq ($(wildcard $O/kmod/*),)
diff --git a/tools/dpdk-pmdinfo.py b/tools/dpdk-pmdinfo.py
new file mode 100755
index 0000000..b8a9be2
--- /dev/null
+++ b/tools/dpdk-pmdinfo.py
@@ -0,0 +1,628 @@
+#!/usr/bin/python
+# -------------------------------------------------------------------------
+#
+# Utility to dump PMD_INFO_STRING support from an object file
+#
+# -------------------------------------------------------------------------
+import os
+import sys
+from optparse import OptionParser
+import string
+import json
+
+# For running from development directory. It should take precedence over the
+# installed pyelftools.
+sys.path.insert(0, '.')
+
+
+from elftools import __version__
+from elftools.common.exceptions import ELFError
+from elftools.common.py3compat import (
+    ifilter, byte2int, bytes2str, itervalues, str2bytes)
+from elftools.elf.elffile import ELFFile
+from elftools.elf.dynamic import DynamicSection, DynamicSegment
+from elftools.elf.enums import ENUM_D_TAG
+from elftools.elf.segments import InterpSegment
+from elftools.elf.sections import SymbolTableSection
+from elftools.elf.gnuversions import (
+    GNUVerSymSection, GNUVerDefSection,
+    GNUVerNeedSection,
+)
+from elftools.elf.relocation import RelocationSection
+from elftools.elf.descriptions import (
+    describe_ei_class, describe_ei_data, describe_ei_version,
+    describe_ei_osabi, describe_e_type, describe_e_machine,
+    describe_e_version_numeric, describe_p_type, describe_p_flags,
+    describe_sh_type, describe_sh_flags,
+    describe_symbol_type, describe_symbol_bind, describe_symbol_visibility,
+    describe_symbol_shndx, describe_reloc_type, describe_dyn_tag,
+    describe_ver_flags,
+)
+from elftools.elf.constants import E_FLAGS
+from elftools.dwarf.dwarfinfo import DWARFInfo
+from elftools.dwarf.descriptions import (
+    describe_reg_name, describe_attr_value, set_global_machine_arch,
+    describe_CFI_instructions, describe_CFI_register_rule,
+    describe_CFI_CFA_rule,
+)
+from elftools.dwarf.constants import (
+    DW_LNS_copy, DW_LNS_set_file, DW_LNE_define_file)
+from elftools.dwarf.callframe import CIE, FDE
+
+raw_output = False
+pcidb = None
+
+# ===========================================
+
+
+class Vendor:
+    """
+    Class for vendors. This is the top level class
+    for the devices belong to a specific vendor.
+    self.devices is the device dictionary
+    subdevices are in each device.
+    """
+
+    def __init__(self, vendorStr):
+        """
+        Class initializes with the raw line from pci.ids
+        Parsing takes place inside __init__
+        """
+        self.ID = vendorStr.split()[0]
+        self.name = vendorStr.replace("%s " % self.ID, "").rstrip()
+        self.devices = {}
+
+    def add_device(self, deviceStr):
+        """
+        Adds a device to self.devices
+        takes the raw line from pci.ids
+        """
+        s = deviceStr.strip()
+        devID = s.split()[0]
+        if devID in self.devices:
+            pass
+        else:
+            self.devices[devID] = Device(deviceStr)
+
+    def report(self):
+        print self.ID, self.name
+        for id, dev in self.devices.items():
+            dev.report()
+
+    def find_device(self, devid):
+        # convert to a hex string and remove 0x
+        devid = hex(devid)[2:]
+        try:
+            return self.devices[devid]
+        except:
+            return Device("%s  Unknown Device" % devid)
+
+
+class Device:
+
+    def __init__(self, deviceStr):
+        """
+        Class for each device.
+        Each vendor has its own devices dictionary.
+        """
+        s = deviceStr.strip()
+        self.ID = s.split()[0]
+        self.name = s.replace("%s  " % self.ID, "")
+        self.subdevices = {}
+
+    def report(self):
+        print "\t%s\t%s" % (self.ID, self.name)
+        for subID, subdev in self.subdevices.items():
+            subdev.report()
+
+    def add_sub_device(self, subDeviceStr):
+        """
+        Adds a subvendor, subdevice to device.
+        Uses raw line from pci.ids
+        """
+        s = subDeviceStr.strip()
+        spl = s.split()
+        subVendorID = spl[0]
+        subDeviceID = spl[1]
+        subDeviceName = s.split("  ")[-1]
+        devID = "%s:%s" % (subVendorID, subDeviceID)
+        self.subdevices[devID] = SubDevice(
+            subVendorID, subDeviceID, subDeviceName)
+
+    def find_subid(self, subven, subdev):
+        subven = hex(subven)[2:]
+        subdev = hex(subdev)[2:]
+        devid = "%s:%s" % (subven, subdev)
+
+        try:
+            return self.subdevices[devid]
+        except:
+            if (subven == "ffff" and subdev == "ffff"):
+                return SubDevice("ffff", "ffff", "(All Subdevices)")
+            else:
+                return SubDevice(subven, subdev, "(Unknown Subdevice)")
+
+
+class SubDevice:
+    """
+    Class for subdevices.
+    """
+
+    def __init__(self, vendor, device, name):
+        """
+        Class initializes with vendorid, deviceid and name
+        """
+        self.vendorID = vendor
+        self.deviceID = device
+        self.name = name
+
+    def report(self):
+        print "\t\t%s\t%s\t%s" % (self.vendorID, self.deviceID, self.name)
+
+
+class PCIIds:
+    """
+    Top class for all pci.ids entries.
+    All queries will be asked to this class.
+    PCIIds.vendors["0e11"].devices["0046"].\
+    subdevices["0e11:4091"].name  =  "Smart Array 6i"
+    """
+
+    def __init__(self, filename):
+        """
+        Prepares the directories.
+        Checks local data file.
+        Tries to load from local, if not found, downloads from web
+        """
+        self.version = ""
+        self.date = ""
+        self.vendors = {}
+        self.contents = None
+        self.read_local(filename)
+        self.parse()
+
+    def report_vendors(self):
+        """Reports the vendors
+        """
+        for vid, v in self.vendors.items():
+            print v.ID, v.name
+
+    def report(self, vendor=None):
+        """
+        Reports everything for all vendors or a specific vendor
+        PCIIds.report()  reports everything
+        PCIIDs.report("0e11") reports only "Compaq Computer Corporation"
+        """
+        if vendor is not None:
+            self.vendors[vendor].report()
+        else:
+            for vID, v in self.vendors.items():
+                v.report()
+
+    def find_vendor(self, vid):
+        # convert vid to a hex string and remove the 0x
+        vid = hex(vid)[2:]
+
+        try:
+            return self.vendors[vid]
+        except:
+            return Vendor("%s Unknown Vendor" % (vid))
+
+    def find_date(self, content):
+        for l in content:
+            if l.find("Date:") > -1:
+                return l.split()[-2].replace("-", "")
+        return None
+
+    def parse(self):
+        if len(self.contents) < 1:
+            print "data/%s-pci.ids not found" % self.date
+        else:
+            vendorID = ""
+            deviceID = ""
+            for l in self.contents:
+                if l[0] == "#":
+                    continue
+                elif len(l.strip()) == 0:
+                    continue
+                else:
+                    if l.find("\t\t") == 0:
+                        self.vendors[vendorID].devices[
+                            deviceID].add_sub_device(l)
+                    elif l.find("\t") == 0:
+                        deviceID = l.strip().split()[0]
+                        self.vendors[vendorID].add_device(l)
+                    else:
+                        vendorID = l.split()[0]
+                        self.vendors[vendorID] = Vendor(l)
+
+    def read_local(self, filename):
+        """
+        Reads the local file
+        """
+        self.contents = open(filename).readlines()
+        self.date = self.find_date(self.contents)
+
+    def load_local(self):
+        """
+        Loads database from local. If there is no file,
+        it creates a new one from web
+        """
+        self.date = idsfile[0].split("/")[1].split("-")[0]
+        self.read_local()
+
+
+# =======================================
+
+def search_file(filename, search_path):
+    """ Given a search path, find file with requested name """
+    for path in string.split(search_path, ":"):
+        candidate = os.path.join(path, filename)
+        if os.path.exists(candidate):
+            return os.path.abspath(candidate)
+    return None
+
+
+class ReadElf(object):
+    """ display_* methods are used to emit output into the output stream
+    """
+
+    def __init__(self, file, output):
+        """ file:
+                stream object with the ELF file to read
+
+            output:
+                output stream to write to
+        """
+        self.elffile = ELFFile(file)
+        self.output = output
+
+        # Lazily initialized if a debug dump is requested
+        self._dwarfinfo = None
+
+        self._versioninfo = None
+
+    def _section_from_spec(self, spec):
+        """ Retrieve a section given a "spec" (either number or name).
+            Return None if no such section exists in the file.
+        """
+        try:
+            num = int(spec)
+            if num < self.elffile.num_sections():
+                return self.elffile.get_section(num)
+            else:
+                return None
+        except ValueError:
+            # Not a number. Must be a name then
+            return self.elffile.get_section_by_name(str2bytes(spec))
+
+    def pretty_print_pmdinfo(self, pmdinfo):
+        global pcidb
+
+        for i in pmdinfo["pci_ids"]:
+            vendor = pcidb.find_vendor(i[0])
+            device = vendor.find_device(i[1])
+            subdev = device.find_subid(i[2], i[3])
+            print("%s (%s) : %s (%s) %s" %
+                  (vendor.name, vendor.ID, device.name,
+                   device.ID, subdev.name))
+
+    def parse_pmd_info_string(self, mystring):
+        global raw_output
+        global pcidb
+
+        optional_pmd_info = [{'id': 'params', 'tag': 'PMD PARAMETERS'}]
+
+        i = mystring.index("=")
+        mystring = mystring[i + 2:]
+        pmdinfo = json.loads(mystring)
+
+        if raw_output:
+            print(pmdinfo)
+            return
+
+        print("PMD NAME: " + pmdinfo["name"])
+        for i in optional_pmd_info:
+            try:
+                print("%s: %s" % (i['tag'], pmdinfo[i['id']]))
+            except KeyError as e:
+                continue
+
+        if (len(pmdinfo["pci_ids"]) != 0):
+            print("PMD HW SUPPORT:")
+            if pcidb is not None:
+                self.pretty_print_pmdinfo(pmdinfo)
+            else:
+                print("VENDOR\t DEVICE\t SUBVENDOR\t SUBDEVICE")
+                for i in pmdinfo["pci_ids"]:
+                    print("0x%04x\t 0x%04x\t 0x%04x\t\t 0x%04x" %
+                          (i[0], i[1], i[2], i[3]))
+
+        print("")
+
+    def display_pmd_info_strings(self, section_spec):
+        """ Display a strings dump of a section. section_spec is either a
+            section number or a name.
+        """
+        section = self._section_from_spec(section_spec)
+        if section is None:
+            return
+
+        data = section.data()
+        dataptr = 0
+
+        while dataptr < len(data):
+            while (dataptr < len(data) and
+                    not (32 <= byte2int(data[dataptr]) <= 127)):
+                dataptr += 1
+
+            if dataptr >= len(data):
+                break
+
+            endptr = dataptr
+            while endptr < len(data) and byte2int(data[endptr]) != 0:
+                endptr += 1
+
+            mystring = bytes2str(data[dataptr:endptr])
+            rc = mystring.find("PMD_INFO_STRING")
+            if (rc != -1):
+                self.parse_pmd_info_string(mystring)
+
+            dataptr = endptr
+
+    def find_librte_eal(self, section):
+        for tag in section.iter_tags():
+            if tag.entry.d_tag == 'DT_NEEDED':
+                if "librte_eal" in tag.needed:
+                    return tag.needed
+        return None
+
+    def search_for_autoload_path(self):
+        scanelf = self
+        scanfile = None
+        library = None
+
+        section = self._section_from_spec(".dynamic")
+        try:
+            eallib = self.find_librte_eal(section)
+            if eallib is not None:
+                ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+                if ldlibpath is None:
+                    ldlibpath = ""
+                dtr = self.get_dt_runpath(section)
+                library = search_file(eallib,
+                                      dtr + ":" + ldlibpath +
+                                      ":/usr/lib64:/lib64:/usr/lib:/lib")
+                if library is None:
+                    return (None, None)
+                if raw_output is False:
+                    print("Scanning for autoload path in %s" % library)
+                scanfile = open(library, 'rb')
+                scanelf = ReadElf(scanfile, sys.stdout)
+        except AttributeError:
+            # Not a dynamic binary
+            pass
+        except ELFError:
+            scanfile.close()
+            return (None, None)
+
+        section = scanelf._section_from_spec(".rodata")
+        if section is None:
+            if scanfile is not None:
+                scanfile.close()
+            return (None, None)
+
+        data = section.data()
+        dataptr = 0
+
+        while dataptr < len(data):
+            while (dataptr < len(data) and
+                    not (32 <= byte2int(data[dataptr]) <= 127)):
+                dataptr += 1
+
+            if dataptr >= len(data):
+                break
+
+            endptr = dataptr
+            while endptr < len(data) and byte2int(data[endptr]) != 0:
+                endptr += 1
+
+            mystring = bytes2str(data[dataptr:endptr])
+            rc = mystring.find("DPDK_PLUGIN_PATH")
+            if (rc != -1):
+                rc = mystring.find("=")
+                return (mystring[rc + 1:], library)
+
+            dataptr = endptr
+        if scanfile is not None:
+            scanfile.close()
+        return (None, None)
+
+    def get_dt_runpath(self, dynsec):
+        for tag in dynsec.iter_tags():
+            if tag.entry.d_tag == 'DT_RUNPATH':
+                return tag.runpath
+        return ""
+
+    def process_dt_needed_entries(self):
+        """ Look to see if there are any DT_NEEDED entries in the binary
+            And process those if there are
+        """
+        global raw_output
+        runpath = ""
+        ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+        if ldlibpath is None:
+            ldlibpath = ""
+
+        dynsec = self._section_from_spec(".dynamic")
+        try:
+            runpath = self.get_dt_runpath(dynsec)
+        except AttributeError:
+            # dynsec is None, just return
+            return
+
+        for tag in dynsec.iter_tags():
+            if tag.entry.d_tag == 'DT_NEEDED':
+                rc = tag.needed.find("librte_pmd")
+                if (rc != -1):
+                    library = search_file(tag.needed,
+                                          runpath + ":" + ldlibpath +
+                                          ":/usr/lib64:/lib64:/usr/lib:/lib")
+                    if library is not None:
+                        if raw_output is False:
+                            print("Scanning %s for pmd information" % library)
+                        with open(library, 'rb') as file:
+                            try:
+                                libelf = ReadElf(file, sys.stdout)
+                            except ELFError as e:
+                                print("%s is no an ELF file" % library)
+                                continue
+                            libelf.process_dt_needed_entries()
+                            libelf.display_pmd_info_strings(".rodata")
+                            file.close()
+
+
+def scan_autoload_path(autoload_path):
+    global raw_output
+
+    if os.path.exists(autoload_path) is False:
+        return
+
+    try:
+        dirs = os.listdir(autoload_path)
+    except OSError as e:
+        # Couldn't read the directory, give up
+        return
+
+    for d in dirs:
+        dpath = os.path.join(autoload_path, d)
+        if os.path.isdir(dpath):
+            scan_autoload_path(dpath)
+        if os.path.isfile(dpath):
+            try:
+                file = open(dpath, 'rb')
+                readelf = ReadElf(file, sys.stdout)
+            except ELFError as e:
+                # this is likely not an elf file, skip it
+                continue
+            except IOError as e:
+                # No permission to read the file, skip it
+                continue
+
+            if raw_output is False:
+                print("Hw Support for library %s" % d)
+            readelf.display_pmd_info_strings(".rodata")
+            file.close()
+
+
+def scan_for_autoload_pmds(dpdk_path):
+    """
+    search the specified application or path for a pmd autoload path
+    then scan said path for pmds and report hw support
+    """
+    global raw_output
+
+    if (os.path.isfile(dpdk_path) is False):
+        if raw_output is False:
+            print("Must specify a file name")
+        return
+
+    file = open(dpdk_path, 'rb')
+    try:
+        readelf = ReadElf(file, sys.stdout)
+    except ElfError as e:
+        if raw_output is False:
+            print("Unable to parse %s" % file)
+        return
+
+    (autoload_path, scannedfile) = readelf.search_for_autoload_path()
+    if (autoload_path is None or autoload_path is ""):
+        if (raw_output is False):
+            print("No autoload path configured in %s" % dpdk_path)
+        return
+    if (raw_output is False):
+        if (scannedfile is None):
+            scannedfile = dpdk_path
+        print("Found autoload path %s in %s" % (autoload_path, scannedfile))
+
+    file.close()
+    if (raw_output is False):
+        print("Discovered Autoload HW Support:")
+    scan_autoload_path(autoload_path)
+    return
+
+
+def main(stream=None):
+    global raw_output
+    global pcidb
+
+    optparser = OptionParser(
+        usage='usage: %prog [-hrtp] [-d <pci id file] <elf-file>',
+        description="Dump pmd hardware support info",
+        add_help_option=True,
+        prog='pmdinfo.py')
+    optparser.add_option('-r', '--raw',
+                         action='store_true', dest='raw_output',
+                         help='Dump raw json strings')
+    optparser.add_option("-d", "--pcidb", dest="pcifile",
+                         help="specify a pci database "
+                              "to get vendor names from",
+                         default="/usr/share/hwdata/pci.ids", metavar="FILE")
+    optparser.add_option("-t", "--table", dest="tblout",
+                         help="output information on hw support as a hex table",
+                         action='store_true')
+    optparser.add_option("-p", "--plugindir", dest="pdir",
+                         help="scan dpdk for autoload plugins",
+                         action='store_true')
+
+    options, args = optparser.parse_args()
+
+    if options.raw_output:
+        raw_output = True
+
+    if options.pcifile:
+        pcidb = PCIIds(options.pcifile)
+        if pcidb is None:
+            print("Pci DB file not found")
+            exit(1)
+
+    if options.tblout:
+        options.pcifile = None
+        pcidb = None
+
+    if (len(args) == 0):
+        optparser.print_usage()
+        exit(1)
+
+    if options.pdir is True:
+        exit(scan_for_autoload_pmds(args[0]))
+
+    ldlibpath = os.environ.get('LD_LIBRARY_PATH')
+    if (ldlibpath is None):
+        ldlibpath = ""
+
+    if (os.path.exists(args[0]) is True):
+        myelffile = args[0]
+    else:
+        myelffile = search_file(
+            args[0], ldlibpath + ":/usr/lib64:/lib64:/usr/lib:/lib")
+
+    if (myelffile is None):
+        print("File not found")
+        sys.exit(1)
+
+    with open(myelffile, 'rb') as file:
+        try:
+            readelf = ReadElf(file, sys.stdout)
+            readelf.process_dt_needed_entries()
+            readelf.display_pmd_info_strings(".rodata")
+            sys.exit(0)
+
+        except ELFError as ex:
+            sys.stderr.write('ELF error: %s\n' % ex)
+            sys.exit(1)
+
+
+# -------------------------------------------------------------------------
+if __name__ == '__main__':
+    main()
-- 
2.7.0

Re: [PATCH v9 0/7] export PMD infos

[Neil Horman]

On Mon, Jul 04, 2016 at 03:13:58AM +0200, Thomas Monjalon wrote:
> This is a respin of the series from Neil.
> It was planned to be integrated in 16.07-rc1 but the discovered issues
> make a new revision needed.
> There are still few things which could be improved but it is not mandatory
> to fix them for an integration in 16.07-rc2:
>   - fix make clean after pmdinfogen
Clean works just fine, whats wrong with it?
>   - build installable pmdinfogen for target
Again, this seems to be working just fine for me, what do you see as wrong with
it?
>   - convert pmdinfo.py to Python 3
This was discussed weeks ago earlier in this thread, since there is no
requirement for python3 we decided not to use it.  Why renege on that now?
>   - document dependency pyelftools
In what capacity beyond simply noting the fact that it imports that package?

> 
> Changes done in this v9:
>   - fix build dependency of drivers on pmdinfogen
In what way is it broken?  The drivers directory is dependent on building
pmdinfogen, what more do you want?

>   - fix build of mlx4, mlx5, aesni
They weren't broken as of V8, not sure what you're getting at here

>   - fix new drivers bnxt, thunderx, kasumi
>   - fix MAINTAINERS file
>   - fix coding style in pmdinfogen
>   - add compiler checks for pmdinfogen
>   - remove useless functions in pmdinfogen
>   - fail build if pmdinfogen fails (set -e)
>   - fix verbose pmdinfogen run
>   - build pmdinfogen in buildtools directory (was app)
Are you kidding me!?  We just went 10 rounds over where to build this stupid
application, and after I finally gave in to your demands, you change it one
last time and call it a fix?  Thats a real slap in the face, thanks for that.

>   - install pmdinfogen in sdk package (was runtime)
>   - fix CamelCase in pmdinfo.py
>   - prefix executables with dpdk-
Again, seriously?  Are you just trying to assert some sort of not invented here
mentality?  We had this conversation over and over again weeks ago, but you
couldn't possibly chime in then?  And you have to slip it in now, because this
is the way you want it, as though I did it wrong previously?  

>   - rename PMD_REGISTER_DRIVER -> RTE_REGISTER_DRIVER
Annnd...one more, of course.  There simply no way you could leave this alone,
and modify subsequent pending patches to match the existing macro format, is
there? 

>   - separate commit for hostapp.mk refresh
>   - remove useless hostlib.mk
What does this have to do with 'fixes' for this series?

>   - spread doc in appropriate patches
> 
Again, why?  Whats wrong with adding documentation in its own patch?  To modify
my work to fit your sensibilities and call it a fix is really quite an insult.

Honestly, at this point, I'm done.  Do what you want to it, I don't plan on
making any more changes.

> Please review carefully.
> 
> Older changes:
> 
> v8)
>  * Modified symlink for pmdinfo to use dpdk_
>  * Added dpdk_pmdinfogen for pmdinfogen binary
> 
> v7)
>  * Fixed up copyright
>  * Switched buildtool makefile to use hostapp.mk
>  * Modified arch check to use RTE_ARCH_64
>  * Modifed hostapp.mk to drop output in build/app
>  * Additional testing on ppc64 to ensure big endian works
>  * Modified TO_NATIVE macro to use rte_byteorder inlines 
>    based on endianess of target ELF file
>  * Ran checkpatch on commits
>  * Fixed some typos in doc
> 
> v6)
>  * Added some programming guide documentation
>  * Reformatted python script with pep8
> 
> v5)
>  * Added a dpdk- prefix to pmdinfo symlink
>  * Renamed build announcement to PMDINFOGEN/BUILD
>  * Removed erroneous true statement from makefile
>  * Removed duplicate rte.hostapp.mk makefile
>  * Fixed some whitespace
>  * Whitespace fixups
>  * Fixed makefile if; then style
>  * Renamed module string C file
>  * Removed duplicate rte_pci_id definition
>  * Clarified macro names
>  * Removed PMD type attribute
>  * Fixed tools usage for 32 bit arches
>  * Removed some unused code
>  * Added a few comments
> 
> v4)
>  * Modified the operation of the -p option. As much as I don't like implying
> that autoloaded pmds are guaranteed to be there at run time, I'm having a hard
> time seeing how we can avoid specifying the application file to scan for the
> autoload directory.  Without it we can't determine which library the user means
> in a multiversion installation
>  * Cleaned up the help text
>  * Added a rule for an install target for pmdinfo
>  * Guarded against some tracebacks in pmdinfo
>  * Use DT_NEEDED entries to get versioned libraries in -p mode
>  * Fixed traceback that occurs on lack of input arguments
>  * Fixed some erroneous macro usage in drivers that aren't in the default build
> 
> v3)
>  * Made table mode printing the default mode
>  * Added a default path to the pci.ids file
>  * Modifed pmdinfo to use python rather than python3
>  * Cleaned up some changelog entries
>  * Added an export for RTE_EAL_PMD_PATH
>  * Added a plugin option to pmdinfo to scan for autoloaded DSO's
> 
> v2)
>  * Made the export macros a bit easier to expand
>  * Added a macro to optionally export command line strings
>  * Renamed utilties to be more appropriate
>    (pmdinfo -> pmdinfogen, pmd_hw_support.py -> pmdinfo.py)
>  * Added search capabilities to pmdinfo.py so that we search for libraries
>    linked using DT_NEEDINFO entries.  We search DT_RUNPATH, LD_LIBRARY_PATH,
>    /usr/lib and /lib
>  * Added serch abilities if the specified binary to pmdinfo isn't found, we
>    search LD_LIBRARY_PATH, /usr/lib and /lib
>  * Added an option to pmdinfo.py to pretty-print hardware support info using the
>    pci.ids database
> 
> ----------------
> Original summary
> ----------------
> 
>         So heres attempt number 2 at a method for exporting PMD hardware support
> information.  As we discussed previously, the consensus seems to be that pmd
> information should be:
> 
> 1) Able to be interrogated on any ELF binary (application binary or individual
> DSO)
> 2) Equally functional on statically linked applications or on DSO's
> 3) Resilient to symbol stripping
> 4) Script friendly
> 5) Show kernel dependencies
> 6) List driver options
> 7) Show driver name
> 8) Offer human readable output
> 9) Show DPDK version
> 10) Show driver version
> 11) Allow for expansion
> 12) Not place additional build environment dependencies on an application
> 
> I added item 12 myself, because I don't think its reasonable to require
> applications to use a specific linker script to get hardware support information
> (which precludes the use of a special .modinfo section like the kernel uses)
> 
> However, we still can use some tricks from the kernel to make this work.  In
> this approach, what I've done is the following:
> 
> A) Modified the driver registration macro to also define a variable:
> this_pmd_name<n>= "name"
> 
> Based on the unique name string pointed to by the above variable, we can
> query for an arbitrary number of other symbols following the pattern:
> <name>_<tag>
> 
> Where tag is some well known identifier for information we wish to export
> 
> B) Added a utility called pmdinfogen.  This utility is not meant for general use,
> but rather is used by the dpdk build environment itself when compiling pmds.
> for each object that uses the PMD_REGISTER_DRIVER macro, this utiity is run.  It
> searches for the symbols in (A), and using those, extracts the hardware support
> info, and module name from the object, using that to produce a new c file
> containing a single variable in the following format:
> 
> static const char <name>[] __attribute__((used)) = "PMD_DRIVER_INFO=<json>";
> 
> The <name> is arbitrary, as its static and not referenced.  The relevant bit is
> the string value assigned to it.  The <json> is a json encoded string of the
> extracted hardware support information pmdinfo found for the corresponding
> object.  This C file is suitable for compilation and relocatable linking back
> into the parent object file.  The result of this operation is that the object
> string table now contains a string that will not e removed by stripping, whos
> leading text (PMD_DRIVER_INFO) can be easily searched for at any time weather
> the symbol referring to it is stripped or not.
> 
> C) Added a utilty called pmdinfo.py.  This python script, searches the
> string table of the .rodata section of any provided ELF file looking for the
> PMD_DRIVER_INFO prefix on a string.  When found, it will interpret the remainder
> of the string as json, and output the hardware support for that ELF file (if
> any).
> 
> 
> This approach ticks most of the above boxes:
> 1) Impervious to stripping
> 2) Works on static and dynamic binaries
> 3) Human and script friendly
> 4) Allows for expansion
> 
> Because of (4) the other items should be pretty easy to implement, as its just a
> matter of modifying the macros to export the info, pmdinfo to encode it to json,
> and pmd_hw_support.py to read it.  I'm not adding them now, as theres alot of
> rote work to do to get some of them in place (e.g. DPDK has no current global
> VERSION macro, drivers don't have a consistent version scheme, command line
> strings have to be built for each driver, etc).  But once this is accepted,
> those items can be done as time allows and should be fairly easy to implement.
> 
> ----------------
> 
> Neil Horman (4):
>   drivers: export infos as string symbols
>   pmdinfogen: parse driver to generate code to export
>   eal: export default plugin path to external tools
>   tools: query binaries for support information
> 
> Thomas Monjalon (3):
>   mk: remove recipe for tool library
>   mk: refresh recipe for any host application
>   mk: link infos generated by pmdinfogen
> 
>  GNUmakefile                                    |   2 +-
>  MAINTAINERS                                    |   5 +
>  GNUmakefile => buildtools/Makefile             |  17 +-
>  GNUmakefile => buildtools/pmdinfogen/Makefile  |  21 +-
>  buildtools/pmdinfogen/pmdinfogen.c             | 451 ++++++++++++++++++
>  buildtools/pmdinfogen/pmdinfogen.h             |  99 ++++
>  doc/guides/freebsd_gsg/build_dpdk.rst          |   2 +-
>  doc/guides/linux_gsg/build_dpdk.rst            |   2 +-
>  doc/guides/prog_guide/dev_kit_build_system.rst |  38 +-
>  drivers/crypto/aesni_gcm/aesni_gcm_pmd.c       |   4 +-
>  drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c     |   4 +-
>  drivers/crypto/kasumi/rte_kasumi_pmd.c         |   4 +-
>  drivers/crypto/null/null_crypto_pmd.c          |   4 +-
>  drivers/crypto/qat/rte_qat_cryptodev.c         |   3 +-
>  drivers/crypto/snow3g/rte_snow3g_pmd.c         |   4 +-
>  drivers/net/af_packet/rte_eth_af_packet.c      |   4 +-
>  drivers/net/bnx2x/bnx2x_ethdev.c               |   6 +-
>  drivers/net/bnxt/bnxt_ethdev.c                 |   3 +-
>  drivers/net/bonding/rte_eth_bond_pmd.c         |   5 +-
>  drivers/net/cxgbe/cxgbe_ethdev.c               |   3 +-
>  drivers/net/e1000/em_ethdev.c                  |   3 +-
>  drivers/net/e1000/igb_ethdev.c                 |   6 +-
>  drivers/net/ena/ena_ethdev.c                   |   3 +-
>  drivers/net/enic/enic_ethdev.c                 |   3 +-
>  drivers/net/fm10k/fm10k_ethdev.c               |   3 +-
>  drivers/net/i40e/i40e_ethdev.c                 |   3 +-
>  drivers/net/i40e/i40e_ethdev_vf.c              |   3 +-
>  drivers/net/ixgbe/ixgbe_ethdev.c               |   6 +-
>  drivers/net/mlx4/mlx4.c                        |   3 +-
>  drivers/net/mlx5/mlx5.c                        |   3 +-
>  drivers/net/mpipe/mpipe_tilegx.c               |   4 +-
>  drivers/net/nfp/nfp_net.c                      |   3 +-
>  drivers/net/null/rte_eth_null.c                |   3 +-
>  drivers/net/pcap/rte_eth_pcap.c                |   4 +-
>  drivers/net/qede/qede_ethdev.c                 |   6 +-
>  drivers/net/ring/rte_eth_ring.c                |   3 +-
>  drivers/net/szedata2/rte_eth_szedata2.c        |   3 +-
>  drivers/net/thunderx/nicvf_ethdev.c            |   3 +-
>  drivers/net/vhost/rte_eth_vhost.c              |   3 +-
>  drivers/net/virtio/virtio_ethdev.c             |   3 +-
>  drivers/net/virtio/virtio_user_ethdev.c        |   2 +-
>  drivers/net/vmxnet3/vmxnet3_ethdev.c           |   3 +-
>  drivers/net/xenvirt/rte_eth_xenvirt.c          |   2 +-
>  lib/librte_eal/common/eal_common_dev.c         |   2 +-
>  lib/librte_eal/common/eal_common_options.c     |   4 +
>  lib/librte_eal/common/include/rte_dev.h        |  30 +-
>  mk/internal/rte.compile-pre.mk                 |  12 +
>  mk/rte.hostapp.mk                              |  23 +-
>  mk/rte.hostlib.mk                              | 116 -----
>  mk/rte.sdkbuild.mk                             |   4 +-
>  mk/rte.sdkconfig.mk                            |   3 +-
>  mk/rte.sdkinstall.mk                           |   5 +
>  tools/dpdk-pmdinfo.py                          | 628 +++++++++++++++++++++++++
>  53 files changed, 1371 insertions(+), 215 deletions(-)
>  copy GNUmakefile => buildtools/Makefile (87%)
>  copy GNUmakefile => buildtools/pmdinfogen/Makefile (84%)
>  create mode 100644 buildtools/pmdinfogen/pmdinfogen.c
>  create mode 100644 buildtools/pmdinfogen/pmdinfogen.h
>  delete mode 100644 mk/rte.hostlib.mk
>  create mode 100755 tools/dpdk-pmdinfo.py
> 
> -- 
> 2.7.0
> 
> 

Re: [PATCH v9 0/7] export PMD infos

[Thomas Monjalon]

Hi Neil,

I don't really understand why you don't accept I contribute to this
patchset. More details below.

2016-07-04 08:34, Neil Horman:
> On Mon, Jul 04, 2016 at 03:13:58AM +0200, Thomas Monjalon wrote:
> > This is a respin of the series from Neil.
> > It was planned to be integrated in 16.07-rc1 but the discovered issues
> > make a new revision needed.
> > There are still few things which could be improved but it is not mandatory
> > to fix them for an integration in 16.07-rc2:
> >   - fix make clean after pmdinfogen
> Clean works just fine, whats wrong with it?

"make clean" do not remove the .pmd.* files.
But it is not a blocking issue.

> >   - build installable pmdinfogen for target
> Again, this seems to be working just fine for me, what do you see as wrong with
> it?

If we install the SDK as a binary package for another arch, we cannot use
pmdinfogen as it is not cross-compiled (on purpose).
It was just a thought for possible future enhancement and is not blocking.

> >   - convert pmdinfo.py to Python 3
> This was discussed weeks ago earlier in this thread, since there is no
> requirement for python3 we decided not to use it.  Why renege on that now?

Yes we can take the script as is. It is just a note for future enhancement.
Not blocking.

> >   - document dependency pyelftools
> In what capacity beyond simply noting the fact that it imports that package?

I don't know where we could document this dependency.
Just a note, not blocking.

> > Changes done in this v9:
> >   - fix build dependency of drivers on pmdinfogen
> In what way is it broken?  The drivers directory is dependent on building
> pmdinfogen, what more do you want?

The drivers directory Makefile is a rte.subdir.mk. It does not handle
DEPDIRS. I've fix it in mk/rte.sdkbuild.mk
	drivers: | buildtools

> >   - fix build of mlx4, mlx5, aesni
> They weren't broken as of V8, not sure what you're getting at here

Yes they were because of typos.
I guess you were not compiling them.

> >   - fix new drivers bnxt, thunderx, kasumi
> >   - fix MAINTAINERS file
> >   - fix coding style in pmdinfogen
> >   - add compiler checks for pmdinfogen
> >   - remove useless functions in pmdinfogen
> >   - fail build if pmdinfogen fails (set -e)
> >   - fix verbose pmdinfogen run
> >   - build pmdinfogen in buildtools directory (was app)
> Are you kidding me!?  We just went 10 rounds over where to build this stupid
> application, and after I finally gave in to your demands, you change it one
> last time and call it a fix?  Thats a real slap in the face, thanks for that.

We were discussing reusing hostapp.mk.
And I was OK to use app/ as build directory.
After checking sdkinstall.mk, I changed my mind to build in buildtools/.
I do not call it a fix, just a change. Where do you see it is a fix? No slap.

> >   - install pmdinfogen in sdk package (was runtime)
> >   - fix CamelCase in pmdinfo.py
> >   - prefix executables with dpdk-
> Again, seriously?  Are you just trying to assert some sort of not invented here
> mentality?  We had this conversation over and over again weeks ago, but you
> couldn't possibly chime in then?  And you have to slip it in now, because this
> is the way you want it, as though I did it wrong previously?

You did not do it wrong.
After more thoughts, I really think we must be careful to have a consistent
namespace with tools we export publically.
I'm thinking to re-use the dpdk- prefix for other tools.
It's open to discussion as everything.

> >   - rename PMD_REGISTER_DRIVER -> RTE_REGISTER_DRIVER
> Annnd...one more, of course.  There simply no way you could leave this alone,
> and modify subsequent pending patches to match the existing macro format, is
> there?

I'm just trying to have a consistent namespace: RTE_ in code,
dpdk- for executables.

> >   - separate commit for hostapp.mk refresh
> >   - remove useless hostlib.mk
> What does this have to do with 'fixes' for this series?

It is not a fix, just a patch that I have added in the series.

> >   - spread doc in appropriate patches
> > 
> Again, why?  Whats wrong with adding documentation in its own patch?  To modify
> my work to fit your sensibilities and call it a fix is really quite an insult.

It is neither a fix nor an insult.
Having the doc and the code in the same patch helps to understand the patch.

> Honestly, at this point, I'm done.  Do what you want to it, I don't plan on
> making any more changes.

Please do not take it personnaly. I'm just trying to work on this
interesting feature.
Ideally you would have say "thank you for the work".
At least it would be interesting to have your review.

Re: [PATCH v9 0/7] export PMD infos

[Bruce Richardson]

On Mon, Jul 04, 2016 at 03:13:58AM +0200, Thomas Monjalon wrote:
> This is a respin of the series from Neil.
> It was planned to be integrated in 16.07-rc1 but the discovered issues
> make a new revision needed.
> There are still few things which could be improved but it is not mandatory
> to fix them for an integration in 16.07-rc2:
>   - fix make clean after pmdinfogen
>   - build installable pmdinfogen for target
>   - convert pmdinfo.py to Python 3
>   - document dependency pyelftools
> 
> Changes done in this v9:
>   - fix build dependency of drivers on pmdinfogen
>   - fix build of mlx4, mlx5, aesni
>   - fix new drivers bnxt, thunderx, kasumi
>   - fix MAINTAINERS file
>   - fix coding style in pmdinfogen
>   - add compiler checks for pmdinfogen
>   - remove useless functions in pmdinfogen
>   - fail build if pmdinfogen fails (set -e)
>   - fix verbose pmdinfogen run
>   - build pmdinfogen in buildtools directory (was app)
>   - install pmdinfogen in sdk package (was runtime)
>   - fix CamelCase in pmdinfo.py
>   - prefix executables with dpdk-
>   - rename PMD_REGISTER_DRIVER -> RTE_REGISTER_DRIVER
>   - separate commit for hostapp.mk refresh
>   - remove useless hostlib.mk
>   - spread doc in appropriate patches
> 
> Please review carefully.
> 
Haven't reviewed, but did test applying these patches on FreeBSD to see what
happens there. Compilation works fine after applying all patches, and when I
run: 
	strings testpmd | grep PMD_INFO_STRING

I get the appropriate metadata output of device ids etc. that shows that the
data is getting written into the static binaries ok.

For a shared library, rather than static build, there was a problem building
with clang - probably unrelated to this set, I haven't checked yet - but a
gcc shared build worked fine. Checking testpmd showed zero PMD_INFO strings
as expected, and librte_pmd_ixgbe.so showed two, again as expected.

For the script in the tools directory, the first problem is that python is
not to be found in "/usr/bin/python" as on Linux. To make it run on FreeBSD,
this should be changed to "/usr/bin/env python", as in dpdk_config.py.
For the "pyelftools" dependency, on FreeBSD, this is available in ports as
"py-pyelftools" and it installed ok for me. The final issue was the hard-coded
path to the pci-ids in /usr/share/hwdata. Patch to fix these script issues is
below.

Regards,
/Bruce

diff --git a/tools/dpdk-pmdinfo.py b/tools/dpdk-pmdinfo.py
index b8a9be2..8d19b90 100755
--- a/tools/dpdk-pmdinfo.py
+++ b/tools/dpdk-pmdinfo.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python
 # -------------------------------------------------------------------------
 #
 # Utility to dump PMD_INFO_STRING support from an object file
@@ -9,6 +9,7 @@ import sys
 from optparse import OptionParser
 import string
 import json
+import platform

 # For running from development directory. It should take precedence over the
 # installed pyelftools.
@@ -556,6 +557,14 @@ def main(stream=None):
     global raw_output
     global pcidb

+    pcifile_default = "./pci.ids" # for unknown OS's assume local file
+    if platform.system() == 'Linux':
+        pcifile_default = "/usr/share/hwdata/pci.ids"
+    elif platform.system() == 'FreeBSD':
+        pcifile_default = "/usr/local/share/pciids/pci.ids"
+        if not os.path.exists(pcifile_default):
+            pcifile_default = "/usr/share/misc/pci_vendors"
+
     optparser = OptionParser(
         usage='usage: %prog [-hrtp] [-d <pci id file] <elf-file>',
         description="Dump pmd hardware support info",
@@ -567,7 +576,7 @@ def main(stream=None):
     optparser.add_option("-d", "--pcidb", dest="pcifile",
                          help="specify a pci database "
                               "to get vendor names from",
-                         default="/usr/share/hwdata/pci.ids", metavar="FILE")
+                         default=pcifile_default, metavar="FILE")
     optparser.add_option("-t", "--table", dest="tblout",
                          help="output information on hw support as a hex table",
                          action='store_true')

Re: [PATCH v9 0/7] export PMD infos

[Neil Horman]

On Mon, Jul 04, 2016 at 03:10:14PM +0200, Thomas Monjalon wrote:
> Hi Neil,
> 
> I don't really understand why you don't accept I contribute to this
> patchset. More details below.
> 
I don't object to your contribution to this patchset.  What I object to is you
making changes contrary to what I just spent I don't even know how many weeks
arguing with you and others about.  Given your role as maintainer, it is
tantamout to a 'my way or the highway' position.

> 2016-07-04 08:34, Neil Horman:
> > On Mon, Jul 04, 2016 at 03:13:58AM +0200, Thomas Monjalon wrote:
> > > This is a respin of the series from Neil.
> > > It was planned to be integrated in 16.07-rc1 but the discovered issues
> > > make a new revision needed.
> > > There are still few things which could be improved but it is not mandatory
> > > to fix them for an integration in 16.07-rc2:
> > >   - fix make clean after pmdinfogen
> > Clean works just fine, whats wrong with it?
> 
> "make clean" do not remove the .pmd.* files.
> But it is not a blocking issue.
> 
They don't really need to be removed, as pmdinfogen will overwrite them
unilaterally when a rebuild is done.

> > >   - build installable pmdinfogen for target
> > Again, this seems to be working just fine for me, what do you see as wrong with
> > it?
> 
> If we install the SDK as a binary package for another arch, we cannot use
> pmdinfogen as it is not cross-compiled (on purpose).
> It was just a thought for possible future enhancement and is not blocking.
> 
Ah, you want cross-compile support, rather than just native target support?  I
can agree with that.

> > >   - convert pmdinfo.py to Python 3
> > This was discussed weeks ago earlier in this thread, since there is no
> > requirement for python3 we decided not to use it.  Why renege on that now?
> 
> Yes we can take the script as is. It is just a note for future enhancement.
> Not blocking.
Ok, but again, this was discussed weeks ago, the way its written currently
should work with python2 or python3 (dependent on the python environment setup).
converting any code to make it work better/more efficiently/etc with python3
seems like a step backwards

> 
> > >   - document dependency pyelftools
> > In what capacity beyond simply noting the fact that it imports that package?
> 
> I don't know where we could document this dependency.
> Just a note, not blocking.
> 
Ok, so you dont have any standard here, you just have some vague desire
for...something.  No other python script bothers with documenting what it
imports, so I think you can understand when you list this as an enhancement for
my work, but don't worry about any of the other work, its both confusing and
frustrating.

> > > Changes done in this v9:
> > >   - fix build dependency of drivers on pmdinfogen
> > In what way is it broken?  The drivers directory is dependent on building
> > pmdinfogen, what more do you want?
> 
> The drivers directory Makefile is a rte.subdir.mk. It does not handle
> DEPDIRS. I've fix it in mk/rte.sdkbuild.mk
> 	drivers: | buildtools
> 
> > >   - fix build of mlx4, mlx5, aesni
> > They weren't broken as of V8, not sure what you're getting at here
> 
> Yes they were because of typos.
> I guess you were not compiling them.
> 
No, I specifically compiled them in v8, as they were called out.

> > >   - fix new drivers bnxt, thunderx, kasumi
> > >   - fix MAINTAINERS file
> > >   - fix coding style in pmdinfogen
> > >   - add compiler checks for pmdinfogen
> > >   - remove useless functions in pmdinfogen
> > >   - fail build if pmdinfogen fails (set -e)
> > >   - fix verbose pmdinfogen run
> > >   - build pmdinfogen in buildtools directory (was app)
> > Are you kidding me!?  We just went 10 rounds over where to build this stupid
> > application, and after I finally gave in to your demands, you change it one
> > last time and call it a fix?  Thats a real slap in the face, thanks for that.
> 
> We were discussing reusing hostapp.mk.
> And I was OK to use app/ as build directory.
> After checking sdkinstall.mk, I changed my mind to build in buildtools/.
> I do not call it a fix, just a change. Where do you see it is a fix? No slap.
> 
No, its decidedly a slap.  You and I argued about this back and forth very
publically over the course of a week, and I finally aquiesced to your demands.
If you've changed your mind, thats fine I suppose, but to do so after you've
gotten your way is fairly frustrating.  To then make the change and repost it
without any prior consultation sends a very clear signal that my opinion on the
subject doesn't really matter.  You've put me in a position where I now either
need to say whatever change you want to make is ok with me, or I need to argue
against the thing that I wanted in the first place.  Its very insulting.

> > >   - install pmdinfogen in sdk package (was runtime)
> > >   - fix CamelCase in pmdinfo.py
> > >   - prefix executables with dpdk-
> > Again, seriously?  Are you just trying to assert some sort of not invented here
> > mentality?  We had this conversation over and over again weeks ago, but you
> > couldn't possibly chime in then?  And you have to slip it in now, because this
> > is the way you want it, as though I did it wrong previously?
> 
> You did not do it wrong.
> After more thoughts, I really think we must be careful to have a consistent
> namespace with tools we export publically.
> I'm thinking to re-use the dpdk- prefix for other tools.
> It's open to discussion as everything.
Except that we already had the discussion!  Panu, myself, and several others
went over this over a week ago, and I made the stated changes in response.  To
have you then post yet another version on my behalf sends a clear signal that
both our opinions and my work is irrelevant.  If you disagree with it, please
say so, but as the maintainer, posting a new version of someone elses work is
a fairly heavy handed way of saying that the default is the way you want it, and
we are all welcome to offer opinions on that.


> 
> > >   - rename PMD_REGISTER_DRIVER -> RTE_REGISTER_DRIVER
> > Annnd...one more, of course.  There simply no way you could leave this alone,
> > and modify subsequent pending patches to match the existing macro format, is
> > there?
> 
> I'm just trying to have a consistent namespace: RTE_ in code,
> dpdk- for executables.
> 
Ok, thats a fine opinion I suppose, but its both:

1) My work
2) the existing pattern

This patchset is my work.  If you want to add features, fix things, ask me to,
and we can have that discussion (I personally disagree with it, but as we've
seen previously, I'm willing to bend).  Point being, its my work, I don't
appreciate you making these sorts of changes on my behalf, as it immediately
implies I'm ok with them (they've still got my sign offs on them)

Its also orthogonal to the purpose of the patchset.  If you want to rewrite the
macro name, fine, submit a patchset that does so, please don't co-opt my work to
get your changes in.  I honestly have no opinion regarding this macro name, so
if you want to change it, its fine, but there was no need to do it as part of
this patchset. 

I'll also note that you changed the REGISTER_DRIVER macro, but left all the new
macros (REGISTER_CMDLINE_STRING and REGISTER_PCI_TBL) with the PMD_ prefix, so
you've created a bifurcation here.

> > >   - separate commit for hostapp.mk refresh
> > >   - remove useless hostlib.mk
> > What does this have to do with 'fixes' for this series?
> 
> It is not a fix, just a patch that I have added in the series.
> 
Again, completely orthogonal to the purpose of the series.

> > >   - spread doc in appropriate patches
> > > 
> > Again, why?  Whats wrong with adding documentation in its own patch?  To modify
> > my work to fit your sensibilities and call it a fix is really quite an insult.
> 
> It is neither a fix nor an insult.
> Having the doc and the code in the same patch helps to understand the patch.
> 
> > Honestly, at this point, I'm done.  Do what you want to it, I don't plan on
> > making any more changes.
> 
> Please do not take it personnaly. I'm just trying to work on this
> interesting feature.
> Ideally you would have say "thank you for the work".
Thomas, you took my work, changed it in ways I don't agree with, reverted things
I fought you on, and eventually aquiesced to, added changes that have no
association with the purpose of this series, and signed my name to it.

I apprecate the desire to help, but the way you've gone about this has
effectively said to me that you feel like I can't do it the way you want, so
you'll just do it for me.  Please understand how insulting that is.

> At least it would be interesting to have your review.
> 
Maybe, I need to take a breath over this.

Neil

Re: [PATCH v9 0/7] export PMD infos

[Thomas Monjalon]

2016-07-04 12:41, Neil Horman:
> On Mon, Jul 04, 2016 at 03:10:14PM +0200, Thomas Monjalon wrote:
> > Hi Neil,
> > 
> > I don't really understand why you don't accept I contribute to this
> > patchset. More details below.
> > 
> I don't object to your contribution to this patchset.  What I object to is you
> making changes contrary to what I just spent I don't even know how many weeks
> arguing with you and others about.  Given your role as maintainer, it is
> tantamout to a 'my way or the highway' position.

Thank you for explaining your feeling so clearly.
I'm sorry for any inconvenience or misunderstanding.
I now understand that you prefer I make some patches separately after
integrating your version.
So I can send a v10 based on your v8 which only fixes compilation.
Or you can do it yourself if you prefer.
Then I'll send another series for the other changes.
Deal?

> > 2016-07-04 08:34, Neil Horman:
> > > On Mon, Jul 04, 2016 at 03:13:58AM +0200, Thomas Monjalon wrote:
> > > > Changes done in this v9:
> > > >   - fix build dependency of drivers on pmdinfogen
> > > In what way is it broken?  The drivers directory is dependent on building
> > > pmdinfogen, what more do you want?
> > 
> > The drivers directory Makefile is a rte.subdir.mk. It does not handle
> > DEPDIRS. I've fix it in mk/rte.sdkbuild.mk
> > 	drivers: | buildtools
> > 
> > > >   - fix build of mlx4, mlx5, aesni
> > > They weren't broken as of V8, not sure what you're getting at here
> > 
> > Yes they were because of typos.
> > I guess you were not compiling them.
> > 
> No, I specifically compiled them in v8, as they were called out.

I didn't want to focus on these typos but as you insist, please check v8:
	http://dpdk.org/ml/archives/dev/2016-June/041994.html
Example:
	-PMD_REGISTER_DRIVER(rte_mlx4_driver)
	+PMD_REGISTER_DRIVER(rte_mlx4_driveri, mlx4)
	+DRIVER_REGISTER_PCI_TABLE(mlx4, mlx4_pci_id_map);
Which suggests that you are using vi: rte_mlx4_driveri
And as far as I remember your rework of PMD_REGISTER_DRIVER requires
to end the line with a semi-colon.

> > > >   - fix new drivers bnxt, thunderx, kasumi
> > > >   - fix MAINTAINERS file
> > > >   - fix coding style in pmdinfogen
> > > >   - add compiler checks for pmdinfogen
> > > >   - remove useless functions in pmdinfogen
> > > >   - fail build if pmdinfogen fails (set -e)
> > > >   - fix verbose pmdinfogen run
> > > >   - build pmdinfogen in buildtools directory (was app)
> > > Are you kidding me!?  We just went 10 rounds over where to build this stupid
> > > application, and after I finally gave in to your demands, you change it one
> > > last time and call it a fix?  Thats a real slap in the face, thanks for that.
> > 
> > We were discussing reusing hostapp.mk.
> > And I was OK to use app/ as build directory.
> > After checking sdkinstall.mk, I changed my mind to build in buildtools/.
> > I do not call it a fix, just a change. Where do you see it is a fix? No slap.
> > 
> No, its decidedly a slap.  You and I argued about this back and forth very
> publically over the course of a week, and I finally aquiesced to your demands.
> If you've changed your mind, thats fine I suppose, but to do so after you've
> gotten your way is fairly frustrating.  To then make the change and repost it
> without any prior consultation sends a very clear signal that my opinion on the
> subject doesn't really matter.  You've put me in a position where I now either
> need to say whatever change you want to make is ok with me, or I need to argue
> against the thing that I wanted in the first place.  Its very insulting.

I was in favor of having a generic hostapp.mk. It's still the case.
I just changed my mind on building in build/buildtools.
I thought it was not a big deal and I'm sorry.

> > > >   - install pmdinfogen in sdk package (was runtime)
> > > >   - fix CamelCase in pmdinfo.py
> > > >   - prefix executables with dpdk-
> > > Again, seriously?  Are you just trying to assert some sort of not invented here
> > > mentality?  We had this conversation over and over again weeks ago, but you
> > > couldn't possibly chime in then?  And you have to slip it in now, because this
> > > is the way you want it, as though I did it wrong previously?
> > 
> > You did not do it wrong.
> > After more thoughts, I really think we must be careful to have a consistent
> > namespace with tools we export publically.
> > I'm thinking to re-use the dpdk- prefix for other tools.
> > It's open to discussion as everything.
> Except that we already had the discussion!  Panu, myself, and several others
> went over this over a week ago, and I made the stated changes in response.

No you said you would prefix pmdinfogen but you did not.
As you said me you would add an entry in MAINTAINERS but you did not.
That's why I did it, thinking you agreed.

> To
> have you then post yet another version on my behalf sends a clear signal that
> both our opinions and my work is irrelevant.  If you disagree with it, please
> say so, but as the maintainer, posting a new version of someone elses work is
> a fairly heavy handed way of saying that the default is the way you want it, and
> we are all welcome to offer opinions on that.

> > > >   - rename PMD_REGISTER_DRIVER -> RTE_REGISTER_DRIVER
> > > Annnd...one more, of course.  There simply no way you could leave this alone,
> > > and modify subsequent pending patches to match the existing macro format, is
> > > there?
> > 
> > I'm just trying to have a consistent namespace: RTE_ in code,
> > dpdk- for executables.
> > 
> Ok, thats a fine opinion I suppose, but its both:
> 
> 1) My work
> 2) the existing pattern

The existing PMD_REGISTER_DRIVER was already your work.
I think it would be better to use RTE_ prefix for consistency but I
will send a separate patch for it.

> This patchset is my work.  If you want to add features, fix things, ask me to,
> and we can have that discussion (I personally disagree with it, but as we've
> seen previously, I'm willing to bend).  Point being, its my work, I don't
> appreciate you making these sorts of changes on my behalf, as it immediately
> implies I'm ok with them (they've still got my sign offs on them)

By rejecting the patches, it clearly shows you are not OK with them.

> Its also orthogonal to the purpose of the patchset.  If you want to rewrite the
> macro name, fine, submit a patchset that does so, please don't co-opt my work to
> get your changes in.  I honestly have no opinion regarding this macro name, so
> if you want to change it, its fine, but there was no need to do it as part of
> this patchset. 

OK

> I'll also note that you changed the REGISTER_DRIVER macro, but left all the new
> macros (REGISTER_CMDLINE_STRING and REGISTER_PCI_TBL) with the PMD_ prefix, so
> you've created a bifurcation here.
[...]
> > Please do not take it personnaly. I'm just trying to work on this
> > interesting feature.
> > Ideally you would have say "thank you for the work".
> Thomas, you took my work, changed it in ways I don't agree with, reverted things
> I fought you on, and eventually aquiesced to, added changes that have no
> association with the purpose of this series, and signed my name to it.

Precision, I co-signed the patches.

> I apprecate the desire to help, but the way you've gone about this has
> effectively said to me that you feel like I can't do it the way you want, so
> you'll just do it for me.  Please understand how insulting that is.

When I've seen your laconic answer to Remy pointing a required change:
	http://dpdk.org/ml/archives/dev/2016-June/042965.html
I've understood you didn't want to fix more things.
While at fixing those compilation errors, I've merged other changes, yes.

> > At least it would be interesting to have your review.
> > 
> Maybe, I need to take a breath over this.

No need to review this v9.
We just need to agree on who makes the v10 based on v8 to fix compilation.

Re: [PATCH v9 0/7] export PMD infos

[Neil Horman]

On Mon, Jul 04, 2016 at 10:10:37PM +0200, Thomas Monjalon wrote:
> 2016-07-04 12:41, Neil Horman:
> > On Mon, Jul 04, 2016 at 03:10:14PM +0200, Thomas Monjalon wrote:
> > > Hi Neil,
> > > 
> > > I don't really understand why you don't accept I contribute to this
> > > patchset. More details below.
> > > 
> > I don't object to your contribution to this patchset.  What I object to is you
> > making changes contrary to what I just spent I don't even know how many weeks
> > arguing with you and others about.  Given your role as maintainer, it is
> > tantamout to a 'my way or the highway' position.
> 
> Thank you for explaining your feeling so clearly.
> I'm sorry for any inconvenience or misunderstanding.
> I now understand that you prefer I make some patches separately after
> integrating your version.
> So I can send a v10 based on your v8 which only fixes compilation.
> Or you can do it yourself if you prefer.
> Then I'll send another series for the other changes.
> Deal?
> 
You go ahead and do it, I'm done.
Neil

Re: [PATCH v9 0/7] export PMD infos

[Thomas Monjalon]

2016-07-05 16:08, Neil Horman:
> On Mon, Jul 04, 2016 at 10:10:37PM +0200, Thomas Monjalon wrote:
> > 2016-07-04 12:41, Neil Horman:
> > > On Mon, Jul 04, 2016 at 03:10:14PM +0200, Thomas Monjalon wrote:
> > > > Hi Neil,
> > > > 
> > > > I don't really understand why you don't accept I contribute to this
> > > > patchset. More details below.
> > > > 
> > > I don't object to your contribution to this patchset.  What I object to is you
> > > making changes contrary to what I just spent I don't even know how many weeks
> > > arguing with you and others about.  Given your role as maintainer, it is
> > > tantamout to a 'my way or the highway' position.
> > 
> > Thank you for explaining your feeling so clearly.
> > I'm sorry for any inconvenience or misunderstanding.
> > I now understand that you prefer I make some patches separately after
> > integrating your version.
> > So I can send a v10 based on your v8 which only fixes compilation.
> > Or you can do it yourself if you prefer.
> > Then I'll send another series for the other changes.
> > Deal?
> > 
> You go ahead and do it, I'm done.

I'm sorry of this answer.
So I'll push the v8 to keep your work intact
and will reconsider sending some additional patches for review later.

Re: [PATCHv8 0/6] Implement pmd hardware support exports

[Thomas Monjalon]

2016-06-30 08:45, Remy Horton:
> On 17/06/2016 19:46, Neil Horman wrote:
> > Hey all-
> > 	So heres attempt number 2 at a method for exporting PMD hardware support
> > information.  As we discussed previously, the consensus seems to be that pmd
> > information should be:
> [..]
> > Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> > Acked-by: Panu Matilainen <pmatilai@redhat.com>
> > CC: Bruce Richardson <bruce.richardson@intel.com>
> > CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> > CC: Stephen Hemminger <stephen@networkplumber.org>
> > CC: Panu Matilainen <pmatilai@redhat.com>
> 
> Acked-by: Remy Horton <remy.horton@intel.com>

v8 applied

The only changes I did are to update new drivers introduced recently:

--- a/drivers/crypto/kasumi/rte_kasumi_pmd.c
+++ b/drivers/crypto/kasumi/rte_kasumi_pmd.c
@@ -654,4 +654,6 @@ static struct rte_driver cryptodev_kasumi_pmd_drv = {
        .uninit = cryptodev_kasumi_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_kasumi_pmd_drv);
+PMD_REGISTER_DRIVER(cryptodev_kasumi_pmd_drv, kasumi);
+DRIVER_REGISTER_PARAM_STRING(kasumi, "max_nb_queue_pairs=<int> "
+"max_nb_sessions=<int> socket_id=<int>");

--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -1046,4 +1046,5 @@ static struct rte_driver bnxt_pmd_drv = {
        .init = bnxt_rte_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(bnxt_pmd_drv);
+PMD_REGISTER_DRIVER(bnxt_pmd_drv, bnxt);
+DRIVER_REGISTER_PCI_TABLE(bnxt, bnxt_pci_id_map);

--- a/drivers/net/thunderx/nicvf_ethdev.c
+++ b/drivers/net/thunderx/nicvf_ethdev.c
@@ -1788,4 +1788,5 @@ static struct rte_driver rte_nicvf_driver = {
        .init = rte_nicvf_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(rte_nicvf_driver);
+PMD_REGISTER_DRIVER(rte_nicvf_driver, thunderx_nicvf);
+DRIVER_REGISTER_PCI_TABLE(thunderx_nicvf, pci_id_nicvf_map);

--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -437,4 +437,4 @@ static struct rte_driver virtio_user_driver = {
        .uninit = virtio_user_pmd_devuninit,
 };
 
-PMD_REGISTER_DRIVER(virtio_user_driver);
+PMD_REGISTER_DRIVER(virtio_user_driver, virtio_user);

Re: [PATCHv8 2/6] drivers: Update driver registration macro usage

[De Lara Guarch, Pablo]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
> Sent: Friday, June 17, 2016 7:46 PM
> To: dev@dpdk.org
> Cc: Neil Horman; Richardson, Bruce; Thomas Monjalon; Stephen Hemminger;
> Panu Matilainen
> Subject: [dpdk-dev] [PATCHv8 2/6] drivers: Update driver registration macro
> usage
> 
> Modify the PMD_REGISTER_DRIVER macro, adding a name argument to it.
> The
> addition of a name argument creates a token that can be used for subsequent
> macros in the creation of unique symbol names to export additional bits of
> information for use by the pmdinfogen tool.  For example:
> 
> PMD_REGISTER_DRIVER(ena_driver, ena);
> 
> registers the ena_driver struct as it always did, and creates a symbol
> const char this_pmd_name0[] __attribute__((used)) = "ena";
> 
> which pmdinfogen can search for and extract.  The subsequent macro
> 
> DRIVER_REGISTER_PCI_TABLE(ena, ena_pci_id_map);
> 
> creates a symbol const char ena_pci_tbl_export[] __attribute__((used)) =
> "ena_pci_id_map";
> 
> Which allows pmdinfogen to find the pci table of this driver
> 
> Using this pattern, we can export arbitrary bits of information.
> 
> pmdinfo uses this information to extract hardware support from an object
> file and create a json string to make hardware support info discoverable
> later.
> 
> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> Acked-by: Panu Matilainen <pmatilai@redhat.com>
> CC: Bruce Richardson <bruce.richardson@intel.com>
> CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> CC: Stephen Hemminger <stephen@networkplumber.org>
> CC: Panu Matilainen <pmatilai@redhat.com>

Hi,

I was running some crypto unit tests and I realized that the tests do not pass anymore,
because the creation of the software crypto device fails, due to this patch.

The reason is that the name of the pmd has changed (some names have changed,
like eth_pcap to pcap, whereas others have remained the same, like eth_null),
so the name passed to rte_eal_vdev_init has to be changed.

I am not sure if that is going to cause any problems, because users will have to
change their code because of this name change.

Also, if now the name is being set with PMD_REGISTER_DRIVER,
then we should remove it from the rte_driver structure, right?

static struct rte_driver pmd_ring_drv = {
        .name = "eth_ring", <-- This is being overwritten by PMD_REGISTER_DRIVER?
        .type = PMD_VDEV,
        .init = rte_pmd_ring_devinit,
        .uninit = rte_pmd_ring_devuninit,
};

PMD_REGISTER_DRIVER(pmd_ring_drv, eth_ring);

Pablo

Re: [PATCHv8 2/6] drivers: Update driver registration macro usage

[Neil Horman]

On Thu, Jul 07, 2016 at 11:00:59AM +0000, De Lara Guarch, Pablo wrote:
> 
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
> > Sent: Friday, June 17, 2016 7:46 PM
> > To: dev@dpdk.org
> > Cc: Neil Horman; Richardson, Bruce; Thomas Monjalon; Stephen Hemminger;
> > Panu Matilainen
> > Subject: [dpdk-dev] [PATCHv8 2/6] drivers: Update driver registration macro
> > usage
> > 
> > Modify the PMD_REGISTER_DRIVER macro, adding a name argument to it.
> > The
> > addition of a name argument creates a token that can be used for subsequent
> > macros in the creation of unique symbol names to export additional bits of
> > information for use by the pmdinfogen tool.  For example:
> > 
> > PMD_REGISTER_DRIVER(ena_driver, ena);
> > 
> > registers the ena_driver struct as it always did, and creates a symbol
> > const char this_pmd_name0[] __attribute__((used)) = "ena";
> > 
> > which pmdinfogen can search for and extract.  The subsequent macro
> > 
> > DRIVER_REGISTER_PCI_TABLE(ena, ena_pci_id_map);
> > 
> > creates a symbol const char ena_pci_tbl_export[] __attribute__((used)) =
> > "ena_pci_id_map";
> > 
> > Which allows pmdinfogen to find the pci table of this driver
> > 
> > Using this pattern, we can export arbitrary bits of information.
> > 
> > pmdinfo uses this information to extract hardware support from an object
> > file and create a json string to make hardware support info discoverable
> > later.
> > 
> > Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> > Acked-by: Panu Matilainen <pmatilai@redhat.com>
> > CC: Bruce Richardson <bruce.richardson@intel.com>
> > CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> > CC: Stephen Hemminger <stephen@networkplumber.org>
> > CC: Panu Matilainen <pmatilai@redhat.com>
> 
> Hi,
> 
> I was running some crypto unit tests and I realized that the tests do not pass anymore,
> because the creation of the software crypto device fails, due to this patch.
> 
> The reason is that the name of the pmd has changed (some names have changed,
> like eth_pcap to pcap, whereas others have remained the same, like eth_null),
> so the name passed to rte_eal_vdev_init has to be changed.
> 
> I am not sure if that is going to cause any problems, because users will have to
> change their code because of this name change.
> 
> Also, if now the name is being set with PMD_REGISTER_DRIVER,
> then we should remove it from the rte_driver structure, right?
> 
> static struct rte_driver pmd_ring_drv = {
>         .name = "eth_ring", <-- This is being overwritten by PMD_REGISTER_DRIVER?
>         .type = PMD_VDEV,
>         .init = rte_pmd_ring_devinit,
>         .uninit = rte_pmd_ring_devuninit,
> };
> 
> PMD_REGISTER_DRIVER(pmd_ring_drv, eth_ring);

The short answer to the question is yes, there is no real need to explicitly
call out the name in the definition of the rt_driver structure definition. I
made this change because some drivers failed to provide a driver name at all
(which was previously acceptable), but with the introduction of this patch the
name field is also used to construct the intermediate strings from which the
exported information is built.

We could restore the origional name of the driver that was renamed, though I'm
not sure which one that is (the example you provide should result in the pmd
having the exact same name).  Or we could update the registration macro to use
the cryptodev name macros.

Neil

> 
> Pablo
> 
> 

[PATCH] crypto: normalize cryptodev pmd names with macros

[Neil Horman]

Recently reported, the introduction of pmd information exports led to a
breakage of cryptodev unit tests because the test infrastructure relies on the
cryptodev names being available in macros.  This patch fixes the pmd naming to
use the macro names.  Note that the macro names were already pre-stringified,
which won't work as the PMD_REGISTER_DRIVER macro requires the name in both a
processing token and stringified form.  As such the names are defined now as
tokens, and converted where needed to stringified form on demand using RTE_STR.

Tested using the null and aesni_mb crypto pmds, as I don't have hardware for any
other device.  Also not build tested on snow3g or kasumi pmd because building
those requires external libraries that appear to not be open source licensed.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: "De Lara Guarch, Pablo" <pablo.de.lara.guarch@intel.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---
 app/test/test_cryptodev.c                          | 20 ++++++++++----------
 app/test/test_cryptodev_perf.c                     | 18 +++++++++---------
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c           |  7 +++----
 drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h   |  6 +++---
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c         |  7 +++----
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h |  2 +-
 drivers/crypto/kasumi/rte_kasumi_pmd.c             |  5 ++---
 drivers/crypto/null/null_crypto_pmd.c              |  7 +++----
 drivers/crypto/null/null_crypto_pmd_private.h      |  6 +++---
 drivers/crypto/qat/rte_qat_cryptodev.c             |  4 ++--
 drivers/crypto/snow3g/rte_snow3g_pmd.c             |  4 ++--
 lib/librte_cryptodev/rte_cryptodev.h               | 12 ++++++------
 12 files changed, 47 insertions(+), 51 deletions(-)

diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index fbfe1d0..1e1f887 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -186,12 +186,12 @@ testsuite_setup(void)
 		if (nb_devs < 2) {
 			for (i = nb_devs; i < 2; i++) {
 				ret = rte_eal_vdev_init(
-					CRYPTODEV_NAME_AESNI_MB_PMD, NULL);
+					RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD), NULL);
 
 				TEST_ASSERT(ret == 0,
 					"Failed to create instance %u of"
 					" pmd : %s",
-					i, CRYPTODEV_NAME_AESNI_MB_PMD);
+					i, RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD));
 			}
 		}
 	}
@@ -203,10 +203,10 @@ testsuite_setup(void)
 		if (nb_devs < 2) {
 			for (i = nb_devs; i < 2; i++) {
 				TEST_ASSERT_SUCCESS(rte_eal_vdev_init(
-					CRYPTODEV_NAME_AESNI_GCM_PMD, NULL),
+					RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD), NULL),
 					"Failed to create instance %u of"
 					" pmd : %s",
-					i, CRYPTODEV_NAME_AESNI_GCM_PMD);
+					i, RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD));
 			}
 		}
 	}
@@ -217,10 +217,10 @@ testsuite_setup(void)
 		if (nb_devs < 2) {
 			for (i = nb_devs; i < 2; i++) {
 				TEST_ASSERT_SUCCESS(rte_eal_vdev_init(
-					CRYPTODEV_NAME_SNOW3G_PMD, NULL),
+					RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD), NULL),
 					"Failed to create instance %u of"
 					" pmd : %s",
-					i, CRYPTODEV_NAME_SNOW3G_PMD);
+					i, RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD));
 			}
 		}
 	}
@@ -231,10 +231,10 @@ testsuite_setup(void)
 		if (nb_devs < 2) {
 			for (i = nb_devs; i < 2; i++) {
 				TEST_ASSERT_SUCCESS(rte_eal_vdev_init(
-					CRYPTODEV_NAME_KASUMI_PMD, NULL),
+					RTE_STR(CRYPTODEV_NAME_KASUMI_PMD), NULL),
 					"Failed to create instance %u of"
 					" pmd : %s",
-					i, CRYPTODEV_NAME_KASUMI_PMD);
+					i, RTE_STR(CRYPTODEV_NAME_KASUMI_PMD));
 			}
 		}
 	}
@@ -246,12 +246,12 @@ testsuite_setup(void)
 		if (nb_devs < 2) {
 			for (i = nb_devs; i < 2; i++) {
 				int dev_id = rte_eal_vdev_init(
-					CRYPTODEV_NAME_NULL_PMD, NULL);
+					RTE_STR(CRYPTODEV_NAME_NULL_PMD), NULL);
 
 				TEST_ASSERT(dev_id >= 0,
 					"Failed to create instance %u of"
 					" pmd : %s",
-					i, CRYPTODEV_NAME_NULL_PMD);
+					i, RTE_STR(CRYPTODEV_NAME_NULL_PMD));
 			}
 		}
 	}
diff --git a/app/test/test_cryptodev_perf.c b/app/test/test_cryptodev_perf.c
index d728211..815c41f 100644
--- a/app/test/test_cryptodev_perf.c
+++ b/app/test/test_cryptodev_perf.c
@@ -120,15 +120,15 @@ static const char *chain_mode_name(enum chain_mode mode)
 static const char *pmd_name(enum rte_cryptodev_type pmd)
 {
 	switch (pmd) {
-	case RTE_CRYPTODEV_NULL_PMD: return CRYPTODEV_NAME_NULL_PMD; break;
+	case RTE_CRYPTODEV_NULL_PMD: return RTE_STR(CRYPTODEV_NAME_NULL_PMD); break;
 	case RTE_CRYPTODEV_AESNI_GCM_PMD:
-		return CRYPTODEV_NAME_AESNI_GCM_PMD;
+		return RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD);
 	case RTE_CRYPTODEV_AESNI_MB_PMD:
-		return CRYPTODEV_NAME_AESNI_MB_PMD;
+		return RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD);
 	case RTE_CRYPTODEV_QAT_SYM_PMD:
-		return CRYPTODEV_NAME_QAT_SYM_PMD;
+		return RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD);
 	case RTE_CRYPTODEV_SNOW3G_PMD:
-		return CRYPTODEV_NAME_SNOW3G_PMD;
+		return RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD);
 	default:
 		return "";
 	}
@@ -249,11 +249,11 @@ testsuite_setup(void)
 		if (nb_devs < 2) {
 			for (i = nb_devs; i < 2; i++) {
 				ret = rte_eal_vdev_init(
-					CRYPTODEV_NAME_AESNI_MB_PMD, NULL);
+					RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD), NULL);
 
 				TEST_ASSERT(ret == 0,
 					"Failed to create instance %u of pmd : %s",
-					i, CRYPTODEV_NAME_AESNI_MB_PMD);
+					i, RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD));
 			}
 		}
 	}
@@ -264,11 +264,11 @@ testsuite_setup(void)
 		if (nb_devs < 2) {
 			for (i = nb_devs; i < 2; i++) {
 				ret = rte_eal_vdev_init(
-					CRYPTODEV_NAME_SNOW3G_PMD, NULL);
+					RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD), NULL);
 
 				TEST_ASSERT(ret == 0,
 					"Failed to create instance %u of pmd : %s",
-					i, CRYPTODEV_NAME_SNOW3G_PMD);
+					i, RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD));
 			}
 		}
 	}
diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
index f43e407..f9be44a 100644
--- a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
+++ b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
@@ -57,7 +57,7 @@ create_unique_device_name(char *name, size_t size)
 	if (name == NULL)
 		return -EINVAL;
 
-	ret = snprintf(name, size, "%s_%u", CRYPTODEV_NAME_AESNI_GCM_PMD,
+	ret = snprintf(name, size, "%s_%u", RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD),
 			unique_name_id++);
 	if (ret < 0)
 		return ret;
@@ -515,12 +515,11 @@ aesni_gcm_uninit(const char *name)
 }
 
 static struct rte_driver aesni_gcm_pmd_drv = {
-	.name = CRYPTODEV_NAME_AESNI_GCM_PMD,
 	.type = PMD_VDEV,
 	.init = aesni_gcm_init,
 	.uninit = aesni_gcm_uninit
 };
 
-PMD_REGISTER_DRIVER(aesni_gcm_pmd_drv, aesni_gcm);
-DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+PMD_REGISTER_DRIVER(aesni_gcm_pmd_drv, CRYPTODEV_NAME_AESNI_GCM_PMD);
+DRIVER_REGISTER_PARAM_STRING(CRYPTODEV_NAME_AESNI_GCM_PMD, "max_nb_queue_pairs=<int> "
 "max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h b/drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h
index a42f941..9878d6e 100644
--- a/drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h
+++ b/drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h
@@ -37,18 +37,18 @@
 
 #define GCM_LOG_ERR(fmt, args...) \
 	RTE_LOG(ERR, CRYPTODEV, "[%s] %s() line %u: " fmt "\n",  \
-			CRYPTODEV_NAME_AESNI_GCM_PMD, \
+			RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD), \
 			__func__, __LINE__, ## args)
 
 #ifdef RTE_LIBRTE_AESNI_MB_DEBUG
 #define GCM_LOG_INFO(fmt, args...) \
 	RTE_LOG(INFO, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
-			CRYPTODEV_NAME_AESNI_GCM_PMD, \
+			RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD), \
 			__func__, __LINE__, ## args)
 
 #define GCM_LOG_DBG(fmt, args...) \
 	RTE_LOG(DEBUG, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
-			CRYPTODEV_NAME_AESNI_GCM_PMD, \
+			RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD), \
 			__func__, __LINE__, ## args)
 #else
 #define GCM_LOG_INFO(fmt, args...)
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
index db3e562..2140b92 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
@@ -54,7 +54,7 @@ create_unique_device_name(char *name, size_t size)
 	if (name == NULL)
 		return -EINVAL;
 
-	ret = snprintf(name, size, "%s_%u", CRYPTODEV_NAME_AESNI_MB_PMD,
+	ret = snprintf(name, size, "%s_%u", RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD),
 			unique_name_id++);
 	if (ret < 0)
 		return ret;
@@ -715,12 +715,11 @@ cryptodev_aesni_mb_uninit(const char *name)
 }
 
 static struct rte_driver cryptodev_aesni_mb_pmd_drv = {
-	.name = CRYPTODEV_NAME_AESNI_MB_PMD,
 	.type = PMD_VDEV,
 	.init = cryptodev_aesni_mb_init,
 	.uninit = cryptodev_aesni_mb_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drvi, aesni_mb);
-DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drv, CRYPTODEV_NAME_AESNI_MB_PMD);
+DRIVER_REGISTER_PARAM_STRING(CRYPTODEV_NAME_AESNI_MB_PMD, "max_nb_queue_pairs=<int> "
 "max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h
index 949d9a6..17f367f 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h
@@ -37,7 +37,7 @@
 
 #define MB_LOG_ERR(fmt, args...) \
 	RTE_LOG(ERR, CRYPTODEV, "[%s] %s() line %u: " fmt "\n",  \
-			CRYPTODEV_NAME_AESNI_MB_PMD, \
+			RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD), \
 			__func__, __LINE__, ## args)
 
 #ifdef RTE_LIBRTE_AESNI_MB_DEBUG
diff --git a/drivers/crypto/kasumi/rte_kasumi_pmd.c b/drivers/crypto/kasumi/rte_kasumi_pmd.c
index 1d5af46..a98314e 100644
--- a/drivers/crypto/kasumi/rte_kasumi_pmd.c
+++ b/drivers/crypto/kasumi/rte_kasumi_pmd.c
@@ -648,12 +648,11 @@ cryptodev_kasumi_uninit(const char *name)
 }
 
 static struct rte_driver cryptodev_kasumi_pmd_drv = {
-	.name = CRYPTODEV_NAME_KASUMI_PMD,
 	.type = PMD_VDEV,
 	.init = cryptodev_kasumi_init,
 	.uninit = cryptodev_kasumi_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_kasumi_pmd_drv, kasumi);
-DRIVER_REGISTER_PARAM_STRING(kasumi, "max_nb_queue_pairs=<int> "
+PMD_REGISTER_DRIVER(cryptodev_kasumi_pmd_drv, CRYPTODEV_NAME_KASUMI_PMD);
+DRIVER_REGISTER_PARAM_STRING(CRYPTODEV_NAME_KASUMI_PMD, "max_nb_queue_pairs=<int> "
 "max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/null/null_crypto_pmd.c b/drivers/crypto/null/null_crypto_pmd.c
index 0a195ed..e03d0eb 100644
--- a/drivers/crypto/null/null_crypto_pmd.c
+++ b/drivers/crypto/null/null_crypto_pmd.c
@@ -51,7 +51,7 @@ create_unique_device_name(char *name, size_t size)
 	if (name == NULL)
 		return -EINVAL;
 
-	ret = snprintf(name, size, "%s_%u", CRYPTODEV_NAME_NULL_PMD,
+	ret = snprintf(name, size, "%s_%u", RTE_STR(CRYPTODEV_NAME_NULL_PMD),
 			unique_name_id++);
 	if (ret < 0)
 		return ret;
@@ -269,12 +269,11 @@ cryptodev_null_uninit(const char *name)
 }
 
 static struct rte_driver cryptodev_null_pmd_drv = {
-	.name = CRYPTODEV_NAME_NULL_PMD,
 	.type = PMD_VDEV,
 	.init = cryptodev_null_init,
 	.uninit = cryptodev_null_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_null_pmd_drv, cryptodev_null_pmd);
-DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+PMD_REGISTER_DRIVER(cryptodev_null_pmd_drv, CRYPTODEV_NAME_NULL_PMD);
+DRIVER_REGISTER_PARAM_STRING(CRYPTODEV_NAME_NULL_PMD, "max_nb_queue_pairs=<int> "
 "max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/null/null_crypto_pmd_private.h b/drivers/crypto/null/null_crypto_pmd_private.h
index 2a4c739..acebc97 100644
--- a/drivers/crypto/null/null_crypto_pmd_private.h
+++ b/drivers/crypto/null/null_crypto_pmd_private.h
@@ -37,18 +37,18 @@
 
 #define NULL_CRYPTO_LOG_ERR(fmt, args...) \
 	RTE_LOG(ERR, CRYPTODEV, "[%s] %s() line %u: " fmt "\n",  \
-			CRYPTODEV_NAME_NULL_PMD, \
+			RTE_STR(CRYPTODEV_NAME_NULL_PMD), \
 			__func__, __LINE__, ## args)
 
 #ifdef RTE_LIBRTE_NULL_CRYPTO_DEBUG
 #define NULL_CRYPTO_LOG_INFO(fmt, args...) \
 	RTE_LOG(INFO, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
-			CRYPTODEV_NAME_NULL_PMD, \
+			RTE_STR(CRYPTODEV_NAME_NULL_PMD), \
 			__func__, __LINE__, ## args)
 
 #define NULL_CRYPTO_LOG_DBG(fmt, args...) \
 	RTE_LOG(DEBUG, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
-			CRYPTODEV_NAME_NULL_PMD, \
+			RTE_STR(CRYPTODEV_NAME_NULL_PMD), \
 			__func__, __LINE__, ## args)
 #else
 #define NULL_CRYPTO_LOG_INFO(fmt, args...)
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index b35267e..345afe3 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -134,6 +134,6 @@ static struct rte_driver pmd_qat_drv = {
 	.init = rte_qat_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(pmd_qat_drv, qat);
-DRIVER_REGISTER_PCI_TABLE(qat, pci_id_qat_map);
+PMD_REGISTER_DRIVER(pmd_qat_drv, CRYPTODEV_NAME_QAT_SYM_PMD);
+DRIVER_REGISTER_PCI_TABLE(CRYPTODEV_NAME_QAT_SYM_PMD, pci_id_qat_map);
 
diff --git a/drivers/crypto/snow3g/rte_snow3g_pmd.c b/drivers/crypto/snow3g/rte_snow3g_pmd.c
index ddb724c..771a0cf 100644
--- a/drivers/crypto/snow3g/rte_snow3g_pmd.c
+++ b/drivers/crypto/snow3g/rte_snow3g_pmd.c
@@ -645,6 +645,6 @@ static struct rte_driver cryptodev_snow3g_pmd_drv = {
 	.uninit = cryptodev_snow3g_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_snow3g_pmd_drv, snow3g);
-DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+PMD_REGISTER_DRIVER(cryptodev_snow3g_pmd_drv, CRYPTODEV_NAME_SNOW3G_PMD);
+DRIVER_REGISTER_PARAM_STRING(CRYPTODEV_NAME_SNOW3G_PMD, "max_nb_queue_pairs=<int> "
 "max_nb_sessions=<int> socket_id=<int>");
diff --git a/lib/librte_cryptodev/rte_cryptodev.h b/lib/librte_cryptodev/rte_cryptodev.h
index 7768f0a..4c02307 100644
--- a/lib/librte_cryptodev/rte_cryptodev.h
+++ b/lib/librte_cryptodev/rte_cryptodev.h
@@ -49,17 +49,17 @@ extern "C" {
 #include "rte_crypto.h"
 #include "rte_dev.h"
 
-#define CRYPTODEV_NAME_NULL_PMD		("cryptodev_null_pmd")
+#define CRYPTODEV_NAME_NULL_PMD		cryptodev_null_pmd
 /**< Null crypto PMD device name */
-#define CRYPTODEV_NAME_AESNI_MB_PMD	("cryptodev_aesni_mb_pmd")
+#define CRYPTODEV_NAME_AESNI_MB_PMD	cryptodev_aesni_mb_pmd
 /**< AES-NI Multi buffer PMD device name */
-#define CRYPTODEV_NAME_AESNI_GCM_PMD	("cryptodev_aesni_gcm_pmd")
+#define CRYPTODEV_NAME_AESNI_GCM_PMD	cryptodev_aesni_gcm_pmd
 /**< AES-NI GCM PMD device name */
-#define CRYPTODEV_NAME_QAT_SYM_PMD	("cryptodev_qat_sym_pmd")
+#define CRYPTODEV_NAME_QAT_SYM_PMD	cryptodev_qat_sym_pmd
 /**< Intel QAT Symmetric Crypto PMD device name */
-#define CRYPTODEV_NAME_SNOW3G_PMD	("cryptodev_snow3g_pmd")
+#define CRYPTODEV_NAME_SNOW3G_PMD	cryptodev_snow3g_pmd
 /**< SNOW 3G PMD device name */
-#define CRYPTODEV_NAME_KASUMI_PMD	("cryptodev_kasumi_pmd")
+#define CRYPTODEV_NAME_KASUMI_PMD	cryptodev_kasumi_pmd
 /**< KASUMI PMD device name */
 
 /** Crypto device type */
-- 
2.5.5

Re: [PATCH] crypto: normalize cryptodev pmd names with macros

[De Lara Guarch, Pablo]

Hi Neil,

> -----Original Message-----
> From: Neil Horman [mailto:nhorman@tuxdriver.com]
> Sent: Thursday, July 07, 2016 4:38 PM
> To: dev@dpdk.org
> Cc: Neil Horman; Richardson, Bruce; Thomas Monjalon; De Lara Guarch,
> Pablo; Stephen Hemminger; Panu Matilainen
> Subject: [PATCH] crypto: normalize cryptodev pmd names with macros
> 
> Recently reported, the introduction of pmd information exports led to a
> breakage of cryptodev unit tests because the test infrastructure relies on the
> cryptodev names being available in macros.  This patch fixes the pmd naming
> to
> use the macro names.  Note that the macro names were already pre-
> stringified,
> which won't work as the PMD_REGISTER_DRIVER macro requires the name in
> both a
> processing token and stringified form.  As such the names are defined now as
> tokens, and converted where needed to stringified form on demand using
> RTE_STR.
> 
> Tested using the null and aesni_mb crypto pmds, as I don't have hardware for
> any
> other device.  Also not build tested on snow3g or kasumi pmd because
> building
> those requires external libraries that appear to not be open source licensed.
> 
> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> CC: Bruce Richardson <bruce.richardson@intel.com>
> CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> CC: "De Lara Guarch, Pablo" <pablo.de.lara.guarch@intel.com>
> CC: Stephen Hemminger <stephen@networkplumber.org>
> CC: Panu Matilainen <pmatilai@redhat.com>
> ---
>  app/test/test_cryptodev.c                          | 20 ++++++++++----------
>  app/test/test_cryptodev_perf.c                     | 18 +++++++++---------
>  drivers/crypto/aesni_gcm/aesni_gcm_pmd.c           |  7 +++----
>  drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h   |  6 +++---
>  drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c         |  7 +++----
>  drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h |  2 +-
>  drivers/crypto/kasumi/rte_kasumi_pmd.c             |  5 ++---
>  drivers/crypto/null/null_crypto_pmd.c              |  7 +++----
>  drivers/crypto/null/null_crypto_pmd_private.h      |  6 +++---
>  drivers/crypto/qat/rte_qat_cryptodev.c             |  4 ++--
>  drivers/crypto/snow3g/rte_snow3g_pmd.c             |  4 ++--
>  lib/librte_cryptodev/rte_cryptodev.h               | 12 ++++++------
>  12 files changed, 47 insertions(+), 51 deletions(-)

Thanks for this patch. I tested snow3g and kasumi, and they don't compile.
I have a fix for that, so I can send a v2 of this patch if it is OK for you.

Also, we should make these changes in the other PMDs as well, right?
I mean, avoid setting the name of the driver directly in the structure and go back to the original name.
I can do that as well, if you want (maybe a separate patch, as this one is only related to crypto).

Thanks,
Pablo

Re: [PATCH] crypto: normalize cryptodev pmd names with macros

[Thomas Monjalon]

2016-07-07 11:37, Neil Horman:
> -PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drvi, aesni_mb);
> -DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
> +PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drv, CRYPTODEV_NAME_AESNI_MB_PMD);
> +DRIVER_REGISTER_PARAM_STRING(CRYPTODEV_NAME_AESNI_MB_PMD, "max_nb_queue_pairs=<int> "
>  "max_nb_sessions=<int> socket_id=<int>");

A coding style opinion:
The line DRIVER_REGISTER_PARAM_STRING is a bit long and I think it would be
nice to have only one param per line with an indent like this:

DRIVER_REGISTER_PARAM_STRING(CRYPTODEV_NAME_AESNI_MB_PMD,
	"max_nb_queue_pairs=<int> "
	"max_nb_sessions=<int> "
	"socket_id=<int>");

Re: [PATCH] crypto: normalize cryptodev pmd names with macros

[Neil Horman]

On Fri, Jul 08, 2016 at 09:09:10AM +0000, De Lara Guarch, Pablo wrote:
> Hi Neil,
> 
> > -----Original Message-----
> > From: Neil Horman [mailto:nhorman@tuxdriver.com]
> > Sent: Thursday, July 07, 2016 4:38 PM
> > To: dev@dpdk.org
> > Cc: Neil Horman; Richardson, Bruce; Thomas Monjalon; De Lara Guarch,
> > Pablo; Stephen Hemminger; Panu Matilainen
> > Subject: [PATCH] crypto: normalize cryptodev pmd names with macros
> > 
> > Recently reported, the introduction of pmd information exports led to a
> > breakage of cryptodev unit tests because the test infrastructure relies on the
> > cryptodev names being available in macros.  This patch fixes the pmd naming
> > to
> > use the macro names.  Note that the macro names were already pre-
> > stringified,
> > which won't work as the PMD_REGISTER_DRIVER macro requires the name in
> > both a
> > processing token and stringified form.  As such the names are defined now as
> > tokens, and converted where needed to stringified form on demand using
> > RTE_STR.
> > 
> > Tested using the null and aesni_mb crypto pmds, as I don't have hardware for
> > any
> > other device.  Also not build tested on snow3g or kasumi pmd because
> > building
> > those requires external libraries that appear to not be open source licensed.
> > 
> > Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> > CC: Bruce Richardson <bruce.richardson@intel.com>
> > CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> > CC: "De Lara Guarch, Pablo" <pablo.de.lara.guarch@intel.com>
> > CC: Stephen Hemminger <stephen@networkplumber.org>
> > CC: Panu Matilainen <pmatilai@redhat.com>
> > ---
> >  app/test/test_cryptodev.c                          | 20 ++++++++++----------
> >  app/test/test_cryptodev_perf.c                     | 18 +++++++++---------
> >  drivers/crypto/aesni_gcm/aesni_gcm_pmd.c           |  7 +++----
> >  drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h   |  6 +++---
> >  drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c         |  7 +++----
> >  drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h |  2 +-
> >  drivers/crypto/kasumi/rte_kasumi_pmd.c             |  5 ++---
> >  drivers/crypto/null/null_crypto_pmd.c              |  7 +++----
> >  drivers/crypto/null/null_crypto_pmd_private.h      |  6 +++---
> >  drivers/crypto/qat/rte_qat_cryptodev.c             |  4 ++--
> >  drivers/crypto/snow3g/rte_snow3g_pmd.c             |  4 ++--
> >  lib/librte_cryptodev/rte_cryptodev.h               | 12 ++++++------
> >  12 files changed, 47 insertions(+), 51 deletions(-)
> 
> Thanks for this patch. I tested snow3g and kasumi, and they don't compile.
> I have a fix for that, so I can send a v2 of this patch if it is OK for you.
> 

I suppose thats fine, sure, though I'm really not comfortable with an open
source project requiring what appears to be non-open source components (though I
can't really tell what the snow3g or kasumi license is).  Regardless, whats the
compilation error?


> Also, we should make these changes in the other PMDs as well, right?
> I mean, avoid setting the name of the driver directly in the structure and go back to the original name.
> I can do that as well, if you want (maybe a separate patch, as this one is only related to crypto).
> 
I think thats kind of two questions:

1) Should we remove the static setting of the name in the pmd_driver structure
in favor of doing it in the registration macro?

2) Should we be consistent in the name conversion (from the setting in the
structure instance definition to the string in the macro parameter)?

The answer to (1) is yes, though having it in both places is harmless, since the
former will just get overridden.  We should definately remove the static
setting, to avoid confusion, but theres not any functional rush to do so.

The answer to (2) is yes, but I think thats already done.  I don't think we
deviated in too many places (if any), as the strings for all the net devices
were directly set (i.e. not through macros), and I just transferred them.

Neil

> Thanks,
> Pablo
> 

Re: [PATCH] crypto: normalize cryptodev pmd names with macros

[Thomas Monjalon]

2016-07-08 08:17, Neil Horman:
> I suppose thats fine, sure, though I'm really not comfortable with an open
> source project requiring what appears to be non-open source components (though I
> can't really tell what the snow3g or kasumi license is).

Yes you're right.
IMHO, it is acceptable because it is free (just need a registration)
but it can be discussed.
I think there are some legal export issues because of crypto aim.
I suggest to avoid discussing this topic here and launch a dedicated thread
(with better visibility).

Re: [PATCH] crypto: normalize cryptodev pmd names with macros

[Neil Horman]

On Fri, Jul 08, 2016 at 02:40:01PM +0200, Thomas Monjalon wrote:
> 2016-07-08 08:17, Neil Horman:
> > I suppose thats fine, sure, though I'm really not comfortable with an open
> > source project requiring what appears to be non-open source components (though I
> > can't really tell what the snow3g or kasumi license is).
> 
> Yes you're right.
> IMHO, it is acceptable because it is free (just need a registration)
> but it can be discussed.
> I think there are some legal export issues because of crypto aim.
Yeah, crypto usually has munitions export restrictions.  I'm fine with that, as
long as the license to the code is GPL.  And I can't tell what the license is
because you have to register to get it before you can see it.

If you, or someone can confirm that the snow3g and kasumi ssl libraries are open
source compatible (or better still if the license can be announced prior to
registration on the web site), then I'm fine with it, and am happy to test with
it, I just don't want to register without knowing that beforehand.

> I suggest to avoid discussing this topic here and launch a dedicated thread
> (with better visibility).
> 
Sounds good to me
Neil

> 

Re: [PATCH] crypto: normalize cryptodev pmd names with macros

[De Lara Guarch, Pablo]

> -----Original Message-----
> From: Neil Horman [mailto:nhorman@tuxdriver.com]
> Sent: Friday, July 08, 2016 1:18 PM
> To: De Lara Guarch, Pablo
> Cc: dev@dpdk.org; Richardson, Bruce; Thomas Monjalon; Stephen
> Hemminger; Panu Matilainen
> Subject: Re: [PATCH] crypto: normalize cryptodev pmd names with macros
> 
> On Fri, Jul 08, 2016 at 09:09:10AM +0000, De Lara Guarch, Pablo wrote:
> > Hi Neil,
> >
> > > -----Original Message-----
> > > From: Neil Horman [mailto:nhorman@tuxdriver.com]
> > > Sent: Thursday, July 07, 2016 4:38 PM
> > > To: dev@dpdk.org
> > > Cc: Neil Horman; Richardson, Bruce; Thomas Monjalon; De Lara Guarch,
> > > Pablo; Stephen Hemminger; Panu Matilainen
> > > Subject: [PATCH] crypto: normalize cryptodev pmd names with macros
> > >
> > > Recently reported, the introduction of pmd information exports led to a
> > > breakage of cryptodev unit tests because the test infrastructure relies on
> the
> > > cryptodev names being available in macros.  This patch fixes the pmd
> naming
> > > to
> > > use the macro names.  Note that the macro names were already pre-
> > > stringified,
> > > which won't work as the PMD_REGISTER_DRIVER macro requires the
> name in
> > > both a
> > > processing token and stringified form.  As such the names are defined now
> as
> > > tokens, and converted where needed to stringified form on demand using
> > > RTE_STR.
> > >
> > > Tested using the null and aesni_mb crypto pmds, as I don't have hardware
> for
> > > any
> > > other device.  Also not build tested on snow3g or kasumi pmd because
> > > building
> > > those requires external libraries that appear to not be open source
> licensed.
> > >
> > > Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> > > CC: Bruce Richardson <bruce.richardson@intel.com>
> > > CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> > > CC: "De Lara Guarch, Pablo" <pablo.de.lara.guarch@intel.com>
> > > CC: Stephen Hemminger <stephen@networkplumber.org>
> > > CC: Panu Matilainen <pmatilai@redhat.com>
> > > ---
> > >  app/test/test_cryptodev.c                          | 20 ++++++++++----------
> > >  app/test/test_cryptodev_perf.c                     | 18 +++++++++---------
> > >  drivers/crypto/aesni_gcm/aesni_gcm_pmd.c           |  7 +++----
> > >  drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h   |  6 +++---
> > >  drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c         |  7 +++----
> > >  drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h |  2 +-
> > >  drivers/crypto/kasumi/rte_kasumi_pmd.c             |  5 ++---
> > >  drivers/crypto/null/null_crypto_pmd.c              |  7 +++----
> > >  drivers/crypto/null/null_crypto_pmd_private.h      |  6 +++---
> > >  drivers/crypto/qat/rte_qat_cryptodev.c             |  4 ++--
> > >  drivers/crypto/snow3g/rte_snow3g_pmd.c             |  4 ++--
> > >  lib/librte_cryptodev/rte_cryptodev.h               | 12 ++++++------
> > >  12 files changed, 47 insertions(+), 51 deletions(-)
> >
> > Thanks for this patch. I tested snow3g and kasumi, and they don't compile.
> > I have a fix for that, so I can send a v2 of this patch if it is OK for you.
> >
> 
> I suppose thats fine, sure, though I'm really not comfortable with an open
> source project requiring what appears to be non-open source components
> (though I
> can't really tell what the snow3g or kasumi license is).  Regardless, whats the
> compilation error?

drivers/crypto/snow3g/rte_snow3g_pmd_ops.c: In function 'snow3g_pmd_qp_create_processed_ops_ring':
drivers/crypto/snow3g/rte_snow3g_pmd_ops.c:208:152: error: 'cryptodev_snow3g_pmd' undeclared (first use in this function)
   SNOW3G_LOG_ERR("Unable to reuse existing ring %s"
                                                                                                                                                        ^     
dpdk-16.04/drivers/crypto/snow3g/rte_snow3g_pmd_ops.c:208:152: note: each undeclared identifier is reported only once for each function it appears in
dpdk-16.04/drivers/crypto/snow3g/rte_snow3g_pmd_ops.c: In function 'snow3g_pmd_session_configure':
dpdk-16.04/drivers/crypto/snow3g/rte_snow3g_pmd_ops.c:296:117: error: 'cryptodev_snow3g_pmd' undeclared (first use in this function)
   SNOW3G_LOG_ERR("invalid session struct");

...

The solution is adding RTE_STR  in SNOW3G_LOG_ERR.

...
> 
> 
> > Also, we should make these changes in the other PMDs as well, right?
> > I mean, avoid setting the name of the driver directly in the structure and go
> back to the original name.
> > I can do that as well, if you want (maybe a separate patch, as this one is
> only related to crypto).
> >
> I think thats kind of two questions:
> 
> 1) Should we remove the static setting of the name in the pmd_driver
> structure
> in favor of doing it in the registration macro?
> 
> 2) Should we be consistent in the name conversion (from the setting in the
> structure instance definition to the string in the macro parameter)?
> 
> The answer to (1) is yes, though having it in both places is harmless, since the
> former will just get overridden.  We should definately remove the static
> setting, to avoid confusion, but theres not any functional rush to do so.

Will do that in a separate patch.

> 
> The answer to (2) is yes, but I think thats already done.  I don't think we
> deviated in too many places (if any), as the strings for all the net devices
> were directly set (i.e. not through macros), and I just transferred them.

Some driver names have changed (like eth_pcap to pcap).
I can revert that to the original name and we can rename them in the next release,
after a deprecation notice, since this is breaking the API.

> 
> Neil
> 
> > Thanks,
> > Pablo
> >

Re: [PATCH] crypto: normalize cryptodev pmd names with macros

[Neil Horman]

On Fri, Jul 08, 2016 at 02:00:20PM +0000, De Lara Guarch, Pablo wrote:
> 
> 
> > -----Original Message-----
> > From: Neil Horman [mailto:nhorman@tuxdriver.com]
> > Sent: Friday, July 08, 2016 1:18 PM
> > To: De Lara Guarch, Pablo
> > Cc: dev@dpdk.org; Richardson, Bruce; Thomas Monjalon; Stephen
> > Hemminger; Panu Matilainen
> > Subject: Re: [PATCH] crypto: normalize cryptodev pmd names with macros
> > 
> > On Fri, Jul 08, 2016 at 09:09:10AM +0000, De Lara Guarch, Pablo wrote:
> > > Hi Neil,
> > >
> > > > -----Original Message-----
> > > > From: Neil Horman [mailto:nhorman@tuxdriver.com]
> > > > Sent: Thursday, July 07, 2016 4:38 PM
> > > > To: dev@dpdk.org
> > > > Cc: Neil Horman; Richardson, Bruce; Thomas Monjalon; De Lara Guarch,
> > > > Pablo; Stephen Hemminger; Panu Matilainen
> > > > Subject: [PATCH] crypto: normalize cryptodev pmd names with macros
> > > >
> > > > Recently reported, the introduction of pmd information exports led to a
> > > > breakage of cryptodev unit tests because the test infrastructure relies on
> > the
> > > > cryptodev names being available in macros.  This patch fixes the pmd
> > naming
> > > > to
> > > > use the macro names.  Note that the macro names were already pre-
> > > > stringified,
> > > > which won't work as the PMD_REGISTER_DRIVER macro requires the
> > name in
> > > > both a
> > > > processing token and stringified form.  As such the names are defined now
> > as
> > > > tokens, and converted where needed to stringified form on demand using
> > > > RTE_STR.
> > > >
> > > > Tested using the null and aesni_mb crypto pmds, as I don't have hardware
> > for
> > > > any
> > > > other device.  Also not build tested on snow3g or kasumi pmd because
> > > > building
> > > > those requires external libraries that appear to not be open source
> > licensed.
> > > >
> > > > Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> > > > CC: Bruce Richardson <bruce.richardson@intel.com>
> > > > CC: Thomas Monjalon <thomas.monjalon@6wind.com>
> > > > CC: "De Lara Guarch, Pablo" <pablo.de.lara.guarch@intel.com>
> > > > CC: Stephen Hemminger <stephen@networkplumber.org>
> > > > CC: Panu Matilainen <pmatilai@redhat.com>
> > > > ---
> > > >  app/test/test_cryptodev.c                          | 20 ++++++++++----------
> > > >  app/test/test_cryptodev_perf.c                     | 18 +++++++++---------
> > > >  drivers/crypto/aesni_gcm/aesni_gcm_pmd.c           |  7 +++----
> > > >  drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h   |  6 +++---
> > > >  drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c         |  7 +++----
> > > >  drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h |  2 +-
> > > >  drivers/crypto/kasumi/rte_kasumi_pmd.c             |  5 ++---
> > > >  drivers/crypto/null/null_crypto_pmd.c              |  7 +++----
> > > >  drivers/crypto/null/null_crypto_pmd_private.h      |  6 +++---
> > > >  drivers/crypto/qat/rte_qat_cryptodev.c             |  4 ++--
> > > >  drivers/crypto/snow3g/rte_snow3g_pmd.c             |  4 ++--
> > > >  lib/librte_cryptodev/rte_cryptodev.h               | 12 ++++++------
> > > >  12 files changed, 47 insertions(+), 51 deletions(-)
> > >
> > > Thanks for this patch. I tested snow3g and kasumi, and they don't compile.
> > > I have a fix for that, so I can send a v2 of this patch if it is OK for you.
> > >
> > 
> > I suppose thats fine, sure, though I'm really not comfortable with an open
> > source project requiring what appears to be non-open source components
> > (though I
> > can't really tell what the snow3g or kasumi license is).  Regardless, whats the
> > compilation error?
> 
> drivers/crypto/snow3g/rte_snow3g_pmd_ops.c: In function 'snow3g_pmd_qp_create_processed_ops_ring':
> drivers/crypto/snow3g/rte_snow3g_pmd_ops.c:208:152: error: 'cryptodev_snow3g_pmd' undeclared (first use in this function)
>    SNOW3G_LOG_ERR("Unable to reuse existing ring %s"
>                                                                                                                                                         ^     
> dpdk-16.04/drivers/crypto/snow3g/rte_snow3g_pmd_ops.c:208:152: note: each undeclared identifier is reported only once for each function it appears in
> dpdk-16.04/drivers/crypto/snow3g/rte_snow3g_pmd_ops.c: In function 'snow3g_pmd_session_configure':
> dpdk-16.04/drivers/crypto/snow3g/rte_snow3g_pmd_ops.c:296:117: error: 'cryptodev_snow3g_pmd' undeclared (first use in this function)
>    SNOW3G_LOG_ERR("invalid session struct");
> 
> ...
> 
> The solution is adding RTE_STR  in SNOW3G_LOG_ERR.
> 
Ah,thanks.  Its odd, I used cscope to do a find and replace for all the other
instances of the RTE_STR conversion, I wonder how I missed that.
Neil

> ...
> > 
> > 
> > > Also, we should make these changes in the other PMDs as well, right?
> > > I mean, avoid setting the name of the driver directly in the structure and go
> > back to the original name.
> > > I can do that as well, if you want (maybe a separate patch, as this one is
> > only related to crypto).
> > >
> > I think thats kind of two questions:
> > 
> > 1) Should we remove the static setting of the name in the pmd_driver
> > structure
> > in favor of doing it in the registration macro?
> > 
> > 2) Should we be consistent in the name conversion (from the setting in the
> > structure instance definition to the string in the macro parameter)?
> > 
> > The answer to (1) is yes, though having it in both places is harmless, since the
> > former will just get overridden.  We should definately remove the static
> > setting, to avoid confusion, but theres not any functional rush to do so.
> 
> Will do that in a separate patch.
> 
> > 
> > The answer to (2) is yes, but I think thats already done.  I don't think we
> > deviated in too many places (if any), as the strings for all the net devices
> > were directly set (i.e. not through macros), and I just transferred them.
> 
> Some driver names have changed (like eth_pcap to pcap).
> I can revert that to the original name and we can rename them in the next release,
> after a deprecation notice, since this is breaking the API.
> 
> > 
> > Neil
> > 
> > > Thanks,
> > > Pablo
> > >
> 

[PATCH v2] crypto: normalize cryptodev pmd names with macros

[Pablo de Lara]

Recently reported, the introduction of pmd information exports led to a
breakage of cryptodev unit tests because the test infrastructure relies on the
cryptodev names being available in macros.  This patch fixes the pmd naming to
use the macro names.  Note that the macro names were already pre-stringified,
which won't work as the PMD_REGISTER_DRIVER macro requires the name in both a
processing token and stringified form.  As such the names are defined now as
tokens, and converted where needed to stringified form on demand using RTE_STR.

Tested using the null and aesni_mb crypto pmds, as I don't have hardware for any
other device.  Also not build tested on snow3g or kasumi pmd because building
those requires external libraries that appear to not be open source licensed.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: "De Lara Guarch, Pablo" <pablo.de.lara.guarch@intel.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---

Changes in v2:
- Fixed compilation issue on snow3g/kasumi pmd
- Removed static name of qat pmd

 app/test/test_cryptodev.c                          | 20 ++++++++++----------
 app/test/test_cryptodev_perf.c                     | 18 +++++++++---------
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c           |  7 +++----
 drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h   |  6 +++---
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c         |  7 +++----
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h |  2 +-
 drivers/crypto/kasumi/rte_kasumi_pmd.c             |  7 +++----
 drivers/crypto/kasumi/rte_kasumi_pmd_private.h     |  6 +++---
 drivers/crypto/null/null_crypto_pmd.c              |  7 +++----
 drivers/crypto/null/null_crypto_pmd_private.h      |  6 +++---
 drivers/crypto/qat/rte_qat_cryptodev.c             |  5 ++---
 drivers/crypto/snow3g/rte_snow3g_pmd.c             |  7 +++----
 drivers/crypto/snow3g/rte_snow3g_pmd_private.h     |  6 +++---
 lib/librte_cryptodev/rte_cryptodev.h               | 12 ++++++------
 14 files changed, 55 insertions(+), 61 deletions(-)

diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index fbfe1d0..1e1f887 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -186,12 +186,12 @@ testsuite_setup(void)
 		if (nb_devs < 2) {
 			for (i = nb_devs; i < 2; i++) {
 				ret = rte_eal_vdev_init(
-					CRYPTODEV_NAME_AESNI_MB_PMD, NULL);
+					RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD), NULL);
 
 				TEST_ASSERT(ret == 0,
 					"Failed to create instance %u of"
 					" pmd : %s",
-					i, CRYPTODEV_NAME_AESNI_MB_PMD);
+					i, RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD));
 			}
 		}
 	}
@@ -203,10 +203,10 @@ testsuite_setup(void)
 		if (nb_devs < 2) {
 			for (i = nb_devs; i < 2; i++) {
 				TEST_ASSERT_SUCCESS(rte_eal_vdev_init(
-					CRYPTODEV_NAME_AESNI_GCM_PMD, NULL),
+					RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD), NULL),
 					"Failed to create instance %u of"
 					" pmd : %s",
-					i, CRYPTODEV_NAME_AESNI_GCM_PMD);
+					i, RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD));
 			}
 		}
 	}
@@ -217,10 +217,10 @@ testsuite_setup(void)
 		if (nb_devs < 2) {
 			for (i = nb_devs; i < 2; i++) {
 				TEST_ASSERT_SUCCESS(rte_eal_vdev_init(
-					CRYPTODEV_NAME_SNOW3G_PMD, NULL),
+					RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD), NULL),
 					"Failed to create instance %u of"
 					" pmd : %s",
-					i, CRYPTODEV_NAME_SNOW3G_PMD);
+					i, RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD));
 			}
 		}
 	}
@@ -231,10 +231,10 @@ testsuite_setup(void)
 		if (nb_devs < 2) {
 			for (i = nb_devs; i < 2; i++) {
 				TEST_ASSERT_SUCCESS(rte_eal_vdev_init(
-					CRYPTODEV_NAME_KASUMI_PMD, NULL),
+					RTE_STR(CRYPTODEV_NAME_KASUMI_PMD), NULL),
 					"Failed to create instance %u of"
 					" pmd : %s",
-					i, CRYPTODEV_NAME_KASUMI_PMD);
+					i, RTE_STR(CRYPTODEV_NAME_KASUMI_PMD));
 			}
 		}
 	}
@@ -246,12 +246,12 @@ testsuite_setup(void)
 		if (nb_devs < 2) {
 			for (i = nb_devs; i < 2; i++) {
 				int dev_id = rte_eal_vdev_init(
-					CRYPTODEV_NAME_NULL_PMD, NULL);
+					RTE_STR(CRYPTODEV_NAME_NULL_PMD), NULL);
 
 				TEST_ASSERT(dev_id >= 0,
 					"Failed to create instance %u of"
 					" pmd : %s",
-					i, CRYPTODEV_NAME_NULL_PMD);
+					i, RTE_STR(CRYPTODEV_NAME_NULL_PMD));
 			}
 		}
 	}
diff --git a/app/test/test_cryptodev_perf.c b/app/test/test_cryptodev_perf.c
index d728211..815c41f 100644
--- a/app/test/test_cryptodev_perf.c
+++ b/app/test/test_cryptodev_perf.c
@@ -120,15 +120,15 @@ static const char *chain_mode_name(enum chain_mode mode)
 static const char *pmd_name(enum rte_cryptodev_type pmd)
 {
 	switch (pmd) {
-	case RTE_CRYPTODEV_NULL_PMD: return CRYPTODEV_NAME_NULL_PMD; break;
+	case RTE_CRYPTODEV_NULL_PMD: return RTE_STR(CRYPTODEV_NAME_NULL_PMD); break;
 	case RTE_CRYPTODEV_AESNI_GCM_PMD:
-		return CRYPTODEV_NAME_AESNI_GCM_PMD;
+		return RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD);
 	case RTE_CRYPTODEV_AESNI_MB_PMD:
-		return CRYPTODEV_NAME_AESNI_MB_PMD;
+		return RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD);
 	case RTE_CRYPTODEV_QAT_SYM_PMD:
-		return CRYPTODEV_NAME_QAT_SYM_PMD;
+		return RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD);
 	case RTE_CRYPTODEV_SNOW3G_PMD:
-		return CRYPTODEV_NAME_SNOW3G_PMD;
+		return RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD);
 	default:
 		return "";
 	}
@@ -249,11 +249,11 @@ testsuite_setup(void)
 		if (nb_devs < 2) {
 			for (i = nb_devs; i < 2; i++) {
 				ret = rte_eal_vdev_init(
-					CRYPTODEV_NAME_AESNI_MB_PMD, NULL);
+					RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD), NULL);
 
 				TEST_ASSERT(ret == 0,
 					"Failed to create instance %u of pmd : %s",
-					i, CRYPTODEV_NAME_AESNI_MB_PMD);
+					i, RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD));
 			}
 		}
 	}
@@ -264,11 +264,11 @@ testsuite_setup(void)
 		if (nb_devs < 2) {
 			for (i = nb_devs; i < 2; i++) {
 				ret = rte_eal_vdev_init(
-					CRYPTODEV_NAME_SNOW3G_PMD, NULL);
+					RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD), NULL);
 
 				TEST_ASSERT(ret == 0,
 					"Failed to create instance %u of pmd : %s",
-					i, CRYPTODEV_NAME_SNOW3G_PMD);
+					i, RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD));
 			}
 		}
 	}
diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
index f43e407..f9be44a 100644
--- a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
+++ b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
@@ -57,7 +57,7 @@ create_unique_device_name(char *name, size_t size)
 	if (name == NULL)
 		return -EINVAL;
 
-	ret = snprintf(name, size, "%s_%u", CRYPTODEV_NAME_AESNI_GCM_PMD,
+	ret = snprintf(name, size, "%s_%u", RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD),
 			unique_name_id++);
 	if (ret < 0)
 		return ret;
@@ -515,12 +515,11 @@ aesni_gcm_uninit(const char *name)
 }
 
 static struct rte_driver aesni_gcm_pmd_drv = {
-	.name = CRYPTODEV_NAME_AESNI_GCM_PMD,
 	.type = PMD_VDEV,
 	.init = aesni_gcm_init,
 	.uninit = aesni_gcm_uninit
 };
 
-PMD_REGISTER_DRIVER(aesni_gcm_pmd_drv, aesni_gcm);
-DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+PMD_REGISTER_DRIVER(aesni_gcm_pmd_drv, CRYPTODEV_NAME_AESNI_GCM_PMD);
+DRIVER_REGISTER_PARAM_STRING(CRYPTODEV_NAME_AESNI_GCM_PMD, "max_nb_queue_pairs=<int> "
 "max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h b/drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h
index a42f941..9878d6e 100644
--- a/drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h
+++ b/drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h
@@ -37,18 +37,18 @@
 
 #define GCM_LOG_ERR(fmt, args...) \
 	RTE_LOG(ERR, CRYPTODEV, "[%s] %s() line %u: " fmt "\n",  \
-			CRYPTODEV_NAME_AESNI_GCM_PMD, \
+			RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD), \
 			__func__, __LINE__, ## args)
 
 #ifdef RTE_LIBRTE_AESNI_MB_DEBUG
 #define GCM_LOG_INFO(fmt, args...) \
 	RTE_LOG(INFO, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
-			CRYPTODEV_NAME_AESNI_GCM_PMD, \
+			RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD), \
 			__func__, __LINE__, ## args)
 
 #define GCM_LOG_DBG(fmt, args...) \
 	RTE_LOG(DEBUG, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
-			CRYPTODEV_NAME_AESNI_GCM_PMD, \
+			RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD), \
 			__func__, __LINE__, ## args)
 #else
 #define GCM_LOG_INFO(fmt, args...)
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
index db3e562..2140b92 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
@@ -54,7 +54,7 @@ create_unique_device_name(char *name, size_t size)
 	if (name == NULL)
 		return -EINVAL;
 
-	ret = snprintf(name, size, "%s_%u", CRYPTODEV_NAME_AESNI_MB_PMD,
+	ret = snprintf(name, size, "%s_%u", RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD),
 			unique_name_id++);
 	if (ret < 0)
 		return ret;
@@ -715,12 +715,11 @@ cryptodev_aesni_mb_uninit(const char *name)
 }
 
 static struct rte_driver cryptodev_aesni_mb_pmd_drv = {
-	.name = CRYPTODEV_NAME_AESNI_MB_PMD,
 	.type = PMD_VDEV,
 	.init = cryptodev_aesni_mb_init,
 	.uninit = cryptodev_aesni_mb_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drvi, aesni_mb);
-DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drv, CRYPTODEV_NAME_AESNI_MB_PMD);
+DRIVER_REGISTER_PARAM_STRING(CRYPTODEV_NAME_AESNI_MB_PMD, "max_nb_queue_pairs=<int> "
 "max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h
index 949d9a6..17f367f 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h
@@ -37,7 +37,7 @@
 
 #define MB_LOG_ERR(fmt, args...) \
 	RTE_LOG(ERR, CRYPTODEV, "[%s] %s() line %u: " fmt "\n",  \
-			CRYPTODEV_NAME_AESNI_MB_PMD, \
+			RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD), \
 			__func__, __LINE__, ## args)
 
 #ifdef RTE_LIBRTE_AESNI_MB_DEBUG
diff --git a/drivers/crypto/kasumi/rte_kasumi_pmd.c b/drivers/crypto/kasumi/rte_kasumi_pmd.c
index 1d5af46..400a248 100644
--- a/drivers/crypto/kasumi/rte_kasumi_pmd.c
+++ b/drivers/crypto/kasumi/rte_kasumi_pmd.c
@@ -61,7 +61,7 @@ create_unique_device_name(char *name, size_t size)
 	if (name == NULL)
 		return -EINVAL;
 
-	ret = snprintf(name, size, "%s_%u", CRYPTODEV_NAME_KASUMI_PMD,
+	ret = snprintf(name, size, "%s_%u", RTE_STR(CRYPTODEV_NAME_KASUMI_PMD),
 			unique_name_id++);
 	if (ret < 0)
 		return ret;
@@ -648,12 +648,11 @@ cryptodev_kasumi_uninit(const char *name)
 }
 
 static struct rte_driver cryptodev_kasumi_pmd_drv = {
-	.name = CRYPTODEV_NAME_KASUMI_PMD,
 	.type = PMD_VDEV,
 	.init = cryptodev_kasumi_init,
 	.uninit = cryptodev_kasumi_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_kasumi_pmd_drv, kasumi);
-DRIVER_REGISTER_PARAM_STRING(kasumi, "max_nb_queue_pairs=<int> "
+PMD_REGISTER_DRIVER(cryptodev_kasumi_pmd_drv, CRYPTODEV_NAME_KASUMI_PMD);
+DRIVER_REGISTER_PARAM_STRING(CRYPTODEV_NAME_KASUMI_PMD, "max_nb_queue_pairs=<int> "
 "max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/kasumi/rte_kasumi_pmd_private.h b/drivers/crypto/kasumi/rte_kasumi_pmd_private.h
index 04e1c43..fb586ca 100644
--- a/drivers/crypto/kasumi/rte_kasumi_pmd_private.h
+++ b/drivers/crypto/kasumi/rte_kasumi_pmd_private.h
@@ -37,18 +37,18 @@
 
 #define KASUMI_LOG_ERR(fmt, args...) \
 	RTE_LOG(ERR, CRYPTODEV, "[%s] %s() line %u: " fmt "\n",  \
-			CRYPTODEV_NAME_KASUMI_PMD, \
+			RTE_STR(CRYPTODEV_NAME_KASUMI_PMD), \
 			__func__, __LINE__, ## args)
 
 #ifdef RTE_LIBRTE_KASUMI_DEBUG
 #define KASUMI_LOG_INFO(fmt, args...) \
 	RTE_LOG(INFO, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
-			CRYPTODEV_NAME_KASUMI_PMD, \
+			RTE_STR(CRYPTODEV_NAME_KASUMI_PMD), \
 			__func__, __LINE__, ## args)
 
 #define KASUMI_LOG_DBG(fmt, args...) \
 	RTE_LOG(DEBUG, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
-			CRYPTODEV_NAME_KASUMI_PMD, \
+			RTE_STR(CRYPTODEV_NAME_KASUMI_PMD), \
 			__func__, __LINE__, ## args)
 #else
 #define KASUMI_LOG_INFO(fmt, args...)
diff --git a/drivers/crypto/null/null_crypto_pmd.c b/drivers/crypto/null/null_crypto_pmd.c
index 0a195ed..e03d0eb 100644
--- a/drivers/crypto/null/null_crypto_pmd.c
+++ b/drivers/crypto/null/null_crypto_pmd.c
@@ -51,7 +51,7 @@ create_unique_device_name(char *name, size_t size)
 	if (name == NULL)
 		return -EINVAL;
 
-	ret = snprintf(name, size, "%s_%u", CRYPTODEV_NAME_NULL_PMD,
+	ret = snprintf(name, size, "%s_%u", RTE_STR(CRYPTODEV_NAME_NULL_PMD),
 			unique_name_id++);
 	if (ret < 0)
 		return ret;
@@ -269,12 +269,11 @@ cryptodev_null_uninit(const char *name)
 }
 
 static struct rte_driver cryptodev_null_pmd_drv = {
-	.name = CRYPTODEV_NAME_NULL_PMD,
 	.type = PMD_VDEV,
 	.init = cryptodev_null_init,
 	.uninit = cryptodev_null_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_null_pmd_drv, cryptodev_null_pmd);
-DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+PMD_REGISTER_DRIVER(cryptodev_null_pmd_drv, CRYPTODEV_NAME_NULL_PMD);
+DRIVER_REGISTER_PARAM_STRING(CRYPTODEV_NAME_NULL_PMD, "max_nb_queue_pairs=<int> "
 "max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/null/null_crypto_pmd_private.h b/drivers/crypto/null/null_crypto_pmd_private.h
index 2a4c739..acebc97 100644
--- a/drivers/crypto/null/null_crypto_pmd_private.h
+++ b/drivers/crypto/null/null_crypto_pmd_private.h
@@ -37,18 +37,18 @@
 
 #define NULL_CRYPTO_LOG_ERR(fmt, args...) \
 	RTE_LOG(ERR, CRYPTODEV, "[%s] %s() line %u: " fmt "\n",  \
-			CRYPTODEV_NAME_NULL_PMD, \
+			RTE_STR(CRYPTODEV_NAME_NULL_PMD), \
 			__func__, __LINE__, ## args)
 
 #ifdef RTE_LIBRTE_NULL_CRYPTO_DEBUG
 #define NULL_CRYPTO_LOG_INFO(fmt, args...) \
 	RTE_LOG(INFO, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
-			CRYPTODEV_NAME_NULL_PMD, \
+			RTE_STR(CRYPTODEV_NAME_NULL_PMD), \
 			__func__, __LINE__, ## args)
 
 #define NULL_CRYPTO_LOG_DBG(fmt, args...) \
 	RTE_LOG(DEBUG, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
-			CRYPTODEV_NAME_NULL_PMD, \
+			RTE_STR(CRYPTODEV_NAME_NULL_PMD), \
 			__func__, __LINE__, ## args)
 #else
 #define NULL_CRYPTO_LOG_INFO(fmt, args...)
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index b35267e..82ab047 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -114,7 +114,6 @@ crypto_qat_dev_init(__attribute__((unused)) struct rte_cryptodev_driver *crypto_
 
 static struct rte_cryptodev_driver rte_qat_pmd = {
 	{
-		.name = "rte_qat_pmd",
 		.id_table = pci_id_qat_map,
 		.drv_flags = RTE_PCI_DRV_NEED_MAPPING,
 	},
@@ -134,6 +133,6 @@ static struct rte_driver pmd_qat_drv = {
 	.init = rte_qat_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(pmd_qat_drv, qat);
-DRIVER_REGISTER_PCI_TABLE(qat, pci_id_qat_map);
+PMD_REGISTER_DRIVER(pmd_qat_drv, CRYPTODEV_NAME_QAT_SYM_PMD);
+DRIVER_REGISTER_PCI_TABLE(CRYPTODEV_NAME_QAT_SYM_PMD, pci_id_qat_map);
 
diff --git a/drivers/crypto/snow3g/rte_snow3g_pmd.c b/drivers/crypto/snow3g/rte_snow3g_pmd.c
index ddb724c..a474a4d 100644
--- a/drivers/crypto/snow3g/rte_snow3g_pmd.c
+++ b/drivers/crypto/snow3g/rte_snow3g_pmd.c
@@ -60,7 +60,7 @@ create_unique_device_name(char *name, size_t size)
 	if (name == NULL)
 		return -EINVAL;
 
-	ret = snprintf(name, size, "%s_%u", CRYPTODEV_NAME_SNOW3G_PMD,
+	ret = snprintf(name, size, "%s_%u", RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD),
 			unique_name_id++);
 	if (ret < 0)
 		return ret;
@@ -639,12 +639,11 @@ cryptodev_snow3g_uninit(const char *name)
 }
 
 static struct rte_driver cryptodev_snow3g_pmd_drv = {
-	.name = CRYPTODEV_NAME_SNOW3G_PMD,
 	.type = PMD_VDEV,
 	.init = cryptodev_snow3g_init,
 	.uninit = cryptodev_snow3g_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_snow3g_pmd_drv, snow3g);
-DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+PMD_REGISTER_DRIVER(cryptodev_snow3g_pmd_drv, CRYPTODEV_NAME_SNOW3G_PMD);
+DRIVER_REGISTER_PARAM_STRING(CRYPTODEV_NAME_SNOW3G_PMD, "max_nb_queue_pairs=<int> "
 "max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/snow3g/rte_snow3g_pmd_private.h b/drivers/crypto/snow3g/rte_snow3g_pmd_private.h
index b383cbc..03973b9 100644
--- a/drivers/crypto/snow3g/rte_snow3g_pmd_private.h
+++ b/drivers/crypto/snow3g/rte_snow3g_pmd_private.h
@@ -37,18 +37,18 @@
 
 #define SNOW3G_LOG_ERR(fmt, args...) \
 	RTE_LOG(ERR, CRYPTODEV, "[%s] %s() line %u: " fmt "\n",  \
-			CRYPTODEV_NAME_SNOW3G_PMD, \
+			RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD), \
 			__func__, __LINE__, ## args)
 
 #ifdef RTE_LIBRTE_SNOW3G_DEBUG
 #define SNOW3G_LOG_INFO(fmt, args...) \
 	RTE_LOG(INFO, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
-			CRYPTODEV_NAME_SNOW3G_PMD, \
+			RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD), \
 			__func__, __LINE__, ## args)
 
 #define SNOW3G_LOG_DBG(fmt, args...) \
 	RTE_LOG(DEBUG, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
-			CRYPTODEV_NAME_SNOW3G_PMD, \
+			RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD), \
 			__func__, __LINE__, ## args)
 #else
 #define SNOW3G_LOG_INFO(fmt, args...)
diff --git a/lib/librte_cryptodev/rte_cryptodev.h b/lib/librte_cryptodev/rte_cryptodev.h
index 7768f0a..4c02307 100644
--- a/lib/librte_cryptodev/rte_cryptodev.h
+++ b/lib/librte_cryptodev/rte_cryptodev.h
@@ -49,17 +49,17 @@ extern "C" {
 #include "rte_crypto.h"
 #include "rte_dev.h"
 
-#define CRYPTODEV_NAME_NULL_PMD		("cryptodev_null_pmd")
+#define CRYPTODEV_NAME_NULL_PMD		cryptodev_null_pmd
 /**< Null crypto PMD device name */
-#define CRYPTODEV_NAME_AESNI_MB_PMD	("cryptodev_aesni_mb_pmd")
+#define CRYPTODEV_NAME_AESNI_MB_PMD	cryptodev_aesni_mb_pmd
 /**< AES-NI Multi buffer PMD device name */
-#define CRYPTODEV_NAME_AESNI_GCM_PMD	("cryptodev_aesni_gcm_pmd")
+#define CRYPTODEV_NAME_AESNI_GCM_PMD	cryptodev_aesni_gcm_pmd
 /**< AES-NI GCM PMD device name */
-#define CRYPTODEV_NAME_QAT_SYM_PMD	("cryptodev_qat_sym_pmd")
+#define CRYPTODEV_NAME_QAT_SYM_PMD	cryptodev_qat_sym_pmd
 /**< Intel QAT Symmetric Crypto PMD device name */
-#define CRYPTODEV_NAME_SNOW3G_PMD	("cryptodev_snow3g_pmd")
+#define CRYPTODEV_NAME_SNOW3G_PMD	cryptodev_snow3g_pmd
 /**< SNOW 3G PMD device name */
-#define CRYPTODEV_NAME_KASUMI_PMD	("cryptodev_kasumi_pmd")
+#define CRYPTODEV_NAME_KASUMI_PMD	cryptodev_kasumi_pmd
 /**< KASUMI PMD device name */
 
 /** Crypto device type */
-- 
2.7.4

[PATCH v3] crypto: normalize cryptodev pmd names with macros

[Pablo de Lara]

Recently reported, the introduction of pmd information exports led to a
breakage of cryptodev unit tests because the test infrastructure relies on the
cryptodev names being available in macros.  This patch fixes the pmd naming to
use the macro names.  Note that the macro names were already pre-stringified,
which won't work as the PMD_REGISTER_DRIVER macro requires the name in both a
processing token and stringified form.  As such the names are defined now as
tokens, and converted where needed to stringified form on demand using RTE_STR.

Tested using the null and aesni_mb crypto pmds, as I don't have hardware for any
other device.  Also not build tested on snow3g or kasumi pmd because building
those requires external libraries that appear to not be open source licensed.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
CC: Bruce Richardson <bruce.richardson@intel.com>
CC: Thomas Monjalon <thomas.monjalon@6wind.com>
CC: "De Lara Guarch, Pablo" <pablo.de.lara.guarch@intel.com>
CC: Stephen Hemminger <stephen@networkplumber.org>
CC: Panu Matilainen <pmatilai@redhat.com>
---

Changes in v3:
- Rebased against HEAD

Changes in v2:
- Fixed compilation issue on snow3g/kasumi pmd
- Removed static name of qat pmd

 app/test/test_cryptodev.c                          | 20 ++++++++++----------
 app/test/test_cryptodev_perf.c                     | 18 +++++++++---------
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c           |  7 +++----
 drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h   |  6 +++---
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c         |  7 +++----
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h |  2 +-
 drivers/crypto/kasumi/rte_kasumi_pmd.c             |  7 +++----
 drivers/crypto/kasumi/rte_kasumi_pmd_private.h     |  6 +++---
 drivers/crypto/null/null_crypto_pmd.c              |  7 +++----
 drivers/crypto/null/null_crypto_pmd_private.h      |  6 +++---
 drivers/crypto/qat/rte_qat_cryptodev.c             |  5 ++---
 drivers/crypto/snow3g/rte_snow3g_pmd.c             |  7 +++----
 drivers/crypto/snow3g/rte_snow3g_pmd_private.h     |  6 +++---
 lib/librte_cryptodev/rte_cryptodev.h               | 12 ++++++------
 14 files changed, 55 insertions(+), 61 deletions(-)

diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index fbfe1d0..1e1f887 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -186,12 +186,12 @@ testsuite_setup(void)
 		if (nb_devs < 2) {
 			for (i = nb_devs; i < 2; i++) {
 				ret = rte_eal_vdev_init(
-					CRYPTODEV_NAME_AESNI_MB_PMD, NULL);
+					RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD), NULL);
 
 				TEST_ASSERT(ret == 0,
 					"Failed to create instance %u of"
 					" pmd : %s",
-					i, CRYPTODEV_NAME_AESNI_MB_PMD);
+					i, RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD));
 			}
 		}
 	}
@@ -203,10 +203,10 @@ testsuite_setup(void)
 		if (nb_devs < 2) {
 			for (i = nb_devs; i < 2; i++) {
 				TEST_ASSERT_SUCCESS(rte_eal_vdev_init(
-					CRYPTODEV_NAME_AESNI_GCM_PMD, NULL),
+					RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD), NULL),
 					"Failed to create instance %u of"
 					" pmd : %s",
-					i, CRYPTODEV_NAME_AESNI_GCM_PMD);
+					i, RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD));
 			}
 		}
 	}
@@ -217,10 +217,10 @@ testsuite_setup(void)
 		if (nb_devs < 2) {
 			for (i = nb_devs; i < 2; i++) {
 				TEST_ASSERT_SUCCESS(rte_eal_vdev_init(
-					CRYPTODEV_NAME_SNOW3G_PMD, NULL),
+					RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD), NULL),
 					"Failed to create instance %u of"
 					" pmd : %s",
-					i, CRYPTODEV_NAME_SNOW3G_PMD);
+					i, RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD));
 			}
 		}
 	}
@@ -231,10 +231,10 @@ testsuite_setup(void)
 		if (nb_devs < 2) {
 			for (i = nb_devs; i < 2; i++) {
 				TEST_ASSERT_SUCCESS(rte_eal_vdev_init(
-					CRYPTODEV_NAME_KASUMI_PMD, NULL),
+					RTE_STR(CRYPTODEV_NAME_KASUMI_PMD), NULL),
 					"Failed to create instance %u of"
 					" pmd : %s",
-					i, CRYPTODEV_NAME_KASUMI_PMD);
+					i, RTE_STR(CRYPTODEV_NAME_KASUMI_PMD));
 			}
 		}
 	}
@@ -246,12 +246,12 @@ testsuite_setup(void)
 		if (nb_devs < 2) {
 			for (i = nb_devs; i < 2; i++) {
 				int dev_id = rte_eal_vdev_init(
-					CRYPTODEV_NAME_NULL_PMD, NULL);
+					RTE_STR(CRYPTODEV_NAME_NULL_PMD), NULL);
 
 				TEST_ASSERT(dev_id >= 0,
 					"Failed to create instance %u of"
 					" pmd : %s",
-					i, CRYPTODEV_NAME_NULL_PMD);
+					i, RTE_STR(CRYPTODEV_NAME_NULL_PMD));
 			}
 		}
 	}
diff --git a/app/test/test_cryptodev_perf.c b/app/test/test_cryptodev_perf.c
index d728211..815c41f 100644
--- a/app/test/test_cryptodev_perf.c
+++ b/app/test/test_cryptodev_perf.c
@@ -120,15 +120,15 @@ static const char *chain_mode_name(enum chain_mode mode)
 static const char *pmd_name(enum rte_cryptodev_type pmd)
 {
 	switch (pmd) {
-	case RTE_CRYPTODEV_NULL_PMD: return CRYPTODEV_NAME_NULL_PMD; break;
+	case RTE_CRYPTODEV_NULL_PMD: return RTE_STR(CRYPTODEV_NAME_NULL_PMD); break;
 	case RTE_CRYPTODEV_AESNI_GCM_PMD:
-		return CRYPTODEV_NAME_AESNI_GCM_PMD;
+		return RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD);
 	case RTE_CRYPTODEV_AESNI_MB_PMD:
-		return CRYPTODEV_NAME_AESNI_MB_PMD;
+		return RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD);
 	case RTE_CRYPTODEV_QAT_SYM_PMD:
-		return CRYPTODEV_NAME_QAT_SYM_PMD;
+		return RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD);
 	case RTE_CRYPTODEV_SNOW3G_PMD:
-		return CRYPTODEV_NAME_SNOW3G_PMD;
+		return RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD);
 	default:
 		return "";
 	}
@@ -249,11 +249,11 @@ testsuite_setup(void)
 		if (nb_devs < 2) {
 			for (i = nb_devs; i < 2; i++) {
 				ret = rte_eal_vdev_init(
-					CRYPTODEV_NAME_AESNI_MB_PMD, NULL);
+					RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD), NULL);
 
 				TEST_ASSERT(ret == 0,
 					"Failed to create instance %u of pmd : %s",
-					i, CRYPTODEV_NAME_AESNI_MB_PMD);
+					i, RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD));
 			}
 		}
 	}
@@ -264,11 +264,11 @@ testsuite_setup(void)
 		if (nb_devs < 2) {
 			for (i = nb_devs; i < 2; i++) {
 				ret = rte_eal_vdev_init(
-					CRYPTODEV_NAME_SNOW3G_PMD, NULL);
+					RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD), NULL);
 
 				TEST_ASSERT(ret == 0,
 					"Failed to create instance %u of pmd : %s",
-					i, CRYPTODEV_NAME_SNOW3G_PMD);
+					i, RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD));
 			}
 		}
 	}
diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
index f43e407..f9be44a 100644
--- a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
+++ b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
@@ -57,7 +57,7 @@ create_unique_device_name(char *name, size_t size)
 	if (name == NULL)
 		return -EINVAL;
 
-	ret = snprintf(name, size, "%s_%u", CRYPTODEV_NAME_AESNI_GCM_PMD,
+	ret = snprintf(name, size, "%s_%u", RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD),
 			unique_name_id++);
 	if (ret < 0)
 		return ret;
@@ -515,12 +515,11 @@ aesni_gcm_uninit(const char *name)
 }
 
 static struct rte_driver aesni_gcm_pmd_drv = {
-	.name = CRYPTODEV_NAME_AESNI_GCM_PMD,
 	.type = PMD_VDEV,
 	.init = aesni_gcm_init,
 	.uninit = aesni_gcm_uninit
 };
 
-PMD_REGISTER_DRIVER(aesni_gcm_pmd_drv, aesni_gcm);
-DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+PMD_REGISTER_DRIVER(aesni_gcm_pmd_drv, CRYPTODEV_NAME_AESNI_GCM_PMD);
+DRIVER_REGISTER_PARAM_STRING(CRYPTODEV_NAME_AESNI_GCM_PMD, "max_nb_queue_pairs=<int> "
 "max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h b/drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h
index a42f941..9878d6e 100644
--- a/drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h
+++ b/drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h
@@ -37,18 +37,18 @@
 
 #define GCM_LOG_ERR(fmt, args...) \
 	RTE_LOG(ERR, CRYPTODEV, "[%s] %s() line %u: " fmt "\n",  \
-			CRYPTODEV_NAME_AESNI_GCM_PMD, \
+			RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD), \
 			__func__, __LINE__, ## args)
 
 #ifdef RTE_LIBRTE_AESNI_MB_DEBUG
 #define GCM_LOG_INFO(fmt, args...) \
 	RTE_LOG(INFO, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
-			CRYPTODEV_NAME_AESNI_GCM_PMD, \
+			RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD), \
 			__func__, __LINE__, ## args)
 
 #define GCM_LOG_DBG(fmt, args...) \
 	RTE_LOG(DEBUG, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
-			CRYPTODEV_NAME_AESNI_GCM_PMD, \
+			RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD), \
 			__func__, __LINE__, ## args)
 #else
 #define GCM_LOG_INFO(fmt, args...)
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
index 859a04b..2140b92 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
@@ -54,7 +54,7 @@ create_unique_device_name(char *name, size_t size)
 	if (name == NULL)
 		return -EINVAL;
 
-	ret = snprintf(name, size, "%s_%u", CRYPTODEV_NAME_AESNI_MB_PMD,
+	ret = snprintf(name, size, "%s_%u", RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD),
 			unique_name_id++);
 	if (ret < 0)
 		return ret;
@@ -715,12 +715,11 @@ cryptodev_aesni_mb_uninit(const char *name)
 }
 
 static struct rte_driver cryptodev_aesni_mb_pmd_drv = {
-	.name = CRYPTODEV_NAME_AESNI_MB_PMD,
 	.type = PMD_VDEV,
 	.init = cryptodev_aesni_mb_init,
 	.uninit = cryptodev_aesni_mb_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drv, aesni_mb);
-DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+PMD_REGISTER_DRIVER(cryptodev_aesni_mb_pmd_drv, CRYPTODEV_NAME_AESNI_MB_PMD);
+DRIVER_REGISTER_PARAM_STRING(CRYPTODEV_NAME_AESNI_MB_PMD, "max_nb_queue_pairs=<int> "
 "max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h
index 949d9a6..17f367f 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h
@@ -37,7 +37,7 @@
 
 #define MB_LOG_ERR(fmt, args...) \
 	RTE_LOG(ERR, CRYPTODEV, "[%s] %s() line %u: " fmt "\n",  \
-			CRYPTODEV_NAME_AESNI_MB_PMD, \
+			RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD), \
 			__func__, __LINE__, ## args)
 
 #ifdef RTE_LIBRTE_AESNI_MB_DEBUG
diff --git a/drivers/crypto/kasumi/rte_kasumi_pmd.c b/drivers/crypto/kasumi/rte_kasumi_pmd.c
index 1d5af46..400a248 100644
--- a/drivers/crypto/kasumi/rte_kasumi_pmd.c
+++ b/drivers/crypto/kasumi/rte_kasumi_pmd.c
@@ -61,7 +61,7 @@ create_unique_device_name(char *name, size_t size)
 	if (name == NULL)
 		return -EINVAL;
 
-	ret = snprintf(name, size, "%s_%u", CRYPTODEV_NAME_KASUMI_PMD,
+	ret = snprintf(name, size, "%s_%u", RTE_STR(CRYPTODEV_NAME_KASUMI_PMD),
 			unique_name_id++);
 	if (ret < 0)
 		return ret;
@@ -648,12 +648,11 @@ cryptodev_kasumi_uninit(const char *name)
 }
 
 static struct rte_driver cryptodev_kasumi_pmd_drv = {
-	.name = CRYPTODEV_NAME_KASUMI_PMD,
 	.type = PMD_VDEV,
 	.init = cryptodev_kasumi_init,
 	.uninit = cryptodev_kasumi_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_kasumi_pmd_drv, kasumi);
-DRIVER_REGISTER_PARAM_STRING(kasumi, "max_nb_queue_pairs=<int> "
+PMD_REGISTER_DRIVER(cryptodev_kasumi_pmd_drv, CRYPTODEV_NAME_KASUMI_PMD);
+DRIVER_REGISTER_PARAM_STRING(CRYPTODEV_NAME_KASUMI_PMD, "max_nb_queue_pairs=<int> "
 "max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/kasumi/rte_kasumi_pmd_private.h b/drivers/crypto/kasumi/rte_kasumi_pmd_private.h
index 04e1c43..fb586ca 100644
--- a/drivers/crypto/kasumi/rte_kasumi_pmd_private.h
+++ b/drivers/crypto/kasumi/rte_kasumi_pmd_private.h
@@ -37,18 +37,18 @@
 
 #define KASUMI_LOG_ERR(fmt, args...) \
 	RTE_LOG(ERR, CRYPTODEV, "[%s] %s() line %u: " fmt "\n",  \
-			CRYPTODEV_NAME_KASUMI_PMD, \
+			RTE_STR(CRYPTODEV_NAME_KASUMI_PMD), \
 			__func__, __LINE__, ## args)
 
 #ifdef RTE_LIBRTE_KASUMI_DEBUG
 #define KASUMI_LOG_INFO(fmt, args...) \
 	RTE_LOG(INFO, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
-			CRYPTODEV_NAME_KASUMI_PMD, \
+			RTE_STR(CRYPTODEV_NAME_KASUMI_PMD), \
 			__func__, __LINE__, ## args)
 
 #define KASUMI_LOG_DBG(fmt, args...) \
 	RTE_LOG(DEBUG, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
-			CRYPTODEV_NAME_KASUMI_PMD, \
+			RTE_STR(CRYPTODEV_NAME_KASUMI_PMD), \
 			__func__, __LINE__, ## args)
 #else
 #define KASUMI_LOG_INFO(fmt, args...)
diff --git a/drivers/crypto/null/null_crypto_pmd.c b/drivers/crypto/null/null_crypto_pmd.c
index 0a195ed..e03d0eb 100644
--- a/drivers/crypto/null/null_crypto_pmd.c
+++ b/drivers/crypto/null/null_crypto_pmd.c
@@ -51,7 +51,7 @@ create_unique_device_name(char *name, size_t size)
 	if (name == NULL)
 		return -EINVAL;
 
-	ret = snprintf(name, size, "%s_%u", CRYPTODEV_NAME_NULL_PMD,
+	ret = snprintf(name, size, "%s_%u", RTE_STR(CRYPTODEV_NAME_NULL_PMD),
 			unique_name_id++);
 	if (ret < 0)
 		return ret;
@@ -269,12 +269,11 @@ cryptodev_null_uninit(const char *name)
 }
 
 static struct rte_driver cryptodev_null_pmd_drv = {
-	.name = CRYPTODEV_NAME_NULL_PMD,
 	.type = PMD_VDEV,
 	.init = cryptodev_null_init,
 	.uninit = cryptodev_null_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_null_pmd_drv, cryptodev_null_pmd);
-DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+PMD_REGISTER_DRIVER(cryptodev_null_pmd_drv, CRYPTODEV_NAME_NULL_PMD);
+DRIVER_REGISTER_PARAM_STRING(CRYPTODEV_NAME_NULL_PMD, "max_nb_queue_pairs=<int> "
 "max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/null/null_crypto_pmd_private.h b/drivers/crypto/null/null_crypto_pmd_private.h
index 2a4c739..acebc97 100644
--- a/drivers/crypto/null/null_crypto_pmd_private.h
+++ b/drivers/crypto/null/null_crypto_pmd_private.h
@@ -37,18 +37,18 @@
 
 #define NULL_CRYPTO_LOG_ERR(fmt, args...) \
 	RTE_LOG(ERR, CRYPTODEV, "[%s] %s() line %u: " fmt "\n",  \
-			CRYPTODEV_NAME_NULL_PMD, \
+			RTE_STR(CRYPTODEV_NAME_NULL_PMD), \
 			__func__, __LINE__, ## args)
 
 #ifdef RTE_LIBRTE_NULL_CRYPTO_DEBUG
 #define NULL_CRYPTO_LOG_INFO(fmt, args...) \
 	RTE_LOG(INFO, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
-			CRYPTODEV_NAME_NULL_PMD, \
+			RTE_STR(CRYPTODEV_NAME_NULL_PMD), \
 			__func__, __LINE__, ## args)
 
 #define NULL_CRYPTO_LOG_DBG(fmt, args...) \
 	RTE_LOG(DEBUG, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
-			CRYPTODEV_NAME_NULL_PMD, \
+			RTE_STR(CRYPTODEV_NAME_NULL_PMD), \
 			__func__, __LINE__, ## args)
 #else
 #define NULL_CRYPTO_LOG_INFO(fmt, args...)
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index b35267e..82ab047 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -114,7 +114,6 @@ crypto_qat_dev_init(__attribute__((unused)) struct rte_cryptodev_driver *crypto_
 
 static struct rte_cryptodev_driver rte_qat_pmd = {
 	{
-		.name = "rte_qat_pmd",
 		.id_table = pci_id_qat_map,
 		.drv_flags = RTE_PCI_DRV_NEED_MAPPING,
 	},
@@ -134,6 +133,6 @@ static struct rte_driver pmd_qat_drv = {
 	.init = rte_qat_pmd_init,
 };
 
-PMD_REGISTER_DRIVER(pmd_qat_drv, qat);
-DRIVER_REGISTER_PCI_TABLE(qat, pci_id_qat_map);
+PMD_REGISTER_DRIVER(pmd_qat_drv, CRYPTODEV_NAME_QAT_SYM_PMD);
+DRIVER_REGISTER_PCI_TABLE(CRYPTODEV_NAME_QAT_SYM_PMD, pci_id_qat_map);
 
diff --git a/drivers/crypto/snow3g/rte_snow3g_pmd.c b/drivers/crypto/snow3g/rte_snow3g_pmd.c
index ddb724c..a474a4d 100644
--- a/drivers/crypto/snow3g/rte_snow3g_pmd.c
+++ b/drivers/crypto/snow3g/rte_snow3g_pmd.c
@@ -60,7 +60,7 @@ create_unique_device_name(char *name, size_t size)
 	if (name == NULL)
 		return -EINVAL;
 
-	ret = snprintf(name, size, "%s_%u", CRYPTODEV_NAME_SNOW3G_PMD,
+	ret = snprintf(name, size, "%s_%u", RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD),
 			unique_name_id++);
 	if (ret < 0)
 		return ret;
@@ -639,12 +639,11 @@ cryptodev_snow3g_uninit(const char *name)
 }
 
 static struct rte_driver cryptodev_snow3g_pmd_drv = {
-	.name = CRYPTODEV_NAME_SNOW3G_PMD,
 	.type = PMD_VDEV,
 	.init = cryptodev_snow3g_init,
 	.uninit = cryptodev_snow3g_uninit
 };
 
-PMD_REGISTER_DRIVER(cryptodev_snow3g_pmd_drv, snow3g);
-DRIVER_REGISTER_PARAM_STRING(aesni_gcm, "max_nb_queue_pairs=<int> "
+PMD_REGISTER_DRIVER(cryptodev_snow3g_pmd_drv, CRYPTODEV_NAME_SNOW3G_PMD);
+DRIVER_REGISTER_PARAM_STRING(CRYPTODEV_NAME_SNOW3G_PMD, "max_nb_queue_pairs=<int> "
 "max_nb_sessions=<int> socket_id=<int>");
diff --git a/drivers/crypto/snow3g/rte_snow3g_pmd_private.h b/drivers/crypto/snow3g/rte_snow3g_pmd_private.h
index b383cbc..03973b9 100644
--- a/drivers/crypto/snow3g/rte_snow3g_pmd_private.h
+++ b/drivers/crypto/snow3g/rte_snow3g_pmd_private.h
@@ -37,18 +37,18 @@
 
 #define SNOW3G_LOG_ERR(fmt, args...) \
 	RTE_LOG(ERR, CRYPTODEV, "[%s] %s() line %u: " fmt "\n",  \
-			CRYPTODEV_NAME_SNOW3G_PMD, \
+			RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD), \
 			__func__, __LINE__, ## args)
 
 #ifdef RTE_LIBRTE_SNOW3G_DEBUG
 #define SNOW3G_LOG_INFO(fmt, args...) \
 	RTE_LOG(INFO, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
-			CRYPTODEV_NAME_SNOW3G_PMD, \
+			RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD), \
 			__func__, __LINE__, ## args)
 
 #define SNOW3G_LOG_DBG(fmt, args...) \
 	RTE_LOG(DEBUG, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
-			CRYPTODEV_NAME_SNOW3G_PMD, \
+			RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD), \
 			__func__, __LINE__, ## args)
 #else
 #define SNOW3G_LOG_INFO(fmt, args...)
diff --git a/lib/librte_cryptodev/rte_cryptodev.h b/lib/librte_cryptodev/rte_cryptodev.h
index 7768f0a..4c02307 100644
--- a/lib/librte_cryptodev/rte_cryptodev.h
+++ b/lib/librte_cryptodev/rte_cryptodev.h
@@ -49,17 +49,17 @@ extern "C" {
 #include "rte_crypto.h"
 #include "rte_dev.h"
 
-#define CRYPTODEV_NAME_NULL_PMD		("cryptodev_null_pmd")
+#define CRYPTODEV_NAME_NULL_PMD		cryptodev_null_pmd
 /**< Null crypto PMD device name */
-#define CRYPTODEV_NAME_AESNI_MB_PMD	("cryptodev_aesni_mb_pmd")
+#define CRYPTODEV_NAME_AESNI_MB_PMD	cryptodev_aesni_mb_pmd
 /**< AES-NI Multi buffer PMD device name */
-#define CRYPTODEV_NAME_AESNI_GCM_PMD	("cryptodev_aesni_gcm_pmd")
+#define CRYPTODEV_NAME_AESNI_GCM_PMD	cryptodev_aesni_gcm_pmd
 /**< AES-NI GCM PMD device name */
-#define CRYPTODEV_NAME_QAT_SYM_PMD	("cryptodev_qat_sym_pmd")
+#define CRYPTODEV_NAME_QAT_SYM_PMD	cryptodev_qat_sym_pmd
 /**< Intel QAT Symmetric Crypto PMD device name */
-#define CRYPTODEV_NAME_SNOW3G_PMD	("cryptodev_snow3g_pmd")
+#define CRYPTODEV_NAME_SNOW3G_PMD	cryptodev_snow3g_pmd
 /**< SNOW 3G PMD device name */
-#define CRYPTODEV_NAME_KASUMI_PMD	("cryptodev_kasumi_pmd")
+#define CRYPTODEV_NAME_KASUMI_PMD	cryptodev_kasumi_pmd
 /**< KASUMI PMD device name */
 
 /** Crypto device type */
-- 
2.7.4

Re: [PATCH v3] crypto: normalize cryptodev pmd names with macros

[Thomas Monjalon]

> Recently reported, the introduction of pmd information exports led to a
> breakage of cryptodev unit tests because the test infrastructure relies on the
> cryptodev names being available in macros.  This patch fixes the pmd naming to
> use the macro names.  Note that the macro names were already pre-stringified,
> which won't work as the PMD_REGISTER_DRIVER macro requires the name in both a
> processing token and stringified form.  As such the names are defined now as
> tokens, and converted where needed to stringified form on demand using RTE_STR.
> 
> Tested using the null and aesni_mb crypto pmds, as I don't have hardware for any
> other device.  Also not build tested on snow3g or kasumi pmd because building
> those requires external libraries that appear to not be open source licensed.
> 
> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
> Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

Applied, thanks

Re: [PATCH] crypto: normalize cryptodev pmd names with macros

[Mcnamara, John]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
> Sent: Friday, July 8, 2016 2:42 PM
> To: Thomas Monjalon <thomas.monjalon@6wind.com>
> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; dev@dpdk.org;
> Richardson, Bruce <bruce.richardson@intel.com>; Stephen Hemminger
> <stephen@networkplumber.org>; Panu Matilainen <pmatilai@redhat.com>
> Subject: Re: [dpdk-dev] [PATCH] crypto: normalize cryptodev pmd names with
> macros
> 
> On Fri, Jul 08, 2016 at 02:40:01PM +0200, Thomas Monjalon wrote:
> > 2016-07-08 08:17, Neil Horman:
> > > I suppose thats fine, sure, though I'm really not comfortable with
> > > an open source project requiring what appears to be non-open source
> > > components (though I can't really tell what the snow3g or kasumi
> license is).
> >
> > Yes you're right.
> > IMHO, it is acceptable because it is free (just need a registration)
> > but it can be discussed.
> > I think there are some legal export issues because of crypto aim.
> Yeah, crypto usually has munitions export restrictions.  I'm fine with
> that, as long as the license to the code is GPL.  And I can't tell what
> the license is because you have to register to get it before you can see
> it.
> 
> If you, or someone can confirm that the snow3g and kasumi ssl libraries
> are open source compatible (or better still if the license can be
> announced prior to registration on the web site), then I'm fine with it,
> and am happy to test with it, I just don't want to register without
> knowing that beforehand.

Hi Neil,

The license is "Intel Pre-Release Software License Agreement". We will send a
patch to make this this clear in the documentation. We will also see if it
can be made visible on the download page. 

I can send you a copy of the license if you wish. Pablo has already tested 
compilation with the latest version of the patch.

John

Re: [PATCH] crypto: normalize cryptodev pmd names with macros

[Neil Horman]

On Fri, Jul 08, 2016 at 07:03:10PM +0000, Mcnamara, John wrote:
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
> > Sent: Friday, July 8, 2016 2:42 PM
> > To: Thomas Monjalon <thomas.monjalon@6wind.com>
> > Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; dev@dpdk.org;
> > Richardson, Bruce <bruce.richardson@intel.com>; Stephen Hemminger
> > <stephen@networkplumber.org>; Panu Matilainen <pmatilai@redhat.com>
> > Subject: Re: [dpdk-dev] [PATCH] crypto: normalize cryptodev pmd names with
> > macros
> > 
> > On Fri, Jul 08, 2016 at 02:40:01PM +0200, Thomas Monjalon wrote:
> > > 2016-07-08 08:17, Neil Horman:
> > > > I suppose thats fine, sure, though I'm really not comfortable with
> > > > an open source project requiring what appears to be non-open source
> > > > components (though I can't really tell what the snow3g or kasumi
> > license is).
> > >
> > > Yes you're right.
> > > IMHO, it is acceptable because it is free (just need a registration)
> > > but it can be discussed.
> > > I think there are some legal export issues because of crypto aim.
> > Yeah, crypto usually has munitions export restrictions.  I'm fine with
> > that, as long as the license to the code is GPL.  And I can't tell what
> > the license is because you have to register to get it before you can see
> > it.
> > 
> > If you, or someone can confirm that the snow3g and kasumi ssl libraries
> > are open source compatible (or better still if the license can be
> > announced prior to registration on the web site), then I'm fine with it,
> > and am happy to test with it, I just don't want to register without
> > knowing that beforehand.
> 
> Hi Neil,
> 
> The license is "Intel Pre-Release Software License Agreement". We will send a
> patch to make this this clear in the documentation. We will also see if it
> can be made visible on the download page. 
> 
> I can send you a copy of the license if you wish. Pablo has already tested 
> compilation with the latest version of the patch.
> 
I understand and appreciate that, but I'd still like to have a copy of the
license.  It seems a bit unreasonable to me to include open source drivers in an
open source project, if they can't be built without a non-open source library
included.

Neil

> John
> 
> 

Re: [PATCH] crypto: normalize cryptodev pmd names with macros

[Mcnamara, John]

> -----Original Message-----
> From: Neil Horman [mailto:nhorman@tuxdriver.com]
> Sent: Saturday, July 9, 2016 2:34 PM
> To: Mcnamara, John <john.mcnamara@intel.com>
> Cc: Thomas Monjalon <thomas.monjalon@6wind.com>; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>; dev@dpdk.org; Richardson, Bruce
> <bruce.richardson@intel.com>; Stephen Hemminger
> <stephen@networkplumber.org>; Panu Matilainen <pmatilai@redhat.com>
> Subject: Re: [dpdk-dev] [PATCH] crypto: normalize cryptodev pmd names with
> macros
> 
> On Fri, Jul 08, 2016 at 07:03:10PM +0000, Mcnamara, John wrote:
> > > -----Original Message-----
> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
> > > Sent: Friday, July 8, 2016 2:42 PM
> > > To: Thomas Monjalon <thomas.monjalon@6wind.com>
> > > Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>;
> > > dev@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>;
> > > Stephen Hemminger <stephen@networkplumber.org>; Panu Matilainen
> > > <pmatilai@redhat.com>
> > > Subject: Re: [dpdk-dev] [PATCH] crypto: normalize cryptodev pmd
> > > names with macros
> > >
> > > On Fri, Jul 08, 2016 at 02:40:01PM +0200, Thomas Monjalon wrote:
> > > > 2016-07-08 08:17, Neil Horman:
> > > > > I suppose thats fine, sure, though I'm really not comfortable
> > > > > with an open source project requiring what appears to be
> > > > > non-open source components (though I can't really tell what the
> > > > > snow3g or kasumi
> > > license is).
> > > >
> > > > Yes you're right.
> > > > IMHO, it is acceptable because it is free (just need a
> > > > registration) but it can be discussed.
> > > > I think there are some legal export issues because of crypto aim.
> > > Yeah, crypto usually has munitions export restrictions.  I'm fine
> > > with that, as long as the license to the code is GPL.  And I can't
> > > tell what the license is because you have to register to get it
> > > before you can see it.
> > >
> > > If you, or someone can confirm that the snow3g and kasumi ssl
> > > libraries are open source compatible (or better still if the license
> > > can be announced prior to registration on the web site), then I'm
> > > fine with it, and am happy to test with it, I just don't want to
> > > register without knowing that beforehand.
> >
> > Hi Neil,
> >
> > The license is "Intel Pre-Release Software License Agreement". We will
> > send a patch to make this this clear in the documentation. We will
> > also see if it can be made visible on the download page.
> >
> > I can send you a copy of the license if you wish. Pablo has already
> > tested compilation with the latest version of the patch.
> >
> I understand and appreciate that, but I'd still like to have a copy of the
> license.  It seems a bit unreasonable to me to include open source drivers
> in an open source project, if they can't be built without a non-open
> source library included.

I'll send you a copy.

John