[RFC 0/4] Include resources in tests

[RFC 0/4] Include resources in tests

[Jan Viktorin]

Hello,

this patch set introduces a mechanism to include a resource (in general a blob)
into the test binary. This allows to make tests less dependent on the target
testing environment. The first use case is testing of PCI bus scan by changing
the hard-coded path (/sys/bus/pci/devices) to something different and provide
a fake tree of devices with the test. It can help with testing of device-tree
parsing as I've proposed in [1] where such mechanism was missing at that time.
I'd like to use such framework for the SoC infra testing as well.

The patch set introduces a struct resource into the app/test. The resource is
generic to include any kind of binary data. The binary data can be created in
C or linked as an object file (created by objcopy). I am not sure where to
place the objcopy logic and how to perform guessing of the objcopy arguments
as they are pretty non-standard.

To include a complex resource (a file hierarchy), the last patch implements
an archive extraction logic. So, it is possible to include a tar archive and
unpack it before a test starts. Any ideas how to do this in a better way are
welcome.

[1] http://comments.gmane.org/gmane.comp.networking.dpdk.devel/36545

Regards
Jan Viktorin

---

Jan Viktorin (4):
  app/test: introduce resources for tests
  app/test: support resources externally linked
  app/test: add functions to create files from resources
  app/test: support resources archived by tar

 app/test/Makefile        |  38 +++++++
 app/test/resource.c      | 276 +++++++++++++++++++++++++++++++++++++++++++++++
 app/test/resource.h      |  99 +++++++++++++++++
 app/test/test_resource.c | 132 +++++++++++++++++++++++
 4 files changed, 545 insertions(+)
 create mode 100644 app/test/resource.c
 create mode 100644 app/test/resource.h
 create mode 100644 app/test/test_resource.c

-- 
2.8.0

[RFC 1/4] app/test: introduce resources for tests

[Jan Viktorin]

Certain internal mechanisms of DPDK access different file system structures
(e.g. /sys/bus/pci/devices). It is difficult to test those cases automatically
by a unit test when such path is not hard-coded and there is no simple way how
to distribute fake ones with the current testing environment.

This patch adds a possibility to declare a resource embedded in the test binary
itself. The structure resource cover the generic situation - it provides a name
for lookup and pointers to the embedded data blob. A resource is registered
in a constructor by the macro REGISTER_RESOURCE.

Some initial tests of simple resources is included.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/Makefile        |  2 ++
 app/test/resource.c      | 61 ++++++++++++++++++++++++++++++++++++++
 app/test/resource.h      | 77 ++++++++++++++++++++++++++++++++++++++++++++++++
 app/test/test_resource.c | 75 ++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 215 insertions(+)
 create mode 100644 app/test/resource.c
 create mode 100644 app/test/resource.h
 create mode 100644 app/test/test_resource.c

diff --git a/app/test/Makefile b/app/test/Makefile
index a4907d5..7fbdd18 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -43,6 +43,8 @@ APP = test
 #
 SRCS-$(CONFIG_RTE_LIBRTE_CMDLINE) := commands.c
 SRCS-y += test.c
+SRCS-y += resource.c
+SRCS-y += test_resource.c
 SRCS-y += test_pci.c
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
diff --git a/app/test/resource.c b/app/test/resource.c
new file mode 100644
index 0000000..50a4510
--- /dev/null
+++ b/app/test/resource.c
@@ -0,0 +1,61 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2016 RehiveTech. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of RehiveTech nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <string.h>
+#include <errno.h>
+#include <sys/queue.h>
+
+#include <rte_debug.h>
+
+#include "resource.h"
+
+struct resource_list resource_list = TAILQ_HEAD_INITIALIZER(resource_list);
+
+const struct resource *resource_find(const char *name)
+{
+	struct resource *r;
+
+	TAILQ_FOREACH(r, &resource_list, next) {
+		RTE_VERIFY(r->name);
+
+		if (!strcmp(r->name, name))
+			return r;
+	}
+
+	return NULL;
+}
+
+void __resource_register(struct resource *r)
+{
+	TAILQ_INSERT_TAIL(&resource_list, r, next);
+}
diff --git a/app/test/resource.h b/app/test/resource.h
new file mode 100644
index 0000000..a50a973
--- /dev/null
+++ b/app/test/resource.h
@@ -0,0 +1,77 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2016 RehiveTech. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of RehiveTech nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RESOURCE_H_
+#define _RESOURCE_H_
+
+#include <sys/queue.h>
+#include <stddef.h>
+
+#include <rte_eal.h>
+#include <rte_common.h>
+
+TAILQ_HEAD(resource_list, resource);
+extern struct resource_list resource_list;
+
+struct resource {
+	const char *name;
+	const char *beg;
+	const char *end;
+	TAILQ_ENTRY(resource) next;
+};
+
+static inline size_t resource_size(const struct resource *r)
+{
+	return r->end - r->beg;
+}
+
+const struct resource *resource_find(const char *name);
+
+void __resource_register(struct resource *r);
+
+#define REGISTER_RESOURCE(_n, _b, _e) \
+static struct resource linkres_ ##_n = {       \
+	.name = RTE_STR(_n),     \
+	.beg = _b,               \
+	.end = _e,               \
+};                               \
+__REGISTER_RESOURCE(linkres_ ##_n)
+
+#define __REGISTER_RESOURCE(name)     \
+RTE_INIT(resinitfn_ ##name);        \
+static void resinitfn_ ##name(void) \
+{                                   \
+	__resource_register(&name); \
+}
+
+#endif
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
new file mode 100644
index 0000000..e3d2486
--- /dev/null
+++ b/app/test/test_resource.c
@@ -0,0 +1,75 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2016 RehiveTech. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of RehiveTech nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <stdio.h>
+#include <string.h>
+
+#include "test.h"
+#include "resource.h"
+
+const char test_resource_dpdk_blob[] = {
+	'\x44', '\x50', '\x44', '\x4b', '\x00'
+};
+
+REGISTER_RESOURCE(test_resource_dpdk,
+		test_resource_dpdk_blob, test_resource_dpdk_blob + 4);
+
+static int test_resource_dpdk(void)
+{
+	const struct resource *r;
+
+	r = resource_find("test_resource_dpdk");
+	TEST_ASSERT_NOT_NULL(r, "Could not find test_resource_dpdk");
+	TEST_ASSERT(!strcmp(r->name, "test_resource_dpdk"),
+			"Found resource %s, expected test_resource_dpdk",
+			r->name);
+
+	TEST_ASSERT(!strncmp("DPDK", r->beg, 4),
+			"Unexpected payload: %.4s...", r->beg);
+
+	return 0;
+}
+
+static int test_resource(void)
+{
+	if (test_resource_dpdk())
+		return -1;
+
+	return 0;
+}
+
+static struct test_command resource_cmd = {
+	.command = "resource_autotest",
+	.callback = test_resource,
+};
+REGISTER_TEST_COMMAND(resource_cmd);
-- 
2.8.0

[RFC 2/4] app/test: support resources externally linked

[Jan Viktorin]

To include resources from other source that the C source code we can take
advantage of the objcopy behaviour, i.e. packing of an arbitrary file as an
object file that is linked to the target program.

A linked object file is always accessible as a pair

extern const char beg_<name>;
extern const char end_<name>;
(extern const char siz_<name>;)

The objcopy however accepts architecture parameters in a very tricky way.
Try to translate as many arguments as possible from the RTE_ARCH into the
objcopy specific names.

We pack the resource.c source file as an example for testing.

*** CAUTION: ***
The objcopy and resource creation is a subject of change. Any comments of how
to integrate those is welcome.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/Makefile        | 32 ++++++++++++++++++++++++++++++++
 app/test/resource.h      |  5 +++++
 app/test/test_resource.c | 18 ++++++++++++++++++
 3 files changed, 55 insertions(+)

diff --git a/app/test/Makefile b/app/test/Makefile
index 7fbdd18..a9502f1 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -33,6 +33,37 @@ include $(RTE_SDK)/mk/rte.vars.mk
 
 ifeq ($(CONFIG_RTE_APP_TEST),y)
 
+ifeq ($(RTE_ARCH),arm)
+RTE_OBJCOPY_O = elf32-littlearm
+RTE_OBJCOPY_B = arm
+else ifeq ($(RTE_ARCH),arm64)
+RTE_OBJCOPY_O = elf64-littleaarch64
+RTE_OBJCOPY_B = aarch64
+else ifeq ($(RTE_ARCH),i686)
+RTE_OBJCOPY_O = elf32-i386
+RTE_OBJCOPY_B = i386
+else ifeq ($(RTE_ARCH),x86_64)
+RTE_OBJCOPY_O = elf64-x86-64
+RTE_OBJCOPY_B = i386:x86-64
+else ifeq ($(RTE_ARCH),x86_x32)
+RTE_OBJCOPY_O = elf32-x86-64
+RTE_OBJCOPY_B = i386:x86-64
+else
+$(error Unrecognized RTE_ARCH: $(RTE_ARCH))
+endif
+
+define resource
+SRCS-y += $(1).res.o
+$(1).res.o: $(2)
+	$(OBJCOPY) -I binary -B $(RTE_OBJCOPY_B) -O $(RTE_OBJCOPY_O)     \
+		--rename-section                                         \
+			.data=.rodata,alloc,load,data,contents,readonly  \
+		--redefine-sym _binary__dev_stdin_start=beg_$(1)         \
+		--redefine-sym _binary__dev_stdin_end=end_$(1)           \
+		--redefine-sym _binary__dev_stdin_size=siz_$(1)          \
+		/dev/stdin $$@ < $$<
+endef
+
 #
 # library name
 #
@@ -45,6 +76,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_CMDLINE) := commands.c
 SRCS-y += test.c
 SRCS-y += resource.c
 SRCS-y += test_resource.c
+$(eval $(call resource,test_resource_c,resource.c))
 SRCS-y += test_pci.c
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
diff --git a/app/test/resource.h b/app/test/resource.h
index a50a973..7978e77 100644
--- a/app/test/resource.h
+++ b/app/test/resource.h
@@ -59,6 +59,11 @@ const struct resource *resource_find(const char *name);
 
 void __resource_register(struct resource *r);
 
+#define REGISTER_LINKED_RESOURCE(_n) \
+extern const char beg_ ##_n;         \
+extern const char end_ ##_n;         \
+REGISTER_RESOURCE(_n, &beg_ ##_n, &end_ ##_n); \
+
 #define REGISTER_RESOURCE(_n, _b, _e) \
 static struct resource linkres_ ##_n = {       \
 	.name = RTE_STR(_n),     \
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
index e3d2486..7752522 100644
--- a/app/test/test_resource.c
+++ b/app/test/test_resource.c
@@ -60,11 +60,29 @@ static int test_resource_dpdk(void)
 	return 0;
 }
 
+REGISTER_LINKED_RESOURCE(test_resource_c);
+
+static int test_resource_c(void)
+{
+	const struct resource *r;
+
+	r = resource_find("test_resource_c");
+	TEST_ASSERT_NOT_NULL(r, "No test_resource_c found");
+	TEST_ASSERT(!strcmp(r->name, "test_resource_c"),
+			"Found resource %s, expected test_resource_c",
+			r->name);
+
+	return 0;
+}
+
 static int test_resource(void)
 {
 	if (test_resource_dpdk())
 		return -1;
 
+	if (test_resource_c())
+		return -1;
+
 	return 0;
 }
 
-- 
2.8.0

[RFC 3/4] app/test: add functions to create files from resources

[Jan Viktorin]

A resource can be written into the target filesystem by calling resource_fwrite
or resource_fwrite_file. Such file can be created before a test is started and
removed after the test finishes.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/resource.c      | 35 +++++++++++++++++++++++++++++++++++
 app/test/resource.h      |  4 ++++
 app/test/test_resource.c | 10 ++++++++++
 3 files changed, 49 insertions(+)

diff --git a/app/test/resource.c b/app/test/resource.c
index 50a4510..a11c86e 100644
--- a/app/test/resource.c
+++ b/app/test/resource.c
@@ -31,6 +31,7 @@
  *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+#include <stdio.h>
 #include <string.h>
 #include <errno.h>
 #include <sys/queue.h>
@@ -55,6 +56,40 @@ const struct resource *resource_find(const char *name)
 	return NULL;
 }
 
+int resource_fwrite(const struct resource *r, FILE *f)
+{
+	const size_t goal = resource_size(r);
+	size_t total = 0;
+
+	while (total < goal) {
+		size_t wlen = fwrite(r->beg + total, 1, goal - total, f);
+		if (wlen == 0) {
+			perror(__func__);
+			return -1;
+		}
+
+		total += wlen;
+	}
+
+	return 0;
+}
+
+int resource_fwrite_file(const struct resource *r, const char *fname)
+{
+	FILE *f;
+	int ret;
+
+	f = fopen(fname, "w");
+	if (f == NULL) {
+		perror(__func__);
+		return -1;
+	}
+
+	ret = resource_fwrite(r, f);
+	fclose(f);
+	return ret;
+}
+
 void __resource_register(struct resource *r)
 {
 	TAILQ_INSERT_TAIL(&resource_list, r, next);
diff --git a/app/test/resource.h b/app/test/resource.h
index 7978e77..096633c 100644
--- a/app/test/resource.h
+++ b/app/test/resource.h
@@ -35,6 +35,7 @@
 #define _RESOURCE_H_
 
 #include <sys/queue.h>
+#include <stdio.h>
 #include <stddef.h>
 
 #include <rte_eal.h>
@@ -57,6 +58,9 @@ static inline size_t resource_size(const struct resource *r)
 
 const struct resource *resource_find(const char *name);
 
+int resource_fwrite(const struct resource *r, FILE *f);
+int resource_fwrite_file(const struct resource *r, const char *fname);
+
 void __resource_register(struct resource *r);
 
 #define REGISTER_LINKED_RESOURCE(_n) \
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
index 7752522..148964e 100644
--- a/app/test/test_resource.c
+++ b/app/test/test_resource.c
@@ -65,6 +65,7 @@ REGISTER_LINKED_RESOURCE(test_resource_c);
 static int test_resource_c(void)
 {
 	const struct resource *r;
+	FILE *f;
 
 	r = resource_find("test_resource_c");
 	TEST_ASSERT_NOT_NULL(r, "No test_resource_c found");
@@ -72,6 +73,15 @@ static int test_resource_c(void)
 			"Found resource %s, expected test_resource_c",
 			r->name);
 
+	TEST_ASSERT_SUCCESS(resource_fwrite_file(r, "test_resource.c"),
+			"Failed to to write file %s", r->name);
+
+	f = fopen("test_resource.c", "r");
+	TEST_ASSERT_NOT_NULL(f,
+			"Missing extracted file resource.c");
+	fclose(f);
+	remove("test_resource.c");
+
 	return 0;
 }
 
-- 
2.8.0

[RFC 4/4] app/test: support resources archived by tar

[Jan Viktorin]

When needing a more complex resource (a file hierarchy), packing every single
file as a single resource would be very ineffective. For that purpose, it is
possible to pack the files into a tar archive, extract it before test from the
resource and finally clean up all the created files.

This patch introduces functions resource_untar and resource_rm_by_tar to
perform those tasks. An example of using those functions is included as a test.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/Makefile        |   4 ++
 app/test/resource.c      | 180 +++++++++++++++++++++++++++++++++++++++++++++++
 app/test/resource.h      |  13 ++++
 app/test/test_resource.c |  29 ++++++++
 4 files changed, 226 insertions(+)

diff --git a/app/test/Makefile b/app/test/Makefile
index a9502f1..90acd63 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -77,6 +77,9 @@ SRCS-y += test.c
 SRCS-y += resource.c
 SRCS-y += test_resource.c
 $(eval $(call resource,test_resource_c,resource.c))
+$(eval $(call resource,test_resource_tar,resource.tar))
+resource.tar: test_resource.c
+	tar -C $(dir $<) -cf $@ $(notdir $<)
 SRCS-y += test_pci.c
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
@@ -196,6 +199,7 @@ CFLAGS += $(WERROR_FLAGS)
 CFLAGS += -D_GNU_SOURCE
 
 LDLIBS += -lm
+LDLIBS += -larchive
 
 # Disable VTA for memcpy test
 ifeq ($(CC), gcc)
diff --git a/app/test/resource.c b/app/test/resource.c
index a11c86e..7732bc7 100644
--- a/app/test/resource.c
+++ b/app/test/resource.c
@@ -33,6 +33,8 @@
 
 #include <stdio.h>
 #include <string.h>
+#include <archive.h>
+#include <archive_entry.h>
 #include <errno.h>
 #include <sys/queue.h>
 
@@ -90,6 +92,184 @@ int resource_fwrite_file(const struct resource *r, const char *fname)
 	return ret;
 }
 
+static int do_copy(struct archive *r, struct archive *w)
+{
+	const void *buf;
+	size_t len;
+	off_t off;
+	int ret;
+
+	while (1) {
+		ret = archive_read_data_block(r, &buf, &len, &off);
+		if (ret == ARCHIVE_RETRY)
+			continue;
+
+		if (ret == ARCHIVE_EOF)
+			return 0;
+
+		if (ret != ARCHIVE_OK)
+			return ret;
+
+		do {
+			ret = archive_write_data_block(w, buf, len, off);
+			if (ret != ARCHIVE_OK && ret != ARCHIVE_RETRY)
+				return ret;
+		} while (ret != ARCHIVE_OK);
+	}
+}
+
+int resource_untar(const struct resource *res)
+{
+	struct archive *r;
+	struct archive *w;
+	struct archive_entry *e;
+	void *p;
+	int flags = 0;
+	int ret;
+
+	p = malloc(resource_size(res));
+	if (p == NULL)
+		rte_panic("Failed to malloc %zu B\n", resource_size(res));
+
+	memcpy(p, res->beg, resource_size(res));
+
+	r = archive_read_new();
+	if (r == NULL) {
+		free(p);
+		return -1;
+	}
+
+	archive_read_support_format_all(r);
+	archive_read_support_filter_all(r);
+
+	w = archive_write_disk_new();
+	if (w == NULL) {
+		archive_read_free(r);
+		free(p);
+		return -1;
+	}
+
+	flags |= ARCHIVE_EXTRACT_PERM;
+	flags |= ARCHIVE_EXTRACT_FFLAGS;
+	archive_write_disk_set_options(w, flags);
+	archive_write_disk_set_standard_lookup(w);
+
+	ret = archive_read_open_memory(r, p, resource_size(res));
+	if (ret != ARCHIVE_OK)
+		goto fail;
+
+	while (1) {
+		ret = archive_read_next_header(r, &e);
+		if (ret == ARCHIVE_EOF)
+			break;
+		if (ret != ARCHIVE_OK)
+			goto fail;
+
+		ret = archive_write_header(w, e);
+		if (ret == ARCHIVE_EOF)
+			break;
+		if (ret != ARCHIVE_OK)
+			goto fail;
+
+		if (archive_entry_size(e) == 0)
+			continue;
+
+		ret = do_copy(r, w);
+		if (ret != ARCHIVE_OK)
+			goto fail;
+
+		ret = archive_write_finish_entry(w);
+		if (ret != ARCHIVE_OK)
+			goto fail;
+	}
+
+	archive_write_free(w);
+	archive_read_free(r);
+	free(p);
+	return 0;
+
+fail:
+	archive_write_free(w);
+	archive_read_free(r);
+	free(p);
+	rte_panic("Failed: %s\n", archive_error_string(r));
+	return -1;
+}
+
+int resource_rm_by_tar(const struct resource *res)
+{
+	struct archive *r;
+	struct archive_entry *e;
+	void *p;
+	int try_again = 1;
+	int ret;
+
+	p = malloc(resource_size(res));
+	if (p == NULL)
+		rte_panic("Failed to malloc %zu B\n", resource_size(res));
+
+	memcpy(p, res->beg, resource_size(res));
+
+	while (try_again) {
+		r = archive_read_new();
+		if (r == NULL) {
+			free(p);
+			return -1;
+		}
+
+		archive_read_support_format_all(r);
+		archive_read_support_filter_all(r);
+
+		ret = archive_read_open_memory(r, p, resource_size(res));
+		if (ret != ARCHIVE_OK) {
+			fprintf(stderr, "Failed: %s\n",
+					archive_error_string(r));
+			goto fail;
+		}
+
+		try_again = 0;
+
+		while (1) {
+			ret = archive_read_next_header(r, &e);
+			if (ret == ARCHIVE_EOF)
+				break;
+			if (ret != ARCHIVE_OK)
+				goto fail;
+
+			ret = remove(archive_entry_pathname(e));
+			if (ret < 0) {
+				switch (errno) {
+				case ENOTEMPTY:
+				case EEXIST:
+					try_again = 1;
+					break;
+
+				/* should not usually happen: */
+				case ENOENT:
+				case ENOTDIR:
+				case EROFS:
+					continue;
+				default:
+					perror("Failed to remove file");
+					goto fail;
+				}
+			}
+		}
+
+		archive_read_free(r);
+	}
+
+	free(p);
+	return 0;
+
+fail:
+	archive_read_free(r);
+	free(p);
+
+	rte_panic("Failed: %s\n", archive_error_string(r));
+	return -1;
+}
+
 void __resource_register(struct resource *r)
 {
 	TAILQ_INSERT_TAIL(&resource_list, r, next);
diff --git a/app/test/resource.h b/app/test/resource.h
index 096633c..61ccea2 100644
--- a/app/test/resource.h
+++ b/app/test/resource.h
@@ -61,6 +61,19 @@ const struct resource *resource_find(const char *name);
 int resource_fwrite(const struct resource *r, FILE *f);
 int resource_fwrite_file(const struct resource *r, const char *fname);
 
+/**
+ * Treat the given resource as a tar archive. Extract
+ * the archive to the current directory.
+ */
+int resource_untar(const struct resource *res);
+
+/**
+ * Treat the given resource as a tar archive. Remove
+ * all files (related to the current directory) listed
+ * in the tar archive.
+ */
+int resource_rm_by_tar(const struct resource *res);
+
 void __resource_register(struct resource *r);
 
 #define REGISTER_LINKED_RESOURCE(_n) \
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
index 148964e..7b9ed31 100644
--- a/app/test/test_resource.c
+++ b/app/test/test_resource.c
@@ -85,6 +85,32 @@ static int test_resource_c(void)
 	return 0;
 }
 
+REGISTER_LINKED_RESOURCE(test_resource_tar);
+
+static int test_resource_tar(void)
+{
+	const struct resource *r;
+	FILE *f;
+
+	r = resource_find("test_resource_tar");
+	TEST_ASSERT_NOT_NULL(r, "No test_resource_tar found");
+	TEST_ASSERT(!strcmp(r->name, "test_resource_tar"),
+			"Found resource %s, expected test_resource_tar",
+			r->name);
+
+	TEST_ASSERT_SUCCESS(resource_untar(r),
+			"Failed to to untar %s", r->name);
+
+	f = fopen("test_resource.c", "r");
+	TEST_ASSERT_NOT_NULL(f,
+			"Missing extracted file test_resource.c");
+	fclose(f);
+
+	TEST_ASSERT_SUCCESS(resource_rm_by_tar(r),
+			"Failed to remove extracted contents of %s", r->name);
+	return 0;
+}
+
 static int test_resource(void)
 {
 	if (test_resource_dpdk())
@@ -93,6 +119,9 @@ static int test_resource(void)
 	if (test_resource_c())
 		return -1;
 
+	if (test_resource_tar())
+		return -1;
+
 	return 0;
 }
 
-- 
2.8.0

Re: [RFC 4/4] app/test: support resources archived by tar

[Bruce Richardson]

On Fri, Apr 29, 2016 at 03:11:36PM +0200, Jan Viktorin wrote:
> When needing a more complex resource (a file hierarchy), packing every single
> file as a single resource would be very ineffective. For that purpose, it is
> possible to pack the files into a tar archive, extract it before test from the
> resource and finally clean up all the created files.
> 
> This patch introduces functions resource_untar and resource_rm_by_tar to
> perform those tasks. An example of using those functions is included as a test.
> 
> Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
We might want to make this configurable on/off at build time, since it adds a new
dependency for building dpdk - libarchive-devel.

/Bruce

Re: [RFC 4/4] app/test: support resources archived by tar

[Jan Viktorin]

On Thu, 5 May 2016 14:33:01 +0100
Bruce Richardson <bruce.richardson@intel.com> wrote:

> On Fri, Apr 29, 2016 at 03:11:36PM +0200, Jan Viktorin wrote:
> > When needing a more complex resource (a file hierarchy), packing every single
> > file as a single resource would be very ineffective. For that purpose, it is
> > possible to pack the files into a tar archive, extract it before test from the
> > resource and finally clean up all the created files.
> > 
> > This patch introduces functions resource_untar and resource_rm_by_tar to
> > perform those tasks. An example of using those functions is included as a test.
> > 
> > Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>  
> We might want to make this configurable on/off at build time, since it adds a new
> dependency for building dpdk - libarchive-devel.

This is true. Unfortunately...

Is it a problem to introduce such dependency into the test app?
Would it be help to use another similar (more spread) library (zlib)?

Jan

> 
> /Bruce



-- 
   Jan Viktorin                  E-mail: Viktorin@RehiveTech.com
   System Architect              Web:    www.RehiveTech.com
   RehiveTech
   Brno, Czech Republic

Re: [RFC 4/4] app/test: support resources archived by tar

[Bruce Richardson]

On Thu, May 05, 2016 at 07:05:40PM +0200, Jan Viktorin wrote:
> On Thu, 5 May 2016 14:33:01 +0100
> Bruce Richardson <bruce.richardson@intel.com> wrote:
> 
> > On Fri, Apr 29, 2016 at 03:11:36PM +0200, Jan Viktorin wrote:
> > > When needing a more complex resource (a file hierarchy), packing every single
> > > file as a single resource would be very ineffective. For that purpose, it is
> > > possible to pack the files into a tar archive, extract it before test from the
> > > resource and finally clean up all the created files.
> > > 
> > > This patch introduces functions resource_untar and resource_rm_by_tar to
> > > perform those tasks. An example of using those functions is included as a test.
> > > 
> > > Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>  
> > We might want to make this configurable on/off at build time, since it adds a new
> > dependency for building dpdk - libarchive-devel.
> 
> This is true. Unfortunately...
> 
> Is it a problem to introduce such dependency into the test app?
> Would it be help to use another similar (more spread) library (zlib)?
>
Not sure it would make much difference, to be honest. It all depends on what
-devel packages are installed by default on most distros when you install the
development tools sets. I'm not concerned too much either way, but if we add
in a new dependency like this, we should also take the opportunity to turn on
pcap PMD by default too and make libpcap-devel a dependency.

/Bruce

Re: [RFC 0/4] Include resources in tests

[Bruce Richardson]

On Fri, Apr 29, 2016 at 03:11:32PM +0200, Jan Viktorin wrote:
> Hello,
> 
> this patch set introduces a mechanism to include a resource (in general a blob)
> into the test binary. This allows to make tests less dependent on the target
> testing environment. The first use case is testing of PCI bus scan by changing
> the hard-coded path (/sys/bus/pci/devices) to something different and provide
> a fake tree of devices with the test. It can help with testing of device-tree
> parsing as I've proposed in [1] where such mechanism was missing at that time.
> I'd like to use such framework for the SoC infra testing as well.
> 
> The patch set introduces a struct resource into the app/test. The resource is
> generic to include any kind of binary data. The binary data can be created in
> C or linked as an object file (created by objcopy). I am not sure where to
> place the objcopy logic and how to perform guessing of the objcopy arguments
> as they are pretty non-standard.
> 
> To include a complex resource (a file hierarchy), the last patch implements
> an archive extraction logic. So, it is possible to include a tar archive and
> unpack it before a test starts. Any ideas how to do this in a better way are
> welcome.
> 
> [1] http://comments.gmane.org/gmane.comp.networking.dpdk.devel/36545
> 
> Regards
> Jan Viktorin
> 

Hi Jan,

this looks really interesting, especially since just yesterday I was looking at
taking the million-entry lpm test routing table out of the C code and into a
separate resource file in this case an ini file.

In terms of a solution, I'm not convinced of the placing of the blobs inside the
test binary. I think a better solution would be to allow the different autotests
to take parameters from the commandline, so that the user can specify the path
to the file to use for the test. What would be your opinion of such a scheme?

/Bruce

Re: [RFC 0/4] Include resources in tests

[Jan Viktorin]

On Fri, 29 Apr 2016 15:42:27 +0100
Bruce Richardson <bruce.richardson@intel.com> wrote:

> On Fri, Apr 29, 2016 at 03:11:32PM +0200, Jan Viktorin wrote:
> > Hello,
> > 
> > this patch set introduces a mechanism to include a resource (in general a blob)
> > into the test binary. This allows to make tests less dependent on the target
> > testing environment. The first use case is testing of PCI bus scan by changing
> > the hard-coded path (/sys/bus/pci/devices) to something different and provide
> > a fake tree of devices with the test. It can help with testing of device-tree
> > parsing as I've proposed in [1] where such mechanism was missing at that time.
> > I'd like to use such framework for the SoC infra testing as well.
> > 
> > The patch set introduces a struct resource into the app/test. The resource is
> > generic to include any kind of binary data. The binary data can be created in
> > C or linked as an object file (created by objcopy). I am not sure where to
> > place the objcopy logic and how to perform guessing of the objcopy arguments
> > as they are pretty non-standard.
> > 
> > To include a complex resource (a file hierarchy), the last patch implements
> > an archive extraction logic. So, it is possible to include a tar archive and
> > unpack it before a test starts. Any ideas how to do this in a better way are
> > welcome.
> > 
> > [1] http://comments.gmane.org/gmane.comp.networking.dpdk.devel/36545
> > 
> > Regards
> > Jan Viktorin
> >   
> 
> Hi Jan,
> 
> this looks really interesting, especially since just yesterday I was looking at
> taking the million-entry lpm test routing table out of the C code and into a
> separate resource file in this case an ini file.

If it is an automatic test case which can show some regressions over
time then it is a good example of the resource usage.

> 
> In terms of a solution, I'm not convinced of the placing of the blobs inside the
> test binary. I think a better solution would be to allow the different autotests
> to take parameters from the commandline, so that the user can specify the path
> to the file to use for the test. What would be your opinion of such a scheme?

I think that we are already at this stage. You can read parameters e.g.
from the environment vars (so in fact, no changes to the DPDK testing
code are needed). The thing is that I want to say "run tests" and don't
care about any parameters to receive the result.

It might be possible to pass parameters to the tests (optionally) to
reuse the testing code base for different cases. However, initially,
the test (in my opinion) should run on any architecture on any device
without any configuration and return consistent results.

How can I test probing of PCI devices on my PC when I have different
network card then the author of the test? I cannot or I have to pass
parameters which is not what I want as it complicates every single
such test (and I have to understand those tests or understand the
parameters syntax even though the code is unrelated to my work). I just
want to check that the DPDK works as expected - without regressions.

And what about the case when I have no such card (running automatic
tests on a build server)?

I don't like including the resources in binaries but I didn't find
any better way yet. I need to easily install the tests together with
the testing data (resources) to an embedded device and perform the
tests on the target system. When you are testing x86 code on your x86
you don't have to care too much about the location of resources. But I
do. And there is no standard way (at the moment) how to install the
resources together with the testing code.

What I like on this solution is that the DPDK git repository would
contain the testing data as a real file hierarchy (e.g. the
fake /sys/bus/...). After build and install of DPDK, the file hierachy
is packed (and compressed if needed) together with the tests. So moving
to the target platform works without any other changes to the
testing/install infrastructure. I could potentially run PCI tests on my
ARM board without any PCI available and don't have to ignore the
failures of those tests (as they simply pass).

Regards
Jan

> 
> /Bruce



-- 
  Jan Viktorin                E-mail: Viktorin@RehiveTech.com
  System Architect            Web:    www.RehiveTech.com
  RehiveTech
  Brno, Czech Republic

Re: [RFC 0/4] Include resources in tests

[Bruce Richardson]

On Fri, Apr 29, 2016 at 03:11:32PM +0200, Jan Viktorin wrote:
> Hello,
> 
> this patch set introduces a mechanism to include a resource (in general a blob)
> into the test binary. This allows to make tests less dependent on the target
> testing environment. The first use case is testing of PCI bus scan by changing
> the hard-coded path (/sys/bus/pci/devices) to something different and provide
> a fake tree of devices with the test. It can help with testing of device-tree
> parsing as I've proposed in [1] where such mechanism was missing at that time.
> I'd like to use such framework for the SoC infra testing as well.
> 
> The patch set introduces a struct resource into the app/test. The resource is
> generic to include any kind of binary data. The binary data can be created in
> C or linked as an object file (created by objcopy). I am not sure where to
> place the objcopy logic and how to perform guessing of the objcopy arguments
> as they are pretty non-standard.
> 
> To include a complex resource (a file hierarchy), the last patch implements
> an archive extraction logic. So, it is possible to include a tar archive and
> unpack it before a test starts. Any ideas how to do this in a better way are
> welcome.
> 
> [1] http://comments.gmane.org/gmane.comp.networking.dpdk.devel/36545
> 
> Regards
> Jan Viktorin
> 
BTW: It looks like your patch has a dependency on the 17-patch dev cleanup set from
David, [and now on the header include bug fix I submitted yesterday too], so
you should probably note that in any future revs of the patch you do. Save
some head-scratching from those of us testing it out. :-)

Thanks,
/Bruce

Re: [RFC 0/4] Include resources in tests

[Jan Viktorin]

On Thu, 5 May 2016 14:29:30 +0100
Bruce Richardson <bruce.richardson@intel.com> wrote:

> On Fri, Apr 29, 2016 at 03:11:32PM +0200, Jan Viktorin wrote:
> > Hello,
> > 
> > this patch set introduces a mechanism to include a resource (in general a blob)
> > into the test binary. This allows to make tests less dependent on the target
> > testing environment. The first use case is testing of PCI bus scan by changing
> > the hard-coded path (/sys/bus/pci/devices) to something different and provide
> > a fake tree of devices with the test. It can help with testing of device-tree
> > parsing as I've proposed in [1] where such mechanism was missing at that time.
> > I'd like to use such framework for the SoC infra testing as well.
> > 
> > The patch set introduces a struct resource into the app/test. The resource is
> > generic to include any kind of binary data. The binary data can be created in
> > C or linked as an object file (created by objcopy). I am not sure where to
> > place the objcopy logic and how to perform guessing of the objcopy arguments
> > as they are pretty non-standard.
> > 
> > To include a complex resource (a file hierarchy), the last patch implements
> > an archive extraction logic. So, it is possible to include a tar archive and
> > unpack it before a test starts. Any ideas how to do this in a better way are
> > welcome.
> > 
> > [1] http://comments.gmane.org/gmane.comp.networking.dpdk.devel/36545
> > 
> > Regards
> > Jan Viktorin
> >   
> BTW: It looks like your patch has a dependency on the 17-patch dev cleanup set from
> David, [and now on the header include bug fix I submitted yesterday too], so
> you should probably note that in any future revs of the patch you do. Save
> some head-scratching from those of us testing it out. :-)

Hello, well, the patch set should be independent on that cleanup, so I take it as a
bug report. There should be no dependency, sorry. I'll fix it and repost.

Thanks
Jan

> 
> Thanks,
> /Bruce
> 



-- 
   Jan Viktorin                  E-mail: Viktorin@RehiveTech.com
   System Architect              Web:    www.RehiveTech.com
   RehiveTech
   Brno, Czech Republic

Re: [RFC 0/4] Include resources in tests

[Bruce Richardson]

On Thu, May 05, 2016 at 07:03:00PM +0200, Jan Viktorin wrote:
> On Thu, 5 May 2016 14:29:30 +0100
> Bruce Richardson <bruce.richardson@intel.com> wrote:
> 
> > On Fri, Apr 29, 2016 at 03:11:32PM +0200, Jan Viktorin wrote:
> > > Hello,
> > > 
> > > this patch set introduces a mechanism to include a resource (in general a blob)
> > > into the test binary. This allows to make tests less dependent on the target
> > > testing environment. The first use case is testing of PCI bus scan by changing
> > > the hard-coded path (/sys/bus/pci/devices) to something different and provide
> > > a fake tree of devices with the test. It can help with testing of device-tree
> > > parsing as I've proposed in [1] where such mechanism was missing at that time.
> > > I'd like to use such framework for the SoC infra testing as well.
> > > 
> > > The patch set introduces a struct resource into the app/test. The resource is
> > > generic to include any kind of binary data. The binary data can be created in
> > > C or linked as an object file (created by objcopy). I am not sure where to
> > > place the objcopy logic and how to perform guessing of the objcopy arguments
> > > as they are pretty non-standard.
> > > 
> > > To include a complex resource (a file hierarchy), the last patch implements
> > > an archive extraction logic. So, it is possible to include a tar archive and
> > > unpack it before a test starts. Any ideas how to do this in a better way are
> > > welcome.
> > > 
> > > [1] http://comments.gmane.org/gmane.comp.networking.dpdk.devel/36545
> > > 
> > > Regards
> > > Jan Viktorin
> > >   
> > BTW: It looks like your patch has a dependency on the 17-patch dev cleanup set from
> > David, [and now on the header include bug fix I submitted yesterday too], so
> > you should probably note that in any future revs of the patch you do. Save
> > some head-scratching from those of us testing it out. :-)
> 
> Hello, well, the patch set should be independent on that cleanup, so I take it as a
> bug report. There should be no dependency, sorry. I'll fix it and repost.
> 
> Thanks
> Jan
> 
It's the RTE_INIT macro is being missed otherwise.
I'm trying out this new infrastructure by reworking the lpm tests to use this
for the big routing table data. I'll let you know any feedback when I'm done.

	/Bruce

[PATCH v1 00/10] Include resources in tests

[Jan Viktorin]

Hello,

the second round of the approach to include resources in the test application.
I've tested on top of 84c9b5a so it should work now as expected without any
dependencies.

I've included 5 patches more related to the PCI and PCI tests. They should
probably go in their own patch series but I wanted to show how the resource
"API" can be used for current tests. The PCI tests should now work everywhere.
I recommend to execute as ./test --no-pci (optionally --no-huge) to avoid
scanning of the real devices on startup.

The resource "API" is still not very nice...

Regards
Jan

---
v1
* fix non-existing RTE_INIT, using raw __attribute__ approach instead
* included PCI test changes to demonstrate resource API

Jan Viktorin (10):
  app/test: introduce resources for tests
  app/test: support resources externally linked
  app/test: add functions to create files from resources
  app/test: support resources archived by tar
  app/test: use linked list to store PCI drivers
  app/test: extract test_pci_setup and test_pci_cleanup
  app/test: convert current pci_test into a single test case
  eal/pci: replace SYSFS_PCI_DEVICES with pci_get_sysfs_path()
  app/test: scan PCI bus using a fake sysfs
  app/test: do not dump PCI devices in blacklist test

 app/test/Makefile                                  |  42 ++++
 app/test/resource.c                                | 276 +++++++++++++++++++++
 app/test/resource.h                                |  99 ++++++++
 app/test/test_pci.c                                | 148 +++++++++--
 .../bus/pci/devices/0000:01:00.0/class             |   1 +
 .../bus/pci/devices/0000:01:00.0/config            | Bin 0 -> 64 bytes
 .../devices/0000:01:00.0/consistent_dma_mask_bits  |   1 +
 .../bus/pci/devices/0000:01:00.0/device            |   1 +
 .../bus/pci/devices/0000:01:00.0/dma_mask_bits     |   1 +
 .../bus/pci/devices/0000:01:00.0/enable            |   1 +
 .../bus/pci/devices/0000:01:00.0/irq               |   1 +
 .../bus/pci/devices/0000:01:00.0/modalias          |   1 +
 .../bus/pci/devices/0000:01:00.0/msi_bus           |   1 +
 .../bus/pci/devices/0000:01:00.0/numa_node         |   1 +
 .../bus/pci/devices/0000:01:00.0/resource          |  13 +
 .../bus/pci/devices/0000:01:00.0/sriov_numvfs      |   1 +
 .../bus/pci/devices/0000:01:00.0/sriov_totalvfs    |   1 +
 .../bus/pci/devices/0000:01:00.0/subsystem_device  |   1 +
 .../bus/pci/devices/0000:01:00.0/subsystem_vendor  |   1 +
 .../bus/pci/devices/0000:01:00.0/uevent            |   6 +
 .../bus/pci/devices/0000:01:00.0/vendor            |   1 +
 app/test/test_resource.c                           | 132 ++++++++++
 drivers/net/szedata2/rte_eth_szedata2.c            |   2 +-
 drivers/net/virtio/virtio_pci.c                    |   2 +-
 lib/librte_eal/bsdapp/eal/rte_eal_version.map      |   6 +
 lib/librte_eal/common/eal_common_pci.c             |  13 +
 lib/librte_eal/common/include/rte_pci.h            |   2 +-
 lib/librte_eal/linuxapp/eal/eal_pci.c              |   6 +-
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c          |   7 +-
 lib/librte_eal/linuxapp/eal/eal_pci_vfio.c         |   2 +-
 lib/librte_eal/linuxapp/eal/rte_eal_version.map    |   6 +
 31 files changed, 750 insertions(+), 26 deletions(-)
 create mode 100644 app/test/resource.c
 create mode 100644 app/test/resource.h
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/config
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor
 create mode 100644 app/test/test_resource.c

-- 
2.8.0

[PATCH v1 01/10] app/test: introduce resources for tests

[Jan Viktorin]

Certain internal mechanisms of DPDK access different file system structures
(e.g. /sys/bus/pci/devices). It is difficult to test those cases automatically
by a unit test when such path is not hard-coded and there is no simple way how
to distribute fake ones with the current testing environment.

This patch adds a possibility to declare a resource embedded in the test binary
itself. The structure resource cover the generic situation - it provides a name
for lookup and pointers to the embedded data blob. A resource is registered
in a constructor by the macro REGISTER_RESOURCE.

Some initial tests of simple resources is included.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/Makefile        |  2 ++
 app/test/resource.c      | 61 ++++++++++++++++++++++++++++++++++++++
 app/test/resource.h      | 77 ++++++++++++++++++++++++++++++++++++++++++++++++
 app/test/test_resource.c | 75 ++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 215 insertions(+)
 create mode 100644 app/test/resource.c
 create mode 100644 app/test/resource.h
 create mode 100644 app/test/test_resource.c

diff --git a/app/test/Makefile b/app/test/Makefile
index a4907d5..7fbdd18 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -43,6 +43,8 @@ APP = test
 #
 SRCS-$(CONFIG_RTE_LIBRTE_CMDLINE) := commands.c
 SRCS-y += test.c
+SRCS-y += resource.c
+SRCS-y += test_resource.c
 SRCS-y += test_pci.c
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
diff --git a/app/test/resource.c b/app/test/resource.c
new file mode 100644
index 0000000..50a4510
--- /dev/null
+++ b/app/test/resource.c
@@ -0,0 +1,61 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2016 RehiveTech. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of RehiveTech nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <string.h>
+#include <errno.h>
+#include <sys/queue.h>
+
+#include <rte_debug.h>
+
+#include "resource.h"
+
+struct resource_list resource_list = TAILQ_HEAD_INITIALIZER(resource_list);
+
+const struct resource *resource_find(const char *name)
+{
+	struct resource *r;
+
+	TAILQ_FOREACH(r, &resource_list, next) {
+		RTE_VERIFY(r->name);
+
+		if (!strcmp(r->name, name))
+			return r;
+	}
+
+	return NULL;
+}
+
+void __resource_register(struct resource *r)
+{
+	TAILQ_INSERT_TAIL(&resource_list, r, next);
+}
diff --git a/app/test/resource.h b/app/test/resource.h
new file mode 100644
index 0000000..ce9588e
--- /dev/null
+++ b/app/test/resource.h
@@ -0,0 +1,77 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2016 RehiveTech. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of RehiveTech nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RESOURCE_H_
+#define _RESOURCE_H_
+
+#include <sys/queue.h>
+#include <stddef.h>
+
+#include <rte_eal.h>
+#include <rte_common.h>
+
+TAILQ_HEAD(resource_list, resource);
+extern struct resource_list resource_list;
+
+struct resource {
+	const char *name;
+	const char *beg;
+	const char *end;
+	TAILQ_ENTRY(resource) next;
+};
+
+static inline size_t resource_size(const struct resource *r)
+{
+	return r->end - r->beg;
+}
+
+const struct resource *resource_find(const char *name);
+
+void __resource_register(struct resource *r);
+
+#define REGISTER_RESOURCE(_n, _b, _e) \
+static struct resource linkres_ ##_n = {       \
+	.name = RTE_STR(_n),     \
+	.beg = _b,               \
+	.end = _e,               \
+};                               \
+__REGISTER_RESOURCE(linkres_ ##_n)
+
+#define __REGISTER_RESOURCE(name)     \
+static void __attribute__((constructor,used)) resinitfn_ ##name(void); \
+static void __attribute__((constructor,used)) resinitfn_ ##name(void) \
+{                                   \
+	__resource_register(&name); \
+}
+
+#endif
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
new file mode 100644
index 0000000..e3d2486
--- /dev/null
+++ b/app/test/test_resource.c
@@ -0,0 +1,75 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2016 RehiveTech. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of RehiveTech nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <stdio.h>
+#include <string.h>
+
+#include "test.h"
+#include "resource.h"
+
+const char test_resource_dpdk_blob[] = {
+	'\x44', '\x50', '\x44', '\x4b', '\x00'
+};
+
+REGISTER_RESOURCE(test_resource_dpdk,
+		test_resource_dpdk_blob, test_resource_dpdk_blob + 4);
+
+static int test_resource_dpdk(void)
+{
+	const struct resource *r;
+
+	r = resource_find("test_resource_dpdk");
+	TEST_ASSERT_NOT_NULL(r, "Could not find test_resource_dpdk");
+	TEST_ASSERT(!strcmp(r->name, "test_resource_dpdk"),
+			"Found resource %s, expected test_resource_dpdk",
+			r->name);
+
+	TEST_ASSERT(!strncmp("DPDK", r->beg, 4),
+			"Unexpected payload: %.4s...", r->beg);
+
+	return 0;
+}
+
+static int test_resource(void)
+{
+	if (test_resource_dpdk())
+		return -1;
+
+	return 0;
+}
+
+static struct test_command resource_cmd = {
+	.command = "resource_autotest",
+	.callback = test_resource,
+};
+REGISTER_TEST_COMMAND(resource_cmd);
-- 
2.8.0

Re: [PATCH v1 01/10] app/test: introduce resources for tests

[Thomas Monjalon]

2016-05-06 12:48, Jan Viktorin:
> --- /dev/null
> +++ b/app/test/resource.h
> @@ -0,0 +1,61 @@
> +/*-
> + *   BSD LICENSE
[...]
> + */

Please include a multi-line comment here to explain what is a resource
and why it is needed.

> +
> +#ifndef _RESOURCE_H_
> +#define _RESOURCE_H_
> +
|...]
|> +
> +TAILQ_HEAD(resource_list, resource);
> +extern struct resource_list resource_list;

Why extern?

> +
> +struct resource {
> +	const char *name;
> +	const char *beg;

begin?
Bruce is removing some megabytes from lpm tests, so we have
some room for field names ;)

> +	const char *end;
> +	TAILQ_ENTRY(resource) next;
> +};
> +
> +static inline size_t resource_size(const struct resource *r)

Why inline? It could be in .c

> +{
> +	return r->end - r->beg;
> +}
> +
> +const struct resource *resource_find(const char *name);
> +
> +void __resource_register(struct resource *r);

A comment is needed to explain the role of this function.
Why a double underscore?

> +#define REGISTER_RESOURCE(_n, _b, _e) \

I'm not a big fan of the underscores.

> +static struct resource linkres_ ##_n = {       \
> +	.name = RTE_STR(_n),     \
> +	.beg = _b,               \
> +	.end = _e,               \
> +};                               \
> +__REGISTER_RESOURCE(linkres_ ##_n)

Please avoid nested macros.

> +#define __REGISTER_RESOURCE(name)     \
> +static void __attribute__((constructor,used)) resinitfn_ ##name(void); \

Why declaring the function just before its implementation?

> +static void __attribute__((constructor,used)) resinitfn_ ##name(void) \
> +{                                   \
> +	__resource_register(&name); \
> +}

Re: [PATCH v1 01/10] app/test: introduce resources for tests

[Jan Viktorin]

On Fri, 06 May 2016 16:01:08 +0200
Thomas Monjalon <thomas.monjalon@6wind.com> wrote:

> 2016-05-06 12:48, Jan Viktorin:
> > --- /dev/null
> > +++ b/app/test/resource.h
> > @@ -0,0 +1,61 @@
> > +/*-
> > + *   BSD LICENSE  
> [...]
> > + */  
> 
> Please include a multi-line comment here to explain what is a resource
> and why it is needed.

Will fix.

> 
> > +
> > +#ifndef _RESOURCE_H_
> > +#define _RESOURCE_H_
> > +  
> |...]
> |> +
> > +TAILQ_HEAD(resource_list, resource);
> > +extern struct resource_list resource_list;  
> 
> Why extern?

Will fix.

> 
> > +
> > +struct resource {
> > +	const char *name;
> > +	const char *beg;  
> 
> begin?
> Bruce is removing some megabytes from lpm tests, so we have
> some room for field names ;)

I love 3-chars identifiers... ;)

I will rework this, thanks.

> 
> > +	const char *end;
> > +	TAILQ_ENTRY(resource) next;
> > +};
> > +
> > +static inline size_t resource_size(const struct resource *r)  
> 
> Why inline? It could be in .c

OK.

> 
> > +{
> > +	return r->end - r->beg;
> > +}
> > +
> > +const struct resource *resource_find(const char *name);
> > +
> > +void __resource_register(struct resource *r);  
> 
> A comment is needed to explain the role of this function.
> Why a double underscore?

A non-API function. It shouldn't be called by user. Nothing more.

> 
> > +#define REGISTER_RESOURCE(_n, _b, _e) \  
> 
> I'm not a big fan of the underscores.

+1

> 
> > +static struct resource linkres_ ##_n = {       \
> > +	.name = RTE_STR(_n),     \
> > +	.beg = _b,               \
> > +	.end = _e,               \
> > +};                               \
> > +__REGISTER_RESOURCE(linkres_ ##_n)  
> 
> Please avoid nested macros.

I don't understand this. I am avoiding code duplication. Is it wrong?

> 
> > +#define __REGISTER_RESOURCE(name)     \
> > +static void __attribute__((constructor,used)) resinitfn_ ##name(void); \  
> 
> Why declaring the function just before its implementation?

Wrote in a hurry to work out-of-the box. Will fix...

> 
> > +static void __attribute__((constructor,used)) resinitfn_ ##name(void) \
> > +{                                   \
> > +	__resource_register(&name); \
> > +}  
> 



-- 
   Jan Viktorin                  E-mail: Viktorin@RehiveTech.com
   System Architect              Web:    www.RehiveTech.com
   RehiveTech
   Brno, Czech Republic

Re: [PATCH v1 01/10] app/test: introduce resources for tests

[Thomas Monjalon]

2016-05-06 18:20, Jan Viktorin:
> On Fri, 06 May 2016 16:01:08 +0200
> Thomas Monjalon <thomas.monjalon@6wind.com> wrote:
> > 2016-05-06 12:48, Jan Viktorin:
> > > +static struct resource linkres_ ##_n = {       \
> > > +	.name = RTE_STR(_n),     \
> > > +	.beg = _b,               \
> > > +	.end = _e,               \
> > > +};                               \
> > > +__REGISTER_RESOURCE(linkres_ ##_n)  
> > 
> > Please avoid nested macros.
> 
> I don't understand this. I am avoiding code duplication. Is it wrong?

Which code duplication?
Is __REGISTER_RESOURCE used elsewhere?

Re: [PATCH v1 01/10] app/test: introduce resources for tests

[Jan Viktorin]

On Thu, 12 May 2016 16:58:31 +0200
Thomas Monjalon <thomas.monjalon@6wind.com> wrote:

> 2016-05-06 18:20, Jan Viktorin:
> > On Fri, 06 May 2016 16:01:08 +0200
> > Thomas Monjalon <thomas.monjalon@6wind.com> wrote:  
> > > 2016-05-06 12:48, Jan Viktorin:  
> > > > +static struct resource linkres_ ##_n = {       \
> > > > +	.name = RTE_STR(_n),     \
> > > > +	.beg = _b,               \
> > > > +	.end = _e,               \
> > > > +};                               \
> > > > +__REGISTER_RESOURCE(linkres_ ##_n)    
> > > 
> > > Please avoid nested macros.  
> > 
> > I don't understand this. I am avoiding code duplication. Is it wrong?  
> 
> Which code duplication?
> Is __REGISTER_RESOURCE used elsewhere?

No :). I've already posted a fix of this.

-- 
   Jan Viktorin                  E-mail: Viktorin@RehiveTech.com
   System Architect              Web:    www.RehiveTech.com
   RehiveTech
   Brno, Czech Republic

Re: [PATCH v1 01/10] app/test: introduce resources for tests

[Jan Viktorin]

On Fri, 06 May 2016 16:01:08 +0200
Thomas Monjalon <thomas.monjalon@6wind.com> wrote:

> 2016-05-06 12:48, Jan Viktorin:
> > --- /dev/null
> > +++ b/app/test/resource.h
> > @@ -0,0 +1,61 @@
> > +/*-
> > + *   BSD LICENSE  
> [...]
> > + */  
> 
> Please include a multi-line comment here to explain what is a resource
> and why it is needed.
> 
> > +
> > +#ifndef _RESOURCE_H_
> > +#define _RESOURCE_H_
> > +  
> |...]
> |> +
> > +TAILQ_HEAD(resource_list, resource);
> > +extern struct resource_list resource_list;  
> 
> Why extern?

It is not a definition. There is a single global resource_list and
it is defined in the resource.c. A common practice in DPDK (eg. rte_pci.h).
Do I miss something?

Regards
Jan

> 

[...]


-- 
   Jan Viktorin                  E-mail: Viktorin@RehiveTech.com
   System Architect              Web:    www.RehiveTech.com
   RehiveTech
   Brno, Czech Republic

[PATCH v1 02/10] app/test: support resources externally linked

[Jan Viktorin]

To include resources from other source that the C source code we can take
advantage of the objcopy behaviour, i.e. packing of an arbitrary file as an
object file that is linked to the target program.

A linked object file is always accessible as a pair

extern const char beg_<name>;
extern const char end_<name>;
(extern const char siz_<name>;)

The objcopy however accepts architecture parameters in a very tricky way.
Try to translate as many arguments as possible from the RTE_ARCH into the
objcopy specific names.

We pack the resource.c source file as an example for testing.

*** CAUTION: ***
The objcopy and resource creation is a subject of change. Any comments of how
to integrate those are welcome.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/Makefile        | 32 ++++++++++++++++++++++++++++++++
 app/test/resource.h      |  5 +++++
 app/test/test_resource.c | 18 ++++++++++++++++++
 3 files changed, 55 insertions(+)

diff --git a/app/test/Makefile b/app/test/Makefile
index 7fbdd18..a9502f1 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -33,6 +33,37 @@ include $(RTE_SDK)/mk/rte.vars.mk
 
 ifeq ($(CONFIG_RTE_APP_TEST),y)
 
+ifeq ($(RTE_ARCH),arm)
+RTE_OBJCOPY_O = elf32-littlearm
+RTE_OBJCOPY_B = arm
+else ifeq ($(RTE_ARCH),arm64)
+RTE_OBJCOPY_O = elf64-littleaarch64
+RTE_OBJCOPY_B = aarch64
+else ifeq ($(RTE_ARCH),i686)
+RTE_OBJCOPY_O = elf32-i386
+RTE_OBJCOPY_B = i386
+else ifeq ($(RTE_ARCH),x86_64)
+RTE_OBJCOPY_O = elf64-x86-64
+RTE_OBJCOPY_B = i386:x86-64
+else ifeq ($(RTE_ARCH),x86_x32)
+RTE_OBJCOPY_O = elf32-x86-64
+RTE_OBJCOPY_B = i386:x86-64
+else
+$(error Unrecognized RTE_ARCH: $(RTE_ARCH))
+endif
+
+define resource
+SRCS-y += $(1).res.o
+$(1).res.o: $(2)
+	$(OBJCOPY) -I binary -B $(RTE_OBJCOPY_B) -O $(RTE_OBJCOPY_O)     \
+		--rename-section                                         \
+			.data=.rodata,alloc,load,data,contents,readonly  \
+		--redefine-sym _binary__dev_stdin_start=beg_$(1)         \
+		--redefine-sym _binary__dev_stdin_end=end_$(1)           \
+		--redefine-sym _binary__dev_stdin_size=siz_$(1)          \
+		/dev/stdin $$@ < $$<
+endef
+
 #
 # library name
 #
@@ -45,6 +76,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_CMDLINE) := commands.c
 SRCS-y += test.c
 SRCS-y += resource.c
 SRCS-y += test_resource.c
+$(eval $(call resource,test_resource_c,resource.c))
 SRCS-y += test_pci.c
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
diff --git a/app/test/resource.h b/app/test/resource.h
index ce9588e..84ee6b5 100644
--- a/app/test/resource.h
+++ b/app/test/resource.h
@@ -59,6 +59,11 @@ const struct resource *resource_find(const char *name);
 
 void __resource_register(struct resource *r);
 
+#define REGISTER_LINKED_RESOURCE(_n) \
+extern const char beg_ ##_n;         \
+extern const char end_ ##_n;         \
+REGISTER_RESOURCE(_n, &beg_ ##_n, &end_ ##_n); \
+
 #define REGISTER_RESOURCE(_n, _b, _e) \
 static struct resource linkres_ ##_n = {       \
 	.name = RTE_STR(_n),     \
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
index e3d2486..7752522 100644
--- a/app/test/test_resource.c
+++ b/app/test/test_resource.c
@@ -60,11 +60,29 @@ static int test_resource_dpdk(void)
 	return 0;
 }
 
+REGISTER_LINKED_RESOURCE(test_resource_c);
+
+static int test_resource_c(void)
+{
+	const struct resource *r;
+
+	r = resource_find("test_resource_c");
+	TEST_ASSERT_NOT_NULL(r, "No test_resource_c found");
+	TEST_ASSERT(!strcmp(r->name, "test_resource_c"),
+			"Found resource %s, expected test_resource_c",
+			r->name);
+
+	return 0;
+}
+
 static int test_resource(void)
 {
 	if (test_resource_dpdk())
 		return -1;
 
+	if (test_resource_c())
+		return -1;
+
 	return 0;
 }
 
-- 
2.8.0

Re: [PATCH v1 02/10] app/test: support resources externally linked

[Thomas Monjalon]

It looks a lot too much tricky to be integrated without code comments ;)
Please make a documentation effort.

2016-05-06 12:48, Jan Viktorin:
> --- a/app/test/Makefile
> +++ b/app/test/Makefile
> @@ -33,6 +33,37 @@ include $(RTE_SDK)/mk/rte.vars.mk
>  
>  ifeq ($(CONFIG_RTE_APP_TEST),y)

A comment is needed here to explain the intent of the following code.

> +ifeq ($(RTE_ARCH),arm)
> +RTE_OBJCOPY_O = elf32-littlearm
> +RTE_OBJCOPY_B = arm
> +else ifeq ($(RTE_ARCH),arm64)
> +RTE_OBJCOPY_O = elf64-littleaarch64
> +RTE_OBJCOPY_B = aarch64
> +else ifeq ($(RTE_ARCH),i686)
> +RTE_OBJCOPY_O = elf32-i386
> +RTE_OBJCOPY_B = i386
> +else ifeq ($(RTE_ARCH),x86_64)
> +RTE_OBJCOPY_O = elf64-x86-64
> +RTE_OBJCOPY_B = i386:x86-64
> +else ifeq ($(RTE_ARCH),x86_x32)
> +RTE_OBJCOPY_O = elf32-x86-64
> +RTE_OBJCOPY_B = i386:x86-64
> +else
> +$(error Unrecognized RTE_ARCH: $(RTE_ARCH))
> +endif

RTE_OBJCOPY_O could be renamed RTE_OBJCOPY_TARGET.
RTE_OBJCOPY_B could be renamed RTE_OBJCOPY_ARCH.

These definitions could be done in mk/arch/

> +
> +define resource

When defining a makefile macro, the arguments must be documented on
the previous line. Otherwise, nobody (including you) will be able
to read it without parsing the code below.

It is not a simple resource, so the name must avoid the confusion.
Why not linked_resource?

> +SRCS-y += $(1).res.o
> +$(1).res.o: $(2)
> +       $(OBJCOPY) -I binary -B $(RTE_OBJCOPY_B) -O $(RTE_OBJCOPY_O)     \
> +               --rename-section                                         \
> +                       .data=.rodata,alloc,load,data,contents,readonly  \
> +               --redefine-sym _binary__dev_stdin_start=beg_$(1)         \
> +               --redefine-sym _binary__dev_stdin_end=end_$(1)           \
> +               --redefine-sym _binary__dev_stdin_size=siz_$(1)          \
> +               /dev/stdin $$@ < $$<
> +endef

[...]

> +#define REGISTER_LINKED_RESOURCE(_n) \
> +extern const char beg_ ##_n;         \
> +extern const char end_ ##_n;         \
> +REGISTER_RESOURCE(_n, &beg_ ##_n, &end_ ##_n); \

Please explain the begin/end trick.
Thanks

Re: [PATCH v1 02/10] app/test: support resources externally linked

[Jan Viktorin]

Hello Thomas,

On Fri, 06 May 2016 16:32:53 +0200
Thomas Monjalon <thomas.monjalon@6wind.com> wrote:

> It looks a lot too much tricky to be integrated without code comments ;)
> Please make a documentation effort.

I know it's not the brightly shining code... I focused mainly on the C API of
this. Thanks a lot for comments. I will fix those and come back with something
better.

> 
> 2016-05-06 12:48, Jan Viktorin:
> > --- a/app/test/Makefile
> > +++ b/app/test/Makefile
> > @@ -33,6 +33,37 @@ include $(RTE_SDK)/mk/rte.vars.mk
> >  
> >  ifeq ($(CONFIG_RTE_APP_TEST),y)  
> 
> A comment is needed here to explain the intent of the following code.

Sure.

> 
> > +ifeq ($(RTE_ARCH),arm)
> > +RTE_OBJCOPY_O = elf32-littlearm
> > +RTE_OBJCOPY_B = arm
> > +else ifeq ($(RTE_ARCH),arm64)
> > +RTE_OBJCOPY_O = elf64-littleaarch64
> > +RTE_OBJCOPY_B = aarch64
> > +else ifeq ($(RTE_ARCH),i686)
> > +RTE_OBJCOPY_O = elf32-i386
> > +RTE_OBJCOPY_B = i386
> > +else ifeq ($(RTE_ARCH),x86_64)
> > +RTE_OBJCOPY_O = elf64-x86-64
> > +RTE_OBJCOPY_B = i386:x86-64
> > +else ifeq ($(RTE_ARCH),x86_x32)
> > +RTE_OBJCOPY_O = elf32-x86-64
> > +RTE_OBJCOPY_B = i386:x86-64
> > +else
> > +$(error Unrecognized RTE_ARCH: $(RTE_ARCH))
> > +endif  
> 
> RTE_OBJCOPY_O could be renamed RTE_OBJCOPY_TARGET.
> RTE_OBJCOPY_B could be renamed RTE_OBJCOPY_ARCH.
> 
> These definitions could be done in mk/arch/

Cool, I will move those. However, I don't know all the cryptic objcopy
names for all platforms. I've tested just arm, arm64 and x86_64. I guessed
the rest (or copied from some web page).

> 
> > +
> > +define resource  
> 
> When defining a makefile macro, the arguments must be documented on
> the previous line. Otherwise, nobody (including you) will be able
> to read it without parsing the code below.

Sure!

What about a better name (RTE_TEST_RESOURCE, TEST_RESOURCE)? Should this
macro stay in this Makefile?

> 
> It is not a simple resource, so the name must avoid the confusion.
> Why not linked_resource?
> 
> > +SRCS-y += $(1).res.o
> > +$(1).res.o: $(2)
> > +       $(OBJCOPY) -I binary -B $(RTE_OBJCOPY_B) -O $(RTE_OBJCOPY_O)     \
> > +               --rename-section                                         \
> > +                       .data=.rodata,alloc,load,data,contents,readonly  \
> > +               --redefine-sym _binary__dev_stdin_start=beg_$(1)         \
> > +               --redefine-sym _binary__dev_stdin_end=end_$(1)           \
> > +               --redefine-sym _binary__dev_stdin_size=siz_$(1)          \
> > +               /dev/stdin $$@ < $$<
> > +endef  
> 
> [...]
> 
> > +#define REGISTER_LINKED_RESOURCE(_n) \
> > +extern const char beg_ ##_n;         \
> > +extern const char end_ ##_n;         \
> > +REGISTER_RESOURCE(_n, &beg_ ##_n, &end_ ##_n); \  
> 
> Please explain the begin/end trick.

The objcopy creates an object file with our data located at _binary__dev_stdin_start
until _binary__dev_stdin_end. The names are derrived from the input file name - this is
so so stupid! I used /dev/stdin here to have a deterministic name of the *_start/_end
labels. Otherwise, the code would be very messy (trying to santize the aboslute!!! path
passed by make).

In our C code, we declare

extern const char _binary__dev_stdin_start;
extern const char _binary__dev_stdin_end;

...variables placed exactly on addresses where our data were put by objcopy.

This &_binary__dev_stdin_start is address of our data.

Those names are however changed by --redefine-sym to be unique and to reflect the
name we use to refer to the data from our C code by REGISTER_LINKED_RESOURCE(<name>).

Regards
Jan

> Thanks



-- 
   Jan Viktorin                  E-mail: Viktorin@RehiveTech.com
   System Architect              Web:    www.RehiveTech.com
   RehiveTech
   Brno, Czech Republic

Re: [PATCH v1 02/10] app/test: support resources externally linked

[Jan Viktorin]

On Fri, 06 May 2016 16:32:53 +0200
Thomas Monjalon <thomas.monjalon@6wind.com> wrote:

> It looks a lot too much tricky to be integrated without code comments ;)
> Please make a documentation effort.
> 
[...]

Thomas,

is it OK to include the PCI test enhancements? Or should I post only the
fixed "resource framework"?

Regards
Jan

-- 
   Jan Viktorin                  E-mail: Viktorin@RehiveTech.com
   System Architect              Web:    www.RehiveTech.com
   RehiveTech
   Brno, Czech Republic

Re: [PATCH v1 02/10] app/test: support resources externally linked

[Thomas Monjalon]

2016-05-09 17:19, Jan Viktorin:
> On Fri, 06 May 2016 16:32:53 +0200
> Thomas Monjalon <thomas.monjalon@6wind.com> wrote:
> 
> > It looks a lot too much tricky to be integrated without code comments ;)
> > Please make a documentation effort.
> > 
> [...]
> 
> Thomas,
> 
> is it OK to include the PCI test enhancements? Or should I post only the
> fixed "resource framework"?

Yes it is OK.

[PATCH v1 03/10] app/test: add functions to create files from resources

[Jan Viktorin]

A resource can be written into the target filesystem by calling resource_fwrite
or resource_fwrite_file. Such file can be created before a test is started and
removed after the test finishes.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/resource.c      | 35 +++++++++++++++++++++++++++++++++++
 app/test/resource.h      |  4 ++++
 app/test/test_resource.c | 10 ++++++++++
 3 files changed, 49 insertions(+)

diff --git a/app/test/resource.c b/app/test/resource.c
index 50a4510..a11c86e 100644
--- a/app/test/resource.c
+++ b/app/test/resource.c
@@ -31,6 +31,7 @@
  *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+#include <stdio.h>
 #include <string.h>
 #include <errno.h>
 #include <sys/queue.h>
@@ -55,6 +56,40 @@ const struct resource *resource_find(const char *name)
 	return NULL;
 }
 
+int resource_fwrite(const struct resource *r, FILE *f)
+{
+	const size_t goal = resource_size(r);
+	size_t total = 0;
+
+	while (total < goal) {
+		size_t wlen = fwrite(r->beg + total, 1, goal - total, f);
+		if (wlen == 0) {
+			perror(__func__);
+			return -1;
+		}
+
+		total += wlen;
+	}
+
+	return 0;
+}
+
+int resource_fwrite_file(const struct resource *r, const char *fname)
+{
+	FILE *f;
+	int ret;
+
+	f = fopen(fname, "w");
+	if (f == NULL) {
+		perror(__func__);
+		return -1;
+	}
+
+	ret = resource_fwrite(r, f);
+	fclose(f);
+	return ret;
+}
+
 void __resource_register(struct resource *r)
 {
 	TAILQ_INSERT_TAIL(&resource_list, r, next);
diff --git a/app/test/resource.h b/app/test/resource.h
index 84ee6b5..9f85c16 100644
--- a/app/test/resource.h
+++ b/app/test/resource.h
@@ -35,6 +35,7 @@
 #define _RESOURCE_H_
 
 #include <sys/queue.h>
+#include <stdio.h>
 #include <stddef.h>
 
 #include <rte_eal.h>
@@ -57,6 +58,9 @@ static inline size_t resource_size(const struct resource *r)
 
 const struct resource *resource_find(const char *name);
 
+int resource_fwrite(const struct resource *r, FILE *f);
+int resource_fwrite_file(const struct resource *r, const char *fname);
+
 void __resource_register(struct resource *r);
 
 #define REGISTER_LINKED_RESOURCE(_n) \
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
index 7752522..148964e 100644
--- a/app/test/test_resource.c
+++ b/app/test/test_resource.c
@@ -65,6 +65,7 @@ REGISTER_LINKED_RESOURCE(test_resource_c);
 static int test_resource_c(void)
 {
 	const struct resource *r;
+	FILE *f;
 
 	r = resource_find("test_resource_c");
 	TEST_ASSERT_NOT_NULL(r, "No test_resource_c found");
@@ -72,6 +73,15 @@ static int test_resource_c(void)
 			"Found resource %s, expected test_resource_c",
 			r->name);
 
+	TEST_ASSERT_SUCCESS(resource_fwrite_file(r, "test_resource.c"),
+			"Failed to to write file %s", r->name);
+
+	f = fopen("test_resource.c", "r");
+	TEST_ASSERT_NOT_NULL(f,
+			"Missing extracted file resource.c");
+	fclose(f);
+	remove("test_resource.c");
+
 	return 0;
 }
 
-- 
2.8.0

[PATCH v1 04/10] app/test: support resources archived by tar

[Jan Viktorin]

When needing a more complex resource (a file hierarchy), packing every single
file as a single resource would be very ineffective. For that purpose, it is
possible to pack the files into a tar archive, extract it before test from the
resource and finally clean up all the created files.

This patch introduces functions resource_untar and resource_rm_by_tar to
perform those tasks. An example of using those functions is included as a test.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/Makefile        |   4 ++
 app/test/resource.c      | 180 +++++++++++++++++++++++++++++++++++++++++++++++
 app/test/resource.h      |  13 ++++
 app/test/test_resource.c |  29 ++++++++
 4 files changed, 226 insertions(+)

diff --git a/app/test/Makefile b/app/test/Makefile
index a9502f1..90acd63 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -77,6 +77,9 @@ SRCS-y += test.c
 SRCS-y += resource.c
 SRCS-y += test_resource.c
 $(eval $(call resource,test_resource_c,resource.c))
+$(eval $(call resource,test_resource_tar,resource.tar))
+resource.tar: test_resource.c
+	tar -C $(dir $<) -cf $@ $(notdir $<)
 SRCS-y += test_pci.c
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
@@ -196,6 +199,7 @@ CFLAGS += $(WERROR_FLAGS)
 CFLAGS += -D_GNU_SOURCE
 
 LDLIBS += -lm
+LDLIBS += -larchive
 
 # Disable VTA for memcpy test
 ifeq ($(CC), gcc)
diff --git a/app/test/resource.c b/app/test/resource.c
index a11c86e..7732bc7 100644
--- a/app/test/resource.c
+++ b/app/test/resource.c
@@ -33,6 +33,8 @@
 
 #include <stdio.h>
 #include <string.h>
+#include <archive.h>
+#include <archive_entry.h>
 #include <errno.h>
 #include <sys/queue.h>
 
@@ -90,6 +92,184 @@ int resource_fwrite_file(const struct resource *r, const char *fname)
 	return ret;
 }
 
+static int do_copy(struct archive *r, struct archive *w)
+{
+	const void *buf;
+	size_t len;
+	off_t off;
+	int ret;
+
+	while (1) {
+		ret = archive_read_data_block(r, &buf, &len, &off);
+		if (ret == ARCHIVE_RETRY)
+			continue;
+
+		if (ret == ARCHIVE_EOF)
+			return 0;
+
+		if (ret != ARCHIVE_OK)
+			return ret;
+
+		do {
+			ret = archive_write_data_block(w, buf, len, off);
+			if (ret != ARCHIVE_OK && ret != ARCHIVE_RETRY)
+				return ret;
+		} while (ret != ARCHIVE_OK);
+	}
+}
+
+int resource_untar(const struct resource *res)
+{
+	struct archive *r;
+	struct archive *w;
+	struct archive_entry *e;
+	void *p;
+	int flags = 0;
+	int ret;
+
+	p = malloc(resource_size(res));
+	if (p == NULL)
+		rte_panic("Failed to malloc %zu B\n", resource_size(res));
+
+	memcpy(p, res->beg, resource_size(res));
+
+	r = archive_read_new();
+	if (r == NULL) {
+		free(p);
+		return -1;
+	}
+
+	archive_read_support_format_all(r);
+	archive_read_support_filter_all(r);
+
+	w = archive_write_disk_new();
+	if (w == NULL) {
+		archive_read_free(r);
+		free(p);
+		return -1;
+	}
+
+	flags |= ARCHIVE_EXTRACT_PERM;
+	flags |= ARCHIVE_EXTRACT_FFLAGS;
+	archive_write_disk_set_options(w, flags);
+	archive_write_disk_set_standard_lookup(w);
+
+	ret = archive_read_open_memory(r, p, resource_size(res));
+	if (ret != ARCHIVE_OK)
+		goto fail;
+
+	while (1) {
+		ret = archive_read_next_header(r, &e);
+		if (ret == ARCHIVE_EOF)
+			break;
+		if (ret != ARCHIVE_OK)
+			goto fail;
+
+		ret = archive_write_header(w, e);
+		if (ret == ARCHIVE_EOF)
+			break;
+		if (ret != ARCHIVE_OK)
+			goto fail;
+
+		if (archive_entry_size(e) == 0)
+			continue;
+
+		ret = do_copy(r, w);
+		if (ret != ARCHIVE_OK)
+			goto fail;
+
+		ret = archive_write_finish_entry(w);
+		if (ret != ARCHIVE_OK)
+			goto fail;
+	}
+
+	archive_write_free(w);
+	archive_read_free(r);
+	free(p);
+	return 0;
+
+fail:
+	archive_write_free(w);
+	archive_read_free(r);
+	free(p);
+	rte_panic("Failed: %s\n", archive_error_string(r));
+	return -1;
+}
+
+int resource_rm_by_tar(const struct resource *res)
+{
+	struct archive *r;
+	struct archive_entry *e;
+	void *p;
+	int try_again = 1;
+	int ret;
+
+	p = malloc(resource_size(res));
+	if (p == NULL)
+		rte_panic("Failed to malloc %zu B\n", resource_size(res));
+
+	memcpy(p, res->beg, resource_size(res));
+
+	while (try_again) {
+		r = archive_read_new();
+		if (r == NULL) {
+			free(p);
+			return -1;
+		}
+
+		archive_read_support_format_all(r);
+		archive_read_support_filter_all(r);
+
+		ret = archive_read_open_memory(r, p, resource_size(res));
+		if (ret != ARCHIVE_OK) {
+			fprintf(stderr, "Failed: %s\n",
+					archive_error_string(r));
+			goto fail;
+		}
+
+		try_again = 0;
+
+		while (1) {
+			ret = archive_read_next_header(r, &e);
+			if (ret == ARCHIVE_EOF)
+				break;
+			if (ret != ARCHIVE_OK)
+				goto fail;
+
+			ret = remove(archive_entry_pathname(e));
+			if (ret < 0) {
+				switch (errno) {
+				case ENOTEMPTY:
+				case EEXIST:
+					try_again = 1;
+					break;
+
+				/* should not usually happen: */
+				case ENOENT:
+				case ENOTDIR:
+				case EROFS:
+					continue;
+				default:
+					perror("Failed to remove file");
+					goto fail;
+				}
+			}
+		}
+
+		archive_read_free(r);
+	}
+
+	free(p);
+	return 0;
+
+fail:
+	archive_read_free(r);
+	free(p);
+
+	rte_panic("Failed: %s\n", archive_error_string(r));
+	return -1;
+}
+
 void __resource_register(struct resource *r)
 {
 	TAILQ_INSERT_TAIL(&resource_list, r, next);
diff --git a/app/test/resource.h b/app/test/resource.h
index 9f85c16..ed91d4e 100644
--- a/app/test/resource.h
+++ b/app/test/resource.h
@@ -61,6 +61,19 @@ const struct resource *resource_find(const char *name);
 int resource_fwrite(const struct resource *r, FILE *f);
 int resource_fwrite_file(const struct resource *r, const char *fname);
 
+/**
+ * Treat the given resource as a tar archive. Extract
+ * the archive to the current directory.
+ */
+int resource_untar(const struct resource *res);
+
+/**
+ * Treat the given resource as a tar archive. Remove
+ * all files (related to the current directory) listed
+ * in the tar archive.
+ */
+int resource_rm_by_tar(const struct resource *res);
+
 void __resource_register(struct resource *r);
 
 #define REGISTER_LINKED_RESOURCE(_n) \
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
index 148964e..7b9ed31 100644
--- a/app/test/test_resource.c
+++ b/app/test/test_resource.c
@@ -85,6 +85,32 @@ static int test_resource_c(void)
 	return 0;
 }
 
+REGISTER_LINKED_RESOURCE(test_resource_tar);
+
+static int test_resource_tar(void)
+{
+	const struct resource *r;
+	FILE *f;
+
+	r = resource_find("test_resource_tar");
+	TEST_ASSERT_NOT_NULL(r, "No test_resource_tar found");
+	TEST_ASSERT(!strcmp(r->name, "test_resource_tar"),
+			"Found resource %s, expected test_resource_tar",
+			r->name);
+
+	TEST_ASSERT_SUCCESS(resource_untar(r),
+			"Failed to to untar %s", r->name);
+
+	f = fopen("test_resource.c", "r");
+	TEST_ASSERT_NOT_NULL(f,
+			"Missing extracted file test_resource.c");
+	fclose(f);
+
+	TEST_ASSERT_SUCCESS(resource_rm_by_tar(r),
+			"Failed to remove extracted contents of %s", r->name);
+	return 0;
+}
+
 static int test_resource(void)
 {
 	if (test_resource_dpdk())
@@ -93,6 +119,9 @@ static int test_resource(void)
 	if (test_resource_c())
 		return -1;
 
+	if (test_resource_tar())
+		return -1;
+
 	return 0;
 }
 
-- 
2.8.0

Re: [PATCH v1 04/10] app/test: support resources archived by tar

[Thomas Monjalon]

2016-05-06 12:48, Jan Viktorin:
> When needing a more complex resource (a file hierarchy), packing every single
> file as a single resource would be very ineffective. For that purpose, it is
> possible to pack the files into a tar archive, extract it before test from the
> resource and finally clean up all the created files.
> 
> This patch introduces functions resource_untar and resource_rm_by_tar to
> perform those tasks. An example of using those functions is included as a test.
> 
> Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
> ---
>  app/test/Makefile        |   4 ++
>  app/test/resource.c      | 180 +++++++++++++++++++++++++++++++++++++++++++++++
>  app/test/resource.h      |  13 ++++
>  app/test/test_resource.c |  29 ++++++++
>  4 files changed, 226 insertions(+)
> 
> diff --git a/app/test/Makefile b/app/test/Makefile
> index a9502f1..90acd63 100644
> --- a/app/test/Makefile
> +++ b/app/test/Makefile
> @@ -77,6 +77,9 @@ SRCS-y += test.c
>  SRCS-y += resource.c
>  SRCS-y += test_resource.c
>  $(eval $(call resource,test_resource_c,resource.c))
> +$(eval $(call resource,test_resource_tar,resource.tar))

This tar resource is not provided. Should it be created in the Makefile?

Re: [PATCH v1 04/10] app/test: support resources archived by tar

[Thomas Monjalon]

2016-05-12 17:26, Thomas Monjalon:
> 2016-05-06 12:48, Jan Viktorin:
> > +$(eval $(call resource,test_resource_tar,resource.tar))
> 
> This tar resource is not provided. Should it be created in the Makefile?

Sorry I was looking at v1 while it is implemented in v2.

[PATCH v1 05/10] app/test: use linked list to store PCI drivers

[Jan Viktorin]

The test unregisters all real drivers before starting into an array. This
inflexiable as we can use a linked list for this purpose.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/test_pci.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 0ed357e..cf82373 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -144,21 +144,24 @@ static void free_devargs_list(void)
 	}
 }
 
+/* real drivers (not used for testing) */
+struct pci_driver_list real_pci_driver_list =
+	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
+
 int
 test_pci(void)
 {
 	struct rte_devargs_list save_devargs_list;
 	struct rte_pci_driver *dr = NULL;
-	struct rte_pci_driver *save_pci_driver_list[NUM_MAX_DRIVERS];
-	unsigned i, num_drivers = 0;
 
 	printf("Dump all devices\n");
 	rte_eal_pci_dump(stdout);
 
 	/* Unregister all previous drivers */
-	TAILQ_FOREACH(dr, &pci_driver_list, next) {
+	while (!TAILQ_EMPTY(&pci_driver_list)) {
+		dr = TAILQ_FIRST(&pci_driver_list);
 		rte_eal_pci_unregister(dr);
-		save_pci_driver_list[num_drivers++] = dr;
+		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
 	}
 
 	rte_eal_pci_register(&my_driver);
@@ -197,8 +200,11 @@ test_pci(void)
 	rte_eal_pci_unregister(&my_driver2);
 
 	/* Restore original driver list */
-	for (i = 0; i < num_drivers; i++)
-		rte_eal_pci_register(save_pci_driver_list[i]);
+	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
+		dr = TAILQ_FIRST(&real_pci_driver_list);
+		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
+		rte_eal_pci_register(dr);
+	}
 
 	return 0;
 }
-- 
2.8.0

[PATCH v1 06/10] app/test: extract test_pci_setup and test_pci_cleanup

[Jan Viktorin]

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/test_pci.c | 47 ++++++++++++++++++++++++++++++++++-------------
 1 file changed, 34 insertions(+), 13 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index cf82373..9d53ba5 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -148,21 +148,46 @@ static void free_devargs_list(void)
 struct pci_driver_list real_pci_driver_list =
 	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
 
+static int
+test_pci_setup(void)
+{
+	struct rte_pci_driver *dr;
+
+	/* Unregister original driver list */
+	while (!TAILQ_EMPTY(&pci_driver_list)) {
+		dr = TAILQ_FIRST(&pci_driver_list);
+		rte_eal_pci_unregister(dr);
+		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
+	}
+
+	return 0;
+}
+
+static int
+test_pci_cleanup(void)
+{
+	struct rte_pci_driver *dr;
+
+	/* Restore original driver list */
+	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
+		dr = TAILQ_FIRST(&real_pci_driver_list);
+		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
+		rte_eal_pci_register(dr);
+	}
+
+	return 0;
+}
+
 int
 test_pci(void)
 {
 	struct rte_devargs_list save_devargs_list;
-	struct rte_pci_driver *dr = NULL;
 
 	printf("Dump all devices\n");
 	rte_eal_pci_dump(stdout);
 
-	/* Unregister all previous drivers */
-	while (!TAILQ_EMPTY(&pci_driver_list)) {
-		dr = TAILQ_FIRST(&pci_driver_list);
-		rte_eal_pci_unregister(dr);
-		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
-	}
+	if (test_pci_setup())
+		return -1;
 
 	rte_eal_pci_register(&my_driver);
 	rte_eal_pci_register(&my_driver2);
@@ -199,12 +224,8 @@ test_pci(void)
 	rte_eal_pci_unregister(&my_driver);
 	rte_eal_pci_unregister(&my_driver2);
 
-	/* Restore original driver list */
-	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
-		dr = TAILQ_FIRST(&real_pci_driver_list);
-		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
-		rte_eal_pci_register(dr);
-	}
+	if (test_pci_cleanup())
+		return -1;
 
 	return 0;
 }
-- 
2.8.0

[PATCH v1 07/10] app/test: convert current pci_test into a single test case

[Jan Viktorin]

The current test_pci is just a single test case that tests the blacklisting
of devices. Rename it to test_pci_blacklist and call it from the test_pci.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/test_pci.c | 85 +++++++++++++++++++++++++++++------------------------
 1 file changed, 47 insertions(+), 38 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 9d53ba5..2e2fd70 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -144,51 +144,14 @@ static void free_devargs_list(void)
 	}
 }
 
-/* real drivers (not used for testing) */
-struct pci_driver_list real_pci_driver_list =
-	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
-
 static int
-test_pci_setup(void)
-{
-	struct rte_pci_driver *dr;
-
-	/* Unregister original driver list */
-	while (!TAILQ_EMPTY(&pci_driver_list)) {
-		dr = TAILQ_FIRST(&pci_driver_list);
-		rte_eal_pci_unregister(dr);
-		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
-	}
-
-	return 0;
-}
-
-static int
-test_pci_cleanup(void)
-{
-	struct rte_pci_driver *dr;
-
-	/* Restore original driver list */
-	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
-		dr = TAILQ_FIRST(&real_pci_driver_list);
-		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
-		rte_eal_pci_register(dr);
-	}
-
-	return 0;
-}
-
-int
-test_pci(void)
+test_pci_blacklist(void)
 {
 	struct rte_devargs_list save_devargs_list;
 
 	printf("Dump all devices\n");
 	rte_eal_pci_dump(stdout);
 
-	if (test_pci_setup())
-		return -1;
-
 	rte_eal_pci_register(&my_driver);
 	rte_eal_pci_register(&my_driver2);
 
@@ -224,6 +187,52 @@ test_pci(void)
 	rte_eal_pci_unregister(&my_driver);
 	rte_eal_pci_unregister(&my_driver2);
 
+	return 0;
+}
+
+/* real drivers (not used for testing) */
+struct pci_driver_list real_pci_driver_list =
+	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
+
+static int
+test_pci_setup(void)
+{
+	struct rte_pci_driver *dr;
+
+	/* Unregister original driver list */
+	while (!TAILQ_EMPTY(&pci_driver_list)) {
+		dr = TAILQ_FIRST(&pci_driver_list);
+		rte_eal_pci_unregister(dr);
+		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
+	}
+
+	return 0;
+}
+
+static int
+test_pci_cleanup(void)
+{
+	struct rte_pci_driver *dr;
+
+	/* Restore original driver list */
+	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
+		dr = TAILQ_FIRST(&real_pci_driver_list);
+		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
+		rte_eal_pci_register(dr);
+	}
+
+	return 0;
+}
+
+int
+test_pci(void)
+{
+	if (test_pci_setup())
+		return -1;
+
+	if (test_pci_blacklist())
+		return -1;
+
 	if (test_pci_cleanup())
 		return -1;
 
-- 
2.8.0

[PATCH v1 08/10] eal/pci: replace SYSFS_PCI_DEVICES with pci_get_sysfs_path()

[Jan Viktorin]

The SYSFS_PCI_DEVICES is a constant that makes the PCI testing difficult as
it points to an absolute path. We remove using this constant and introducing
a function pci_get_sysfs_path that gives the same value. However, the user can
pass a SYSFS_PCI_DEVICES env variable to override the path. It is now possible
to create a fake sysfs hierarchy for testing.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/test_pci.c                             | 28 +++++++++++++++++++++++++
 drivers/net/szedata2/rte_eth_szedata2.c         |  2 +-
 drivers/net/virtio/virtio_pci.c                 |  2 +-
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  6 ++++++
 lib/librte_eal/common/eal_common_pci.c          | 13 ++++++++++++
 lib/librte_eal/common/include/rte_pci.h         |  2 +-
 lib/librte_eal/linuxapp/eal/eal_pci.c           |  6 +++---
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c       |  7 ++++---
 lib/librte_eal/linuxapp/eal/eal_pci_vfio.c      |  2 +-
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |  6 ++++++
 10 files changed, 64 insertions(+), 10 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 2e2fd70..df12bb2 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -190,6 +190,31 @@ test_pci_blacklist(void)
 	return 0;
 }
 
+static int test_pci_sysfs(void)
+{
+	const char *orig;
+	const char *path;
+	int ret;
+
+	orig = pci_get_sysfs_path();
+	ret = setenv("SYSFS_PCI_DEVICES", "My Documents", 1);
+	TEST_ASSERT_SUCCESS(ret, "Failed setenv to My Documents");
+
+	path = pci_get_sysfs_path();
+	TEST_ASSERT(strcmp(orig, path),
+			"orig must be different from path: "
+			"%s (orig: %s)", path, orig);
+
+	ret = setenv("SYSFS_PCI_DEVICES", orig, 1);
+	TEST_ASSERT_SUCCESS(ret, "Failed setenv back to '%s'", orig);
+
+	path = pci_get_sysfs_path();
+	TEST_ASSERT(!strcmp(orig, path),
+			"pci_get_sysfs_path returned unexpected path: "
+			"%s (orig: %s)", path, orig);
+	return 0;
+}
+
 /* real drivers (not used for testing) */
 struct pci_driver_list real_pci_driver_list =
 	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
@@ -227,6 +252,9 @@ test_pci_cleanup(void)
 int
 test_pci(void)
 {
+	if (test_pci_sysfs())
+		return -1;
+
 	if (test_pci_setup())
 		return -1;
 
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 78c43b0..985a8d6 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1481,7 +1481,7 @@ rte_szedata2_eth_dev_init(struct rte_eth_dev *dev)
 		return -EINVAL;
 	}
 	snprintf(rsc_filename, PATH_MAX,
-		SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/resource%u",
+		"%s/" PCI_PRI_FMT "/resource%u", pci_get_sysfs_path(),
 		pci_addr->domain, pci_addr->bus,
 		pci_addr->devid, pci_addr->function, PCI_RESOURCE_NUMBER);
 	fd = open(rsc_filename, O_RDWR);
diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
index c007959..4bc058d 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -179,7 +179,7 @@ legacy_virtio_has_msix(const struct rte_pci_addr *loc)
 	char dirname[PATH_MAX];
 
 	snprintf(dirname, sizeof(dirname),
-		     SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/msi_irqs",
+		     "%s/" PCI_PRI_FMT "/msi_irqs", pci_get_sysfs_path(),
 		     loc->domain, loc->bus, loc->devid, loc->function);
 
 	d = opendir(dirname);
diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
index 58c2951..e37a175 100644
--- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
@@ -151,3 +151,9 @@ DPDK_16.04 {
 	rte_eal_primary_proc_alive;
 
 } DPDK_2.2;
+
+DPDK_16.07 {
+	global:
+
+	pci_get_sysfs_path;
+} DPDK_16.04;
diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c
index 3cae4cb..0ec3b61 100644
--- a/lib/librte_eal/common/eal_common_pci.c
+++ b/lib/librte_eal/common/eal_common_pci.c
@@ -85,6 +85,19 @@
 struct pci_driver_list pci_driver_list;
 struct pci_device_list pci_device_list;
 
+#define SYSFS_PCI_DEVICES "/sys/bus/pci/devices"
+
+const char *pci_get_sysfs_path(void)
+{
+	const char *path = NULL;
+
+	path = getenv("SYSFS_PCI_DEVICES");
+	if (path == NULL)
+		return SYSFS_PCI_DEVICES;
+
+	return path;
+}
+
 static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev)
 {
 	struct rte_devargs *devargs;
diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index 8fa2712..7669fd7 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -91,7 +91,7 @@ extern struct pci_driver_list pci_driver_list; /**< Global list of PCI drivers.
 extern struct pci_device_list pci_device_list; /**< Global list of PCI devices. */
 
 /** Pathname of PCI devices directory. */
-#define SYSFS_PCI_DEVICES "/sys/bus/pci/devices"
+const char *pci_get_sysfs_path(void);
 
 /** Formatting string for PCI device identifier: Ex: 0000:00:01.0 */
 #define PCI_PRI_FMT "%.4" PRIx16 ":%.2" PRIx8 ":%.2" PRIx8 ".%" PRIx8
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index bdc08a0..8e130a2 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -66,7 +66,7 @@ pci_unbind_kernel_driver(struct rte_pci_device *dev)
 
 	/* open /sys/bus/pci/devices/AAAA:BB:CC.D/driver */
 	snprintf(filename, sizeof(filename),
-	         SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/driver/unbind",
+	         "%s/" PCI_PRI_FMT "/driver/unbind", pci_get_sysfs_path(),
 	         loc->domain, loc->bus, loc->devid, loc->function);
 
 	f = fopen(filename, "w");
@@ -453,7 +453,7 @@ rte_eal_pci_scan(void)
 	uint16_t domain;
 	uint8_t bus, devid, function;
 
-	dir = opendir(SYSFS_PCI_DEVICES);
+	dir = opendir(pci_get_sysfs_path());
 	if (dir == NULL) {
 		RTE_LOG(ERR, EAL, "%s(): opendir failed: %s\n",
 			__func__, strerror(errno));
@@ -468,7 +468,7 @@ rte_eal_pci_scan(void)
 				&bus, &devid, &function) != 0)
 			continue;
 
-		snprintf(dirname, sizeof(dirname), "%s/%s", SYSFS_PCI_DEVICES,
+		snprintf(dirname, sizeof(dirname), "%s/%s", pci_get_sysfs_path(),
 			 e->d_name);
 		if (pci_scan_one(dirname, domain, bus, devid, function) < 0)
 			goto error;
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
index 068694d..b833244 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
@@ -161,14 +161,14 @@ pci_get_uio_dev(struct rte_pci_device *dev, char *dstbuf,
 	 * or uio:uioX */
 
 	snprintf(dirname, sizeof(dirname),
-			SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/uio",
+			"%s/" PCI_PRI_FMT "/uio", pci_get_sysfs_path(),
 			loc->domain, loc->bus, loc->devid, loc->function);
 
 	dir = opendir(dirname);
 	if (dir == NULL) {
 		/* retry with the parent directory */
 		snprintf(dirname, sizeof(dirname),
-				SYSFS_PCI_DEVICES "/" PCI_PRI_FMT,
+				"%s/" PCI_PRI_FMT, pci_get_sysfs_path(),
 				loc->domain, loc->bus, loc->devid, loc->function);
 		dir = opendir(dirname);
 
@@ -319,7 +319,8 @@ pci_uio_map_resource_by_index(struct rte_pci_device *dev, int res_idx,
 
 	/* update devname for mmap  */
 	snprintf(devname, sizeof(devname),
-			SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/resource%d",
+			"%s/" PCI_PRI_FMT "/resource%d",
+			pci_get_sysfs_path(),
 			loc->domain, loc->bus, loc->devid,
 			loc->function, res_idx);
 
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c b/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c
index 10266f8..f91b924 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c
@@ -602,7 +602,7 @@ pci_vfio_get_group_no(const char *pci_addr, int *iommu_group_no)
 
 	/* try to find out IOMMU group for this device */
 	snprintf(linkname, sizeof(linkname),
-			 SYSFS_PCI_DEVICES "/%s/iommu_group", pci_addr);
+			 "%s/%s/iommu_group", pci_get_sysfs_path(), pci_addr);
 
 	ret = readlink(linkname, filename, sizeof(filename));
 
diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
index 12503ef..905c6ee 100644
--- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
@@ -154,3 +154,9 @@ DPDK_16.04 {
 	rte_eal_primary_proc_alive;
 
 } DPDK_2.2;
+
+DPDK_16.07 {
+	global:
+
+	pci_get_sysfs_path;
+} DPDK_16.04;
-- 
2.8.0

[PATCH v1 09/10] app/test: scan PCI bus using a fake sysfs

[Jan Viktorin]

Scan the PCI bus by providing a fake sysfs with a PCI device. The fake sysfs
is a packed file hierarchy linked into the test.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/Makefile                                  |   4 ++
 app/test/test_pci.c                                |  58 +++++++++++++++++++--
 .../bus/pci/devices/0000:01:00.0/class             |   1 +
 .../bus/pci/devices/0000:01:00.0/config            | Bin 0 -> 64 bytes
 .../devices/0000:01:00.0/consistent_dma_mask_bits  |   1 +
 .../bus/pci/devices/0000:01:00.0/device            |   1 +
 .../bus/pci/devices/0000:01:00.0/dma_mask_bits     |   1 +
 .../bus/pci/devices/0000:01:00.0/enable            |   1 +
 .../bus/pci/devices/0000:01:00.0/irq               |   1 +
 .../bus/pci/devices/0000:01:00.0/modalias          |   1 +
 .../bus/pci/devices/0000:01:00.0/msi_bus           |   1 +
 .../bus/pci/devices/0000:01:00.0/numa_node         |   1 +
 .../bus/pci/devices/0000:01:00.0/resource          |  13 +++++
 .../bus/pci/devices/0000:01:00.0/sriov_numvfs      |   1 +
 .../bus/pci/devices/0000:01:00.0/sriov_totalvfs    |   1 +
 .../bus/pci/devices/0000:01:00.0/subsystem_device  |   1 +
 .../bus/pci/devices/0000:01:00.0/subsystem_vendor  |   1 +
 .../bus/pci/devices/0000:01:00.0/uevent            |   6 +++
 .../bus/pci/devices/0000:01:00.0/vendor            |   1 +
 19 files changed, 92 insertions(+), 3 deletions(-)
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/config
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor

diff --git a/app/test/Makefile b/app/test/Makefile
index 90acd63..6c3d8c2 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -81,6 +81,10 @@ $(eval $(call resource,test_resource_tar,resource.tar))
 resource.tar: test_resource.c
 	tar -C $(dir $<) -cf $@ $(notdir $<)
 SRCS-y += test_pci.c
+$(eval $(call resource,test_pci_sysfs,test_pci_sysfs.tar))
+test_pci_sysfs.tar: test_pci_sysfs
+	tar -C $(dir $<) -cf $@ $(notdir $<)
+
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
 SRCS-y += test_per_lcore.c
diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index df12bb2..3c6c955 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -43,6 +43,7 @@
 #include <rte_devargs.h>
 
 #include "test.h"
+#include "resource.h"
 
 /* Generic maximum number of drivers to have room to allocate all drivers */
 #define NUM_MAX_DRIVERS 256
@@ -215,37 +216,88 @@ static int test_pci_sysfs(void)
 	return 0;
 }
 
-/* real drivers (not used for testing) */
+/* real devices & drivers (not used for testing) */
 struct pci_driver_list real_pci_driver_list =
 	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
+struct pci_device_list real_pci_device_list =
+	TAILQ_HEAD_INITIALIZER(real_pci_device_list);
+
+REGISTER_LINKED_RESOURCE(test_pci_sysfs);
 
 static int
 test_pci_setup(void)
 {
+	struct rte_pci_device *dev;
 	struct rte_pci_driver *dr;
+	const struct resource *r;
+	int ret;
+
+	r = resource_find("test_pci_sysfs");
+	TEST_ASSERT_NOT_NULL(r, "missing resource test_pci_sysfs");
+
+	ret = resource_untar(r);
+	TEST_ASSERT_SUCCESS(ret, "failed to untar %s", r->name);
+
+	ret = setenv("SYSFS_PCI_DEVICES", "test_pci_sysfs/bus/pci/devices", 1);
+	TEST_ASSERT_SUCCESS(ret, "failed to setenv");
 
-	/* Unregister original driver list */
+	/* Unregister original devices & drivers lists */
 	while (!TAILQ_EMPTY(&pci_driver_list)) {
 		dr = TAILQ_FIRST(&pci_driver_list);
 		rte_eal_pci_unregister(dr);
 		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
 	}
 
+	while (!TAILQ_EMPTY(&pci_device_list)) {
+		dev = TAILQ_FIRST(&pci_device_list);
+		TAILQ_REMOVE(&pci_device_list, dev, next);
+		TAILQ_INSERT_TAIL(&real_pci_device_list, dev, next);
+	}
+
+	ret = rte_eal_pci_scan();
+	TEST_ASSERT_SUCCESS(ret, "failed to scan PCI bus");
+	rte_eal_pci_dump(stdout);
+
 	return 0;
 }
 
 static int
 test_pci_cleanup(void)
 {
+	struct rte_pci_device *dev;
 	struct rte_pci_driver *dr;
+	const struct resource *r;
+	int ret;
+
+	unsetenv("SYSFS_PCI_DEVICES");
+
+	r = resource_find("test_pci_sysfs");
+	TEST_ASSERT_NOT_NULL(r, "missing resource test_pci_sysfs");
+
+	ret = resource_rm_by_tar(r);
+	TEST_ASSERT_SUCCESS(ret, "Failed to delete resource %s", r->name);
 
-	/* Restore original driver list */
+	/* FIXME: there is no API in DPDK to free a rte_pci_device so we
+	   cannot free the devices in the right way. Let's assume that we
+	   don't care for tests. */
+	while (!TAILQ_EMPTY(&pci_device_list)) {
+		dev = TAILQ_FIRST(&pci_device_list);
+		TAILQ_REMOVE(&pci_device_list, dev, next);
+	}
+
+	/* Restore original devices & drivers lists */
 	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
 		dr = TAILQ_FIRST(&real_pci_driver_list);
 		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
 		rte_eal_pci_register(dr);
 	}
 
+	while (!TAILQ_EMPTY(&real_pci_device_list)) {
+		dev = TAILQ_FIRST(&real_pci_device_list);
+		TAILQ_REMOVE(&real_pci_device_list, dev, next);
+		TAILQ_INSERT_TAIL(&pci_device_list, dev, next);
+	}
+
 	return 0;
 }
 
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class
new file mode 100644
index 0000000..2f9c1da
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class
@@ -0,0 +1 @@
+0x020000
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/config b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/config
new file mode 100644
index 0000000000000000000000000000000000000000..7752421cf13a5aa00a28eaee02cac2add9ce5566
GIT binary patch
literal 64
zcmZo`_$|QBBEZ1Nz`(@7(7?dMz;S^A2oxWHNCpNT2LUi2#BOU~22l(SV3L7>8>k5Y
DD^Ld(

literal 0
HcmV?d00001

diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits
new file mode 100644
index 0000000..900731f
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits
@@ -0,0 +1 @@
+64
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device
new file mode 100644
index 0000000..9e4789e
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device
@@ -0,0 +1 @@
+0x10fb
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits
new file mode 100644
index 0000000..900731f
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits
@@ -0,0 +1 @@
+64
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable
@@ -0,0 +1 @@
+1
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq
@@ -0,0 +1 @@
+0
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias
new file mode 100644
index 0000000..f4c76ed
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias
@@ -0,0 +1 @@
+pci:v00008086d000010FBsv00008086sd00000003bc02sc00i00
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus
@@ -0,0 +1 @@
+1
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node
new file mode 100644
index 0000000..3a2e3f4
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node
@@ -0,0 +1 @@
+-1
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource
new file mode 100644
index 0000000..f388929
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource
@@ -0,0 +1,13 @@
+0x00000000d0080000 0x00000000d00fffff 0x000000000014220c
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x000000000000e020 0x000000000000e03f 0x0000000000040101
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x00000000d0104000 0x00000000d0107fff 0x000000000014220c
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x00000000ab000000 0x00000000ab0fffff 0x0000000000140204
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x00000000ab100000 0x00000000ab1fffff 0x0000000000140204
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x0000000000000000 0x0000000000000000 0x0000000000000000
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs
@@ -0,0 +1 @@
+0
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs
new file mode 100644
index 0000000..4b9026d
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs
@@ -0,0 +1 @@
+63
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device
new file mode 100644
index 0000000..89a932c
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device
@@ -0,0 +1 @@
+0x0003
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor
new file mode 100644
index 0000000..ce6dc4d
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor
@@ -0,0 +1 @@
+0x8086
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent
new file mode 100644
index 0000000..1dbe34d
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent
@@ -0,0 +1,6 @@
+DRIVER=ixgbe
+PCI_CLASS=20000
+PCI_ID=8086:10FB
+PCI_SUBSYS_ID=8086:0003
+PCI_SLOT_NAME=0000:01:00.0
+MODALIAS=pci:v00008086d000010FBsv00008086sd00000003bc02sc00i00
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor
new file mode 100644
index 0000000..ce6dc4d
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor
@@ -0,0 +1 @@
+0x8086
-- 
2.8.0

[PATCH v1 10/10] app/test: do not dump PCI devices in blacklist test

[Jan Viktorin]

Dumping of devices in a unittest is useless. Instead, test whether the test
has been set up well - i.e. there are no devices.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/test_pci.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 3c6c955..942fa09 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -150,8 +150,8 @@ test_pci_blacklist(void)
 {
 	struct rte_devargs_list save_devargs_list;
 
-	printf("Dump all devices\n");
-	rte_eal_pci_dump(stdout);
+	TEST_ASSERT(TAILQ_EMPTY(&pci_driver_list),
+			"pci_driver_list not empty");
 
 	rte_eal_pci_register(&my_driver);
 	rte_eal_pci_register(&my_driver2);
-- 
2.8.0

[PATCH v2 00/11] Include resources in tests

[Jan Viktorin]

Hello,

another round. The code now looks better. Brief introduction:

This patch set introduces a mechanism to include a resource (in general a blob)
into the test binary. This allows to make tests less dependent on the target
testing environment. The first use case is testing of PCI bus scan by changing
the hard-coded path (/sys/bus/pci/devices) to something different and provide
a fake tree of devices with the test.

(CC: B. Richardson (accidently avoided in v1).)
---
v1:
* included 5 patches improving the PCI tests
* fixed using of non-existing RTE_INIT macro

v2:
* Makefile macro resource renamed to linked_resource
* introduced macro linked_tar_resource
* added more comments
* clarified relation between REGISTER_LINKED_RESOURCE and the Makefile
* untar is checked to not loop infinitely
* improved commit messages
* objcopy params extracted to a separated patcha (missing tile and ppc)
* few random bits (usually suggested by T. Monjalon)
* included a note about the new dependency libarchive in the particular commit

Jan Viktorin (11):
  app/test: introduce resources for tests
  mk: define objcopy-specific target and arch
  app/test: support resources externally linked
  app/test: add functions to create files from resources
  app/test: support resources archived by tar
  app/test: use linked list to store PCI drivers
  app/test: extract test_pci_setup and test_pci_cleanup
  app/test: convert current pci_test into a single test case
  eal/pci: replace SYSFS_PCI_DEVICES with pci_get_sysfs_path()
  app/test: scan PCI bus using a fake sysfs
  app/test: do not dump PCI devices in blacklist test

 app/test/Makefile                                  |  31 +++
 app/test/resource.c                                | 296 +++++++++++++++++++++
 app/test/resource.h                                | 135 ++++++++++
 app/test/test_pci.c                                | 148 +++++++++--
 .../bus/pci/devices/0000:01:00.0/class             |   1 +
 .../bus/pci/devices/0000:01:00.0/config            | Bin 0 -> 64 bytes
 .../devices/0000:01:00.0/consistent_dma_mask_bits  |   1 +
 .../bus/pci/devices/0000:01:00.0/device            |   1 +
 .../bus/pci/devices/0000:01:00.0/dma_mask_bits     |   1 +
 .../bus/pci/devices/0000:01:00.0/enable            |   1 +
 .../bus/pci/devices/0000:01:00.0/irq               |   1 +
 .../bus/pci/devices/0000:01:00.0/modalias          |   1 +
 .../bus/pci/devices/0000:01:00.0/msi_bus           |   1 +
 .../bus/pci/devices/0000:01:00.0/numa_node         |   1 +
 .../bus/pci/devices/0000:01:00.0/resource          |  13 +
 .../bus/pci/devices/0000:01:00.0/sriov_numvfs      |   1 +
 .../bus/pci/devices/0000:01:00.0/sriov_totalvfs    |   1 +
 .../bus/pci/devices/0000:01:00.0/subsystem_device  |   1 +
 .../bus/pci/devices/0000:01:00.0/subsystem_vendor  |   1 +
 .../bus/pci/devices/0000:01:00.0/uevent            |   6 +
 .../bus/pci/devices/0000:01:00.0/vendor            |   1 +
 app/test/test_resource.c                           | 132 +++++++++
 drivers/net/szedata2/rte_eth_szedata2.c            |   2 +-
 drivers/net/virtio/virtio_pci.c                    |   2 +-
 lib/librte_eal/bsdapp/eal/rte_eal_version.map      |   6 +
 lib/librte_eal/common/eal_common_pci.c             |  13 +
 lib/librte_eal/common/include/rte_pci.h            |   2 +-
 lib/librte_eal/linuxapp/eal/eal_pci.c              |   6 +-
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c          |   7 +-
 lib/librte_eal/linuxapp/eal/eal_pci_vfio.c         |   2 +-
 lib/librte_eal/linuxapp/eal/rte_eal_version.map    |   6 +
 mk/arch/arm/rte.vars.mk                            |   5 +
 mk/arch/arm64/rte.vars.mk                          |   5 +
 mk/arch/i686/rte.vars.mk                           |   5 +
 mk/arch/x86_64/rte.vars.mk                         |   5 +
 mk/arch/x86_x32/rte.vars.mk                        |   5 +
 36 files changed, 820 insertions(+), 26 deletions(-)
 create mode 100644 app/test/resource.c
 create mode 100644 app/test/resource.h
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/config
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor
 create mode 100644 app/test/test_resource.c

-- 
2.8.0

[PATCH v2 01/11] app/test: introduce resources for tests

[Jan Viktorin]

Certain internal mechanisms of DPDK access different file system structures
(e.g. /sys/bus/pci/devices). It is difficult to test those cases automatically
by a unit test when such path is not hard-coded and there is no simple way how
to distribute fake ones with the current testing environment.

This patch adds a possibility to declare a resource embedded in the test binary
itself. The structure resource cover the generic situation - it provides a name
for lookup and pointers to the embedded data blob. A resource is registered
in a constructor by the macro REGISTER_RESOURCE.

Some initial tests of simple resources is included.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v2:
* added comments
* resource_size is not inline anymore
* REGISTER_RESOURCE is a single macro now
* fixed double constructor declaration
* __resource_register renamed to resource_register
* less number of underscores...
---
 app/test/Makefile        |  2 +
 app/test/resource.c      | 66 ++++++++++++++++++++++++++++++++
 app/test/resource.h      | 98 ++++++++++++++++++++++++++++++++++++++++++++++++
 app/test/test_resource.c | 75 ++++++++++++++++++++++++++++++++++++
 4 files changed, 241 insertions(+)
 create mode 100644 app/test/resource.c
 create mode 100644 app/test/resource.h
 create mode 100644 app/test/test_resource.c

diff --git a/app/test/Makefile b/app/test/Makefile
index a4907d5..7fbdd18 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -43,6 +43,8 @@ APP = test
 #
 SRCS-$(CONFIG_RTE_LIBRTE_CMDLINE) := commands.c
 SRCS-y += test.c
+SRCS-y += resource.c
+SRCS-y += test_resource.c
 SRCS-y += test_pci.c
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
diff --git a/app/test/resource.c b/app/test/resource.c
new file mode 100644
index 0000000..30513db
--- /dev/null
+++ b/app/test/resource.c
@@ -0,0 +1,66 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2016 RehiveTech. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of RehiveTech nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <string.h>
+#include <errno.h>
+#include <sys/queue.h>
+
+#include <rte_debug.h>
+
+#include "resource.h"
+
+struct resource_list resource_list = TAILQ_HEAD_INITIALIZER(resource_list);
+
+size_t resource_size(const struct resource *r)
+{
+	return r->end - r->begin;
+}
+
+const struct resource *resource_find(const char *name)
+{
+	struct resource *r;
+
+	TAILQ_FOREACH(r, &resource_list, next) {
+		RTE_VERIFY(r->name);
+
+		if (!strcmp(r->name, name))
+			return r;
+	}
+
+	return NULL;
+}
+
+void resource_register(struct resource *r)
+{
+	TAILQ_INSERT_TAIL(&resource_list, r, next);
+}
diff --git a/app/test/resource.h b/app/test/resource.h
new file mode 100644
index 0000000..ae4f547
--- /dev/null
+++ b/app/test/resource.h
@@ -0,0 +1,98 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2016 RehiveTech. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of RehiveTech nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RESOURCE_H_
+#define _RESOURCE_H_
+
+/**
+ * @file
+ *
+ * Test Resource API
+ *
+ * Each test can require and use some external resources. Usually, an external
+ * resource is a file or a filesystem sub-hierarchy. A resource is included
+ * inside the test executable.
+ */
+
+#include <sys/queue.h>
+#include <stddef.h>
+
+#include <rte_eal.h>
+#include <rte_common.h>
+
+TAILQ_HEAD(resource_list, resource);
+extern struct resource_list resource_list;
+
+/**
+ * Representation of a resource. It points to the resource's binary data.
+ * The semantics of the binary data are defined by the target test.
+ */
+struct resource {
+	const char *name;  /** Unique name of the resource */
+	const char *begin; /** Start of resource data */
+	const char *end;   /** End of resource data */
+	TAILQ_ENTRY(resource) next;
+};
+
+/**
+ * @return size of the given resource
+ */
+size_t resource_size(const struct resource *r);
+
+/**
+ * Find a resource by name in the global list of resources.
+ */
+const struct resource *resource_find(const char *name);
+
+/**
+ * Register a resource in the global list of resources.
+ * Not intended for direct use, please check the REGISTER_RESOURCE
+ * macro.
+ */
+void resource_register(struct resource *r);
+
+/**
+ * Definition of a resource described by its name, and pointers begin, end.
+ */
+#define REGISTER_RESOURCE(n, b, e) \
+static struct resource linkres_ ##n = {       \
+	.name = RTE_STR(n),     \
+	.begin = b,             \
+	.end = e,               \
+};                              \
+static void __attribute__((constructor,used)) resinitfn_ ##n(void) \
+{                               \
+	resource_register(&linkres_ ##n);  \
+}
+
+#endif
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
new file mode 100644
index 0000000..69391ad
--- /dev/null
+++ b/app/test/test_resource.c
@@ -0,0 +1,75 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2016 RehiveTech. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of RehiveTech nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <stdio.h>
+#include <string.h>
+
+#include "test.h"
+#include "resource.h"
+
+const char test_resource_dpdk_blob[] = {
+	'\x44', '\x50', '\x44', '\x4b', '\x00'
+};
+
+REGISTER_RESOURCE(test_resource_dpdk,
+		test_resource_dpdk_blob, test_resource_dpdk_blob + 4);
+
+static int test_resource_dpdk(void)
+{
+	const struct resource *r;
+
+	r = resource_find("test_resource_dpdk");
+	TEST_ASSERT_NOT_NULL(r, "Could not find test_resource_dpdk");
+	TEST_ASSERT(!strcmp(r->name, "test_resource_dpdk"),
+			"Found resource %s, expected test_resource_dpdk",
+			r->name);
+
+	TEST_ASSERT(!strncmp("DPDK", r->begin, 4),
+			"Unexpected payload: %.4s...", r->begin);
+
+	return 0;
+}
+
+static int test_resource(void)
+{
+	if (test_resource_dpdk())
+		return -1;
+
+	return 0;
+}
+
+static struct test_command resource_cmd = {
+	.command = "resource_autotest",
+	.callback = test_resource,
+};
+REGISTER_TEST_COMMAND(resource_cmd);
-- 
2.8.0

Re: [PATCH v2 01/11] app/test: introduce resources for tests

[Thomas Monjalon]

2016-05-10 20:13, Jan Viktorin:
> +struct resource {
> +	const char *name;  /** Unique name of the resource */
> +	const char *begin; /** Start of resource data */
> +	const char *end;   /** End of resource data */
> +	TAILQ_ENTRY(resource) next;
> +};

There is no doxygen generated from this file, but you can keep
this format in case we decide to generate one.
Here the comments after the fields should start with /**<

Re: [PATCH v2 01/11] app/test: introduce resources for tests

[Thomas Monjalon]

2016-05-10 20:13, Jan Viktorin:
> +REGISTER_TEST_COMMAND(resource_cmd);

Should you add this test in group 1 of autotest_data.py?

Re: [PATCH v2 01/11] app/test: introduce resources for tests

[Jan Viktorin]

On Thu, 12 May 2016 17:19:21 +0200
Thomas Monjalon <thomas.monjalon@6wind.com> wrote:

> 2016-05-10 20:13, Jan Viktorin:
> > +REGISTER_TEST_COMMAND(resource_cmd);  
> 
> Should you add this test in group 1 of autotest_data.py?

Will do for v3. This way:

...                                                              
 84                 {                                                               
 85                  "Name" :       "Common autotest",                              
 86                  "Command" :    "common_autotest",                              
 87                  "Func" :       default_autotest,                               
 88                  "Report" :     None,                                           
 89                 },                                                              
 90                 {                                                               
 91                  "Name" :       "Resource autotest",                            
 92                  "Command" :    "resource_autotest",                            
 93                  "Func" :       default_autotest,                               
 94                  "Report" :     None,                                           
 95                 },                                                              
 96                 {                                                               
 97                  "Name" :       "Dump log history",                             
 98                  "Command" :    "dump_log_history",                             
 99                  "Func" :       dump_autotest,                                  
100                  "Report" :     None,                                           
101                 },
...

-- 
   Jan Viktorin                  E-mail: Viktorin@RehiveTech.com
   System Architect              Web:    www.RehiveTech.com
   RehiveTech
   Brno, Czech Republic

[PATCH v2 02/11] mk: define objcopy-specific target and arch

[Jan Viktorin]

The program objcopy uses non-standard conventions to name the target and arch.
Define the values for supported architectures.

FIXME: tile and ppc_64 are not present.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 mk/arch/arm/rte.vars.mk     | 5 +++++
 mk/arch/arm64/rte.vars.mk   | 5 +++++
 mk/arch/i686/rte.vars.mk    | 5 +++++
 mk/arch/x86_64/rte.vars.mk  | 5 +++++
 mk/arch/x86_x32/rte.vars.mk | 5 +++++
 5 files changed, 25 insertions(+)

diff --git a/mk/arch/arm/rte.vars.mk b/mk/arch/arm/rte.vars.mk
index bd85140..2f8cf7c 100644
--- a/mk/arch/arm/rte.vars.mk
+++ b/mk/arch/arm/rte.vars.mk
@@ -37,3 +37,8 @@ CPU_LDFLAGS ?=
 CPU_ASFLAGS ?= -felf
 
 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf32-littlearm
+RTE_OBJCOPY_ARCH = arm
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
diff --git a/mk/arch/arm64/rte.vars.mk b/mk/arch/arm64/rte.vars.mk
index 32e3a5f..c168426 100644
--- a/mk/arch/arm64/rte.vars.mk
+++ b/mk/arch/arm64/rte.vars.mk
@@ -56,3 +56,8 @@ CPU_LDFLAGS ?=
 CPU_ASFLAGS ?= -felf
 
 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf64-littleaarch64
+RTE_OBJCOPY_ARCH = aarch64
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
diff --git a/mk/arch/i686/rte.vars.mk b/mk/arch/i686/rte.vars.mk
index 8ba9a23..6a25312 100644
--- a/mk/arch/i686/rte.vars.mk
+++ b/mk/arch/i686/rte.vars.mk
@@ -57,3 +57,8 @@ CPU_LDFLAGS ?= -melf_i386
 CPU_ASFLAGS ?= -felf
 
 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf32-i386
+RTE_OBJCOPY_ARCH = i386
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
diff --git a/mk/arch/x86_64/rte.vars.mk b/mk/arch/x86_64/rte.vars.mk
index b986f04..83723c8 100644
--- a/mk/arch/x86_64/rte.vars.mk
+++ b/mk/arch/x86_64/rte.vars.mk
@@ -57,3 +57,8 @@ CPU_LDFLAGS ?=
 CPU_ASFLAGS ?= -felf64
 
 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf64-x86-64
+RTE_OBJCOPY_ARCH = i386:x86-64
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
diff --git a/mk/arch/x86_x32/rte.vars.mk b/mk/arch/x86_x32/rte.vars.mk
index 3103dfc..676f316 100644
--- a/mk/arch/x86_x32/rte.vars.mk
+++ b/mk/arch/x86_x32/rte.vars.mk
@@ -61,3 +61,8 @@ ifneq ($(shell echo | $(CC) $(CPU_CFLAGS) -E - 2>/dev/null 1>/dev/null && echo 0
 endif
 
 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf32-x86-64
+RTE_OBJCOPY_ARCH = i386:x86-64
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
-- 
2.8.0

[PATCH v2 03/11] app/test: support resources externally linked

[Jan Viktorin]

To include resources from other source that the C source code we can take
advantage of the objcopy behaviour, i.e. packing of an arbitrary file as an
object file that is linked to the target program.

A linked object file is always accessible as a pair

extern const char beg_<name>;
extern const char end_<name>;
(extern const char siz_<name>;)

A unit test that packs the resource.c source file is included.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v2:
* macro resource renamed to linked_resource and documented
* the linking principle is better explained now
---
 app/test/Makefile        | 19 +++++++++++++++++++
 app/test/resource.h      | 10 ++++++++++
 app/test/test_resource.c | 18 ++++++++++++++++++
 3 files changed, 47 insertions(+)

diff --git a/app/test/Makefile b/app/test/Makefile
index 7fbdd18..cfe68a6 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -33,6 +33,24 @@ include $(RTE_SDK)/mk/rte.vars.mk
 
 ifeq ($(CONFIG_RTE_APP_TEST),y)
 
+# Define an externally linked resource. A linked resource is an arbitrary
+# file that is linked into the test binary. The application refers to this
+# resource by name. The linked generates identifiers beg_<name> and end_<name>
+# for referencing by the C code.
+#
+# Parameters: <unique name>, <file to be linked>
+define linked_resource
+SRCS-y += $(1).res.o
+$(1).res.o: $(2)
+	$(OBJCOPY) -I binary -B $(RTE_OBJCOPY_ARCH) -O $(RTE_OBJCOPY_TARGET) \
+		--rename-section                                         \
+			.data=.rodata,alloc,load,data,contents,readonly  \
+		--redefine-sym _binary__dev_stdin_start=beg_$(1)         \
+		--redefine-sym _binary__dev_stdin_end=end_$(1)           \
+		--redefine-sym _binary__dev_stdin_size=siz_$(1)          \
+		/dev/stdin $$@ < $$<
+endef
+
 #
 # library name
 #
@@ -45,6 +63,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_CMDLINE) := commands.c
 SRCS-y += test.c
 SRCS-y += resource.c
 SRCS-y += test_resource.c
+$(eval $(call linked_resource,test_resource_c,resource.c))
 SRCS-y += test_pci.c
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
diff --git a/app/test/resource.h b/app/test/resource.h
index ae4f547..c44aae9 100644
--- a/app/test/resource.h
+++ b/app/test/resource.h
@@ -82,6 +82,16 @@ const struct resource *resource_find(const char *name);
 void resource_register(struct resource *r);
 
 /**
+ * Definition of a resource linked externally (by means of the used toolchain).
+ * Only the base name of the resource is expected. The name refers to the
+ * linked pointers beg_<name> and end_<name> provided externally.
+ */
+#define REGISTER_LINKED_RESOURCE(n) \
+extern const char beg_ ##n;         \
+extern const char end_ ##n;         \
+REGISTER_RESOURCE(n, &beg_ ##n, &end_ ##n); \
+
+/**
  * Definition of a resource described by its name, and pointers begin, end.
  */
 #define REGISTER_RESOURCE(n, b, e) \
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
index 69391ad..b397fa8 100644
--- a/app/test/test_resource.c
+++ b/app/test/test_resource.c
@@ -60,11 +60,29 @@ static int test_resource_dpdk(void)
 	return 0;
 }
 
+REGISTER_LINKED_RESOURCE(test_resource_c);
+
+static int test_resource_c(void)
+{
+	const struct resource *r;
+
+	r = resource_find("test_resource_c");
+	TEST_ASSERT_NOT_NULL(r, "No test_resource_c found");
+	TEST_ASSERT(!strcmp(r->name, "test_resource_c"),
+			"Found resource %s, expected test_resource_c",
+			r->name);
+
+	return 0;
+}
+
 static int test_resource(void)
 {
 	if (test_resource_dpdk())
 		return -1;
 
+	if (test_resource_c())
+		return -1;
+
 	return 0;
 }
 
-- 
2.8.0

[PATCH v2 04/11] app/test: add functions to create files from resources

[Jan Viktorin]

A resource can be written into the target filesystem by calling resource_fwrite
or resource_fwrite_file. Such file can be created before a test is started and
removed after the test finishes.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/resource.c      | 35 +++++++++++++++++++++++++++++++++++
 app/test/resource.h      | 14 ++++++++++++++
 app/test/test_resource.c | 10 ++++++++++
 3 files changed, 59 insertions(+)

diff --git a/app/test/resource.c b/app/test/resource.c
index 30513db..acb63c1 100644
--- a/app/test/resource.c
+++ b/app/test/resource.c
@@ -31,6 +31,7 @@
  *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+#include <stdio.h>
 #include <string.h>
 #include <errno.h>
 #include <sys/queue.h>
@@ -60,6 +61,40 @@ const struct resource *resource_find(const char *name)
 	return NULL;
 }
 
+int resource_fwrite(const struct resource *r, FILE *f)
+{
+	const size_t goal = resource_size(r);
+	size_t total = 0;
+
+	while (total < goal) {
+		size_t wlen = fwrite(r->begin + total, 1, goal - total, f);
+		if (wlen == 0) {
+			perror(__func__);
+			return -1;
+		}
+
+		total += wlen;
+	}
+
+	return 0;
+}
+
+int resource_fwrite_file(const struct resource *r, const char *fname)
+{
+	FILE *f;
+	int ret;
+
+	f = fopen(fname, "w");
+	if (f == NULL) {
+		perror(__func__);
+		return -1;
+	}
+
+	ret = resource_fwrite(r, f);
+	fclose(f);
+	return ret;
+}
+
 void resource_register(struct resource *r)
 {
 	TAILQ_INSERT_TAIL(&resource_list, r, next);
diff --git a/app/test/resource.h b/app/test/resource.h
index c44aae9..3253d86 100644
--- a/app/test/resource.h
+++ b/app/test/resource.h
@@ -45,6 +45,7 @@
  */
 
 #include <sys/queue.h>
+#include <stdio.h>
 #include <stddef.h>
 
 #include <rte_eal.h>
@@ -75,6 +76,19 @@ size_t resource_size(const struct resource *r);
 const struct resource *resource_find(const char *name);
 
 /**
+ * Write the raw data of the resource to the given file.
+ * @return 0 on success
+ */
+int resource_fwrite(const struct resource *r, FILE *f);
+
+/**
+ * Write the raw data of the resource to the given file given by name.
+ * The name is relative to the current working directory.
+ * @return 0 on success
+ */
+int resource_fwrite_file(const struct resource *r, const char *fname);
+
+/**
  * Register a resource in the global list of resources.
  * Not intended for direct use, please check the REGISTER_RESOURCE
  * macro.
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
index b397fa8..3d1bf00 100644
--- a/app/test/test_resource.c
+++ b/app/test/test_resource.c
@@ -65,6 +65,7 @@ REGISTER_LINKED_RESOURCE(test_resource_c);
 static int test_resource_c(void)
 {
 	const struct resource *r;
+	FILE *f;
 
 	r = resource_find("test_resource_c");
 	TEST_ASSERT_NOT_NULL(r, "No test_resource_c found");
@@ -72,6 +73,15 @@ static int test_resource_c(void)
 			"Found resource %s, expected test_resource_c",
 			r->name);
 
+	TEST_ASSERT_SUCCESS(resource_fwrite_file(r, "test_resource.c"),
+			"Failed to to write file %s", r->name);
+
+	f = fopen("test_resource.c", "r");
+	TEST_ASSERT_NOT_NULL(f,
+			"Missing extracted file resource.c");
+	fclose(f);
+	remove("test_resource.c");
+
 	return 0;
 }
 
-- 
2.8.0

[PATCH v2 05/11] app/test: support resources archived by tar

[Jan Viktorin]

When a more complex resource (a file hierarchy) is needed, packing every single
file as a single resource would be very ineffective. For that purpose, it is
possible to pack the files into a tar archive, extract it before test from the
resource and finally clean up all the created files.

This patch introduces functions resource_untar and resource_rm_by_tar to
perform those tasks. An example of using those functions is included as a test.

A new dependency is required to build the app/test: libarchive.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v2:
* mentioned the libarchive dependency in commit log
* resource_rm_by_tar is prevented to loop infinitely
* introduced Makefile macro linked_tar_resource
---
 app/test/Makefile        |   9 +++
 app/test/resource.c      | 195 +++++++++++++++++++++++++++++++++++++++++++++++
 app/test/resource.h      |  13 ++++
 app/test/test_resource.c |  29 +++++++
 4 files changed, 246 insertions(+)

diff --git a/app/test/Makefile b/app/test/Makefile
index cfe68a6..de5fa50 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -51,6 +51,13 @@ $(1).res.o: $(2)
 		/dev/stdin $$@ < $$<
 endef
 
+define linked_tar_resource
+$(1).tar: $(2)
+	tar -C $$(dir $$<) -cf $$@ $$(notdir $$<)
+
+$(call linked_resource,$(1),$(1).tar)
+endef
+
 #
 # library name
 #
@@ -64,6 +71,7 @@ SRCS-y += test.c
 SRCS-y += resource.c
 SRCS-y += test_resource.c
 $(eval $(call linked_resource,test_resource_c,resource.c))
+$(eval $(call linked_tar_resource,test_resource_tar,test_resource.c))
 SRCS-y += test_pci.c
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
@@ -183,6 +191,7 @@ CFLAGS += $(WERROR_FLAGS)
 CFLAGS += -D_GNU_SOURCE
 
 LDLIBS += -lm
+LDLIBS += -larchive
 
 # Disable VTA for memcpy test
 ifeq ($(CC), gcc)
diff --git a/app/test/resource.c b/app/test/resource.c
index acb63c1..668fc52 100644
--- a/app/test/resource.c
+++ b/app/test/resource.c
@@ -33,6 +33,8 @@
 
 #include <stdio.h>
 #include <string.h>
+#include <archive.h>
+#include <archive_entry.h>
 #include <errno.h>
 #include <sys/queue.h>
 
@@ -95,6 +97,199 @@ int resource_fwrite_file(const struct resource *r, const char *fname)
 	return ret;
 }
 
+static int do_copy(struct archive *r, struct archive *w)
+{
+	const void *buf;
+	size_t len;
+	off_t off;
+	int ret;
+
+	while (1) {
+		ret = archive_read_data_block(r, &buf, &len, &off);
+		if (ret == ARCHIVE_RETRY)
+			continue;
+
+		if (ret == ARCHIVE_EOF)
+			return 0;
+
+		if (ret != ARCHIVE_OK)
+			return ret;
+
+		do {
+			ret = archive_write_data_block(w, buf, len, off);
+			if (ret != ARCHIVE_OK && ret != ARCHIVE_RETRY)
+				return ret;
+		} while (ret != ARCHIVE_OK);
+	}
+}
+
+int resource_untar(const struct resource *res)
+{
+	struct archive *r;
+	struct archive *w;
+	struct archive_entry *e;
+	void *p;
+	int flags = 0;
+	int ret;
+
+	p = malloc(resource_size(res));
+	if (p == NULL)
+		rte_panic("Failed to malloc %zu B\n", resource_size(res));
+
+	memcpy(p, res->begin, resource_size(res));
+
+	r = archive_read_new();
+	if (r == NULL) {
+		free(p);
+		return -1;
+	}
+
+	archive_read_support_format_all(r);
+	archive_read_support_filter_all(r);
+
+	w = archive_write_disk_new();
+	if (w == NULL) {
+		archive_read_free(r);
+		free(p);
+		return -1;
+	}
+
+	flags |= ARCHIVE_EXTRACT_PERM;
+	flags |= ARCHIVE_EXTRACT_FFLAGS;
+	archive_write_disk_set_options(w, flags);
+	archive_write_disk_set_standard_lookup(w);
+
+	ret = archive_read_open_memory(r, p, resource_size(res));
+	if (ret != ARCHIVE_OK)
+		goto fail;
+
+	while (1) {
+		ret = archive_read_next_header(r, &e);
+		if (ret == ARCHIVE_EOF)
+			break;
+		if (ret != ARCHIVE_OK)
+			goto fail;
+
+		ret = archive_write_header(w, e);
+		if (ret == ARCHIVE_EOF)
+			break;
+		if (ret != ARCHIVE_OK)
+			goto fail;
+
+		if (archive_entry_size(e) == 0)
+			continue;
+
+		ret = do_copy(r, w);
+		if (ret != ARCHIVE_OK)
+			goto fail;
+
+		ret = archive_write_finish_entry(w);
+		if (ret != ARCHIVE_OK)
+			goto fail;
+	}
+
+	archive_write_free(w);
+	archive_read_free(r);
+	free(p);
+	return 0;
+
+fail:
+	archive_write_free(w);
+	archive_read_free(r);
+	free(p);
+	rte_panic("Failed: %s\n", archive_error_string(r));
+	return -1;
+}
+
+int resource_rm_by_tar(const struct resource *res)
+{
+	struct archive *r;
+	struct archive_entry *e;
+	void *p;
+	int try_again = 1;
+	int attempts = 0;
+	int ret;
+
+	p = malloc(resource_size(res));
+	if (p == NULL)
+		rte_panic("Failed to malloc %zu B\n", resource_size(res));
+
+	memcpy(p, res->begin, resource_size(res));
+
+	/* If somebody creates a file somewhere inside the extracted TAR
+	   hierarchy during a test the resource_rm_by_tar might loop
+	   infinitely. We prevent this by adding the attempts counter there.
+	   In normal case, max N iteration is done where N is the depth of
+	   the file-hierarchy.
+	 */
+	while (try_again && attempts < 10000) {
+		r = archive_read_new();
+		if (r == NULL) {
+			free(p);
+			return -1;
+		}
+
+		archive_read_support_format_all(r);
+		archive_read_support_filter_all(r);
+
+		ret = archive_read_open_memory(r, p, resource_size(res));
+		if (ret != ARCHIVE_OK) {
+			fprintf(stderr, "Failed: %s\n",
+					archive_error_string(r));
+			goto fail;
+		}
+
+		try_again = 0;
+
+		while (1) {
+			ret = archive_read_next_header(r, &e);
+			if (ret == ARCHIVE_EOF)
+				break;
+			if (ret != ARCHIVE_OK)
+				goto fail;
+
+			ret = remove(archive_entry_pathname(e));
+			if (ret < 0) {
+				switch (errno) {
+				case ENOTEMPTY:
+				case EEXIST:
+					try_again = 1;
+					break;
+
+				/* should not usually happen: */
+				case ENOENT:
+				case ENOTDIR:
+				case EROFS:
+					attempts += 1;
+					continue;
+				default:
+					perror("Failed to remove file");
+					goto fail;
+				}
+			}
+		}
+
+		archive_read_free(r);
+		attempts += 1;
+	}
+
+	if (attempts >= 10000) {
+		fprintf(stderr, "Failed to remove archive\n");
+		free(p);
+		return -1;
+	}
+
+	free(p);
+	return 0;
+
+fail:
+	archive_read_free(r);
+	free(p);
+
+	rte_panic("Failed: %s\n", archive_error_string(r));
+	return -1;
+}
+
 void resource_register(struct resource *r)
 {
 	TAILQ_INSERT_TAIL(&resource_list, r, next);
diff --git a/app/test/resource.h b/app/test/resource.h
index 3253d86..0f41814 100644
--- a/app/test/resource.h
+++ b/app/test/resource.h
@@ -89,6 +89,19 @@ int resource_fwrite(const struct resource *r, FILE *f);
 int resource_fwrite_file(const struct resource *r, const char *fname);
 
 /**
+ * Treat the given resource as a tar archive. Extract
+ * the archive to the current directory.
+ */
+int resource_untar(const struct resource *res);
+
+/**
+ * Treat the given resource as a tar archive. Remove
+ * all files (related to the current directory) listed
+ * in the tar archive.
+ */
+int resource_rm_by_tar(const struct resource *res);
+
+/**
  * Register a resource in the global list of resources.
  * Not intended for direct use, please check the REGISTER_RESOURCE
  * macro.
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
index 3d1bf00..1e85040 100644
--- a/app/test/test_resource.c
+++ b/app/test/test_resource.c
@@ -85,6 +85,32 @@ static int test_resource_c(void)
 	return 0;
 }
 
+REGISTER_LINKED_RESOURCE(test_resource_tar);
+
+static int test_resource_tar(void)
+{
+	const struct resource *r;
+	FILE *f;
+
+	r = resource_find("test_resource_tar");
+	TEST_ASSERT_NOT_NULL(r, "No test_resource_tar found");
+	TEST_ASSERT(!strcmp(r->name, "test_resource_tar"),
+			"Found resource %s, expected test_resource_tar",
+			r->name);
+
+	TEST_ASSERT_SUCCESS(resource_untar(r),
+			"Failed to to untar %s", r->name);
+
+	f = fopen("test_resource.c", "r");
+	TEST_ASSERT_NOT_NULL(f,
+			"Missing extracted file test_resource.c");
+	fclose(f);
+
+	TEST_ASSERT_SUCCESS(resource_rm_by_tar(r),
+			"Failed to remove extracted contents of %s", r->name);
+	return 0;
+}
+
 static int test_resource(void)
 {
 	if (test_resource_dpdk())
@@ -93,6 +119,9 @@ static int test_resource(void)
 	if (test_resource_c())
 		return -1;
 
+	if (test_resource_tar())
+		return -1;
+
 	return 0;
 }
 
-- 
2.8.0

[PATCH v2 06/11] app/test: use linked list to store PCI drivers

[Jan Viktorin]

The test unregisters all real drivers before starting into an array. This
inflexiable as we can use a linked list for this purpose.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/test_pci.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 0ed357e..cf82373 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -144,21 +144,24 @@ static void free_devargs_list(void)
 	}
 }
 
+/* real drivers (not used for testing) */
+struct pci_driver_list real_pci_driver_list =
+	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
+
 int
 test_pci(void)
 {
 	struct rte_devargs_list save_devargs_list;
 	struct rte_pci_driver *dr = NULL;
-	struct rte_pci_driver *save_pci_driver_list[NUM_MAX_DRIVERS];
-	unsigned i, num_drivers = 0;
 
 	printf("Dump all devices\n");
 	rte_eal_pci_dump(stdout);
 
 	/* Unregister all previous drivers */
-	TAILQ_FOREACH(dr, &pci_driver_list, next) {
+	while (!TAILQ_EMPTY(&pci_driver_list)) {
+		dr = TAILQ_FIRST(&pci_driver_list);
 		rte_eal_pci_unregister(dr);
-		save_pci_driver_list[num_drivers++] = dr;
+		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
 	}
 
 	rte_eal_pci_register(&my_driver);
@@ -197,8 +200,11 @@ test_pci(void)
 	rte_eal_pci_unregister(&my_driver2);
 
 	/* Restore original driver list */
-	for (i = 0; i < num_drivers; i++)
-		rte_eal_pci_register(save_pci_driver_list[i]);
+	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
+		dr = TAILQ_FIRST(&real_pci_driver_list);
+		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
+		rte_eal_pci_register(dr);
+	}
 
 	return 0;
 }
-- 
2.8.0

Re: [PATCH v2 06/11] app/test: use linked list to store PCI drivers

[Thomas Monjalon]

2016-05-10 20:13, Jan Viktorin:
> The test unregisters all real drivers before starting into an array. This
> inflexiable as we can use a linked list for this purpose.

I don't understand this. Maybe some words are missing.

> +/* real drivers (not used for testing) */

What do mean by "not used for testing"?

> +struct pci_driver_list real_pci_driver_list =
> +	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);

Re: [PATCH v2 06/11] app/test: use linked list to store PCI drivers

[Jan Viktorin]

On Thu, 12 May 2016 17:31:28 +0200
Thomas Monjalon <thomas.monjalon@6wind.com> wrote:

> 2016-05-10 20:13, Jan Viktorin:
> > The test unregisters all real drivers before starting into an array. This
> > inflexiable as we can use a linked list for this purpose.  
> 
> I don't understand this. Maybe some words are missing.

Better?

The test unregisters all real drivers before starting (stored into an array).
This is inflexiable (due to its fixed size) and we can use a linked list for
this purpose.  

> 
> > +/* real drivers (not used for testing) */  
> 
> What do mean by "not used for testing"?

The test now avoids the DPDK builtin drivers. It only considers its
internal fake drivers my_driver and my_driver2. So the real drivers
are temporarily store into the real_pci_driver_list and returned back
after the test finishes.

It is the linked list mentioned in the commit log. It replaces the
original fixed-size array.

(For drivers, it does not matter that much. But for devices, I think,
it is not a good practice to consider them in autotests. Every PC
where you execute the tests have different set of PCI devices.)

> 
> > +struct pci_driver_list real_pci_driver_list =
> > +	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);  
> 



-- 
   Jan Viktorin                  E-mail: Viktorin@RehiveTech.com
   System Architect              Web:    www.RehiveTech.com
   RehiveTech
   Brno, Czech Republic

Re: [PATCH v2 06/11] app/test: use linked list to store PCI drivers

[Thomas Monjalon]

2016-05-12 17:53, Jan Viktorin:
> On Thu, 12 May 2016 17:31:28 +0200
> Thomas Monjalon <thomas.monjalon@6wind.com> wrote:
> 
> > 2016-05-10 20:13, Jan Viktorin:
> > > The test unregisters all real drivers before starting into an array. This
> > > inflexiable as we can use a linked list for this purpose.  
> > 
> > I don't understand this. Maybe some words are missing.
> 
> Better?
> 
> The test unregisters all real drivers before starting (stored into an array).
> This is inflexiable (due to its fixed size) and we can use a linked list for
> this purpose.  

Better with a past tense? "The test was unregistering..."

> > > +/* real drivers (not used for testing) */  
> > 
> > What do mean by "not used for testing"?
> 
> The test now avoids the DPDK builtin drivers. It only considers its
> internal fake drivers my_driver and my_driver2. So the real drivers
> are temporarily store into the real_pci_driver_list and returned back
> after the test finishes.

Maybe adding "Save" or "Backup" would make it clear.

> It is the linked list mentioned in the commit log. It replaces the
> original fixed-size array.
> 
> (For drivers, it does not matter that much. But for devices, I think,
> it is not a good practice to consider them in autotests. Every PC
> where you execute the tests have different set of PCI devices.)

Yes

Re: [PATCH v2 06/11] app/test: use linked list to store PCI drivers

[Jan Viktorin]

On Thu, 12 May 2016 18:08:16 +0200
Thomas Monjalon <thomas.monjalon@6wind.com> wrote:

> 2016-05-12 17:53, Jan Viktorin:
> > On Thu, 12 May 2016 17:31:28 +0200
> > Thomas Monjalon <thomas.monjalon@6wind.com> wrote:
> >   
> > > 2016-05-10 20:13, Jan Viktorin:  
> > > > The test unregisters all real drivers before starting into an array. This
> > > > inflexiable as we can use a linked list for this purpose.    
> > > 
> > > I don't understand this. Maybe some words are missing.  
> > 
> > Better?
> > 
> > The test unregisters all real drivers before starting (stored into an array).
> > This is inflexiable (due to its fixed size) and we can use a linked list for
> > this purpose.    
> 
> Better with a past tense? "The test was unregistering..."

No. This patch does not change the semantics (well, it does, because
the list of drivers become global but that's all). The test still
unregisters the drivers. That is the important fact to consider.

Perhaps:

The test unregisters all drivers before start. The drivers were stored
into a fixed-sized array. This is inflexible. This patch change this to
utilize a linked list for the same purpose.

> 
> > > > +/* real drivers (not used for testing) */    
> > > 
> > > What do mean by "not used for testing"?  
> > 
> > The test now avoids the DPDK builtin drivers. It only considers its
> > internal fake drivers my_driver and my_driver2. So the real drivers
> > are temporarily store into the real_pci_driver_list and returned back
> > after the test finishes.  
> 
> Maybe adding "Save" or "Backup" would make it clear.

Ok, makes sense.

> 
> > It is the linked list mentioned in the commit log. It replaces the
> > original fixed-size array.
> > 
> > (For drivers, it does not matter that much. But for devices, I think,
> > it is not a good practice to consider them in autotests. Every PC
> > where you execute the tests have different set of PCI devices.)  
> 
> Yes

Thanks for comments!

Jan

-- 
  Jan Viktorin                E-mail: Viktorin@RehiveTech.com
  System Architect            Web:    www.RehiveTech.com
  RehiveTech
  Brno, Czech Republic

Re: [PATCH v2 06/11] app/test: use linked list to store PCI drivers

[Thomas Monjalon]

2016-05-12 23:00, Jan Viktorin:
> On Thu, 12 May 2016 18:08:16 +0200
> Thomas Monjalon <thomas.monjalon@6wind.com> wrote:
> 
> > 2016-05-12 17:53, Jan Viktorin:
> > > On Thu, 12 May 2016 17:31:28 +0200
> > > Thomas Monjalon <thomas.monjalon@6wind.com> wrote:
> > >   
> > > > 2016-05-10 20:13, Jan Viktorin:  
> > > > > The test unregisters all real drivers before starting into an array. This
> > > > > inflexiable as we can use a linked list for this purpose.    
> > > > 
> > > > I don't understand this. Maybe some words are missing.  
> > > 
> > > Better?
> > > 
> > > The test unregisters all real drivers before starting (stored into an array).
> > > This is inflexiable (due to its fixed size) and we can use a linked list for
> > > this purpose.    
> > 
> > Better with a past tense? "The test was unregistering..."
> 
> No. This patch does not change the semantics (well, it does, because
> the list of drivers become global but that's all). The test still
> unregisters the drivers. That is the important fact to consider.
> 
> Perhaps:
> 
> The test unregisters all drivers before start. The drivers were stored
> into a fixed-sized array. This is inflexible. This patch change this to
> utilize a linked list for the same purpose.

Yes better. Thanks

[PATCH v2 07/11] app/test: extract test_pci_setup and test_pci_cleanup

[Jan Viktorin]

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/test_pci.c | 47 ++++++++++++++++++++++++++++++++++-------------
 1 file changed, 34 insertions(+), 13 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index cf82373..9d53ba5 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -148,21 +148,46 @@ static void free_devargs_list(void)
 struct pci_driver_list real_pci_driver_list =
 	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
 
+static int
+test_pci_setup(void)
+{
+	struct rte_pci_driver *dr;
+
+	/* Unregister original driver list */
+	while (!TAILQ_EMPTY(&pci_driver_list)) {
+		dr = TAILQ_FIRST(&pci_driver_list);
+		rte_eal_pci_unregister(dr);
+		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
+	}
+
+	return 0;
+}
+
+static int
+test_pci_cleanup(void)
+{
+	struct rte_pci_driver *dr;
+
+	/* Restore original driver list */
+	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
+		dr = TAILQ_FIRST(&real_pci_driver_list);
+		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
+		rte_eal_pci_register(dr);
+	}
+
+	return 0;
+}
+
 int
 test_pci(void)
 {
 	struct rte_devargs_list save_devargs_list;
-	struct rte_pci_driver *dr = NULL;
 
 	printf("Dump all devices\n");
 	rte_eal_pci_dump(stdout);
 
-	/* Unregister all previous drivers */
-	while (!TAILQ_EMPTY(&pci_driver_list)) {
-		dr = TAILQ_FIRST(&pci_driver_list);
-		rte_eal_pci_unregister(dr);
-		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
-	}
+	if (test_pci_setup())
+		return -1;
 
 	rte_eal_pci_register(&my_driver);
 	rte_eal_pci_register(&my_driver2);
@@ -199,12 +224,8 @@ test_pci(void)
 	rte_eal_pci_unregister(&my_driver);
 	rte_eal_pci_unregister(&my_driver2);
 
-	/* Restore original driver list */
-	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
-		dr = TAILQ_FIRST(&real_pci_driver_list);
-		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
-		rte_eal_pci_register(dr);
-	}
+	if (test_pci_cleanup())
+		return -1;
 
 	return 0;
 }
-- 
2.8.0

[PATCH v2 08/11] app/test: convert current pci_test into a single test case

[Jan Viktorin]

The current test_pci is just a single test case that tests the blacklisting
of devices. Rename it to test_pci_blacklist and call it from the test_pci.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/test_pci.c | 85 +++++++++++++++++++++++++++++------------------------
 1 file changed, 47 insertions(+), 38 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 9d53ba5..2e2fd70 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -144,51 +144,14 @@ static void free_devargs_list(void)
 	}
 }
 
-/* real drivers (not used for testing) */
-struct pci_driver_list real_pci_driver_list =
-	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
-
 static int
-test_pci_setup(void)
-{
-	struct rte_pci_driver *dr;
-
-	/* Unregister original driver list */
-	while (!TAILQ_EMPTY(&pci_driver_list)) {
-		dr = TAILQ_FIRST(&pci_driver_list);
-		rte_eal_pci_unregister(dr);
-		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
-	}
-
-	return 0;
-}
-
-static int
-test_pci_cleanup(void)
-{
-	struct rte_pci_driver *dr;
-
-	/* Restore original driver list */
-	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
-		dr = TAILQ_FIRST(&real_pci_driver_list);
-		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
-		rte_eal_pci_register(dr);
-	}
-
-	return 0;
-}
-
-int
-test_pci(void)
+test_pci_blacklist(void)
 {
 	struct rte_devargs_list save_devargs_list;
 
 	printf("Dump all devices\n");
 	rte_eal_pci_dump(stdout);
 
-	if (test_pci_setup())
-		return -1;
-
 	rte_eal_pci_register(&my_driver);
 	rte_eal_pci_register(&my_driver2);
 
@@ -224,6 +187,52 @@ test_pci(void)
 	rte_eal_pci_unregister(&my_driver);
 	rte_eal_pci_unregister(&my_driver2);
 
+	return 0;
+}
+
+/* real drivers (not used for testing) */
+struct pci_driver_list real_pci_driver_list =
+	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
+
+static int
+test_pci_setup(void)
+{
+	struct rte_pci_driver *dr;
+
+	/* Unregister original driver list */
+	while (!TAILQ_EMPTY(&pci_driver_list)) {
+		dr = TAILQ_FIRST(&pci_driver_list);
+		rte_eal_pci_unregister(dr);
+		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
+	}
+
+	return 0;
+}
+
+static int
+test_pci_cleanup(void)
+{
+	struct rte_pci_driver *dr;
+
+	/* Restore original driver list */
+	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
+		dr = TAILQ_FIRST(&real_pci_driver_list);
+		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
+		rte_eal_pci_register(dr);
+	}
+
+	return 0;
+}
+
+int
+test_pci(void)
+{
+	if (test_pci_setup())
+		return -1;
+
+	if (test_pci_blacklist())
+		return -1;
+
 	if (test_pci_cleanup())
 		return -1;
 
-- 
2.8.0

Re: [PATCH v2 08/11] app/test: convert current pci_test into a single test case

[Thomas Monjalon]

2016-05-10 20:13, Jan Viktorin:
> The current test_pci is just a single test case that tests the blacklisting
> of devices. Rename it to test_pci_blacklist and call it from the test_pci.

The functions are also moved. It is confusing.
Maybe this patch can be squashed with the previous one.

Re: [PATCH v2 08/11] app/test: convert current pci_test into a single test case

[Jan Viktorin]

On Thu, 12 May 2016 17:34:13 +0200
Thomas Monjalon <thomas.monjalon@6wind.com> wrote:

> 2016-05-10 20:13, Jan Viktorin:
> > The current test_pci is just a single test case that tests the blacklisting
> > of devices. Rename it to test_pci_blacklist and call it from the test_pci.  
> 
> The functions are also moved. It is confusing.
> Maybe this patch can be squashed with the previous one.
> 

Well, I wanted to separate the functional changes from cosmetic ones.
The goal here is to extract the setup and cleanup phases from the orignal
test. Would be better to just change the description?

---

app/test: extract setup and cleanup phase from pci_test

The original test_pci contains the setup and cleanup phase in its body.
By extracting this code out, we can take advantage of this code for other
(future) tests as well.

The test_pci is renamed to test_pci_blacklist - it's only a single test
case (others will follow) now.

---

Probably, it can be splitted (instead of squashed with a previous one) to
"extract setup and cleanup" and "rename to test_pci_blacklist".

Jan

-- 
   Jan Viktorin                  E-mail: Viktorin@RehiveTech.com
   System Architect              Web:    www.RehiveTech.com
   RehiveTech
   Brno, Czech Republic

Re: [PATCH v2 08/11] app/test: convert current pci_test into a single test case

[Thomas Monjalon]

2016-05-13 17:19, Jan Viktorin:
> On Thu, 12 May 2016 17:34:13 +0200
> Thomas Monjalon <thomas.monjalon@6wind.com> wrote:
> 
> > 2016-05-10 20:13, Jan Viktorin:
> > > The current test_pci is just a single test case that tests the blacklisting
> > > of devices. Rename it to test_pci_blacklist and call it from the test_pci.  
> > 
> > The functions are also moved. It is confusing.
> > Maybe this patch can be squashed with the previous one.
> > 
> 
> Well, I wanted to separate the functional changes from cosmetic ones.
> The goal here is to extract the setup and cleanup phases from the orignal
> test. Would be better to just change the description?
> 
> ---
> 
> app/test: extract setup and cleanup phase from pci_test
> 
> The original test_pci contains the setup and cleanup phase in its body.
> By extracting this code out, we can take advantage of this code for other
> (future) tests as well.
> 
> The test_pci is renamed to test_pci_blacklist - it's only a single test
> case (others will follow) now.
> 
> ---
> 
> Probably, it can be splitted (instead of squashed with a previous one) to
> "extract setup and cleanup" and "rename to test_pci_blacklist".

Not a big deal, I will check in v3.
Thanks

[PATCH v2 09/11] eal/pci: replace SYSFS_PCI_DEVICES with pci_get_sysfs_path()

[Jan Viktorin]

The SYSFS_PCI_DEVICES is a constant that makes the PCI testing difficult as
it points to an absolute path. We remove using this constant and introducing
a function pci_get_sysfs_path that gives the same value. However, the user can
pass a SYSFS_PCI_DEVICES env variable to override the path. It is now possible
to create a fake sysfs hierarchy for testing.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/test_pci.c                             | 28 +++++++++++++++++++++++++
 drivers/net/szedata2/rte_eth_szedata2.c         |  2 +-
 drivers/net/virtio/virtio_pci.c                 |  2 +-
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  6 ++++++
 lib/librte_eal/common/eal_common_pci.c          | 13 ++++++++++++
 lib/librte_eal/common/include/rte_pci.h         |  2 +-
 lib/librte_eal/linuxapp/eal/eal_pci.c           |  6 +++---
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c       |  7 ++++---
 lib/librte_eal/linuxapp/eal/eal_pci_vfio.c      |  2 +-
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |  6 ++++++
 10 files changed, 64 insertions(+), 10 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 2e2fd70..df12bb2 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -190,6 +190,31 @@ test_pci_blacklist(void)
 	return 0;
 }
 
+static int test_pci_sysfs(void)
+{
+	const char *orig;
+	const char *path;
+	int ret;
+
+	orig = pci_get_sysfs_path();
+	ret = setenv("SYSFS_PCI_DEVICES", "My Documents", 1);
+	TEST_ASSERT_SUCCESS(ret, "Failed setenv to My Documents");
+
+	path = pci_get_sysfs_path();
+	TEST_ASSERT(strcmp(orig, path),
+			"orig must be different from path: "
+			"%s (orig: %s)", path, orig);
+
+	ret = setenv("SYSFS_PCI_DEVICES", orig, 1);
+	TEST_ASSERT_SUCCESS(ret, "Failed setenv back to '%s'", orig);
+
+	path = pci_get_sysfs_path();
+	TEST_ASSERT(!strcmp(orig, path),
+			"pci_get_sysfs_path returned unexpected path: "
+			"%s (orig: %s)", path, orig);
+	return 0;
+}
+
 /* real drivers (not used for testing) */
 struct pci_driver_list real_pci_driver_list =
 	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
@@ -227,6 +252,9 @@ test_pci_cleanup(void)
 int
 test_pci(void)
 {
+	if (test_pci_sysfs())
+		return -1;
+
 	if (test_pci_setup())
 		return -1;
 
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 78c43b0..985a8d6 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1481,7 +1481,7 @@ rte_szedata2_eth_dev_init(struct rte_eth_dev *dev)
 		return -EINVAL;
 	}
 	snprintf(rsc_filename, PATH_MAX,
-		SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/resource%u",
+		"%s/" PCI_PRI_FMT "/resource%u", pci_get_sysfs_path(),
 		pci_addr->domain, pci_addr->bus,
 		pci_addr->devid, pci_addr->function, PCI_RESOURCE_NUMBER);
 	fd = open(rsc_filename, O_RDWR);
diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
index c007959..4bc058d 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -179,7 +179,7 @@ legacy_virtio_has_msix(const struct rte_pci_addr *loc)
 	char dirname[PATH_MAX];
 
 	snprintf(dirname, sizeof(dirname),
-		     SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/msi_irqs",
+		     "%s/" PCI_PRI_FMT "/msi_irqs", pci_get_sysfs_path(),
 		     loc->domain, loc->bus, loc->devid, loc->function);
 
 	d = opendir(dirname);
diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
index 58c2951..e37a175 100644
--- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
@@ -151,3 +151,9 @@ DPDK_16.04 {
 	rte_eal_primary_proc_alive;
 
 } DPDK_2.2;
+
+DPDK_16.07 {
+	global:
+
+	pci_get_sysfs_path;
+} DPDK_16.04;
diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c
index 3cae4cb..0ec3b61 100644
--- a/lib/librte_eal/common/eal_common_pci.c
+++ b/lib/librte_eal/common/eal_common_pci.c
@@ -85,6 +85,19 @@
 struct pci_driver_list pci_driver_list;
 struct pci_device_list pci_device_list;
 
+#define SYSFS_PCI_DEVICES "/sys/bus/pci/devices"
+
+const char *pci_get_sysfs_path(void)
+{
+	const char *path = NULL;
+
+	path = getenv("SYSFS_PCI_DEVICES");
+	if (path == NULL)
+		return SYSFS_PCI_DEVICES;
+
+	return path;
+}
+
 static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev)
 {
 	struct rte_devargs *devargs;
diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index 8fa2712..7669fd7 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -91,7 +91,7 @@ extern struct pci_driver_list pci_driver_list; /**< Global list of PCI drivers.
 extern struct pci_device_list pci_device_list; /**< Global list of PCI devices. */
 
 /** Pathname of PCI devices directory. */
-#define SYSFS_PCI_DEVICES "/sys/bus/pci/devices"
+const char *pci_get_sysfs_path(void);
 
 /** Formatting string for PCI device identifier: Ex: 0000:00:01.0 */
 #define PCI_PRI_FMT "%.4" PRIx16 ":%.2" PRIx8 ":%.2" PRIx8 ".%" PRIx8
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index bdc08a0..8e130a2 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -66,7 +66,7 @@ pci_unbind_kernel_driver(struct rte_pci_device *dev)
 
 	/* open /sys/bus/pci/devices/AAAA:BB:CC.D/driver */
 	snprintf(filename, sizeof(filename),
-	         SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/driver/unbind",
+	         "%s/" PCI_PRI_FMT "/driver/unbind", pci_get_sysfs_path(),
 	         loc->domain, loc->bus, loc->devid, loc->function);
 
 	f = fopen(filename, "w");
@@ -453,7 +453,7 @@ rte_eal_pci_scan(void)
 	uint16_t domain;
 	uint8_t bus, devid, function;
 
-	dir = opendir(SYSFS_PCI_DEVICES);
+	dir = opendir(pci_get_sysfs_path());
 	if (dir == NULL) {
 		RTE_LOG(ERR, EAL, "%s(): opendir failed: %s\n",
 			__func__, strerror(errno));
@@ -468,7 +468,7 @@ rte_eal_pci_scan(void)
 				&bus, &devid, &function) != 0)
 			continue;
 
-		snprintf(dirname, sizeof(dirname), "%s/%s", SYSFS_PCI_DEVICES,
+		snprintf(dirname, sizeof(dirname), "%s/%s", pci_get_sysfs_path(),
 			 e->d_name);
 		if (pci_scan_one(dirname, domain, bus, devid, function) < 0)
 			goto error;
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
index 068694d..b833244 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
@@ -161,14 +161,14 @@ pci_get_uio_dev(struct rte_pci_device *dev, char *dstbuf,
 	 * or uio:uioX */
 
 	snprintf(dirname, sizeof(dirname),
-			SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/uio",
+			"%s/" PCI_PRI_FMT "/uio", pci_get_sysfs_path(),
 			loc->domain, loc->bus, loc->devid, loc->function);
 
 	dir = opendir(dirname);
 	if (dir == NULL) {
 		/* retry with the parent directory */
 		snprintf(dirname, sizeof(dirname),
-				SYSFS_PCI_DEVICES "/" PCI_PRI_FMT,
+				"%s/" PCI_PRI_FMT, pci_get_sysfs_path(),
 				loc->domain, loc->bus, loc->devid, loc->function);
 		dir = opendir(dirname);
 
@@ -319,7 +319,8 @@ pci_uio_map_resource_by_index(struct rte_pci_device *dev, int res_idx,
 
 	/* update devname for mmap  */
 	snprintf(devname, sizeof(devname),
-			SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/resource%d",
+			"%s/" PCI_PRI_FMT "/resource%d",
+			pci_get_sysfs_path(),
 			loc->domain, loc->bus, loc->devid,
 			loc->function, res_idx);
 
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c b/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c
index 10266f8..f91b924 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c
@@ -602,7 +602,7 @@ pci_vfio_get_group_no(const char *pci_addr, int *iommu_group_no)
 
 	/* try to find out IOMMU group for this device */
 	snprintf(linkname, sizeof(linkname),
-			 SYSFS_PCI_DEVICES "/%s/iommu_group", pci_addr);
+			 "%s/%s/iommu_group", pci_get_sysfs_path(), pci_addr);
 
 	ret = readlink(linkname, filename, sizeof(filename));
 
diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
index 12503ef..905c6ee 100644
--- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
@@ -154,3 +154,9 @@ DPDK_16.04 {
 	rte_eal_primary_proc_alive;
 
 } DPDK_2.2;
+
+DPDK_16.07 {
+	global:
+
+	pci_get_sysfs_path;
+} DPDK_16.04;
-- 
2.8.0

Re: [PATCH v2 09/11] eal/pci: replace SYSFS_PCI_DEVICES with pci_get_sysfs_path()

[Thomas Monjalon]

2016-05-10 20:13, Jan Viktorin:
> The SYSFS_PCI_DEVICES is a constant that makes the PCI testing difficult as
> it points to an absolute path. We remove using this constant and introducing
> a function pci_get_sysfs_path that gives the same value. However, the user can
> pass a SYSFS_PCI_DEVICES env variable to override the path. It is now possible
> to create a fake sysfs hierarchy for testing.

Yeah!

> +	orig = pci_get_sysfs_path();
> +	ret = setenv("SYSFS_PCI_DEVICES", "My Documents", 1);

Oh no!

> +	TEST_ASSERT_SUCCESS(ret, "Failed setenv to My Documents");
> +
> +	path = pci_get_sysfs_path();
> +	TEST_ASSERT(strcmp(orig, path),
> +			"orig must be different from path: "

I missed something here. Why different?

> +DPDK_16.07 {
> +	global:
> +
> +	pci_get_sysfs_path;
> +} DPDK_16.04;

I don't know why but we are used to put a blank line after the last symbol.

Re: [PATCH v2 09/11] eal/pci: replace SYSFS_PCI_DEVICES with pci_get_sysfs_path()

[Jan Viktorin]

On Thu, 12 May 2016 17:41:22 +0200
Thomas Monjalon <thomas.monjalon@6wind.com> wrote:

> 2016-05-10 20:13, Jan Viktorin:
> > The SYSFS_PCI_DEVICES is a constant that makes the PCI testing difficult as
> > it points to an absolute path. We remove using this constant and introducing
> > a function pci_get_sysfs_path that gives the same value. However, the user can
> > pass a SYSFS_PCI_DEVICES env variable to override the path. It is now possible
> > to create a fake sysfs hierarchy for testing.  
> 
> Yeah!

:)

> 
> > +	orig = pci_get_sysfs_path();
> > +	ret = setenv("SYSFS_PCI_DEVICES", "My Documents", 1);  
> 
> Oh no!

Not sure about your reaction...

> 
> > +	TEST_ASSERT_SUCCESS(ret, "Failed setenv to My Documents");
> > +
> > +	path = pci_get_sysfs_path();
> > +	TEST_ASSERT(strcmp(orig, path),
> > +			"orig must be different from path: "  
> 
> I missed something here. Why different?

Because I've set it to "My Documents" and want to be sure that the
pci_get_sysfs_path() returns the new path instead of the default
one.

Perhaps, !strcmp(path, "My Documents") would be better here...

> 
> > +DPDK_16.07 {
> > +	global:
> > +
> > +	pci_get_sysfs_path;
> > +} DPDK_16.04;  
> 
> I don't know why but we are used to put a blank line after the last symbol.

Will fix.

> 



-- 
   Jan Viktorin                  E-mail: Viktorin@RehiveTech.com
   System Architect              Web:    www.RehiveTech.com
   RehiveTech
   Brno, Czech Republic

Re: [PATCH v2 09/11] eal/pci: replace SYSFS_PCI_DEVICES with pci_get_sysfs_path()

[Thomas Monjalon]

2016-05-12 17:46, Jan Viktorin:
> On Thu, 12 May 2016 17:41:22 +0200
> Thomas Monjalon <thomas.monjalon@6wind.com> wrote:
> > 2016-05-10 20:13, Jan Viktorin:
> > > +	orig = pci_get_sysfs_path();
> > > +	ret = setenv("SYSFS_PCI_DEVICES", "My Documents", 1);  
> > 
> > Oh no!
> 
> Not sure about your reaction...

MS reference... ;)

> > 
> > > +	TEST_ASSERT_SUCCESS(ret, "Failed setenv to My Documents");
> > > +
> > > +	path = pci_get_sysfs_path();
> > > +	TEST_ASSERT(strcmp(orig, path),
> > > +			"orig must be different from path: "  
> > 
> > I missed something here. Why different?
> 
> Because I've set it to "My Documents" and want to be sure that the
> pci_get_sysfs_path() returns the new path instead of the default
> one.
> 
> Perhaps, !strcmp(path, "My Documents") would be better here...

No, just rethink the variable names maybe.

Re: [PATCH v2 09/11] eal/pci: replace SYSFS_PCI_DEVICES with pci_get_sysfs_path()

[Jan Viktorin]

On Thu, 12 May 2016 18:10:07 +0200
Thomas Monjalon <thomas.monjalon@6wind.com> wrote:

> 2016-05-12 17:46, Jan Viktorin:
> > On Thu, 12 May 2016 17:41:22 +0200
> > Thomas Monjalon <thomas.monjalon@6wind.com> wrote:  
> > > 2016-05-10 20:13, Jan Viktorin:  
> > > > +	orig = pci_get_sysfs_path();
> > > > +	ret = setenv("SYSFS_PCI_DEVICES", "My Documents", 1);    
> > > 
> > > Oh no!  
> > 
> > Not sure about your reaction...  
> 
> MS reference... ;)
> 
> > >   
> > > > +	TEST_ASSERT_SUCCESS(ret, "Failed setenv to My Documents");
> > > > +
> > > > +	path = pci_get_sysfs_path();
> > > > +	TEST_ASSERT(strcmp(orig, path),
> > > > +			"orig must be different from path: "    
> > > 
> > > I missed something here. Why different?  
> > 
> > Because I've set it to "My Documents" and want to be sure that the
> > pci_get_sysfs_path() returns the new path instead of the default
> > one.
> > 
> > Perhaps, !strcmp(path, "My Documents") would be better here...  
> 
> No, just rethink the variable names maybe.

Fixed for v3 as a combination of both.



-- 
   Jan Viktorin                  E-mail: Viktorin@RehiveTech.com
   System Architect              Web:    www.RehiveTech.com
   RehiveTech
   Brno, Czech Republic

Re: [PATCH v2 09/11] eal/pci: replace SYSFS_PCI_DEVICES with pci_get_sysfs_path()

[Thomas Monjalon]

2016-05-10 20:13, Jan Viktorin:
> The SYSFS_PCI_DEVICES is a constant that makes the PCI testing difficult as
> it points to an absolute path. We remove using this constant and introducing
> a function pci_get_sysfs_path that gives the same value. However, the user can
> pass a SYSFS_PCI_DEVICES env variable to override the path. It is now possible
> to create a fake sysfs hierarchy for testing.

The headline do not convey the intent. It could be:
	pci: allow to override sysfs

[PATCH v2 10/11] app/test: scan PCI bus using a fake sysfs

[Jan Viktorin]

Scan the PCI bus by providing a fake sysfs with a PCI device. The fake sysfs
is a packed file hierarchy linked into the test.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/Makefile                                  |   1 +
 app/test/test_pci.c                                |  58 +++++++++++++++++++--
 .../bus/pci/devices/0000:01:00.0/class             |   1 +
 .../bus/pci/devices/0000:01:00.0/config            | Bin 0 -> 64 bytes
 .../devices/0000:01:00.0/consistent_dma_mask_bits  |   1 +
 .../bus/pci/devices/0000:01:00.0/device            |   1 +
 .../bus/pci/devices/0000:01:00.0/dma_mask_bits     |   1 +
 .../bus/pci/devices/0000:01:00.0/enable            |   1 +
 .../bus/pci/devices/0000:01:00.0/irq               |   1 +
 .../bus/pci/devices/0000:01:00.0/modalias          |   1 +
 .../bus/pci/devices/0000:01:00.0/msi_bus           |   1 +
 .../bus/pci/devices/0000:01:00.0/numa_node         |   1 +
 .../bus/pci/devices/0000:01:00.0/resource          |  13 +++++
 .../bus/pci/devices/0000:01:00.0/sriov_numvfs      |   1 +
 .../bus/pci/devices/0000:01:00.0/sriov_totalvfs    |   1 +
 .../bus/pci/devices/0000:01:00.0/subsystem_device  |   1 +
 .../bus/pci/devices/0000:01:00.0/subsystem_vendor  |   1 +
 .../bus/pci/devices/0000:01:00.0/uevent            |   6 +++
 .../bus/pci/devices/0000:01:00.0/vendor            |   1 +
 19 files changed, 89 insertions(+), 3 deletions(-)
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/config
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor

diff --git a/app/test/Makefile b/app/test/Makefile
index de5fa50..1c6c29e 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -73,6 +73,7 @@ SRCS-y += test_resource.c
 $(eval $(call linked_resource,test_resource_c,resource.c))
 $(eval $(call linked_tar_resource,test_resource_tar,test_resource.c))
 SRCS-y += test_pci.c
+$(eval $(call linked_tar_resource,test_pci_sysfs,test_pci_sysfs))
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
 SRCS-y += test_per_lcore.c
diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index df12bb2..3c6c955 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -43,6 +43,7 @@
 #include <rte_devargs.h>
 
 #include "test.h"
+#include "resource.h"
 
 /* Generic maximum number of drivers to have room to allocate all drivers */
 #define NUM_MAX_DRIVERS 256
@@ -215,37 +216,88 @@ static int test_pci_sysfs(void)
 	return 0;
 }
 
-/* real drivers (not used for testing) */
+/* real devices & drivers (not used for testing) */
 struct pci_driver_list real_pci_driver_list =
 	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
+struct pci_device_list real_pci_device_list =
+	TAILQ_HEAD_INITIALIZER(real_pci_device_list);
+
+REGISTER_LINKED_RESOURCE(test_pci_sysfs);
 
 static int
 test_pci_setup(void)
 {
+	struct rte_pci_device *dev;
 	struct rte_pci_driver *dr;
+	const struct resource *r;
+	int ret;
+
+	r = resource_find("test_pci_sysfs");
+	TEST_ASSERT_NOT_NULL(r, "missing resource test_pci_sysfs");
+
+	ret = resource_untar(r);
+	TEST_ASSERT_SUCCESS(ret, "failed to untar %s", r->name);
+
+	ret = setenv("SYSFS_PCI_DEVICES", "test_pci_sysfs/bus/pci/devices", 1);
+	TEST_ASSERT_SUCCESS(ret, "failed to setenv");
 
-	/* Unregister original driver list */
+	/* Unregister original devices & drivers lists */
 	while (!TAILQ_EMPTY(&pci_driver_list)) {
 		dr = TAILQ_FIRST(&pci_driver_list);
 		rte_eal_pci_unregister(dr);
 		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
 	}
 
+	while (!TAILQ_EMPTY(&pci_device_list)) {
+		dev = TAILQ_FIRST(&pci_device_list);
+		TAILQ_REMOVE(&pci_device_list, dev, next);
+		TAILQ_INSERT_TAIL(&real_pci_device_list, dev, next);
+	}
+
+	ret = rte_eal_pci_scan();
+	TEST_ASSERT_SUCCESS(ret, "failed to scan PCI bus");
+	rte_eal_pci_dump(stdout);
+
 	return 0;
 }
 
 static int
 test_pci_cleanup(void)
 {
+	struct rte_pci_device *dev;
 	struct rte_pci_driver *dr;
+	const struct resource *r;
+	int ret;
+
+	unsetenv("SYSFS_PCI_DEVICES");
+
+	r = resource_find("test_pci_sysfs");
+	TEST_ASSERT_NOT_NULL(r, "missing resource test_pci_sysfs");
+
+	ret = resource_rm_by_tar(r);
+	TEST_ASSERT_SUCCESS(ret, "Failed to delete resource %s", r->name);
 
-	/* Restore original driver list */
+	/* FIXME: there is no API in DPDK to free a rte_pci_device so we
+	   cannot free the devices in the right way. Let's assume that we
+	   don't care for tests. */
+	while (!TAILQ_EMPTY(&pci_device_list)) {
+		dev = TAILQ_FIRST(&pci_device_list);
+		TAILQ_REMOVE(&pci_device_list, dev, next);
+	}
+
+	/* Restore original devices & drivers lists */
 	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
 		dr = TAILQ_FIRST(&real_pci_driver_list);
 		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
 		rte_eal_pci_register(dr);
 	}
 
+	while (!TAILQ_EMPTY(&real_pci_device_list)) {
+		dev = TAILQ_FIRST(&real_pci_device_list);
+		TAILQ_REMOVE(&real_pci_device_list, dev, next);
+		TAILQ_INSERT_TAIL(&pci_device_list, dev, next);
+	}
+
 	return 0;
 }
 
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class
new file mode 100644
index 0000000..2f9c1da
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class
@@ -0,0 +1 @@
+0x020000
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/config b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/config
new file mode 100644
index 0000000000000000000000000000000000000000..7752421cf13a5aa00a28eaee02cac2add9ce5566
GIT binary patch
literal 64
zcmZo`_$|QBBEZ1Nz`(@7(7?dMz;S^A2oxWHNCpNT2LUi2#BOU~22l(SV3L7>8>k5Y
DD^Ld(

literal 0
HcmV?d00001

diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits
new file mode 100644
index 0000000..900731f
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits
@@ -0,0 +1 @@
+64
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device
new file mode 100644
index 0000000..9e4789e
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device
@@ -0,0 +1 @@
+0x10fb
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits
new file mode 100644
index 0000000..900731f
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits
@@ -0,0 +1 @@
+64
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable
@@ -0,0 +1 @@
+1
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq
@@ -0,0 +1 @@
+0
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias
new file mode 100644
index 0000000..f4c76ed
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias
@@ -0,0 +1 @@
+pci:v00008086d000010FBsv00008086sd00000003bc02sc00i00
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus
@@ -0,0 +1 @@
+1
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node
new file mode 100644
index 0000000..3a2e3f4
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node
@@ -0,0 +1 @@
+-1
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource
new file mode 100644
index 0000000..f388929
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource
@@ -0,0 +1,13 @@
+0x00000000d0080000 0x00000000d00fffff 0x000000000014220c
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x000000000000e020 0x000000000000e03f 0x0000000000040101
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x00000000d0104000 0x00000000d0107fff 0x000000000014220c
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x00000000ab000000 0x00000000ab0fffff 0x0000000000140204
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x00000000ab100000 0x00000000ab1fffff 0x0000000000140204
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x0000000000000000 0x0000000000000000 0x0000000000000000
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs
@@ -0,0 +1 @@
+0
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs
new file mode 100644
index 0000000..4b9026d
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs
@@ -0,0 +1 @@
+63
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device
new file mode 100644
index 0000000..89a932c
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device
@@ -0,0 +1 @@
+0x0003
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor
new file mode 100644
index 0000000..ce6dc4d
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor
@@ -0,0 +1 @@
+0x8086
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent
new file mode 100644
index 0000000..1dbe34d
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent
@@ -0,0 +1,6 @@
+DRIVER=ixgbe
+PCI_CLASS=20000
+PCI_ID=8086:10FB
+PCI_SUBSYS_ID=8086:0003
+PCI_SLOT_NAME=0000:01:00.0
+MODALIAS=pci:v00008086d000010FBsv00008086sd00000003bc02sc00i00
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor
new file mode 100644
index 0000000..ce6dc4d
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor
@@ -0,0 +1 @@
+0x8086
-- 
2.8.0

[PATCH v2 11/11] app/test: do not dump PCI devices in blacklist test

[Jan Viktorin]

Dumping of devices in a unittest is useless. Instead, test whether the test
has been set up well - i.e. there are no devices.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/test_pci.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 3c6c955..942fa09 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -150,8 +150,8 @@ test_pci_blacklist(void)
 {
 	struct rte_devargs_list save_devargs_list;
 
-	printf("Dump all devices\n");
-	rte_eal_pci_dump(stdout);
+	TEST_ASSERT(TAILQ_EMPTY(&pci_driver_list),
+			"pci_driver_list not empty");
 
 	rte_eal_pci_register(&my_driver);
 	rte_eal_pci_register(&my_driver2);
-- 
2.8.0

[PATCH v3 00/11] Include resources in tests

[Jan Viktorin]

Hello,

the third round with fixes based on suggestions by T. Monjalon.

This patch set introduces a mechanism to include a resource (in general a blob)
into the test binary. This allows to make tests less dependent on the target
testing environment. The first use case is testing of PCI bus scan by changing
the hard-coded path (/sys/bus/pci/devices) to something different and provide
a fake tree of devices with the test.

Regards
J. Viktorin

---
v1:
* included 5 patches improving the PCI tests
* fixed using of non-existing RTE_INIT macro

v2:
* Makefile macro resource renamed to linked_resource
* introduced macro linked_tar_resource
* added more comments
* clarified relation between REGISTER_LINKED_RESOURCE and the Makefile
* untar is checked to not loop infinitely
* improved commit messages
* objcopy params extracted to a separated patcha (missing tile and ppc)
* few random bits (usually suggested by T. Monjalon)
* included a note about the new dependency libarchive in the particular commit

v3:
* resource_autotest added to autotest_data.py
* improved test of affecting pci_get_sysfs_path() by setenv
* few other bits...
---
Jan Viktorin (11):
  app/test: introduce resources for tests
  mk: define objcopy-specific target and arch
  app/test: support resources externally linked
  app/test: add functions to create files from resources
  app/test: support resources archived by tar
  app/test: use linked list to store PCI drivers
  app/test: extract test_pci_setup and test_pci_cleanup
  app/test: convert current pci_test into a single test case
  eal/pci: allow to override sysfs
  app/test: scan PCI bus using a fake sysfs
  app/test: do not dump PCI devices in blacklist test

 app/test/Makefile                                  |  31 +++
 app/test/autotest_data.py                          |   6 +
 app/test/resource.c                                | 296 +++++++++++++++++++++
 app/test/resource.h                                | 135 ++++++++++
 app/test/test_pci.c                                | 148 +++++++++--
 .../bus/pci/devices/0000:01:00.0/class             |   1 +
 .../bus/pci/devices/0000:01:00.0/config            | Bin 0 -> 64 bytes
 .../devices/0000:01:00.0/consistent_dma_mask_bits  |   1 +
 .../bus/pci/devices/0000:01:00.0/device            |   1 +
 .../bus/pci/devices/0000:01:00.0/dma_mask_bits     |   1 +
 .../bus/pci/devices/0000:01:00.0/enable            |   1 +
 .../bus/pci/devices/0000:01:00.0/irq               |   1 +
 .../bus/pci/devices/0000:01:00.0/modalias          |   1 +
 .../bus/pci/devices/0000:01:00.0/msi_bus           |   1 +
 .../bus/pci/devices/0000:01:00.0/numa_node         |   1 +
 .../bus/pci/devices/0000:01:00.0/resource          |  13 +
 .../bus/pci/devices/0000:01:00.0/sriov_numvfs      |   1 +
 .../bus/pci/devices/0000:01:00.0/sriov_totalvfs    |   1 +
 .../bus/pci/devices/0000:01:00.0/subsystem_device  |   1 +
 .../bus/pci/devices/0000:01:00.0/subsystem_vendor  |   1 +
 .../bus/pci/devices/0000:01:00.0/uevent            |   6 +
 .../bus/pci/devices/0000:01:00.0/vendor            |   1 +
 app/test/test_resource.c                           | 132 +++++++++
 drivers/net/szedata2/rte_eth_szedata2.c            |   2 +-
 drivers/net/virtio/virtio_pci.c                    |   2 +-
 lib/librte_eal/bsdapp/eal/rte_eal_version.map      |   7 +
 lib/librte_eal/common/eal_common_pci.c             |  13 +
 lib/librte_eal/common/include/rte_pci.h            |   2 +-
 lib/librte_eal/linuxapp/eal/eal_pci.c              |   6 +-
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c          |   7 +-
 lib/librte_eal/linuxapp/eal/eal_pci_vfio.c         |   2 +-
 lib/librte_eal/linuxapp/eal/rte_eal_version.map    |   7 +
 mk/arch/arm/rte.vars.mk                            |   5 +
 mk/arch/arm64/rte.vars.mk                          |   5 +
 mk/arch/i686/rte.vars.mk                           |   5 +
 mk/arch/x86_64/rte.vars.mk                         |   5 +
 mk/arch/x86_x32/rte.vars.mk                        |   5 +
 37 files changed, 828 insertions(+), 26 deletions(-)
 create mode 100644 app/test/resource.c
 create mode 100644 app/test/resource.h
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/config
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor
 create mode 100644 app/test/test_resource.c

-- 
2.8.0

[PATCH v4 00/10] Include resources in tests

[Jan Viktorin]

Hello,

the fourth version with cosmetic changes according to the short IRC discussion.
Note, we are adding the libarchive dependency by this patch set.

---
v1:
* included 5 patches improving the PCI tests
* fixed using of non-existing RTE_INIT macro

v2:
* Makefile macro resource renamed to linked_resource
* introduced macro linked_tar_resource
* added more comments
* clarified relation between REGISTER_LINKED_RESOURCE and the Makefile
* untar is checked to not loop infinitely
* improved commit messages
* objcopy params extracted to a separated patcha (missing tile and ppc)
	* few random bits (usually suggested by T. Monjalon)
	* included a note about the new dependency libarchive in the particular commit

v3:
* resource_autotest added to autotest_data.py
* improved test of affecting pci_get_sysfs_path() by setenv
* few other bits...

v4:
* fix checkpatch issues
* state that tile and ppc_64 are not supported in patch 0002
* avoid moving functions test_pci_setup and test_pci_cleanup in patch 0008


Jan Viktorin (10):
  app/test: introduce resources for tests
  mk: define objcopy-specific target and arch
  app/test: support resources externally linked
  app/test: add functions to create files from resources
  app/test: support resources archived by tar
  app/test: use linked list to store PCI drivers
  app/test: extract test_pci_setup and test_pci_cleanup
  app/test: convert current pci_test into a single test case
  eal/pci: allow to override sysfs
  app/test: do not dump PCI devices in blacklist test

 app/test/Makefile                                  |  31 +++
 app/test/autotest_data.py                          |   6 +
 app/test/resource.c                                | 297 +++++++++++++++++++++
 app/test/resource.h                                | 135 ++++++++++
 app/test/test_pci.c                                | 147 +++++++++-
 .../bus/pci/devices/0000:01:00.0/class             |   1 +
 .../bus/pci/devices/0000:01:00.0/config            | Bin 0 -> 64 bytes
 .../devices/0000:01:00.0/consistent_dma_mask_bits  |   1 +
 .../bus/pci/devices/0000:01:00.0/device            |   1 +
 .../bus/pci/devices/0000:01:00.0/dma_mask_bits     |   1 +
 .../bus/pci/devices/0000:01:00.0/enable            |   1 +
 .../bus/pci/devices/0000:01:00.0/irq               |   1 +
 .../bus/pci/devices/0000:01:00.0/modalias          |   1 +
 .../bus/pci/devices/0000:01:00.0/msi_bus           |   1 +
 .../bus/pci/devices/0000:01:00.0/numa_node         |   1 +
 .../bus/pci/devices/0000:01:00.0/resource          |  13 +
 .../bus/pci/devices/0000:01:00.0/sriov_numvfs      |   1 +
 .../bus/pci/devices/0000:01:00.0/sriov_totalvfs    |   1 +
 .../bus/pci/devices/0000:01:00.0/subsystem_device  |   1 +
 .../bus/pci/devices/0000:01:00.0/subsystem_vendor  |   1 +
 .../bus/pci/devices/0000:01:00.0/uevent            |   6 +
 .../bus/pci/devices/0000:01:00.0/vendor            |   1 +
 app/test/test_resource.c                           | 132 +++++++++
 drivers/net/szedata2/rte_eth_szedata2.c            |   2 +-
 drivers/net/virtio/virtio_pci.c                    |   2 +-
 lib/librte_eal/bsdapp/eal/rte_eal_version.map      |   7 +
 lib/librte_eal/common/eal_common_pci.c             |  13 +
 lib/librte_eal/common/include/rte_pci.h            |   2 +-
 lib/librte_eal/linuxapp/eal/eal_pci.c              |  10 +-
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c          |   7 +-
 lib/librte_eal/linuxapp/eal/eal_pci_vfio.c         |   2 +-
 lib/librte_eal/linuxapp/eal/rte_eal_version.map    |   7 +
 mk/arch/arm/rte.vars.mk                            |   5 +
 mk/arch/arm64/rte.vars.mk                          |   5 +
 mk/arch/i686/rte.vars.mk                           |   5 +
 mk/arch/x86_64/rte.vars.mk                         |   5 +
 mk/arch/x86_x32/rte.vars.mk                        |   5 +
 37 files changed, 832 insertions(+), 26 deletions(-)
 create mode 100644 app/test/resource.c
 create mode 100644 app/test/resource.h
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/config
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor
 create mode 100644 app/test/test_resource.c

-- 
2.8.0

Re: [PATCH v4 00/10] Include resources in tests

[Jan Viktorin]

Oh, I forgot to rebase on the very last HEAD, there is a trivial
merge conflict with

commit a3f34a98b7217f4ff2a8636096984f566a4e7cab
Author: Thomas Monjalon <thomas.monjalon@6wind.com>
Date:   Thu Jun 9 15:49:48 2016 +0200

    log: deprecate history dump


Should I repost it?

Regards
Jan

On Mon, 13 Jun 2016 10:12:20 +0200
Jan Viktorin <viktorin@rehivetech.com> wrote:

> Hello,
> 
> the fourth version with cosmetic changes according to the short IRC discussion.
> Note, we are adding the libarchive dependency by this patch set.
> 
> ---
> v1:
> * included 5 patches improving the PCI tests
> * fixed using of non-existing RTE_INIT macro
> 
> v2:
> * Makefile macro resource renamed to linked_resource
> * introduced macro linked_tar_resource
> * added more comments
> * clarified relation between REGISTER_LINKED_RESOURCE and the Makefile
> * untar is checked to not loop infinitely
> * improved commit messages
> * objcopy params extracted to a separated patcha (missing tile and ppc)
> 	* few random bits (usually suggested by T. Monjalon)
> 	* included a note about the new dependency libarchive in the particular commit
> 
> v3:
> * resource_autotest added to autotest_data.py
> * improved test of affecting pci_get_sysfs_path() by setenv
> * few other bits...
> 
> v4:
> * fix checkpatch issues
> * state that tile and ppc_64 are not supported in patch 0002
> * avoid moving functions test_pci_setup and test_pci_cleanup in patch 0008
> 
> 
> Jan Viktorin (10):
>   app/test: introduce resources for tests
>   mk: define objcopy-specific target and arch
>   app/test: support resources externally linked
>   app/test: add functions to create files from resources
>   app/test: support resources archived by tar
>   app/test: use linked list to store PCI drivers
>   app/test: extract test_pci_setup and test_pci_cleanup
>   app/test: convert current pci_test into a single test case
>   eal/pci: allow to override sysfs
>   app/test: do not dump PCI devices in blacklist test
> 
>  app/test/Makefile                                  |  31 +++
>  app/test/autotest_data.py                          |   6 +
>  app/test/resource.c                                | 297 +++++++++++++++++++++
>  app/test/resource.h                                | 135 ++++++++++
>  app/test/test_pci.c                                | 147 +++++++++-
>  .../bus/pci/devices/0000:01:00.0/class             |   1 +
>  .../bus/pci/devices/0000:01:00.0/config            | Bin 0 -> 64 bytes
>  .../devices/0000:01:00.0/consistent_dma_mask_bits  |   1 +
>  .../bus/pci/devices/0000:01:00.0/device            |   1 +
>  .../bus/pci/devices/0000:01:00.0/dma_mask_bits     |   1 +
>  .../bus/pci/devices/0000:01:00.0/enable            |   1 +
>  .../bus/pci/devices/0000:01:00.0/irq               |   1 +
>  .../bus/pci/devices/0000:01:00.0/modalias          |   1 +
>  .../bus/pci/devices/0000:01:00.0/msi_bus           |   1 +
>  .../bus/pci/devices/0000:01:00.0/numa_node         |   1 +
>  .../bus/pci/devices/0000:01:00.0/resource          |  13 +
>  .../bus/pci/devices/0000:01:00.0/sriov_numvfs      |   1 +
>  .../bus/pci/devices/0000:01:00.0/sriov_totalvfs    |   1 +
>  .../bus/pci/devices/0000:01:00.0/subsystem_device  |   1 +
>  .../bus/pci/devices/0000:01:00.0/subsystem_vendor  |   1 +
>  .../bus/pci/devices/0000:01:00.0/uevent            |   6 +
>  .../bus/pci/devices/0000:01:00.0/vendor            |   1 +
>  app/test/test_resource.c                           | 132 +++++++++
>  drivers/net/szedata2/rte_eth_szedata2.c            |   2 +-
>  drivers/net/virtio/virtio_pci.c                    |   2 +-
>  lib/librte_eal/bsdapp/eal/rte_eal_version.map      |   7 +
>  lib/librte_eal/common/eal_common_pci.c             |  13 +
>  lib/librte_eal/common/include/rte_pci.h            |   2 +-
>  lib/librte_eal/linuxapp/eal/eal_pci.c              |  10 +-
>  lib/librte_eal/linuxapp/eal/eal_pci_uio.c          |   7 +-
>  lib/librte_eal/linuxapp/eal/eal_pci_vfio.c         |   2 +-
>  lib/librte_eal/linuxapp/eal/rte_eal_version.map    |   7 +
>  mk/arch/arm/rte.vars.mk                            |   5 +
>  mk/arch/arm64/rte.vars.mk                          |   5 +
>  mk/arch/i686/rte.vars.mk                           |   5 +
>  mk/arch/x86_64/rte.vars.mk                         |   5 +
>  mk/arch/x86_x32/rte.vars.mk                        |   5 +
>  37 files changed, 832 insertions(+), 26 deletions(-)
>  create mode 100644 app/test/resource.c
>  create mode 100644 app/test/resource.h
>  create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class
>  create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/config
>  create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits
>  create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device
>  create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits
>  create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable
>  create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq
>  create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias
>  create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus
>  create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node
>  create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource
>  create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs
>  create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs
>  create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device
>  create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor
>  create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent
>  create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor
>  create mode 100644 app/test/test_resource.c
> 



-- 
   Jan Viktorin                  E-mail: Viktorin@RehiveTech.com
   System Architect              Web:    www.RehiveTech.com
   RehiveTech
   Brno, Czech Republic

[PATCH v5 00/10] Include resources in tests

[Jan Viktorin]

Hello,

the fifth version fixing the libarchive API usage.

Regards
Jan

---
v1:
* included 5 patches improving the PCI tests
* fixed using of non-existing RTE_INIT macro

v2:
* Makefile macro resource renamed to linked_resource
* introduced macro linked_tar_resource
* added more comments
* clarified relation between REGISTER_LINKED_RESOURCE and the Makefile
* untar is checked to not loop infinitely
* improved commit messages
* objcopy params extracted to a separated patcha (missing tile and ppc)
	* few random bits (usually suggested by T. Monjalon)
	* included a note about the new dependency libarchive in the particular commit

v3:
* resource_autotest added to autotest_data.py
* improved test of affecting pci_get_sysfs_path() by setenv
* few other bits...

v4:
* fix checkpatch issues
* state that tile and ppc_64 are not supported in patch 0002
* avoid moving functions test_pci_setup and test_pci_cleanup in patch 0008

v5:
* fixed off argument of archive_read_data_block


Jan Viktorin (10):
  app/test: introduce resources for tests
  mk: define objcopy-specific target and arch
  app/test: support resources externally linked
  app/test: add functions to create files from resources
  app/test: support resources archived by tar
  app/test: use linked list to store PCI drivers
  app/test: extract test_pci_setup and test_pci_cleanup
  app/test: convert current pci_test into a single test case
  eal/pci: allow to override sysfs
  app/test: do not dump PCI devices in blacklist test

 app/test/Makefile                                  |  31 +++
 app/test/autotest_data.py                          |   6 +
 app/test/resource.c                                | 301 +++++++++++++++++++++
 app/test/resource.h                                | 135 +++++++++
 app/test/test_pci.c                                | 147 +++++++++-
 .../bus/pci/devices/0000:01:00.0/class             |   1 +
 .../bus/pci/devices/0000:01:00.0/config            | Bin 0 -> 64 bytes
 .../devices/0000:01:00.0/consistent_dma_mask_bits  |   1 +
 .../bus/pci/devices/0000:01:00.0/device            |   1 +
 .../bus/pci/devices/0000:01:00.0/dma_mask_bits     |   1 +
 .../bus/pci/devices/0000:01:00.0/enable            |   1 +
 .../bus/pci/devices/0000:01:00.0/irq               |   1 +
 .../bus/pci/devices/0000:01:00.0/modalias          |   1 +
 .../bus/pci/devices/0000:01:00.0/msi_bus           |   1 +
 .../bus/pci/devices/0000:01:00.0/numa_node         |   1 +
 .../bus/pci/devices/0000:01:00.0/resource          |  13 +
 .../bus/pci/devices/0000:01:00.0/sriov_numvfs      |   1 +
 .../bus/pci/devices/0000:01:00.0/sriov_totalvfs    |   1 +
 .../bus/pci/devices/0000:01:00.0/subsystem_device  |   1 +
 .../bus/pci/devices/0000:01:00.0/subsystem_vendor  |   1 +
 .../bus/pci/devices/0000:01:00.0/uevent            |   6 +
 .../bus/pci/devices/0000:01:00.0/vendor            |   1 +
 app/test/test_resource.c                           | 132 +++++++++
 drivers/net/szedata2/rte_eth_szedata2.c            |   2 +-
 drivers/net/virtio/virtio_pci.c                    |   2 +-
 lib/librte_eal/bsdapp/eal/rte_eal_version.map      |   7 +
 lib/librte_eal/common/eal_common_pci.c             |  13 +
 lib/librte_eal/common/include/rte_pci.h            |   2 +-
 lib/librte_eal/linuxapp/eal/eal_pci.c              |  10 +-
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c          |   7 +-
 lib/librte_eal/linuxapp/eal/eal_pci_vfio.c         |   2 +-
 lib/librte_eal/linuxapp/eal/rte_eal_version.map    |   7 +
 mk/arch/arm/rte.vars.mk                            |   5 +
 mk/arch/arm64/rte.vars.mk                          |   5 +
 mk/arch/i686/rte.vars.mk                           |   5 +
 mk/arch/x86_64/rte.vars.mk                         |   5 +
 mk/arch/x86_x32/rte.vars.mk                        |   5 +
 37 files changed, 836 insertions(+), 26 deletions(-)
 create mode 100644 app/test/resource.c
 create mode 100644 app/test/resource.h
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/config
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor
 create mode 100644 app/test/test_resource.c

-- 
2.8.0

Re: [PATCH v5 00/10] Include resources in tests

[Thomas Monjalon]

> Jan Viktorin (10):
>   app/test: introduce resources for tests
>   mk: define objcopy-specific target and arch
>   app/test: support resources externally linked
>   app/test: add functions to create files from resources
>   app/test: support resources archived by tar
>   app/test: use linked list to store PCI drivers
>   app/test: extract test_pci_setup and test_pci_cleanup
>   app/test: convert current pci_test into a single test case
>   eal/pci: allow to override sysfs
>   app/test: do not dump PCI devices in blacklist test

Applied, thanks

Re: [PATCH v5 00/10] Include resources in tests

[Hunt, David]

On 13/6/2016 7:32 PM, Thomas Monjalon wrote:
>> Jan Viktorin (10):
>>    app/test: introduce resources for tests
>>    mk: define objcopy-specific target and arch
>>    app/test: support resources externally linked
>>    app/test: add functions to create files from resources
>>    app/test: support resources archived by tar
>>    app/test: use linked list to store PCI drivers
>>    app/test: extract test_pci_setup and test_pci_cleanup
>>    app/test: convert current pci_test into a single test case
>>    eal/pci: allow to override sysfs
>>    app/test: do not dump PCI devices in blacklist test
> Applied, thanks

Hi all,
Since this patch adds a dependency on libarchive-devel, should we have an
entry in sys_reqs.rst? My fedora build was broken until I added that 
package.
Regards,
Dave.

Re: [PATCH v5 00/10] Include resources in tests

[Thomas Monjalon]

2016-06-14 09:23, Hunt, David:
> 
> On 13/6/2016 7:32 PM, Thomas Monjalon wrote:
> >> Jan Viktorin (10):
> >>    app/test: introduce resources for tests
> >>    mk: define objcopy-specific target and arch
> >>    app/test: support resources externally linked
> >>    app/test: add functions to create files from resources
> >>    app/test: support resources archived by tar
> >>    app/test: use linked list to store PCI drivers
> >>    app/test: extract test_pci_setup and test_pci_cleanup
> >>    app/test: convert current pci_test into a single test case
> >>    eal/pci: allow to override sysfs
> >>    app/test: do not dump PCI devices in blacklist test
> > Applied, thanks
> 
> Hi all,
> Since this patch adds a dependency on libarchive-devel, should we have an
> entry in sys_reqs.rst? My fedora build was broken until I added that 
> package.

You're right.
I'm working on a patch to make it optional.

[PATCH] config: make libarchive optional

[Thomas Monjalon]

The commit 66819e6 has introduced a dependency on libarchive to be able
to use some tar resources in the unit tests.
It is now an optional dependency because some systems do not have it
installed.

If CONFIG_RTE_APP_TEST_RESOURCE_TAR is disabled, the PCI test will not
be run. When a "configure" script will be integrated, the libarchive
availability could be checked to automatically enable the option.

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
---
 app/test/Makefile                 | 11 +++++++----
 app/test/resource.c               |  8 ++++++--
 app/test/test_mp_secondary.c      |  4 ++++
 app/test/test_resource.c          |  5 +++++
 config/common_base                |  1 +
 doc/guides/linux_gsg/sys_reqs.rst |  3 +++
 scripts/test-build.sh             |  4 ++++
 7 files changed, 30 insertions(+), 6 deletions(-)

diff --git a/app/test/Makefile b/app/test/Makefile
index 7e4d484..5ca5c0b 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -71,9 +71,6 @@ SRCS-y += test.c
 SRCS-y += resource.c
 SRCS-y += test_resource.c
 $(eval $(call linked_resource,test_resource_c,resource.c))
-$(eval $(call linked_tar_resource,test_resource_tar,test_resource.c))
-SRCS-y += test_pci.c
-$(eval $(call linked_tar_resource,test_pci_sysfs,test_pci_sysfs))
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
 SRCS-y += test_per_lcore.c
@@ -88,6 +85,13 @@ SRCS-y += test_ring.c
 SRCS-y += test_ring_perf.c
 SRCS-y += test_pmd_perf.c
 
+ifeq ($(CONFIG_RTE_APP_TEST_RESOURCE_TAR),y)
+$(eval $(call linked_tar_resource,test_resource_tar,test_resource.c))
+SRCS-y += test_pci.c
+$(eval $(call linked_tar_resource,test_pci_sysfs,test_pci_sysfs))
+LDLIBS += -larchive
+endif
+
 ifeq ($(CONFIG_RTE_LIBRTE_TABLE),y)
 SRCS-y += test_table.c
 SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += test_table_pipeline.c
@@ -194,7 +198,6 @@ CFLAGS += $(WERROR_FLAGS)
 CFLAGS += -D_GNU_SOURCE
 
 LDLIBS += -lm
-LDLIBS += -larchive
 
 # Disable VTA for memcpy test
 ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y)
diff --git a/app/test/resource.c b/app/test/resource.c
index 8c42eea..0e2b62c 100644
--- a/app/test/resource.c
+++ b/app/test/resource.c
@@ -33,8 +33,6 @@
 
 #include <stdio.h>
 #include <string.h>
-#include <archive.h>
-#include <archive_entry.h>
 #include <errno.h>
 #include <sys/queue.h>
 
@@ -97,6 +95,10 @@ int resource_fwrite_file(const struct resource *r, const char *fname)
 	return ret;
 }
 
+#ifdef RTE_APP_TEST_RESOURCE_TAR
+#include <archive.h>
+#include <archive_entry.h>
+
 static int do_copy(struct archive *r, struct archive *w)
 {
 	const void *buf;
@@ -295,6 +297,8 @@ fail:
 	return -1;
 }
 
+#endif /* RTE_APP_TEST_RESOURCE_TAR */
+
 void resource_register(struct resource *r)
 {
 	TAILQ_INSERT_TAIL(&resource_list, r, next);
diff --git a/app/test/test_mp_secondary.c b/app/test/test_mp_secondary.c
index 4dfe418..f66b68f 100644
--- a/app/test/test_mp_secondary.c
+++ b/app/test/test_mp_secondary.c
@@ -245,6 +245,7 @@ run_object_creation_tests(void)
 	printf("# Checked rte_lpm_create() OK\n");
 #endif
 
+#ifdef RTE_APP_TEST_RESOURCE_TAR
 	/* Run a test_pci call */
 	if (test_pci() != 0) {
 		printf("PCI scan failed in secondary\n");
@@ -252,6 +253,7 @@ run_object_creation_tests(void)
 			return -1;
 	} else
 		printf("PCI scan succeeded in secondary\n");
+#endif
 
 	return 0;
 }
@@ -266,9 +268,11 @@ test_mp_secondary(void)
 {
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
 		if (!test_pci_run) {
+#ifdef RTE_APP_TEST_RESOURCE_TAR
 			printf("=== Running pre-requisite test of test_pci\n");
 			test_pci();
 			printf("=== Requisite test done\n");
+#endif
 		}
 		return run_secondary_instances();
 	}
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
index 1e85040..39a6468 100644
--- a/app/test/test_resource.c
+++ b/app/test/test_resource.c
@@ -85,6 +85,7 @@ static int test_resource_c(void)
 	return 0;
 }
 
+#ifdef RTE_APP_TEST_RESOURCE_TAR
 REGISTER_LINKED_RESOURCE(test_resource_tar);
 
 static int test_resource_tar(void)
@@ -111,6 +112,8 @@ static int test_resource_tar(void)
 	return 0;
 }
 
+#endif /* RTE_APP_TEST_RESOURCE_TAR */
+
 static int test_resource(void)
 {
 	if (test_resource_dpdk())
@@ -119,8 +122,10 @@ static int test_resource(void)
 	if (test_resource_c())
 		return -1;
 
+#ifdef RTE_APP_TEST_RESOURCE_TAR
 	if (test_resource_tar())
 		return -1;
+#endif /* RTE_APP_TEST_RESOURCE_TAR */
 
 	return 0;
 }
diff --git a/config/common_base b/config/common_base
index 47c26f6..b9ba405 100644
--- a/config/common_base
+++ b/config/common_base
@@ -546,6 +546,7 @@ CONFIG_RTE_INSECURE_FUNCTION_WARNING=n
 # Compile the test application
 #
 CONFIG_RTE_APP_TEST=y
+CONFIG_RTE_APP_TEST_RESOURCE_TAR=n
 
 #
 # Compile the PMD test application
diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst
index 959709e..b321544 100644
--- a/doc/guides/linux_gsg/sys_reqs.rst
+++ b/doc/guides/linux_gsg/sys_reqs.rst
@@ -101,6 +101,9 @@ Compilation of the DPDK
 *   libpcap headers and libraries (libpcap-devel) to compile and use the libpcap-based poll-mode driver.
     This driver is disabled by default and can be enabled by setting ``CONFIG_RTE_LIBRTE_PMD_PCAP=y`` in the build time config file.
 
+*   libarchive headers and library are needed for some unit tests using tar to get their resources.
+
+
 Running DPDK Applications
 -------------------------
 
diff --git a/scripts/test-build.sh b/scripts/test-build.sh
index 48539c1..9a11f94 100755
--- a/scripts/test-build.sh
+++ b/scripts/test-build.sh
@@ -35,6 +35,7 @@ default_path=$PATH
 # Load config options:
 # - AESNI_MULTI_BUFFER_LIB_PATH
 # - DPDK_BUILD_TEST_CONFIGS (defconfig1+option1+option2 defconfig2)
+# - DPDK_DEP_ARCHIVE
 # - DPDK_DEP_CFLAGS
 # - DPDK_DEP_LDFLAGS
 # - DPDK_DEP_MOFED (y/[n])
@@ -111,6 +112,7 @@ reset_env ()
 {
 	export PATH=$default_path
 	unset CROSS
+	unset DPDK_DEP_ARCHIVE
 	unset DPDK_DEP_CFLAGS
 	unset DPDK_DEP_LDFLAGS
 	unset DPDK_DEP_MOFED
@@ -149,6 +151,8 @@ config () # <directory> <target> <options>
 		sed -ri         's,(PCI_CONFIG=)n,\1y,' $1/.config
 		sed -ri    's,(LIBRTE_IEEE1588=)n,\1y,' $1/.config
 		sed -ri             's,(BYPASS=)n,\1y,' $1/.config
+		test "$DPDK_DEP_ARCHIVE" != y || \
+		sed -ri       's,(RESOURCE_TAR=)n,\1y,' $1/.config
 		test "$DPDK_DEP_MOFED" != y || \
 		sed -ri           's,(MLX._PMD=)n,\1y,' $1/.config
 		test "$DPDK_DEP_SZE" != y || \
-- 
2.7.0

Re: [PATCH] config: make libarchive optional

[Jan Viktorin]

Hello Thomas,

On Tue, 14 Jun 2016 10:59:49 +0200
Thomas Monjalon <thomas.monjalon@6wind.com> wrote:

> The commit 66819e6 has introduced a dependency on libarchive to be able
> to use some tar resources in the unit tests.
> It is now an optional dependency because some systems do not have it
> installed.

I am surprised how big deal is this. So, let's live with this fact.
Thank you, Thomas, for proposing a solution.

> 
> If CONFIG_RTE_APP_TEST_RESOURCE_TAR is disabled, the PCI test will not
> be run. When a "configure" script will be integrated, the libarchive
> availability could be checked to automatically enable the option.
> 
> Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
> ---
>  app/test/Makefile                 | 11 +++++++----
>  app/test/resource.c               |  8 ++++++--
>  app/test/test_mp_secondary.c      |  4 ++++
>  app/test/test_resource.c          |  5 +++++
>  config/common_base                |  1 +
>  doc/guides/linux_gsg/sys_reqs.rst |  3 +++
>  scripts/test-build.sh             |  4 ++++
>  7 files changed, 30 insertions(+), 6 deletions(-)
> 
> diff --git a/app/test/Makefile b/app/test/Makefile
> index 7e4d484..5ca5c0b 100644
> --- a/app/test/Makefile
> +++ b/app/test/Makefile
> @@ -71,9 +71,6 @@ SRCS-y += test.c
>  SRCS-y += resource.c
>  SRCS-y += test_resource.c
>  $(eval $(call linked_resource,test_resource_c,resource.c))
> -$(eval $(call linked_tar_resource,test_resource_tar,test_resource.c))
> -SRCS-y += test_pci.c
> -$(eval $(call linked_tar_resource,test_pci_sysfs,test_pci_sysfs))
>  SRCS-y += test_prefetch.c
>  SRCS-y += test_byteorder.c
>  SRCS-y += test_per_lcore.c
> @@ -88,6 +85,13 @@ SRCS-y += test_ring.c
>  SRCS-y += test_ring_perf.c
>  SRCS-y += test_pmd_perf.c
>  
> +ifeq ($(CONFIG_RTE_APP_TEST_RESOURCE_TAR),y)
> +$(eval $(call linked_tar_resource,test_resource_tar,test_resource.c))
> +SRCS-y += test_pci.c
> +$(eval $(call linked_tar_resource,test_pci_sysfs,test_pci_sysfs))
> +LDLIBS += -larchive
> +endif
> +

I don't like this very much. I think, the linked_tar_resource can be
disabled at the place of its definition. What about:

ifeq ($(CONFIG_RTE_APP_TEST_RESOURCE_TAR),y)
define linked_tar_resource
...
endef
else
linked_tar_resource =
endif

...

SRCS-$(CONFIG_RTE_APP_TEST_RESOURCE_TAR) += test_pci.c

...

ifeq ($(CONFIG_RTE_APP_TEST_RESOURCE_TAR),y)
LDLIBS += -larchive
endif

>  ifeq ($(CONFIG_RTE_LIBRTE_TABLE),y)
>  SRCS-y += test_table.c
>  SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += test_table_pipeline.c
> @@ -194,7 +198,6 @@ CFLAGS += $(WERROR_FLAGS)
>  CFLAGS += -D_GNU_SOURCE
>  
>  LDLIBS += -lm
> -LDLIBS += -larchive
>  
>  # Disable VTA for memcpy test
>  ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y)
> diff --git a/app/test/resource.c b/app/test/resource.c
> index 8c42eea..0e2b62c 100644
> --- a/app/test/resource.c
> +++ b/app/test/resource.c
> @@ -33,8 +33,6 @@
>  
>  #include <stdio.h>
>  #include <string.h>
> -#include <archive.h>
> -#include <archive_entry.h>
>  #include <errno.h>
>  #include <sys/queue.h>
>  
> @@ -97,6 +95,10 @@ int resource_fwrite_file(const struct resource *r, const char *fname)
>  	return ret;
>  }
>  
> +#ifdef RTE_APP_TEST_RESOURCE_TAR
> +#include <archive.h>
> +#include <archive_entry.h>
> +
>  static int do_copy(struct archive *r, struct archive *w)
>  {
>  	const void *buf;
> @@ -295,6 +297,8 @@ fail:
>  	return -1;
>  }
>  
> +#endif /* RTE_APP_TEST_RESOURCE_TAR */
> +

This looks OK.

>  void resource_register(struct resource *r)
>  {
>  	TAILQ_INSERT_TAIL(&resource_list, r, next);
> diff --git a/app/test/test_mp_secondary.c b/app/test/test_mp_secondary.c
> index 4dfe418..f66b68f 100644
> --- a/app/test/test_mp_secondary.c
> +++ b/app/test/test_mp_secondary.c
> @@ -245,6 +245,7 @@ run_object_creation_tests(void)
>  	printf("# Checked rte_lpm_create() OK\n");
>  #endif
>  
> +#ifdef RTE_APP_TEST_RESOURCE_TAR
>  	/* Run a test_pci call */
>  	if (test_pci() != 0) {
>  		printf("PCI scan failed in secondary\n");
> @@ -252,6 +253,7 @@ run_object_creation_tests(void)
>  			return -1;
>  	} else
>  		printf("PCI scan succeeded in secondary\n");
> +#endif

Is it right to call a test from another test? I think this is
wrong... A user should first test the PCI and then the mp_seconday...
Or?

>  
>  	return 0;
>  }
> @@ -266,9 +268,11 @@ test_mp_secondary(void)
>  {
>  	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
>  		if (!test_pci_run) {
> +#ifdef RTE_APP_TEST_RESOURCE_TAR
>  			printf("=== Running pre-requisite test of test_pci\n");
>  			test_pci();
>  			printf("=== Requisite test done\n");
> +#endif

Similar here.

>  		}
>  		return run_secondary_instances();
>  	}
> diff --git a/app/test/test_resource.c b/app/test/test_resource.c
> index 1e85040..39a6468 100644
> --- a/app/test/test_resource.c
> +++ b/app/test/test_resource.c
> @@ -85,6 +85,7 @@ static int test_resource_c(void)
>  	return 0;
>  }
>  
> +#ifdef RTE_APP_TEST_RESOURCE_TAR
>  REGISTER_LINKED_RESOURCE(test_resource_tar);
>  
>  static int test_resource_tar(void)
> @@ -111,6 +112,8 @@ static int test_resource_tar(void)
>  	return 0;
>  }
>  
> +#endif /* RTE_APP_TEST_RESOURCE_TAR */
> +

This looks OK.

>  static int test_resource(void)
>  {
>  	if (test_resource_dpdk())
> @@ -119,8 +122,10 @@ static int test_resource(void)
>  	if (test_resource_c())
>  		return -1;
>  
> +#ifdef RTE_APP_TEST_RESOURCE_TAR
>  	if (test_resource_tar())
>  		return -1;
> +#endif /* RTE_APP_TEST_RESOURCE_TAR */

This looks OK.

>  
>  	return 0;
>  }
> diff --git a/config/common_base b/config/common_base
> index 47c26f6..b9ba405 100644
> --- a/config/common_base
> +++ b/config/common_base
> @@ -546,6 +546,7 @@ CONFIG_RTE_INSECURE_FUNCTION_WARNING=n
>  # Compile the test application
>  #
>  CONFIG_RTE_APP_TEST=y
> +CONFIG_RTE_APP_TEST_RESOURCE_TAR=n
>  
>  #
>  # Compile the PMD test application
> diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst
> index 959709e..b321544 100644
> --- a/doc/guides/linux_gsg/sys_reqs.rst
> +++ b/doc/guides/linux_gsg/sys_reqs.rst
> @@ -101,6 +101,9 @@ Compilation of the DPDK
>  *   libpcap headers and libraries (libpcap-devel) to compile and use the libpcap-based poll-mode driver.
>      This driver is disabled by default and can be enabled by setting ``CONFIG_RTE_LIBRTE_PMD_PCAP=y`` in the build time config file.
>  
> +*   libarchive headers and library are needed for some unit tests using tar to get their resources.
> +
> +
>  Running DPDK Applications
>  -------------------------
>  
> diff --git a/scripts/test-build.sh b/scripts/test-build.sh
> index 48539c1..9a11f94 100755
> --- a/scripts/test-build.sh
> +++ b/scripts/test-build.sh
> @@ -35,6 +35,7 @@ default_path=$PATH
>  # Load config options:
>  # - AESNI_MULTI_BUFFER_LIB_PATH
>  # - DPDK_BUILD_TEST_CONFIGS (defconfig1+option1+option2 defconfig2)
> +# - DPDK_DEP_ARCHIVE
>  # - DPDK_DEP_CFLAGS
>  # - DPDK_DEP_LDFLAGS
>  # - DPDK_DEP_MOFED (y/[n])
> @@ -111,6 +112,7 @@ reset_env ()
>  {
>  	export PATH=$default_path
>  	unset CROSS
> +	unset DPDK_DEP_ARCHIVE
>  	unset DPDK_DEP_CFLAGS
>  	unset DPDK_DEP_LDFLAGS
>  	unset DPDK_DEP_MOFED
> @@ -149,6 +151,8 @@ config () # <directory> <target> <options>
>  		sed -ri         's,(PCI_CONFIG=)n,\1y,' $1/.config
>  		sed -ri    's,(LIBRTE_IEEE1588=)n,\1y,' $1/.config
>  		sed -ri             's,(BYPASS=)n,\1y,' $1/.config
> +		test "$DPDK_DEP_ARCHIVE" != y || \
> +		sed -ri       's,(RESOURCE_TAR=)n,\1y,' $1/.config
>  		test "$DPDK_DEP_MOFED" != y || \
>  		sed -ri           's,(MLX._PMD=)n,\1y,' $1/.config
>  		test "$DPDK_DEP_SZE" != y || \

This look OK.

Regards
Jan

Re: [PATCH] config: make libarchive optional

[Thomas Monjalon]

2016-06-14 11:33, Jan Viktorin:
> Thomas Monjalon <thomas.monjalon@6wind.com> wrote:
> > +ifeq ($(CONFIG_RTE_APP_TEST_RESOURCE_TAR),y)
> > +$(eval $(call linked_tar_resource,test_resource_tar,test_resource.c))
> > +SRCS-y += test_pci.c
> > +$(eval $(call linked_tar_resource,test_pci_sysfs,test_pci_sysfs))
> > +LDLIBS += -larchive
> > +endif
> 
> I don't like this very much. I think, the linked_tar_resource can be
> disabled at the place of its definition. What about:
> 
> ifeq ($(CONFIG_RTE_APP_TEST_RESOURCE_TAR),y)
> define linked_tar_resource
> ...
> endef
> else
> linked_tar_resource =
> endif
> 
> ...
> 
> SRCS-$(CONFIG_RTE_APP_TEST_RESOURCE_TAR) += test_pci.c
> 
> ...
> 
> ifeq ($(CONFIG_RTE_APP_TEST_RESOURCE_TAR),y)
> LDLIBS += -larchive
> endif

Yes, that's better.

> > --- a/app/test/test_mp_secondary.c
> > +++ b/app/test/test_mp_secondary.c
> > @@ -245,6 +245,7 @@ run_object_creation_tests(void)
> >  	printf("# Checked rte_lpm_create() OK\n");
> >  #endif
> >  
> > +#ifdef RTE_APP_TEST_RESOURCE_TAR
> >  	/* Run a test_pci call */
> >  	if (test_pci() != 0) {
> >  		printf("PCI scan failed in secondary\n");
> > @@ -252,6 +253,7 @@ run_object_creation_tests(void)
> >  			return -1;
> >  	} else
> >  		printf("PCI scan succeeded in secondary\n");
> > +#endif
> 
> Is it right to call a test from another test? I think this is
> wrong... A user should first test the PCI and then the mp_seconday...
> Or?

:)
This is out of scope for this patch.
Yes this is a part of app/test/ which could be improved a lot.
We don't even have a maintainer for the autotest architecture.

[PATCH v2] config: make libarchive optional

[Thomas Monjalon]

The commit 66819e6 has introduced a dependency on libarchive to be able
to use some tar resources in the unit tests.
It is now an optional dependency because some systems do not have it
installed.

If CONFIG_RTE_APP_TEST_RESOURCE_TAR is disabled, the PCI test will not
be run. When a "configure" script will be integrated, the libarchive
availability could be checked to automatically enable the option.

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
---
v2: define empty macro linked_tar_resource if disabled
---
 app/test/Makefile                 | 12 +++++++++---
 app/test/resource.c               |  8 ++++++--
 app/test/test_mp_secondary.c      |  4 ++++
 app/test/test_resource.c          |  5 +++++
 config/common_base                |  1 +
 doc/guides/linux_gsg/sys_reqs.rst |  3 +++
 scripts/test-build.sh             |  4 ++++
 7 files changed, 32 insertions(+), 5 deletions(-)

diff --git a/app/test/Makefile b/app/test/Makefile
index 7e4d484..053f3a2 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -51,12 +51,15 @@ $(1).res.o: $(2)
 		/dev/stdin $$@ < $$<
 endef
 
+ifeq ($(CONFIG_RTE_APP_TEST_RESOURCE_TAR),y)
 define linked_tar_resource
 $(1).tar: $(2)
 	tar -C $$(dir $$<) -cf $$@ $$(notdir $$<)
-
 $(call linked_resource,$(1),$(1).tar)
 endef
+else # ! CONFIG_RTE_APP_TEST_RESOURCE_TAR
+linked_tar_resource =
+endif # CONFIG_RTE_APP_TEST_RESOURCE_TAR
 
 #
 # library name
@@ -72,7 +75,7 @@ SRCS-y += resource.c
 SRCS-y += test_resource.c
 $(eval $(call linked_resource,test_resource_c,resource.c))
 $(eval $(call linked_tar_resource,test_resource_tar,test_resource.c))
-SRCS-y += test_pci.c
+SRCS-$(CONFIG_RTE_APP_TEST_RESOURCE_TAR) += test_pci.c
 $(eval $(call linked_tar_resource,test_pci_sysfs,test_pci_sysfs))
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
@@ -194,7 +197,6 @@ CFLAGS += $(WERROR_FLAGS)
 CFLAGS += -D_GNU_SOURCE
 
 LDLIBS += -lm
-LDLIBS += -larchive
 
 # Disable VTA for memcpy test
 ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y)
@@ -218,6 +220,10 @@ endif
 endif
 endif
 
+ifeq ($(CONFIG_RTE_APP_TEST_RESOURCE_TAR),y)
+LDLIBS += -larchive
+endif
+
 include $(RTE_SDK)/mk/rte.app.mk
 
 endif
diff --git a/app/test/resource.c b/app/test/resource.c
index 8c42eea..0e2b62c 100644
--- a/app/test/resource.c
+++ b/app/test/resource.c
@@ -33,8 +33,6 @@
 
 #include <stdio.h>
 #include <string.h>
-#include <archive.h>
-#include <archive_entry.h>
 #include <errno.h>
 #include <sys/queue.h>
 
@@ -97,6 +95,10 @@ int resource_fwrite_file(const struct resource *r, const char *fname)
 	return ret;
 }
 
+#ifdef RTE_APP_TEST_RESOURCE_TAR
+#include <archive.h>
+#include <archive_entry.h>
+
 static int do_copy(struct archive *r, struct archive *w)
 {
 	const void *buf;
@@ -295,6 +297,8 @@ fail:
 	return -1;
 }
 
+#endif /* RTE_APP_TEST_RESOURCE_TAR */
+
 void resource_register(struct resource *r)
 {
 	TAILQ_INSERT_TAIL(&resource_list, r, next);
diff --git a/app/test/test_mp_secondary.c b/app/test/test_mp_secondary.c
index 4dfe418..f66b68f 100644
--- a/app/test/test_mp_secondary.c
+++ b/app/test/test_mp_secondary.c
@@ -245,6 +245,7 @@ run_object_creation_tests(void)
 	printf("# Checked rte_lpm_create() OK\n");
 #endif
 
+#ifdef RTE_APP_TEST_RESOURCE_TAR
 	/* Run a test_pci call */
 	if (test_pci() != 0) {
 		printf("PCI scan failed in secondary\n");
@@ -252,6 +253,7 @@ run_object_creation_tests(void)
 			return -1;
 	} else
 		printf("PCI scan succeeded in secondary\n");
+#endif
 
 	return 0;
 }
@@ -266,9 +268,11 @@ test_mp_secondary(void)
 {
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
 		if (!test_pci_run) {
+#ifdef RTE_APP_TEST_RESOURCE_TAR
 			printf("=== Running pre-requisite test of test_pci\n");
 			test_pci();
 			printf("=== Requisite test done\n");
+#endif
 		}
 		return run_secondary_instances();
 	}
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
index 1e85040..39a6468 100644
--- a/app/test/test_resource.c
+++ b/app/test/test_resource.c
@@ -85,6 +85,7 @@ static int test_resource_c(void)
 	return 0;
 }
 
+#ifdef RTE_APP_TEST_RESOURCE_TAR
 REGISTER_LINKED_RESOURCE(test_resource_tar);
 
 static int test_resource_tar(void)
@@ -111,6 +112,8 @@ static int test_resource_tar(void)
 	return 0;
 }
 
+#endif /* RTE_APP_TEST_RESOURCE_TAR */
+
 static int test_resource(void)
 {
 	if (test_resource_dpdk())
@@ -119,8 +122,10 @@ static int test_resource(void)
 	if (test_resource_c())
 		return -1;
 
+#ifdef RTE_APP_TEST_RESOURCE_TAR
 	if (test_resource_tar())
 		return -1;
+#endif /* RTE_APP_TEST_RESOURCE_TAR */
 
 	return 0;
 }
diff --git a/config/common_base b/config/common_base
index 47c26f6..b9ba405 100644
--- a/config/common_base
+++ b/config/common_base
@@ -546,6 +546,7 @@ CONFIG_RTE_INSECURE_FUNCTION_WARNING=n
 # Compile the test application
 #
 CONFIG_RTE_APP_TEST=y
+CONFIG_RTE_APP_TEST_RESOURCE_TAR=n
 
 #
 # Compile the PMD test application
diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst
index 959709e..b321544 100644
--- a/doc/guides/linux_gsg/sys_reqs.rst
+++ b/doc/guides/linux_gsg/sys_reqs.rst
@@ -101,6 +101,9 @@ Compilation of the DPDK
 *   libpcap headers and libraries (libpcap-devel) to compile and use the libpcap-based poll-mode driver.
     This driver is disabled by default and can be enabled by setting ``CONFIG_RTE_LIBRTE_PMD_PCAP=y`` in the build time config file.
 
+*   libarchive headers and library are needed for some unit tests using tar to get their resources.
+
+
 Running DPDK Applications
 -------------------------
 
diff --git a/scripts/test-build.sh b/scripts/test-build.sh
index 48539c1..9a11f94 100755
--- a/scripts/test-build.sh
+++ b/scripts/test-build.sh
@@ -35,6 +35,7 @@ default_path=$PATH
 # Load config options:
 # - AESNI_MULTI_BUFFER_LIB_PATH
 # - DPDK_BUILD_TEST_CONFIGS (defconfig1+option1+option2 defconfig2)
+# - DPDK_DEP_ARCHIVE
 # - DPDK_DEP_CFLAGS
 # - DPDK_DEP_LDFLAGS
 # - DPDK_DEP_MOFED (y/[n])
@@ -111,6 +112,7 @@ reset_env ()
 {
 	export PATH=$default_path
 	unset CROSS
+	unset DPDK_DEP_ARCHIVE
 	unset DPDK_DEP_CFLAGS
 	unset DPDK_DEP_LDFLAGS
 	unset DPDK_DEP_MOFED
@@ -149,6 +151,8 @@ config () # <directory> <target> <options>
 		sed -ri         's,(PCI_CONFIG=)n,\1y,' $1/.config
 		sed -ri    's,(LIBRTE_IEEE1588=)n,\1y,' $1/.config
 		sed -ri             's,(BYPASS=)n,\1y,' $1/.config
+		test "$DPDK_DEP_ARCHIVE" != y || \
+		sed -ri       's,(RESOURCE_TAR=)n,\1y,' $1/.config
 		test "$DPDK_DEP_MOFED" != y || \
 		sed -ri           's,(MLX._PMD=)n,\1y,' $1/.config
 		test "$DPDK_DEP_SZE" != y || \
-- 
2.7.0

Re: [PATCH v2] config: make libarchive optional

[Jan Viktorin]

On Tue, 14 Jun 2016 12:01:56 +0200
Thomas Monjalon <thomas.monjalon@6wind.com> wrote:

> The commit 66819e6 has introduced a dependency on libarchive to be able
> to use some tar resources in the unit tests.
> It is now an optional dependency because some systems do not have it
> installed.
> 
> If CONFIG_RTE_APP_TEST_RESOURCE_TAR is disabled, the PCI test will not
> be run. When a "configure" script will be integrated, the libarchive
> availability could be checked to automatically enable the option.
> 
> Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
> ---
> v2: define empty macro linked_tar_resource if disabled
> ---
Reviewed-by: Jan Viktorin <viktorin@rehivetech.com>
Acked-by: Jan Viktorin <viktorin@rehivetech.com>

Re: [PATCH v2] config: make libarchive optional

[Thomas Monjalon]

2016-06-14 14:53, Jan Viktorin:
> On Tue, 14 Jun 2016 12:01:56 +0200
> Thomas Monjalon <thomas.monjalon@6wind.com> wrote:
> 
> > The commit 66819e6 has introduced a dependency on libarchive to be able
> > to use some tar resources in the unit tests.
> > It is now an optional dependency because some systems do not have it
> > installed.
> > 
> > If CONFIG_RTE_APP_TEST_RESOURCE_TAR is disabled, the PCI test will not
> > be run. When a "configure" script will be integrated, the libarchive
> > availability could be checked to automatically enable the option.
> > 
> > Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
> > ---
> > v2: define empty macro linked_tar_resource if disabled
> > ---
> Reviewed-by: Jan Viktorin <viktorin@rehivetech.com>
> Acked-by: Jan Viktorin <viktorin@rehivetech.com>

Applied

[PATCH v5 01/10] app/test: introduce resources for tests

[Jan Viktorin]

Certain internal mechanisms of DPDK access different file system
structures (e.g. /sys/bus/pci/devices). It is difficult to test
those cases automatically by a unit test when such path is not
hard-coded and there is no simple way how to distribute fake ones
with the current testing environment.

This patch adds a possibility to declare a resource embedded in
the test binary itself. The structure resource cover the generic
situation - it provides a name for lookup and pointers to the
embedded data blob. A resource is registered in a constructor by
the macro REGISTER_RESOURCE.

Some initial tests of simple resources is included and added into
the group_1.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v3:
* fixed doc comments
---
 app/test/Makefile         |  2 +
 app/test/autotest_data.py |  6 +++
 app/test/resource.c       | 66 +++++++++++++++++++++++++++++++
 app/test/resource.h       | 98 +++++++++++++++++++++++++++++++++++++++++++++++
 app/test/test_resource.c  | 75 ++++++++++++++++++++++++++++++++++++
 5 files changed, 247 insertions(+)
 create mode 100644 app/test/resource.c
 create mode 100644 app/test/resource.h
 create mode 100644 app/test/test_resource.c

diff --git a/app/test/Makefile b/app/test/Makefile
index 6df9c40..1a13fb2 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -43,6 +43,8 @@ APP = test
 #
 SRCS-$(CONFIG_RTE_LIBRTE_CMDLINE) := commands.c
 SRCS-y += test.c
+SRCS-y += resource.c
+SRCS-y += test_resource.c
 SRCS-y += test_pci.c
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
diff --git a/app/test/autotest_data.py b/app/test/autotest_data.py
index 6c87809..1e6b422 100644
--- a/app/test/autotest_data.py
+++ b/app/test/autotest_data.py
@@ -94,6 +94,12 @@ parallel_test_group_list = [
 		 "Report" :	None,
 		},
 		{
+		 "Name" :	"Resource autotest",
+		 "Command" :	"resource_autotest",
+		 "Func" :	default_autotest,
+		 "Report" :	None,
+		},
+		{
 		 "Name" :	"Dump rings",
 		 "Command" :	"dump_ring",
 		 "Func" :	dump_autotest,
diff --git a/app/test/resource.c b/app/test/resource.c
new file mode 100644
index 0000000..30513db
--- /dev/null
+++ b/app/test/resource.c
@@ -0,0 +1,66 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2016 RehiveTech. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of RehiveTech nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <string.h>
+#include <errno.h>
+#include <sys/queue.h>
+
+#include <rte_debug.h>
+
+#include "resource.h"
+
+struct resource_list resource_list = TAILQ_HEAD_INITIALIZER(resource_list);
+
+size_t resource_size(const struct resource *r)
+{
+	return r->end - r->begin;
+}
+
+const struct resource *resource_find(const char *name)
+{
+	struct resource *r;
+
+	TAILQ_FOREACH(r, &resource_list, next) {
+		RTE_VERIFY(r->name);
+
+		if (!strcmp(r->name, name))
+			return r;
+	}
+
+	return NULL;
+}
+
+void resource_register(struct resource *r)
+{
+	TAILQ_INSERT_TAIL(&resource_list, r, next);
+}
diff --git a/app/test/resource.h b/app/test/resource.h
new file mode 100644
index 0000000..9af8415
--- /dev/null
+++ b/app/test/resource.h
@@ -0,0 +1,98 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2016 RehiveTech. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of RehiveTech nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RESOURCE_H_
+#define _RESOURCE_H_
+
+/**
+ * @file
+ *
+ * Test Resource API
+ *
+ * Each test can require and use some external resources. Usually, an external
+ * resource is a file or a filesystem sub-hierarchy. A resource is included
+ * inside the test executable.
+ */
+
+#include <sys/queue.h>
+#include <stddef.h>
+
+#include <rte_eal.h>
+#include <rte_common.h>
+
+TAILQ_HEAD(resource_list, resource);
+extern struct resource_list resource_list;
+
+/**
+ * Representation of a resource. It points to the resource's binary data.
+ * The semantics of the binary data are defined by the target test.
+ */
+struct resource {
+	const char *name;  /**< Unique name of the resource */
+	const char *begin; /**< Start of resource data */
+	const char *end;   /**< End of resource data */
+	TAILQ_ENTRY(resource) next;
+};
+
+/**
+ * @return size of the given resource
+ */
+size_t resource_size(const struct resource *r);
+
+/**
+ * Find a resource by name in the global list of resources.
+ */
+const struct resource *resource_find(const char *name);
+
+/**
+ * Register a resource in the global list of resources.
+ * Not intended for direct use, please check the REGISTER_RESOURCE
+ * macro.
+ */
+void resource_register(struct resource *r);
+
+/**
+ * Definition of a resource described by its name, and pointers begin, end.
+ */
+#define REGISTER_RESOURCE(n, b, e) \
+static struct resource linkres_ ##n = {       \
+	.name = RTE_STR(n),     \
+	.begin = b,             \
+	.end = e,               \
+};                              \
+static void __attribute__((constructor, used)) resinitfn_ ##n(void) \
+{                               \
+	resource_register(&linkres_ ##n);  \
+}
+
+#endif
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
new file mode 100644
index 0000000..69391ad
--- /dev/null
+++ b/app/test/test_resource.c
@@ -0,0 +1,75 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2016 RehiveTech. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of RehiveTech nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <stdio.h>
+#include <string.h>
+
+#include "test.h"
+#include "resource.h"
+
+const char test_resource_dpdk_blob[] = {
+	'\x44', '\x50', '\x44', '\x4b', '\x00'
+};
+
+REGISTER_RESOURCE(test_resource_dpdk,
+		test_resource_dpdk_blob, test_resource_dpdk_blob + 4);
+
+static int test_resource_dpdk(void)
+{
+	const struct resource *r;
+
+	r = resource_find("test_resource_dpdk");
+	TEST_ASSERT_NOT_NULL(r, "Could not find test_resource_dpdk");
+	TEST_ASSERT(!strcmp(r->name, "test_resource_dpdk"),
+			"Found resource %s, expected test_resource_dpdk",
+			r->name);
+
+	TEST_ASSERT(!strncmp("DPDK", r->begin, 4),
+			"Unexpected payload: %.4s...", r->begin);
+
+	return 0;
+}
+
+static int test_resource(void)
+{
+	if (test_resource_dpdk())
+		return -1;
+
+	return 0;
+}
+
+static struct test_command resource_cmd = {
+	.command = "resource_autotest",
+	.callback = test_resource,
+};
+REGISTER_TEST_COMMAND(resource_cmd);
-- 
2.8.0

[PATCH v5 02/10] mk: define objcopy-specific target and arch

[Jan Viktorin]

The program objcopy uses non-standard conventions to name the
target and arch. Define the values for supported architecturesi
(tile and ppc_64 are missing).

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v4:
* fixed commit message (removed FIXME)
---
 mk/arch/arm/rte.vars.mk     | 5 +++++
 mk/arch/arm64/rte.vars.mk   | 5 +++++
 mk/arch/i686/rte.vars.mk    | 5 +++++
 mk/arch/x86_64/rte.vars.mk  | 5 +++++
 mk/arch/x86_x32/rte.vars.mk | 5 +++++
 5 files changed, 25 insertions(+)

diff --git a/mk/arch/arm/rte.vars.mk b/mk/arch/arm/rte.vars.mk
index bd85140..2f8cf7c 100644
--- a/mk/arch/arm/rte.vars.mk
+++ b/mk/arch/arm/rte.vars.mk
@@ -37,3 +37,8 @@ CPU_LDFLAGS ?=
 CPU_ASFLAGS ?= -felf
 
 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf32-littlearm
+RTE_OBJCOPY_ARCH = arm
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
diff --git a/mk/arch/arm64/rte.vars.mk b/mk/arch/arm64/rte.vars.mk
index 32e3a5f..c168426 100644
--- a/mk/arch/arm64/rte.vars.mk
+++ b/mk/arch/arm64/rte.vars.mk
@@ -56,3 +56,8 @@ CPU_LDFLAGS ?=
 CPU_ASFLAGS ?= -felf
 
 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf64-littleaarch64
+RTE_OBJCOPY_ARCH = aarch64
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
diff --git a/mk/arch/i686/rte.vars.mk b/mk/arch/i686/rte.vars.mk
index 8ba9a23..6a25312 100644
--- a/mk/arch/i686/rte.vars.mk
+++ b/mk/arch/i686/rte.vars.mk
@@ -57,3 +57,8 @@ CPU_LDFLAGS ?= -melf_i386
 CPU_ASFLAGS ?= -felf
 
 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf32-i386
+RTE_OBJCOPY_ARCH = i386
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
diff --git a/mk/arch/x86_64/rte.vars.mk b/mk/arch/x86_64/rte.vars.mk
index b986f04..83723c8 100644
--- a/mk/arch/x86_64/rte.vars.mk
+++ b/mk/arch/x86_64/rte.vars.mk
@@ -57,3 +57,8 @@ CPU_LDFLAGS ?=
 CPU_ASFLAGS ?= -felf64
 
 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf64-x86-64
+RTE_OBJCOPY_ARCH = i386:x86-64
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
diff --git a/mk/arch/x86_x32/rte.vars.mk b/mk/arch/x86_x32/rte.vars.mk
index 3103dfc..676f316 100644
--- a/mk/arch/x86_x32/rte.vars.mk
+++ b/mk/arch/x86_x32/rte.vars.mk
@@ -61,3 +61,8 @@ ifneq ($(shell echo | $(CC) $(CPU_CFLAGS) -E - 2>/dev/null 1>/dev/null && echo 0
 endif
 
 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf32-x86-64
+RTE_OBJCOPY_ARCH = i386:x86-64
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
-- 
2.8.0

[PATCH v5 03/10] app/test: support resources externally linked

[Jan Viktorin]

To include resources from other source that the C source code we
can take advantage of the objcopy behaviour, i.e. packing of an
arbitrary file as an object file that is linked to the target program.

A linked object file is always accessible as a pair

extern const char beg_<name>;
extern const char end_<name>;
(extern const char siz_<name>;)

A unit test that packs the resource.c source file is included.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v4
* fixed trailing semicolon in REGISTER_LINKED_RESOURCE
---
 app/test/Makefile        | 19 +++++++++++++++++++
 app/test/resource.h      | 10 ++++++++++
 app/test/test_resource.c | 18 ++++++++++++++++++
 3 files changed, 47 insertions(+)

diff --git a/app/test/Makefile b/app/test/Makefile
index 1a13fb2..fe67eee 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -33,6 +33,24 @@ include $(RTE_SDK)/mk/rte.vars.mk
 
 ifeq ($(CONFIG_RTE_APP_TEST),y)
 
+# Define an externally linked resource. A linked resource is an arbitrary
+# file that is linked into the test binary. The application refers to this
+# resource by name. The linked generates identifiers beg_<name> and end_<name>
+# for referencing by the C code.
+#
+# Parameters: <unique name>, <file to be linked>
+define linked_resource
+SRCS-y += $(1).res.o
+$(1).res.o: $(2)
+	$(OBJCOPY) -I binary -B $(RTE_OBJCOPY_ARCH) -O $(RTE_OBJCOPY_TARGET) \
+		--rename-section                                         \
+			.data=.rodata,alloc,load,data,contents,readonly  \
+		--redefine-sym _binary__dev_stdin_start=beg_$(1)         \
+		--redefine-sym _binary__dev_stdin_end=end_$(1)           \
+		--redefine-sym _binary__dev_stdin_size=siz_$(1)          \
+		/dev/stdin $$@ < $$<
+endef
+
 #
 # library name
 #
@@ -45,6 +63,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_CMDLINE) := commands.c
 SRCS-y += test.c
 SRCS-y += resource.c
 SRCS-y += test_resource.c
+$(eval $(call linked_resource,test_resource_c,resource.c))
 SRCS-y += test_pci.c
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
diff --git a/app/test/resource.h b/app/test/resource.h
index 9af8415..966fc24 100644
--- a/app/test/resource.h
+++ b/app/test/resource.h
@@ -82,6 +82,16 @@ const struct resource *resource_find(const char *name);
 void resource_register(struct resource *r);
 
 /**
+ * Definition of a resource linked externally (by means of the used toolchain).
+ * Only the base name of the resource is expected. The name refers to the
+ * linked pointers beg_<name> and end_<name> provided externally.
+ */
+#define REGISTER_LINKED_RESOURCE(n) \
+extern const char beg_ ##n;         \
+extern const char end_ ##n;         \
+REGISTER_RESOURCE(n, &beg_ ##n, &end_ ##n) \
+
+/**
  * Definition of a resource described by its name, and pointers begin, end.
  */
 #define REGISTER_RESOURCE(n, b, e) \
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
index 69391ad..b397fa8 100644
--- a/app/test/test_resource.c
+++ b/app/test/test_resource.c
@@ -60,11 +60,29 @@ static int test_resource_dpdk(void)
 	return 0;
 }
 
+REGISTER_LINKED_RESOURCE(test_resource_c);
+
+static int test_resource_c(void)
+{
+	const struct resource *r;
+
+	r = resource_find("test_resource_c");
+	TEST_ASSERT_NOT_NULL(r, "No test_resource_c found");
+	TEST_ASSERT(!strcmp(r->name, "test_resource_c"),
+			"Found resource %s, expected test_resource_c",
+			r->name);
+
+	return 0;
+}
+
 static int test_resource(void)
 {
 	if (test_resource_dpdk())
 		return -1;
 
+	if (test_resource_c())
+		return -1;
+
 	return 0;
 }
 
-- 
2.8.0

[PATCH v5 04/10] app/test: add functions to create files from resources

[Jan Viktorin]

A resource can be written into the target filesystem by calling
resource_fwrite or resource_fwrite_file. Such file can be created
before a test is started and removed after the test finishes.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/resource.c      | 35 +++++++++++++++++++++++++++++++++++
 app/test/resource.h      | 14 ++++++++++++++
 app/test/test_resource.c | 10 ++++++++++
 3 files changed, 59 insertions(+)

diff --git a/app/test/resource.c b/app/test/resource.c
index 30513db..acb63c1 100644
--- a/app/test/resource.c
+++ b/app/test/resource.c
@@ -31,6 +31,7 @@
  *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+#include <stdio.h>
 #include <string.h>
 #include <errno.h>
 #include <sys/queue.h>
@@ -60,6 +61,40 @@ const struct resource *resource_find(const char *name)
 	return NULL;
 }
 
+int resource_fwrite(const struct resource *r, FILE *f)
+{
+	const size_t goal = resource_size(r);
+	size_t total = 0;
+
+	while (total < goal) {
+		size_t wlen = fwrite(r->begin + total, 1, goal - total, f);
+		if (wlen == 0) {
+			perror(__func__);
+			return -1;
+		}
+
+		total += wlen;
+	}
+
+	return 0;
+}
+
+int resource_fwrite_file(const struct resource *r, const char *fname)
+{
+	FILE *f;
+	int ret;
+
+	f = fopen(fname, "w");
+	if (f == NULL) {
+		perror(__func__);
+		return -1;
+	}
+
+	ret = resource_fwrite(r, f);
+	fclose(f);
+	return ret;
+}
+
 void resource_register(struct resource *r)
 {
 	TAILQ_INSERT_TAIL(&resource_list, r, next);
diff --git a/app/test/resource.h b/app/test/resource.h
index 966fc24..ac9cae6 100644
--- a/app/test/resource.h
+++ b/app/test/resource.h
@@ -45,6 +45,7 @@
  */
 
 #include <sys/queue.h>
+#include <stdio.h>
 #include <stddef.h>
 
 #include <rte_eal.h>
@@ -75,6 +76,19 @@ size_t resource_size(const struct resource *r);
 const struct resource *resource_find(const char *name);
 
 /**
+ * Write the raw data of the resource to the given file.
+ * @return 0 on success
+ */
+int resource_fwrite(const struct resource *r, FILE *f);
+
+/**
+ * Write the raw data of the resource to the given file given by name.
+ * The name is relative to the current working directory.
+ * @return 0 on success
+ */
+int resource_fwrite_file(const struct resource *r, const char *fname);
+
+/**
  * Register a resource in the global list of resources.
  * Not intended for direct use, please check the REGISTER_RESOURCE
  * macro.
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
index b397fa8..3d1bf00 100644
--- a/app/test/test_resource.c
+++ b/app/test/test_resource.c
@@ -65,6 +65,7 @@ REGISTER_LINKED_RESOURCE(test_resource_c);
 static int test_resource_c(void)
 {
 	const struct resource *r;
+	FILE *f;
 
 	r = resource_find("test_resource_c");
 	TEST_ASSERT_NOT_NULL(r, "No test_resource_c found");
@@ -72,6 +73,15 @@ static int test_resource_c(void)
 			"Found resource %s, expected test_resource_c",
 			r->name);
 
+	TEST_ASSERT_SUCCESS(resource_fwrite_file(r, "test_resource.c"),
+			"Failed to to write file %s", r->name);
+
+	f = fopen("test_resource.c", "r");
+	TEST_ASSERT_NOT_NULL(f,
+			"Missing extracted file resource.c");
+	fclose(f);
+	remove("test_resource.c");
+
 	return 0;
 }
 
-- 
2.8.0

[PATCH v5 05/10] app/test: support resources archived by tar

[Jan Viktorin]

When a more complex resource (a file hierarchy) is needed, packing
every single file as a single resource would be very ineffective. For
that purpose, it is possible to pack the files into a tar archive,
extract it before test from the resource and finally clean up all the
created files.

This patch introduces functions resource_untar and resource_rm_by_tar
to perform those tasks. An example of using those functions is included
as a test.

A new dependency is required to build the app/test: libarchive.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v5:
* fix archive_read_data_block, more info:
  https://github.com/libarchive/libarchive/blob/master/examples/untar.c#L205
  http://markmail.org/message/y4pgi44gqjd4jfuc
---
 app/test/Makefile        |   9 +++
 app/test/resource.c      | 200 +++++++++++++++++++++++++++++++++++++++++++++++
 app/test/resource.h      |  13 +++
 app/test/test_resource.c |  29 +++++++
 4 files changed, 251 insertions(+)

diff --git a/app/test/Makefile b/app/test/Makefile
index fe67eee..1cc6258 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -51,6 +51,13 @@ $(1).res.o: $(2)
 		/dev/stdin $$@ < $$<
 endef
 
+define linked_tar_resource
+$(1).tar: $(2)
+	tar -C $$(dir $$<) -cf $$@ $$(notdir $$<)
+
+$(call linked_resource,$(1),$(1).tar)
+endef
+
 #
 # library name
 #
@@ -64,6 +71,7 @@ SRCS-y += test.c
 SRCS-y += resource.c
 SRCS-y += test_resource.c
 $(eval $(call linked_resource,test_resource_c,resource.c))
+$(eval $(call linked_tar_resource,test_resource_tar,test_resource.c))
 SRCS-y += test_pci.c
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
@@ -185,6 +193,7 @@ CFLAGS += $(WERROR_FLAGS)
 CFLAGS += -D_GNU_SOURCE
 
 LDLIBS += -lm
+LDLIBS += -larchive
 
 # Disable VTA for memcpy test
 ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y)
diff --git a/app/test/resource.c b/app/test/resource.c
index acb63c1..8c42eea 100644
--- a/app/test/resource.c
+++ b/app/test/resource.c
@@ -33,6 +33,8 @@
 
 #include <stdio.h>
 #include <string.h>
+#include <archive.h>
+#include <archive_entry.h>
 #include <errno.h>
 #include <sys/queue.h>
 
@@ -95,6 +97,204 @@ int resource_fwrite_file(const struct resource *r, const char *fname)
 	return ret;
 }
 
+static int do_copy(struct archive *r, struct archive *w)
+{
+	const void *buf;
+	size_t len;
+#if ARCHIVE_VERSION_NUMBER >= 3000000
+	int64_t off;
+#else
+	off_t off;
+#endif
+	int ret;
+
+	while (1) {
+		ret = archive_read_data_block(r, &buf, &len, &off);
+		if (ret == ARCHIVE_RETRY)
+			continue;
+
+		if (ret == ARCHIVE_EOF)
+			return 0;
+
+		if (ret != ARCHIVE_OK)
+			return ret;
+
+		do {
+			ret = archive_write_data_block(w, buf, len, off);
+			if (ret != ARCHIVE_OK && ret != ARCHIVE_RETRY)
+				return ret;
+		} while (ret != ARCHIVE_OK);
+	}
+}
+
+int resource_untar(const struct resource *res)
+{
+	struct archive *r;
+	struct archive *w;
+	struct archive_entry *e;
+	void *p;
+	int flags = 0;
+	int ret;
+
+	p = malloc(resource_size(res));
+	if (p == NULL)
+		rte_panic("Failed to malloc %zu B\n", resource_size(res));
+
+	memcpy(p, res->begin, resource_size(res));
+
+	r = archive_read_new();
+	if (r == NULL) {
+		free(p);
+		return -1;
+	}
+
+	archive_read_support_format_all(r);
+	archive_read_support_filter_all(r);
+
+	w = archive_write_disk_new();
+	if (w == NULL) {
+		archive_read_free(r);
+		free(p);
+		return -1;
+	}
+
+	flags |= ARCHIVE_EXTRACT_PERM;
+	flags |= ARCHIVE_EXTRACT_FFLAGS;
+	archive_write_disk_set_options(w, flags);
+	archive_write_disk_set_standard_lookup(w);
+
+	ret = archive_read_open_memory(r, p, resource_size(res));
+	if (ret != ARCHIVE_OK)
+		goto fail;
+
+	while (1) {
+		ret = archive_read_next_header(r, &e);
+		if (ret == ARCHIVE_EOF)
+			break;
+		if (ret != ARCHIVE_OK)
+			goto fail;
+
+		ret = archive_write_header(w, e);
+		if (ret == ARCHIVE_EOF)
+			break;
+		if (ret != ARCHIVE_OK)
+			goto fail;
+
+		if (archive_entry_size(e) == 0)
+			continue;
+
+		ret = do_copy(r, w);
+		if (ret != ARCHIVE_OK)
+			goto fail;
+
+		ret = archive_write_finish_entry(w);
+		if (ret != ARCHIVE_OK)
+			goto fail;
+	}
+
+	archive_write_free(w);
+	archive_read_free(r);
+	free(p);
+	return 0;
+
+fail:
+	archive_write_free(w);
+	archive_read_free(r);
+	free(p);
+	rte_panic("Failed: %s\n", archive_error_string(r));
+	return -1;
+}
+
+int resource_rm_by_tar(const struct resource *res)
+{
+	struct archive *r;
+	struct archive_entry *e;
+	void *p;
+	int try_again = 1;
+	int attempts = 0;
+	int ret;
+
+	p = malloc(resource_size(res));
+	if (p == NULL)
+		rte_panic("Failed to malloc %zu B\n", resource_size(res));
+
+	memcpy(p, res->begin, resource_size(res));
+
+	/*
+	 * If somebody creates a file somewhere inside the extracted TAR
+	 * hierarchy during a test the resource_rm_by_tar might loop
+	 * infinitely. We prevent this by adding the attempts counter there.
+	 * In normal case, max N iteration is done where N is the depth of
+	 * the file-hierarchy.
+	 */
+	while (try_again && attempts < 10000) {
+		r = archive_read_new();
+		if (r == NULL) {
+			free(p);
+			return -1;
+		}
+
+		archive_read_support_format_all(r);
+		archive_read_support_filter_all(r);
+
+		ret = archive_read_open_memory(r, p, resource_size(res));
+		if (ret != ARCHIVE_OK) {
+			fprintf(stderr, "Failed: %s\n",
+					archive_error_string(r));
+			goto fail;
+		}
+
+		try_again = 0;
+
+		while (1) {
+			ret = archive_read_next_header(r, &e);
+			if (ret == ARCHIVE_EOF)
+				break;
+			if (ret != ARCHIVE_OK)
+				goto fail;
+
+			ret = remove(archive_entry_pathname(e));
+			if (ret < 0) {
+				switch (errno) {
+				case ENOTEMPTY:
+				case EEXIST:
+					try_again = 1;
+					break;
+
+				/* should not usually happen: */
+				case ENOENT:
+				case ENOTDIR:
+				case EROFS:
+					attempts += 1;
+					continue;
+				default:
+					perror("Failed to remove file");
+					goto fail;
+				}
+			}
+		}
+
+		archive_read_free(r);
+		attempts += 1;
+	}
+
+	if (attempts >= 10000) {
+		fprintf(stderr, "Failed to remove archive\n");
+		free(p);
+		return -1;
+	}
+
+	free(p);
+	return 0;
+
+fail:
+	archive_read_free(r);
+	free(p);
+
+	rte_panic("Failed: %s\n", archive_error_string(r));
+	return -1;
+}
+
 void resource_register(struct resource *r)
 {
 	TAILQ_INSERT_TAIL(&resource_list, r, next);
diff --git a/app/test/resource.h b/app/test/resource.h
index ac9cae6..1e96122 100644
--- a/app/test/resource.h
+++ b/app/test/resource.h
@@ -89,6 +89,19 @@ int resource_fwrite(const struct resource *r, FILE *f);
 int resource_fwrite_file(const struct resource *r, const char *fname);
 
 /**
+ * Treat the given resource as a tar archive. Extract
+ * the archive to the current directory.
+ */
+int resource_untar(const struct resource *res);
+
+/**
+ * Treat the given resource as a tar archive. Remove
+ * all files (related to the current directory) listed
+ * in the tar archive.
+ */
+int resource_rm_by_tar(const struct resource *res);
+
+/**
  * Register a resource in the global list of resources.
  * Not intended for direct use, please check the REGISTER_RESOURCE
  * macro.
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
index 3d1bf00..1e85040 100644
--- a/app/test/test_resource.c
+++ b/app/test/test_resource.c
@@ -85,6 +85,32 @@ static int test_resource_c(void)
 	return 0;
 }
 
+REGISTER_LINKED_RESOURCE(test_resource_tar);
+
+static int test_resource_tar(void)
+{
+	const struct resource *r;
+	FILE *f;
+
+	r = resource_find("test_resource_tar");
+	TEST_ASSERT_NOT_NULL(r, "No test_resource_tar found");
+	TEST_ASSERT(!strcmp(r->name, "test_resource_tar"),
+			"Found resource %s, expected test_resource_tar",
+			r->name);
+
+	TEST_ASSERT_SUCCESS(resource_untar(r),
+			"Failed to to untar %s", r->name);
+
+	f = fopen("test_resource.c", "r");
+	TEST_ASSERT_NOT_NULL(f,
+			"Missing extracted file test_resource.c");
+	fclose(f);
+
+	TEST_ASSERT_SUCCESS(resource_rm_by_tar(r),
+			"Failed to remove extracted contents of %s", r->name);
+	return 0;
+}
+
 static int test_resource(void)
 {
 	if (test_resource_dpdk())
@@ -93,6 +119,9 @@ static int test_resource(void)
 	if (test_resource_c())
 		return -1;
 
+	if (test_resource_tar())
+		return -1;
+
 	return 0;
 }
 
-- 
2.8.0

[PATCH v5 06/10] app/test: use linked list to store PCI drivers

[Jan Viktorin]

The test unregisters all drivers before start. The drivers were stored
into a fixed-sized array. This is inflexible. This patch change this to
utilize a linked list for the same purpose.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v3:
* fixed commit message
* used "backup" to describe the real_pci_driver_list
---
 app/test/test_pci.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 0ed357e..8b7c8bb 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -144,21 +144,24 @@ static void free_devargs_list(void)
 	}
 }
 
+/* backup real drivers (not used for testing) */
+struct pci_driver_list real_pci_driver_list =
+	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
+
 int
 test_pci(void)
 {
 	struct rte_devargs_list save_devargs_list;
 	struct rte_pci_driver *dr = NULL;
-	struct rte_pci_driver *save_pci_driver_list[NUM_MAX_DRIVERS];
-	unsigned i, num_drivers = 0;
 
 	printf("Dump all devices\n");
 	rte_eal_pci_dump(stdout);
 
 	/* Unregister all previous drivers */
-	TAILQ_FOREACH(dr, &pci_driver_list, next) {
+	while (!TAILQ_EMPTY(&pci_driver_list)) {
+		dr = TAILQ_FIRST(&pci_driver_list);
 		rte_eal_pci_unregister(dr);
-		save_pci_driver_list[num_drivers++] = dr;
+		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
 	}
 
 	rte_eal_pci_register(&my_driver);
@@ -197,8 +200,11 @@ test_pci(void)
 	rte_eal_pci_unregister(&my_driver2);
 
 	/* Restore original driver list */
-	for (i = 0; i < num_drivers; i++)
-		rte_eal_pci_register(save_pci_driver_list[i]);
+	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
+		dr = TAILQ_FIRST(&real_pci_driver_list);
+		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
+		rte_eal_pci_register(dr);
+	}
 
 	return 0;
 }
-- 
2.8.0

[PATCH v5 07/10] app/test: extract test_pci_setup and test_pci_cleanup

[Jan Viktorin]

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/test_pci.c | 47 ++++++++++++++++++++++++++++++++++-------------
 1 file changed, 34 insertions(+), 13 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 8b7c8bb..50078a0 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -148,21 +148,46 @@ static void free_devargs_list(void)
 struct pci_driver_list real_pci_driver_list =
 	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
 
+static int
+test_pci_setup(void)
+{
+	struct rte_pci_driver *dr;
+
+	/* Unregister original driver list */
+	while (!TAILQ_EMPTY(&pci_driver_list)) {
+		dr = TAILQ_FIRST(&pci_driver_list);
+		rte_eal_pci_unregister(dr);
+		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
+	}
+
+	return 0;
+}
+
+static int
+test_pci_cleanup(void)
+{
+	struct rte_pci_driver *dr;
+
+	/* Restore original driver list */
+	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
+		dr = TAILQ_FIRST(&real_pci_driver_list);
+		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
+		rte_eal_pci_register(dr);
+	}
+
+	return 0;
+}
+
 int
 test_pci(void)
 {
 	struct rte_devargs_list save_devargs_list;
-	struct rte_pci_driver *dr = NULL;
 
 	printf("Dump all devices\n");
 	rte_eal_pci_dump(stdout);
 
-	/* Unregister all previous drivers */
-	while (!TAILQ_EMPTY(&pci_driver_list)) {
-		dr = TAILQ_FIRST(&pci_driver_list);
-		rte_eal_pci_unregister(dr);
-		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
-	}
+	if (test_pci_setup())
+		return -1;
 
 	rte_eal_pci_register(&my_driver);
 	rte_eal_pci_register(&my_driver2);
@@ -199,12 +224,8 @@ test_pci(void)
 	rte_eal_pci_unregister(&my_driver);
 	rte_eal_pci_unregister(&my_driver2);
 
-	/* Restore original driver list */
-	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
-		dr = TAILQ_FIRST(&real_pci_driver_list);
-		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
-		rte_eal_pci_register(dr);
-	}
+	if (test_pci_cleanup())
+		return -1;
 
 	return 0;
 }
-- 
2.8.0

[PATCH v5 08/10] app/test: convert current pci_test into a single test case

[Jan Viktorin]

The current test_pci is just a single test case that tests the
blacklisting of devices. Rename it to test_pci_blacklist and call it
from the test_pci. The setup and cleanup are moved out of the
test_pci_blacklist entirely to cover all other tests.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v4
* the definitions of setup and cleanup functions are NOT moved (T. Monjalon)
---
 app/test/test_pci.c | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 50078a0..28d710b 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -178,17 +178,14 @@ test_pci_cleanup(void)
 	return 0;
 }
 
-int
-test_pci(void)
+static int
+test_pci_blacklist(void)
 {
 	struct rte_devargs_list save_devargs_list;
 
 	printf("Dump all devices\n");
 	rte_eal_pci_dump(stdout);
 
-	if (test_pci_setup())
-		return -1;
-
 	rte_eal_pci_register(&my_driver);
 	rte_eal_pci_register(&my_driver2);
 
@@ -224,6 +221,18 @@ test_pci(void)
 	rte_eal_pci_unregister(&my_driver);
 	rte_eal_pci_unregister(&my_driver2);
 
+	return 0;
+}
+
+int
+test_pci(void)
+{
+	if (test_pci_setup())
+		return -1;
+
+	if (test_pci_blacklist())
+		return -1;
+
 	if (test_pci_cleanup())
 		return -1;
 
-- 
2.8.0

[PATCH v5 09/10] eal/pci: allow to override sysfs

[Jan Viktorin]

The SYSFS_PCI_DEVICES is a constant that makes the PCI testing
difficult as it points to an absolute path. We remove using this
constant and introducing a function pci_get_sysfs_path that gives
the same value. However, the user can pass a SYSFS_PCI_DEVICES env
variable to override the path. It is now possible to create a fake
sysfs hierarchy for testing.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v3:
* changed subject
* test_pci_sysfs has been slightly modified to be more understandable
* fixed whitespace in *version.map files
v4:
* fixed checkpatch whitespace issue
---
 app/test/Makefile                                  |   1 +
 app/test/test_pci.c                                |  88 ++++++++++++++++++++-
 .../bus/pci/devices/0000:01:00.0/class             |   1 +
 .../bus/pci/devices/0000:01:00.0/config            | Bin 0 -> 64 bytes
 .../devices/0000:01:00.0/consistent_dma_mask_bits  |   1 +
 .../bus/pci/devices/0000:01:00.0/device            |   1 +
 .../bus/pci/devices/0000:01:00.0/dma_mask_bits     |   1 +
 .../bus/pci/devices/0000:01:00.0/enable            |   1 +
 .../bus/pci/devices/0000:01:00.0/irq               |   1 +
 .../bus/pci/devices/0000:01:00.0/modalias          |   1 +
 .../bus/pci/devices/0000:01:00.0/msi_bus           |   1 +
 .../bus/pci/devices/0000:01:00.0/numa_node         |   1 +
 .../bus/pci/devices/0000:01:00.0/resource          |  13 +++
 .../bus/pci/devices/0000:01:00.0/sriov_numvfs      |   1 +
 .../bus/pci/devices/0000:01:00.0/sriov_totalvfs    |   1 +
 .../bus/pci/devices/0000:01:00.0/subsystem_device  |   1 +
 .../bus/pci/devices/0000:01:00.0/subsystem_vendor  |   1 +
 .../bus/pci/devices/0000:01:00.0/uevent            |   6 ++
 .../bus/pci/devices/0000:01:00.0/vendor            |   1 +
 drivers/net/szedata2/rte_eth_szedata2.c            |   2 +-
 drivers/net/virtio/virtio_pci.c                    |   2 +-
 lib/librte_eal/bsdapp/eal/rte_eal_version.map      |   7 ++
 lib/librte_eal/common/eal_common_pci.c             |  13 +++
 lib/librte_eal/common/include/rte_pci.h            |   2 +-
 lib/librte_eal/linuxapp/eal/eal_pci.c              |  10 +--
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c          |   7 +-
 lib/librte_eal/linuxapp/eal/eal_pci_vfio.c         |   2 +-
 lib/librte_eal/linuxapp/eal/rte_eal_version.map    |   7 ++
 28 files changed, 159 insertions(+), 15 deletions(-)
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/config
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor

diff --git a/app/test/Makefile b/app/test/Makefile
index 1cc6258..7e4d484 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -73,6 +73,7 @@ SRCS-y += test_resource.c
 $(eval $(call linked_resource,test_resource_c,resource.c))
 $(eval $(call linked_tar_resource,test_resource_tar,test_resource.c))
 SRCS-y += test_pci.c
+$(eval $(call linked_tar_resource,test_pci_sysfs,test_pci_sysfs))
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
 SRCS-y += test_per_lcore.c
diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 28d710b..362ae3e 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -43,6 +43,7 @@
 #include <rte_devargs.h>
 
 #include "test.h"
+#include "resource.h"
 
 /* Generic maximum number of drivers to have room to allocate all drivers */
 #define NUM_MAX_DRIVERS 256
@@ -144,37 +145,90 @@ static void free_devargs_list(void)
 	}
 }
 
-/* backup real drivers (not used for testing) */
+/* backup real devices & drivers (not used for testing) */
 struct pci_driver_list real_pci_driver_list =
 	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
+struct pci_device_list real_pci_device_list =
+	TAILQ_HEAD_INITIALIZER(real_pci_device_list);
+
+REGISTER_LINKED_RESOURCE(test_pci_sysfs);
 
 static int
 test_pci_setup(void)
 {
+	struct rte_pci_device *dev;
 	struct rte_pci_driver *dr;
+	const struct resource *r;
+	int ret;
+
+	r = resource_find("test_pci_sysfs");
+	TEST_ASSERT_NOT_NULL(r, "missing resource test_pci_sysfs");
+
+	ret = resource_untar(r);
+	TEST_ASSERT_SUCCESS(ret, "failed to untar %s", r->name);
 
-	/* Unregister original driver list */
+	ret = setenv("SYSFS_PCI_DEVICES", "test_pci_sysfs/bus/pci/devices", 1);
+	TEST_ASSERT_SUCCESS(ret, "failed to setenv");
+
+	/* Unregister original devices & drivers lists */
 	while (!TAILQ_EMPTY(&pci_driver_list)) {
 		dr = TAILQ_FIRST(&pci_driver_list);
 		rte_eal_pci_unregister(dr);
 		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
 	}
 
+	while (!TAILQ_EMPTY(&pci_device_list)) {
+		dev = TAILQ_FIRST(&pci_device_list);
+		TAILQ_REMOVE(&pci_device_list, dev, next);
+		TAILQ_INSERT_TAIL(&real_pci_device_list, dev, next);
+	}
+
+	ret = rte_eal_pci_scan();
+	TEST_ASSERT_SUCCESS(ret, "failed to scan PCI bus");
+	rte_eal_pci_dump(stdout);
+
 	return 0;
 }
 
 static int
 test_pci_cleanup(void)
 {
+	struct rte_pci_device *dev;
 	struct rte_pci_driver *dr;
+	const struct resource *r;
+	int ret;
+
+	unsetenv("SYSFS_PCI_DEVICES");
+
+	r = resource_find("test_pci_sysfs");
+	TEST_ASSERT_NOT_NULL(r, "missing resource test_pci_sysfs");
+
+	ret = resource_rm_by_tar(r);
+	TEST_ASSERT_SUCCESS(ret, "Failed to delete resource %s", r->name);
 
-	/* Restore original driver list */
+	/*
+	 * FIXME: there is no API in DPDK to free a rte_pci_device so we
+	 * cannot free the devices in the right way. Let's assume that we
+	 * don't care for tests.
+	 */
+	while (!TAILQ_EMPTY(&pci_device_list)) {
+		dev = TAILQ_FIRST(&pci_device_list);
+		TAILQ_REMOVE(&pci_device_list, dev, next);
+	}
+
+	/* Restore original devices & drivers lists */
 	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
 		dr = TAILQ_FIRST(&real_pci_driver_list);
 		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
 		rte_eal_pci_register(dr);
 	}
 
+	while (!TAILQ_EMPTY(&real_pci_device_list)) {
+		dev = TAILQ_FIRST(&real_pci_device_list);
+		TAILQ_REMOVE(&real_pci_device_list, dev, next);
+		TAILQ_INSERT_TAIL(&pci_device_list, dev, next);
+	}
+
 	return 0;
 }
 
@@ -224,9 +278,37 @@ test_pci_blacklist(void)
 	return 0;
 }
 
+static int test_pci_sysfs(void)
+{
+	const char *orig;
+	const char *newpath;
+	int ret;
+
+	orig = pci_get_sysfs_path();
+	ret = setenv("SYSFS_PCI_DEVICES", "My Documents", 1);
+	TEST_ASSERT_SUCCESS(ret, "Failed setenv to My Documents");
+
+	newpath = pci_get_sysfs_path();
+	TEST_ASSERT(!strcmp(newpath, "My Documents"),
+			"pci_get_sysfs_path() should return 'My Documents' "
+			"but gives %s", newpath);
+
+	ret = setenv("SYSFS_PCI_DEVICES", orig, 1);
+	TEST_ASSERT_SUCCESS(ret, "Failed setenv back to '%s'", orig);
+
+	newpath = pci_get_sysfs_path();
+	TEST_ASSERT(!strcmp(orig, newpath),
+			"pci_get_sysfs_path returned unexpected path: "
+			"%s (expected: %s)", newpath, orig);
+	return 0;
+}
+
 int
 test_pci(void)
 {
+	if (test_pci_sysfs())
+		return -1;
+
 	if (test_pci_setup())
 		return -1;
 
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class
new file mode 100644
index 0000000..2f9c1da
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class
@@ -0,0 +1 @@
+0x020000
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/config b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/config
new file mode 100644
index 0000000000000000000000000000000000000000..7752421cf13a5aa00a28eaee02cac2add9ce5566
GIT binary patch
literal 64
zcmZo`_$|QBBEZ1Nz`(@7(7?dMz;S^A2oxWHNCpNT2LUi2#BOU~22l(SV3L7>8>k5Y
DD^Ld(

literal 0
HcmV?d00001

diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits
new file mode 100644
index 0000000..900731f
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits
@@ -0,0 +1 @@
+64
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device
new file mode 100644
index 0000000..9e4789e
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device
@@ -0,0 +1 @@
+0x10fb
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits
new file mode 100644
index 0000000..900731f
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits
@@ -0,0 +1 @@
+64
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable
@@ -0,0 +1 @@
+1
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq
@@ -0,0 +1 @@
+0
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias
new file mode 100644
index 0000000..f4c76ed
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias
@@ -0,0 +1 @@
+pci:v00008086d000010FBsv00008086sd00000003bc02sc00i00
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus
@@ -0,0 +1 @@
+1
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node
new file mode 100644
index 0000000..3a2e3f4
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node
@@ -0,0 +1 @@
+-1
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource
new file mode 100644
index 0000000..f388929
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource
@@ -0,0 +1,13 @@
+0x00000000d0080000 0x00000000d00fffff 0x000000000014220c
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x000000000000e020 0x000000000000e03f 0x0000000000040101
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x00000000d0104000 0x00000000d0107fff 0x000000000014220c
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x00000000ab000000 0x00000000ab0fffff 0x0000000000140204
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x00000000ab100000 0x00000000ab1fffff 0x0000000000140204
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x0000000000000000 0x0000000000000000 0x0000000000000000
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs
@@ -0,0 +1 @@
+0
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs
new file mode 100644
index 0000000..4b9026d
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs
@@ -0,0 +1 @@
+63
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device
new file mode 100644
index 0000000..89a932c
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device
@@ -0,0 +1 @@
+0x0003
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor
new file mode 100644
index 0000000..ce6dc4d
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor
@@ -0,0 +1 @@
+0x8086
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent
new file mode 100644
index 0000000..1dbe34d
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent
@@ -0,0 +1,6 @@
+DRIVER=ixgbe
+PCI_CLASS=20000
+PCI_ID=8086:10FB
+PCI_SUBSYS_ID=8086:0003
+PCI_SLOT_NAME=0000:01:00.0
+MODALIAS=pci:v00008086d000010FBsv00008086sd00000003bc02sc00i00
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor
new file mode 100644
index 0000000..ce6dc4d
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor
@@ -0,0 +1 @@
+0x8086
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 78c43b0..985a8d6 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1481,7 +1481,7 @@ rte_szedata2_eth_dev_init(struct rte_eth_dev *dev)
 		return -EINVAL;
 	}
 	snprintf(rsc_filename, PATH_MAX,
-		SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/resource%u",
+		"%s/" PCI_PRI_FMT "/resource%u", pci_get_sysfs_path(),
 		pci_addr->domain, pci_addr->bus,
 		pci_addr->devid, pci_addr->function, PCI_RESOURCE_NUMBER);
 	fd = open(rsc_filename, O_RDWR);
diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
index 9cdca06..845141b 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -179,7 +179,7 @@ legacy_virtio_has_msix(const struct rte_pci_addr *loc)
 	char dirname[PATH_MAX];
 
 	snprintf(dirname, sizeof(dirname),
-		     SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/msi_irqs",
+		     "%s/" PCI_PRI_FMT "/msi_irqs", pci_get_sysfs_path(),
 		     loc->domain, loc->bus, loc->devid, loc->function);
 
 	d = opendir(dirname);
diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
index 58c2951..f8c3dea 100644
--- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
@@ -151,3 +151,10 @@ DPDK_16.04 {
 	rte_eal_primary_proc_alive;
 
 } DPDK_2.2;
+
+DPDK_16.07 {
+	global:
+
+	pci_get_sysfs_path;
+
+} DPDK_16.04;
diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c
index 3cae4cb..0ec3b61 100644
--- a/lib/librte_eal/common/eal_common_pci.c
+++ b/lib/librte_eal/common/eal_common_pci.c
@@ -85,6 +85,19 @@
 struct pci_driver_list pci_driver_list;
 struct pci_device_list pci_device_list;
 
+#define SYSFS_PCI_DEVICES "/sys/bus/pci/devices"
+
+const char *pci_get_sysfs_path(void)
+{
+	const char *path = NULL;
+
+	path = getenv("SYSFS_PCI_DEVICES");
+	if (path == NULL)
+		return SYSFS_PCI_DEVICES;
+
+	return path;
+}
+
 static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev)
 {
 	struct rte_devargs *devargs;
diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index 8fa2712..7669fd7 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -91,7 +91,7 @@ extern struct pci_driver_list pci_driver_list; /**< Global list of PCI drivers.
 extern struct pci_device_list pci_device_list; /**< Global list of PCI devices. */
 
 /** Pathname of PCI devices directory. */
-#define SYSFS_PCI_DEVICES "/sys/bus/pci/devices"
+const char *pci_get_sysfs_path(void);
 
 /** Formatting string for PCI device identifier: Ex: 0000:00:01.0 */
 #define PCI_PRI_FMT "%.4" PRIx16 ":%.2" PRIx8 ":%.2" PRIx8 ".%" PRIx8
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index bdc08a0..5041228 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -66,8 +66,8 @@ pci_unbind_kernel_driver(struct rte_pci_device *dev)
 
 	/* open /sys/bus/pci/devices/AAAA:BB:CC.D/driver */
 	snprintf(filename, sizeof(filename),
-	         SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/driver/unbind",
-	         loc->domain, loc->bus, loc->devid, loc->function);
+		"%s/" PCI_PRI_FMT "/driver/unbind", pci_get_sysfs_path(),
+		loc->domain, loc->bus, loc->devid, loc->function);
 
 	f = fopen(filename, "w");
 	if (f == NULL) /* device was not bound */
@@ -453,7 +453,7 @@ rte_eal_pci_scan(void)
 	uint16_t domain;
 	uint8_t bus, devid, function;
 
-	dir = opendir(SYSFS_PCI_DEVICES);
+	dir = opendir(pci_get_sysfs_path());
 	if (dir == NULL) {
 		RTE_LOG(ERR, EAL, "%s(): opendir failed: %s\n",
 			__func__, strerror(errno));
@@ -468,8 +468,8 @@ rte_eal_pci_scan(void)
 				&bus, &devid, &function) != 0)
 			continue;
 
-		snprintf(dirname, sizeof(dirname), "%s/%s", SYSFS_PCI_DEVICES,
-			 e->d_name);
+		snprintf(dirname, sizeof(dirname), "%s/%s",
+				pci_get_sysfs_path(), e->d_name);
 		if (pci_scan_one(dirname, domain, bus, devid, function) < 0)
 			goto error;
 	}
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
index 068694d..b833244 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
@@ -161,14 +161,14 @@ pci_get_uio_dev(struct rte_pci_device *dev, char *dstbuf,
 	 * or uio:uioX */
 
 	snprintf(dirname, sizeof(dirname),
-			SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/uio",
+			"%s/" PCI_PRI_FMT "/uio", pci_get_sysfs_path(),
 			loc->domain, loc->bus, loc->devid, loc->function);
 
 	dir = opendir(dirname);
 	if (dir == NULL) {
 		/* retry with the parent directory */
 		snprintf(dirname, sizeof(dirname),
-				SYSFS_PCI_DEVICES "/" PCI_PRI_FMT,
+				"%s/" PCI_PRI_FMT, pci_get_sysfs_path(),
 				loc->domain, loc->bus, loc->devid, loc->function);
 		dir = opendir(dirname);
 
@@ -319,7 +319,8 @@ pci_uio_map_resource_by_index(struct rte_pci_device *dev, int res_idx,
 
 	/* update devname for mmap  */
 	snprintf(devname, sizeof(devname),
-			SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/resource%d",
+			"%s/" PCI_PRI_FMT "/resource%d",
+			pci_get_sysfs_path(),
 			loc->domain, loc->bus, loc->devid,
 			loc->function, res_idx);
 
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c b/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c
index 10266f8..f91b924 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c
@@ -602,7 +602,7 @@ pci_vfio_get_group_no(const char *pci_addr, int *iommu_group_no)
 
 	/* try to find out IOMMU group for this device */
 	snprintf(linkname, sizeof(linkname),
-			 SYSFS_PCI_DEVICES "/%s/iommu_group", pci_addr);
+			 "%s/%s/iommu_group", pci_get_sysfs_path(), pci_addr);
 
 	ret = readlink(linkname, filename, sizeof(filename));
 
diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
index 12503ef..3d0ff93 100644
--- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
@@ -154,3 +154,10 @@ DPDK_16.04 {
 	rte_eal_primary_proc_alive;
 
 } DPDK_2.2;
+
+DPDK_16.07 {
+	global:
+
+	pci_get_sysfs_path;
+
+} DPDK_16.04;
-- 
2.8.0

[PATCH v5 10/10] app/test: do not dump PCI devices in blacklist test

[Jan Viktorin]

Dumping of devices in a unittest is useless. Instead, test whether
the test has been set up well - i.e. there are no devices.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/test_pci.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 362ae3e..8051e53 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -238,7 +238,8 @@ test_pci_blacklist(void)
 	struct rte_devargs_list save_devargs_list;
 
 	printf("Dump all devices\n");
-	rte_eal_pci_dump(stdout);
+	TEST_ASSERT(TAILQ_EMPTY(&pci_driver_list),
+			"pci_driver_list not empty");
 
 	rte_eal_pci_register(&my_driver);
 	rte_eal_pci_register(&my_driver2);
-- 
2.8.0

[PATCH v4 01/10] app/test: introduce resources for tests

[Jan Viktorin]

Certain internal mechanisms of DPDK access different file system
structures (e.g. /sys/bus/pci/devices). It is difficult to test
those cases automatically by a unit test when such path is not
hard-coded and there is no simple way how to distribute fake ones
with the current testing environment.

This patch adds a possibility to declare a resource embedded in
the test binary itself. The structure resource cover the generic
situation - it provides a name for lookup and pointers to the
embedded data blob. A resource is registered in a constructor by
the macro REGISTER_RESOURCE.

Some initial tests of simple resources is included and added into
the group_1.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v3:
* fixed doc comments
---
 app/test/Makefile         |  2 +
 app/test/autotest_data.py |  6 +++
 app/test/resource.c       | 66 +++++++++++++++++++++++++++++++
 app/test/resource.h       | 98 +++++++++++++++++++++++++++++++++++++++++++++++
 app/test/test_resource.c  | 75 ++++++++++++++++++++++++++++++++++++
 5 files changed, 247 insertions(+)
 create mode 100644 app/test/resource.c
 create mode 100644 app/test/resource.h
 create mode 100644 app/test/test_resource.c

diff --git a/app/test/Makefile b/app/test/Makefile
index 6df9c40..1a13fb2 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -43,6 +43,8 @@ APP = test
 #
 SRCS-$(CONFIG_RTE_LIBRTE_CMDLINE) := commands.c
 SRCS-y += test.c
+SRCS-y += resource.c
+SRCS-y += test_resource.c
 SRCS-y += test_pci.c
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
diff --git a/app/test/autotest_data.py b/app/test/autotest_data.py
index 78d2edd..dfebe64 100644
--- a/app/test/autotest_data.py
+++ b/app/test/autotest_data.py
@@ -94,6 +94,12 @@ parallel_test_group_list = [
 		 "Report" :	None,
 		},
 		{
+		 "Name" :	"Resource autotest",
+		 "Command" :	"resource_autotest",
+		 "Func" :	default_autotest,
+		 "Report" :	None,
+		},
+		{
 		 "Name" :	"Dump log history",
 		 "Command" :	"dump_log_history",
 		 "Func" :	dump_autotest,
diff --git a/app/test/resource.c b/app/test/resource.c
new file mode 100644
index 0000000..30513db
--- /dev/null
+++ b/app/test/resource.c
@@ -0,0 +1,66 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2016 RehiveTech. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of RehiveTech nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <string.h>
+#include <errno.h>
+#include <sys/queue.h>
+
+#include <rte_debug.h>
+
+#include "resource.h"
+
+struct resource_list resource_list = TAILQ_HEAD_INITIALIZER(resource_list);
+
+size_t resource_size(const struct resource *r)
+{
+	return r->end - r->begin;
+}
+
+const struct resource *resource_find(const char *name)
+{
+	struct resource *r;
+
+	TAILQ_FOREACH(r, &resource_list, next) {
+		RTE_VERIFY(r->name);
+
+		if (!strcmp(r->name, name))
+			return r;
+	}
+
+	return NULL;
+}
+
+void resource_register(struct resource *r)
+{
+	TAILQ_INSERT_TAIL(&resource_list, r, next);
+}
diff --git a/app/test/resource.h b/app/test/resource.h
new file mode 100644
index 0000000..9af8415
--- /dev/null
+++ b/app/test/resource.h
@@ -0,0 +1,98 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2016 RehiveTech. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of RehiveTech nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RESOURCE_H_
+#define _RESOURCE_H_
+
+/**
+ * @file
+ *
+ * Test Resource API
+ *
+ * Each test can require and use some external resources. Usually, an external
+ * resource is a file or a filesystem sub-hierarchy. A resource is included
+ * inside the test executable.
+ */
+
+#include <sys/queue.h>
+#include <stddef.h>
+
+#include <rte_eal.h>
+#include <rte_common.h>
+
+TAILQ_HEAD(resource_list, resource);
+extern struct resource_list resource_list;
+
+/**
+ * Representation of a resource. It points to the resource's binary data.
+ * The semantics of the binary data are defined by the target test.
+ */
+struct resource {
+	const char *name;  /**< Unique name of the resource */
+	const char *begin; /**< Start of resource data */
+	const char *end;   /**< End of resource data */
+	TAILQ_ENTRY(resource) next;
+};
+
+/**
+ * @return size of the given resource
+ */
+size_t resource_size(const struct resource *r);
+
+/**
+ * Find a resource by name in the global list of resources.
+ */
+const struct resource *resource_find(const char *name);
+
+/**
+ * Register a resource in the global list of resources.
+ * Not intended for direct use, please check the REGISTER_RESOURCE
+ * macro.
+ */
+void resource_register(struct resource *r);
+
+/**
+ * Definition of a resource described by its name, and pointers begin, end.
+ */
+#define REGISTER_RESOURCE(n, b, e) \
+static struct resource linkres_ ##n = {       \
+	.name = RTE_STR(n),     \
+	.begin = b,             \
+	.end = e,               \
+};                              \
+static void __attribute__((constructor, used)) resinitfn_ ##n(void) \
+{                               \
+	resource_register(&linkres_ ##n);  \
+}
+
+#endif
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
new file mode 100644
index 0000000..69391ad
--- /dev/null
+++ b/app/test/test_resource.c
@@ -0,0 +1,75 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2016 RehiveTech. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of RehiveTech nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <stdio.h>
+#include <string.h>
+
+#include "test.h"
+#include "resource.h"
+
+const char test_resource_dpdk_blob[] = {
+	'\x44', '\x50', '\x44', '\x4b', '\x00'
+};
+
+REGISTER_RESOURCE(test_resource_dpdk,
+		test_resource_dpdk_blob, test_resource_dpdk_blob + 4);
+
+static int test_resource_dpdk(void)
+{
+	const struct resource *r;
+
+	r = resource_find("test_resource_dpdk");
+	TEST_ASSERT_NOT_NULL(r, "Could not find test_resource_dpdk");
+	TEST_ASSERT(!strcmp(r->name, "test_resource_dpdk"),
+			"Found resource %s, expected test_resource_dpdk",
+			r->name);
+
+	TEST_ASSERT(!strncmp("DPDK", r->begin, 4),
+			"Unexpected payload: %.4s...", r->begin);
+
+	return 0;
+}
+
+static int test_resource(void)
+{
+	if (test_resource_dpdk())
+		return -1;
+
+	return 0;
+}
+
+static struct test_command resource_cmd = {
+	.command = "resource_autotest",
+	.callback = test_resource,
+};
+REGISTER_TEST_COMMAND(resource_cmd);
-- 
2.8.0

[PATCH v4 02/10] mk: define objcopy-specific target and arch

[Jan Viktorin]

The program objcopy uses non-standard conventions to name the
target and arch. Define the values for supported architecturesi
(tile and ppc_64 are missing).

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v4:
* fixed commit message (removed FIXME)
---
 mk/arch/arm/rte.vars.mk     | 5 +++++
 mk/arch/arm64/rte.vars.mk   | 5 +++++
 mk/arch/i686/rte.vars.mk    | 5 +++++
 mk/arch/x86_64/rte.vars.mk  | 5 +++++
 mk/arch/x86_x32/rte.vars.mk | 5 +++++
 5 files changed, 25 insertions(+)

diff --git a/mk/arch/arm/rte.vars.mk b/mk/arch/arm/rte.vars.mk
index bd85140..2f8cf7c 100644
--- a/mk/arch/arm/rte.vars.mk
+++ b/mk/arch/arm/rte.vars.mk
@@ -37,3 +37,8 @@ CPU_LDFLAGS ?=
 CPU_ASFLAGS ?= -felf
 
 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf32-littlearm
+RTE_OBJCOPY_ARCH = arm
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
diff --git a/mk/arch/arm64/rte.vars.mk b/mk/arch/arm64/rte.vars.mk
index 32e3a5f..c168426 100644
--- a/mk/arch/arm64/rte.vars.mk
+++ b/mk/arch/arm64/rte.vars.mk
@@ -56,3 +56,8 @@ CPU_LDFLAGS ?=
 CPU_ASFLAGS ?= -felf
 
 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf64-littleaarch64
+RTE_OBJCOPY_ARCH = aarch64
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
diff --git a/mk/arch/i686/rte.vars.mk b/mk/arch/i686/rte.vars.mk
index 8ba9a23..6a25312 100644
--- a/mk/arch/i686/rte.vars.mk
+++ b/mk/arch/i686/rte.vars.mk
@@ -57,3 +57,8 @@ CPU_LDFLAGS ?= -melf_i386
 CPU_ASFLAGS ?= -felf
 
 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf32-i386
+RTE_OBJCOPY_ARCH = i386
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
diff --git a/mk/arch/x86_64/rte.vars.mk b/mk/arch/x86_64/rte.vars.mk
index b986f04..83723c8 100644
--- a/mk/arch/x86_64/rte.vars.mk
+++ b/mk/arch/x86_64/rte.vars.mk
@@ -57,3 +57,8 @@ CPU_LDFLAGS ?=
 CPU_ASFLAGS ?= -felf64
 
 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf64-x86-64
+RTE_OBJCOPY_ARCH = i386:x86-64
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
diff --git a/mk/arch/x86_x32/rte.vars.mk b/mk/arch/x86_x32/rte.vars.mk
index 3103dfc..676f316 100644
--- a/mk/arch/x86_x32/rte.vars.mk
+++ b/mk/arch/x86_x32/rte.vars.mk
@@ -61,3 +61,8 @@ ifneq ($(shell echo | $(CC) $(CPU_CFLAGS) -E - 2>/dev/null 1>/dev/null && echo 0
 endif
 
 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf32-x86-64
+RTE_OBJCOPY_ARCH = i386:x86-64
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
-- 
2.8.0

[PATCH v4 03/10] app/test: support resources externally linked

[Jan Viktorin]

To include resources from other source that the C source code we
can take advantage of the objcopy behaviour, i.e. packing of an
arbitrary file as an object file that is linked to the target program.

A linked object file is always accessible as a pair

extern const char beg_<name>;
extern const char end_<name>;
(extern const char siz_<name>;)

A unit test that packs the resource.c source file is included.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v4
* fixed trailing semicolon in REGISTER_LINKED_RESOURCE
---
 app/test/Makefile        | 19 +++++++++++++++++++
 app/test/resource.h      | 10 ++++++++++
 app/test/test_resource.c | 18 ++++++++++++++++++
 3 files changed, 47 insertions(+)

diff --git a/app/test/Makefile b/app/test/Makefile
index 1a13fb2..fe67eee 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -33,6 +33,24 @@ include $(RTE_SDK)/mk/rte.vars.mk
 
 ifeq ($(CONFIG_RTE_APP_TEST),y)
 
+# Define an externally linked resource. A linked resource is an arbitrary
+# file that is linked into the test binary. The application refers to this
+# resource by name. The linked generates identifiers beg_<name> and end_<name>
+# for referencing by the C code.
+#
+# Parameters: <unique name>, <file to be linked>
+define linked_resource
+SRCS-y += $(1).res.o
+$(1).res.o: $(2)
+	$(OBJCOPY) -I binary -B $(RTE_OBJCOPY_ARCH) -O $(RTE_OBJCOPY_TARGET) \
+		--rename-section                                         \
+			.data=.rodata,alloc,load,data,contents,readonly  \
+		--redefine-sym _binary__dev_stdin_start=beg_$(1)         \
+		--redefine-sym _binary__dev_stdin_end=end_$(1)           \
+		--redefine-sym _binary__dev_stdin_size=siz_$(1)          \
+		/dev/stdin $$@ < $$<
+endef
+
 #
 # library name
 #
@@ -45,6 +63,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_CMDLINE) := commands.c
 SRCS-y += test.c
 SRCS-y += resource.c
 SRCS-y += test_resource.c
+$(eval $(call linked_resource,test_resource_c,resource.c))
 SRCS-y += test_pci.c
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
diff --git a/app/test/resource.h b/app/test/resource.h
index 9af8415..966fc24 100644
--- a/app/test/resource.h
+++ b/app/test/resource.h
@@ -82,6 +82,16 @@ const struct resource *resource_find(const char *name);
 void resource_register(struct resource *r);
 
 /**
+ * Definition of a resource linked externally (by means of the used toolchain).
+ * Only the base name of the resource is expected. The name refers to the
+ * linked pointers beg_<name> and end_<name> provided externally.
+ */
+#define REGISTER_LINKED_RESOURCE(n) \
+extern const char beg_ ##n;         \
+extern const char end_ ##n;         \
+REGISTER_RESOURCE(n, &beg_ ##n, &end_ ##n) \
+
+/**
  * Definition of a resource described by its name, and pointers begin, end.
  */
 #define REGISTER_RESOURCE(n, b, e) \
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
index 69391ad..b397fa8 100644
--- a/app/test/test_resource.c
+++ b/app/test/test_resource.c
@@ -60,11 +60,29 @@ static int test_resource_dpdk(void)
 	return 0;
 }
 
+REGISTER_LINKED_RESOURCE(test_resource_c);
+
+static int test_resource_c(void)
+{
+	const struct resource *r;
+
+	r = resource_find("test_resource_c");
+	TEST_ASSERT_NOT_NULL(r, "No test_resource_c found");
+	TEST_ASSERT(!strcmp(r->name, "test_resource_c"),
+			"Found resource %s, expected test_resource_c",
+			r->name);
+
+	return 0;
+}
+
 static int test_resource(void)
 {
 	if (test_resource_dpdk())
 		return -1;
 
+	if (test_resource_c())
+		return -1;
+
 	return 0;
 }
 
-- 
2.8.0

[PATCH v4 04/10] app/test: add functions to create files from resources

[Jan Viktorin]

A resource can be written into the target filesystem by calling
resource_fwrite or resource_fwrite_file. Such file can be created
before a test is started and removed after the test finishes.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/resource.c      | 35 +++++++++++++++++++++++++++++++++++
 app/test/resource.h      | 14 ++++++++++++++
 app/test/test_resource.c | 10 ++++++++++
 3 files changed, 59 insertions(+)

diff --git a/app/test/resource.c b/app/test/resource.c
index 30513db..acb63c1 100644
--- a/app/test/resource.c
+++ b/app/test/resource.c
@@ -31,6 +31,7 @@
  *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+#include <stdio.h>
 #include <string.h>
 #include <errno.h>
 #include <sys/queue.h>
@@ -60,6 +61,40 @@ const struct resource *resource_find(const char *name)
 	return NULL;
 }
 
+int resource_fwrite(const struct resource *r, FILE *f)
+{
+	const size_t goal = resource_size(r);
+	size_t total = 0;
+
+	while (total < goal) {
+		size_t wlen = fwrite(r->begin + total, 1, goal - total, f);
+		if (wlen == 0) {
+			perror(__func__);
+			return -1;
+		}
+
+		total += wlen;
+	}
+
+	return 0;
+}
+
+int resource_fwrite_file(const struct resource *r, const char *fname)
+{
+	FILE *f;
+	int ret;
+
+	f = fopen(fname, "w");
+	if (f == NULL) {
+		perror(__func__);
+		return -1;
+	}
+
+	ret = resource_fwrite(r, f);
+	fclose(f);
+	return ret;
+}
+
 void resource_register(struct resource *r)
 {
 	TAILQ_INSERT_TAIL(&resource_list, r, next);
diff --git a/app/test/resource.h b/app/test/resource.h
index 966fc24..ac9cae6 100644
--- a/app/test/resource.h
+++ b/app/test/resource.h
@@ -45,6 +45,7 @@
  */
 
 #include <sys/queue.h>
+#include <stdio.h>
 #include <stddef.h>
 
 #include <rte_eal.h>
@@ -75,6 +76,19 @@ size_t resource_size(const struct resource *r);
 const struct resource *resource_find(const char *name);
 
 /**
+ * Write the raw data of the resource to the given file.
+ * @return 0 on success
+ */
+int resource_fwrite(const struct resource *r, FILE *f);
+
+/**
+ * Write the raw data of the resource to the given file given by name.
+ * The name is relative to the current working directory.
+ * @return 0 on success
+ */
+int resource_fwrite_file(const struct resource *r, const char *fname);
+
+/**
  * Register a resource in the global list of resources.
  * Not intended for direct use, please check the REGISTER_RESOURCE
  * macro.
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
index b397fa8..3d1bf00 100644
--- a/app/test/test_resource.c
+++ b/app/test/test_resource.c
@@ -65,6 +65,7 @@ REGISTER_LINKED_RESOURCE(test_resource_c);
 static int test_resource_c(void)
 {
 	const struct resource *r;
+	FILE *f;
 
 	r = resource_find("test_resource_c");
 	TEST_ASSERT_NOT_NULL(r, "No test_resource_c found");
@@ -72,6 +73,15 @@ static int test_resource_c(void)
 			"Found resource %s, expected test_resource_c",
 			r->name);
 
+	TEST_ASSERT_SUCCESS(resource_fwrite_file(r, "test_resource.c"),
+			"Failed to to write file %s", r->name);
+
+	f = fopen("test_resource.c", "r");
+	TEST_ASSERT_NOT_NULL(f,
+			"Missing extracted file resource.c");
+	fclose(f);
+	remove("test_resource.c");
+
 	return 0;
 }
 
-- 
2.8.0

[PATCH v4 05/10] app/test: support resources archived by tar

[Jan Viktorin]

When a more complex resource (a file hierarchy) is needed, packing
every single file as a single resource would be very ineffective. For
that purpose, it is possible to pack the files into a tar archive,
extract it before test from the resource and finally clean up all the
created files.

This patch introduces functions resource_untar and resource_rm_by_tar
to perform those tasks. An example of using those functions is included
as a test.

A new dependency is required to build the app/test: libarchive.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/Makefile        |   9 +++
 app/test/resource.c      | 196 +++++++++++++++++++++++++++++++++++++++++++++++
 app/test/resource.h      |  13 ++++
 app/test/test_resource.c |  29 +++++++
 4 files changed, 247 insertions(+)

diff --git a/app/test/Makefile b/app/test/Makefile
index fe67eee..1cc6258 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -51,6 +51,13 @@ $(1).res.o: $(2)
 		/dev/stdin $$@ < $$<
 endef
 
+define linked_tar_resource
+$(1).tar: $(2)
+	tar -C $$(dir $$<) -cf $$@ $$(notdir $$<)
+
+$(call linked_resource,$(1),$(1).tar)
+endef
+
 #
 # library name
 #
@@ -64,6 +71,7 @@ SRCS-y += test.c
 SRCS-y += resource.c
 SRCS-y += test_resource.c
 $(eval $(call linked_resource,test_resource_c,resource.c))
+$(eval $(call linked_tar_resource,test_resource_tar,test_resource.c))
 SRCS-y += test_pci.c
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
@@ -185,6 +193,7 @@ CFLAGS += $(WERROR_FLAGS)
 CFLAGS += -D_GNU_SOURCE
 
 LDLIBS += -lm
+LDLIBS += -larchive
 
 # Disable VTA for memcpy test
 ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y)
diff --git a/app/test/resource.c b/app/test/resource.c
index acb63c1..f407f90 100644
--- a/app/test/resource.c
+++ b/app/test/resource.c
@@ -33,6 +33,8 @@
 
 #include <stdio.h>
 #include <string.h>
+#include <archive.h>
+#include <archive_entry.h>
 #include <errno.h>
 #include <sys/queue.h>
 
@@ -95,6 +97,200 @@ int resource_fwrite_file(const struct resource *r, const char *fname)
 	return ret;
 }
 
+static int do_copy(struct archive *r, struct archive *w)
+{
+	const void *buf;
+	size_t len;
+	off_t off;
+	int ret;
+
+	while (1) {
+		ret = archive_read_data_block(r, &buf, &len, &off);
+		if (ret == ARCHIVE_RETRY)
+			continue;
+
+		if (ret == ARCHIVE_EOF)
+			return 0;
+
+		if (ret != ARCHIVE_OK)
+			return ret;
+
+		do {
+			ret = archive_write_data_block(w, buf, len, off);
+			if (ret != ARCHIVE_OK && ret != ARCHIVE_RETRY)
+				return ret;
+		} while (ret != ARCHIVE_OK);
+	}
+}
+
+int resource_untar(const struct resource *res)
+{
+	struct archive *r;
+	struct archive *w;
+	struct archive_entry *e;
+	void *p;
+	int flags = 0;
+	int ret;
+
+	p = malloc(resource_size(res));
+	if (p == NULL)
+		rte_panic("Failed to malloc %zu B\n", resource_size(res));
+
+	memcpy(p, res->begin, resource_size(res));
+
+	r = archive_read_new();
+	if (r == NULL) {
+		free(p);
+		return -1;
+	}
+
+	archive_read_support_format_all(r);
+	archive_read_support_filter_all(r);
+
+	w = archive_write_disk_new();
+	if (w == NULL) {
+		archive_read_free(r);
+		free(p);
+		return -1;
+	}
+
+	flags |= ARCHIVE_EXTRACT_PERM;
+	flags |= ARCHIVE_EXTRACT_FFLAGS;
+	archive_write_disk_set_options(w, flags);
+	archive_write_disk_set_standard_lookup(w);
+
+	ret = archive_read_open_memory(r, p, resource_size(res));
+	if (ret != ARCHIVE_OK)
+		goto fail;
+
+	while (1) {
+		ret = archive_read_next_header(r, &e);
+		if (ret == ARCHIVE_EOF)
+			break;
+		if (ret != ARCHIVE_OK)
+			goto fail;
+
+		ret = archive_write_header(w, e);
+		if (ret == ARCHIVE_EOF)
+			break;
+		if (ret != ARCHIVE_OK)
+			goto fail;
+
+		if (archive_entry_size(e) == 0)
+			continue;
+
+		ret = do_copy(r, w);
+		if (ret != ARCHIVE_OK)
+			goto fail;
+
+		ret = archive_write_finish_entry(w);
+		if (ret != ARCHIVE_OK)
+			goto fail;
+	}
+
+	archive_write_free(w);
+	archive_read_free(r);
+	free(p);
+	return 0;
+
+fail:
+	archive_write_free(w);
+	archive_read_free(r);
+	free(p);
+	rte_panic("Failed: %s\n", archive_error_string(r));
+	return -1;
+}
+
+int resource_rm_by_tar(const struct resource *res)
+{
+	struct archive *r;
+	struct archive_entry *e;
+	void *p;
+	int try_again = 1;
+	int attempts = 0;
+	int ret;
+
+	p = malloc(resource_size(res));
+	if (p == NULL)
+		rte_panic("Failed to malloc %zu B\n", resource_size(res));
+
+	memcpy(p, res->begin, resource_size(res));
+
+	/*
+	 * If somebody creates a file somewhere inside the extracted TAR
+	 * hierarchy during a test the resource_rm_by_tar might loop
+	 * infinitely. We prevent this by adding the attempts counter there.
+	 * In normal case, max N iteration is done where N is the depth of
+	 * the file-hierarchy.
+	 */
+	while (try_again && attempts < 10000) {
+		r = archive_read_new();
+		if (r == NULL) {
+			free(p);
+			return -1;
+		}
+
+		archive_read_support_format_all(r);
+		archive_read_support_filter_all(r);
+
+		ret = archive_read_open_memory(r, p, resource_size(res));
+		if (ret != ARCHIVE_OK) {
+			fprintf(stderr, "Failed: %s\n",
+					archive_error_string(r));
+			goto fail;
+		}
+
+		try_again = 0;
+
+		while (1) {
+			ret = archive_read_next_header(r, &e);
+			if (ret == ARCHIVE_EOF)
+				break;
+			if (ret != ARCHIVE_OK)
+				goto fail;
+
+			ret = remove(archive_entry_pathname(e));
+			if (ret < 0) {
+				switch (errno) {
+				case ENOTEMPTY:
+				case EEXIST:
+					try_again = 1;
+					break;
+
+				/* should not usually happen: */
+				case ENOENT:
+				case ENOTDIR:
+				case EROFS:
+					attempts += 1;
+					continue;
+				default:
+					perror("Failed to remove file");
+					goto fail;
+				}
+			}
+		}
+
+		archive_read_free(r);
+		attempts += 1;
+	}
+
+	if (attempts >= 10000) {
+		fprintf(stderr, "Failed to remove archive\n");
+		free(p);
+		return -1;
+	}
+
+	free(p);
+	return 0;
+
+fail:
+	archive_read_free(r);
+	free(p);
+
+	rte_panic("Failed: %s\n", archive_error_string(r));
+	return -1;
+}
+
 void resource_register(struct resource *r)
 {
 	TAILQ_INSERT_TAIL(&resource_list, r, next);
diff --git a/app/test/resource.h b/app/test/resource.h
index ac9cae6..1e96122 100644
--- a/app/test/resource.h
+++ b/app/test/resource.h
@@ -89,6 +89,19 @@ int resource_fwrite(const struct resource *r, FILE *f);
 int resource_fwrite_file(const struct resource *r, const char *fname);
 
 /**
+ * Treat the given resource as a tar archive. Extract
+ * the archive to the current directory.
+ */
+int resource_untar(const struct resource *res);
+
+/**
+ * Treat the given resource as a tar archive. Remove
+ * all files (related to the current directory) listed
+ * in the tar archive.
+ */
+int resource_rm_by_tar(const struct resource *res);
+
+/**
  * Register a resource in the global list of resources.
  * Not intended for direct use, please check the REGISTER_RESOURCE
  * macro.
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
index 3d1bf00..1e85040 100644
--- a/app/test/test_resource.c
+++ b/app/test/test_resource.c
@@ -85,6 +85,32 @@ static int test_resource_c(void)
 	return 0;
 }
 
+REGISTER_LINKED_RESOURCE(test_resource_tar);
+
+static int test_resource_tar(void)
+{
+	const struct resource *r;
+	FILE *f;
+
+	r = resource_find("test_resource_tar");
+	TEST_ASSERT_NOT_NULL(r, "No test_resource_tar found");
+	TEST_ASSERT(!strcmp(r->name, "test_resource_tar"),
+			"Found resource %s, expected test_resource_tar",
+			r->name);
+
+	TEST_ASSERT_SUCCESS(resource_untar(r),
+			"Failed to to untar %s", r->name);
+
+	f = fopen("test_resource.c", "r");
+	TEST_ASSERT_NOT_NULL(f,
+			"Missing extracted file test_resource.c");
+	fclose(f);
+
+	TEST_ASSERT_SUCCESS(resource_rm_by_tar(r),
+			"Failed to remove extracted contents of %s", r->name);
+	return 0;
+}
+
 static int test_resource(void)
 {
 	if (test_resource_dpdk())
@@ -93,6 +119,9 @@ static int test_resource(void)
 	if (test_resource_c())
 		return -1;
 
+	if (test_resource_tar())
+		return -1;
+
 	return 0;
 }
 
-- 
2.8.0

Re: [PATCH v4 05/10] app/test: support resources archived by tar

[Thomas Monjalon]

2016-06-13 10:12, Jan Viktorin:
> +static int do_copy(struct archive *r, struct archive *w)
> +{
> +	const void *buf;
> +	size_t len;
> +	off_t off;
> +	int ret;
> +
> +	while (1) {
> +		ret = archive_read_data_block(r, &buf, &len, &off);

There is an error in 32-bit compilation:

app/test/resource.c:108:48: error:
passing argument 4 of ‘archive_read_data_block’ from incompatible pointer type
   ret = archive_read_data_block(r, &buf, &len, &off);
                                                ^

Re: [PATCH v4 05/10] app/test: support resources archived by tar

[Jan Viktorin]

On Mon, 13 Jun 2016 16:40:09 +0200
Thomas Monjalon <thomas.monjalon@6wind.com> wrote:

> 2016-06-13 10:12, Jan Viktorin:
> > +static int do_copy(struct archive *r, struct archive *w)
> > +{
> > +	const void *buf;
> > +	size_t len;
> > +	off_t off;
> > +	int ret;
> > +
> > +	while (1) {
> > +		ret = archive_read_data_block(r, &buf, &len, &off);  
> 
> There is an error in 32-bit compilation:
> 
> app/test/resource.c:108:48: error:
> passing argument 4 of ‘archive_read_data_block’ from incompatible pointer type
>    ret = archive_read_data_block(r, &buf, &len, &off);
>                                                 ^

Hello Thomas,

confirmed in my Jenkins build. But it's strange as I've built it successfully
for 32b many times before by using Buildroot...

So, I am going to resend v5.

Regards
Jan

[PATCH v4 06/10] app/test: use linked list to store PCI drivers

[Jan Viktorin]

The test unregisters all drivers before start. The drivers were stored
into a fixed-sized array. This is inflexible. This patch change this to
utilize a linked list for the same purpose.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v3:
* fixed commit message
* used "backup" to describe the real_pci_driver_list
---
 app/test/test_pci.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 0ed357e..8b7c8bb 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -144,21 +144,24 @@ static void free_devargs_list(void)
 	}
 }
 
+/* backup real drivers (not used for testing) */
+struct pci_driver_list real_pci_driver_list =
+	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
+
 int
 test_pci(void)
 {
 	struct rte_devargs_list save_devargs_list;
 	struct rte_pci_driver *dr = NULL;
-	struct rte_pci_driver *save_pci_driver_list[NUM_MAX_DRIVERS];
-	unsigned i, num_drivers = 0;
 
 	printf("Dump all devices\n");
 	rte_eal_pci_dump(stdout);
 
 	/* Unregister all previous drivers */
-	TAILQ_FOREACH(dr, &pci_driver_list, next) {
+	while (!TAILQ_EMPTY(&pci_driver_list)) {
+		dr = TAILQ_FIRST(&pci_driver_list);
 		rte_eal_pci_unregister(dr);
-		save_pci_driver_list[num_drivers++] = dr;
+		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
 	}
 
 	rte_eal_pci_register(&my_driver);
@@ -197,8 +200,11 @@ test_pci(void)
 	rte_eal_pci_unregister(&my_driver2);
 
 	/* Restore original driver list */
-	for (i = 0; i < num_drivers; i++)
-		rte_eal_pci_register(save_pci_driver_list[i]);
+	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
+		dr = TAILQ_FIRST(&real_pci_driver_list);
+		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
+		rte_eal_pci_register(dr);
+	}
 
 	return 0;
 }
-- 
2.8.0

[PATCH v4 07/10] app/test: extract test_pci_setup and test_pci_cleanup

[Jan Viktorin]

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/test_pci.c | 47 ++++++++++++++++++++++++++++++++++-------------
 1 file changed, 34 insertions(+), 13 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 8b7c8bb..50078a0 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -148,21 +148,46 @@ static void free_devargs_list(void)
 struct pci_driver_list real_pci_driver_list =
 	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
 
+static int
+test_pci_setup(void)
+{
+	struct rte_pci_driver *dr;
+
+	/* Unregister original driver list */
+	while (!TAILQ_EMPTY(&pci_driver_list)) {
+		dr = TAILQ_FIRST(&pci_driver_list);
+		rte_eal_pci_unregister(dr);
+		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
+	}
+
+	return 0;
+}
+
+static int
+test_pci_cleanup(void)
+{
+	struct rte_pci_driver *dr;
+
+	/* Restore original driver list */
+	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
+		dr = TAILQ_FIRST(&real_pci_driver_list);
+		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
+		rte_eal_pci_register(dr);
+	}
+
+	return 0;
+}
+
 int
 test_pci(void)
 {
 	struct rte_devargs_list save_devargs_list;
-	struct rte_pci_driver *dr = NULL;
 
 	printf("Dump all devices\n");
 	rte_eal_pci_dump(stdout);
 
-	/* Unregister all previous drivers */
-	while (!TAILQ_EMPTY(&pci_driver_list)) {
-		dr = TAILQ_FIRST(&pci_driver_list);
-		rte_eal_pci_unregister(dr);
-		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
-	}
+	if (test_pci_setup())
+		return -1;
 
 	rte_eal_pci_register(&my_driver);
 	rte_eal_pci_register(&my_driver2);
@@ -199,12 +224,8 @@ test_pci(void)
 	rte_eal_pci_unregister(&my_driver);
 	rte_eal_pci_unregister(&my_driver2);
 
-	/* Restore original driver list */
-	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
-		dr = TAILQ_FIRST(&real_pci_driver_list);
-		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
-		rte_eal_pci_register(dr);
-	}
+	if (test_pci_cleanup())
+		return -1;
 
 	return 0;
 }
-- 
2.8.0

[PATCH v4 08/10] app/test: convert current pci_test into a single test case

[Jan Viktorin]

The current test_pci is just a single test case that tests the
blacklisting of devices. Rename it to test_pci_blacklist and call it
from the test_pci. The setup and cleanup are moved out of the
test_pci_blacklist entirely to cover all other tests.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v4
* the definitions of setup and cleanup functions are NOT moved (T. Monjalon)
---
 app/test/test_pci.c | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 50078a0..28d710b 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -178,17 +178,14 @@ test_pci_cleanup(void)
 	return 0;
 }
 
-int
-test_pci(void)
+static int
+test_pci_blacklist(void)
 {
 	struct rte_devargs_list save_devargs_list;
 
 	printf("Dump all devices\n");
 	rte_eal_pci_dump(stdout);
 
-	if (test_pci_setup())
-		return -1;
-
 	rte_eal_pci_register(&my_driver);
 	rte_eal_pci_register(&my_driver2);
 
@@ -224,6 +221,18 @@ test_pci(void)
 	rte_eal_pci_unregister(&my_driver);
 	rte_eal_pci_unregister(&my_driver2);
 
+	return 0;
+}
+
+int
+test_pci(void)
+{
+	if (test_pci_setup())
+		return -1;
+
+	if (test_pci_blacklist())
+		return -1;
+
 	if (test_pci_cleanup())
 		return -1;
 
-- 
2.8.0

[PATCH v4 09/10] eal/pci: allow to override sysfs

[Jan Viktorin]

The SYSFS_PCI_DEVICES is a constant that makes the PCI testing
difficult as it points to an absolute path. We remove using this
constant and introducing a function pci_get_sysfs_path that gives
the same value. However, the user can pass a SYSFS_PCI_DEVICES env
variable to override the path. It is now possible to create a fake
sysfs hierarchy for testing.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v3:
* changed subject
* test_pci_sysfs has been slightly modified to be more understandable
* fixed whitespace in *version.map files
v4:
* fixed checkpatch whitespace issues
---
 app/test/Makefile                                  |   1 +
 app/test/test_pci.c                                |  88 ++++++++++++++++++++-
 .../bus/pci/devices/0000:01:00.0/class             |   1 +
 .../bus/pci/devices/0000:01:00.0/config            | Bin 0 -> 64 bytes
 .../devices/0000:01:00.0/consistent_dma_mask_bits  |   1 +
 .../bus/pci/devices/0000:01:00.0/device            |   1 +
 .../bus/pci/devices/0000:01:00.0/dma_mask_bits     |   1 +
 .../bus/pci/devices/0000:01:00.0/enable            |   1 +
 .../bus/pci/devices/0000:01:00.0/irq               |   1 +
 .../bus/pci/devices/0000:01:00.0/modalias          |   1 +
 .../bus/pci/devices/0000:01:00.0/msi_bus           |   1 +
 .../bus/pci/devices/0000:01:00.0/numa_node         |   1 +
 .../bus/pci/devices/0000:01:00.0/resource          |  13 +++
 .../bus/pci/devices/0000:01:00.0/sriov_numvfs      |   1 +
 .../bus/pci/devices/0000:01:00.0/sriov_totalvfs    |   1 +
 .../bus/pci/devices/0000:01:00.0/subsystem_device  |   1 +
 .../bus/pci/devices/0000:01:00.0/subsystem_vendor  |   1 +
 .../bus/pci/devices/0000:01:00.0/uevent            |   6 ++
 .../bus/pci/devices/0000:01:00.0/vendor            |   1 +
 drivers/net/szedata2/rte_eth_szedata2.c            |   2 +-
 drivers/net/virtio/virtio_pci.c                    |   2 +-
 lib/librte_eal/bsdapp/eal/rte_eal_version.map      |   7 ++
 lib/librte_eal/common/eal_common_pci.c             |  13 +++
 lib/librte_eal/common/include/rte_pci.h            |   2 +-
 lib/librte_eal/linuxapp/eal/eal_pci.c              |  10 +--
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c          |   7 +-
 lib/librte_eal/linuxapp/eal/eal_pci_vfio.c         |   2 +-
 lib/librte_eal/linuxapp/eal/rte_eal_version.map    |   7 ++
 28 files changed, 159 insertions(+), 15 deletions(-)
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/config
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor

diff --git a/app/test/Makefile b/app/test/Makefile
index 1cc6258..7e4d484 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -73,6 +73,7 @@ SRCS-y += test_resource.c
 $(eval $(call linked_resource,test_resource_c,resource.c))
 $(eval $(call linked_tar_resource,test_resource_tar,test_resource.c))
 SRCS-y += test_pci.c
+$(eval $(call linked_tar_resource,test_pci_sysfs,test_pci_sysfs))
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
 SRCS-y += test_per_lcore.c
diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 28d710b..362ae3e 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -43,6 +43,7 @@
 #include <rte_devargs.h>
 
 #include "test.h"
+#include "resource.h"
 
 /* Generic maximum number of drivers to have room to allocate all drivers */
 #define NUM_MAX_DRIVERS 256
@@ -144,37 +145,90 @@ static void free_devargs_list(void)
 	}
 }
 
-/* backup real drivers (not used for testing) */
+/* backup real devices & drivers (not used for testing) */
 struct pci_driver_list real_pci_driver_list =
 	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
+struct pci_device_list real_pci_device_list =
+	TAILQ_HEAD_INITIALIZER(real_pci_device_list);
+
+REGISTER_LINKED_RESOURCE(test_pci_sysfs);
 
 static int
 test_pci_setup(void)
 {
+	struct rte_pci_device *dev;
 	struct rte_pci_driver *dr;
+	const struct resource *r;
+	int ret;
+
+	r = resource_find("test_pci_sysfs");
+	TEST_ASSERT_NOT_NULL(r, "missing resource test_pci_sysfs");
+
+	ret = resource_untar(r);
+	TEST_ASSERT_SUCCESS(ret, "failed to untar %s", r->name);
 
-	/* Unregister original driver list */
+	ret = setenv("SYSFS_PCI_DEVICES", "test_pci_sysfs/bus/pci/devices", 1);
+	TEST_ASSERT_SUCCESS(ret, "failed to setenv");
+
+	/* Unregister original devices & drivers lists */
 	while (!TAILQ_EMPTY(&pci_driver_list)) {
 		dr = TAILQ_FIRST(&pci_driver_list);
 		rte_eal_pci_unregister(dr);
 		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
 	}
 
+	while (!TAILQ_EMPTY(&pci_device_list)) {
+		dev = TAILQ_FIRST(&pci_device_list);
+		TAILQ_REMOVE(&pci_device_list, dev, next);
+		TAILQ_INSERT_TAIL(&real_pci_device_list, dev, next);
+	}
+
+	ret = rte_eal_pci_scan();
+	TEST_ASSERT_SUCCESS(ret, "failed to scan PCI bus");
+	rte_eal_pci_dump(stdout);
+
 	return 0;
 }
 
 static int
 test_pci_cleanup(void)
 {
+	struct rte_pci_device *dev;
 	struct rte_pci_driver *dr;
+	const struct resource *r;
+	int ret;
+
+	unsetenv("SYSFS_PCI_DEVICES");
+
+	r = resource_find("test_pci_sysfs");
+	TEST_ASSERT_NOT_NULL(r, "missing resource test_pci_sysfs");
+
+	ret = resource_rm_by_tar(r);
+	TEST_ASSERT_SUCCESS(ret, "Failed to delete resource %s", r->name);
 
-	/* Restore original driver list */
+	/*
+	 * FIXME: there is no API in DPDK to free a rte_pci_device so we
+	 * cannot free the devices in the right way. Let's assume that we
+	 * don't care for tests.
+	 */
+	while (!TAILQ_EMPTY(&pci_device_list)) {
+		dev = TAILQ_FIRST(&pci_device_list);
+		TAILQ_REMOVE(&pci_device_list, dev, next);
+	}
+
+	/* Restore original devices & drivers lists */
 	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
 		dr = TAILQ_FIRST(&real_pci_driver_list);
 		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
 		rte_eal_pci_register(dr);
 	}
 
+	while (!TAILQ_EMPTY(&real_pci_device_list)) {
+		dev = TAILQ_FIRST(&real_pci_device_list);
+		TAILQ_REMOVE(&real_pci_device_list, dev, next);
+		TAILQ_INSERT_TAIL(&pci_device_list, dev, next);
+	}
+
 	return 0;
 }
 
@@ -224,9 +278,37 @@ test_pci_blacklist(void)
 	return 0;
 }
 
+static int test_pci_sysfs(void)
+{
+	const char *orig;
+	const char *newpath;
+	int ret;
+
+	orig = pci_get_sysfs_path();
+	ret = setenv("SYSFS_PCI_DEVICES", "My Documents", 1);
+	TEST_ASSERT_SUCCESS(ret, "Failed setenv to My Documents");
+
+	newpath = pci_get_sysfs_path();
+	TEST_ASSERT(!strcmp(newpath, "My Documents"),
+			"pci_get_sysfs_path() should return 'My Documents' "
+			"but gives %s", newpath);
+
+	ret = setenv("SYSFS_PCI_DEVICES", orig, 1);
+	TEST_ASSERT_SUCCESS(ret, "Failed setenv back to '%s'", orig);
+
+	newpath = pci_get_sysfs_path();
+	TEST_ASSERT(!strcmp(orig, newpath),
+			"pci_get_sysfs_path returned unexpected path: "
+			"%s (expected: %s)", newpath, orig);
+	return 0;
+}
+
 int
 test_pci(void)
 {
+	if (test_pci_sysfs())
+		return -1;
+
 	if (test_pci_setup())
 		return -1;
 
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class
new file mode 100644
index 0000000..2f9c1da
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class
@@ -0,0 +1 @@
+0x020000
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/config b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/config
new file mode 100644
index 0000000000000000000000000000000000000000..7752421cf13a5aa00a28eaee02cac2add9ce5566
GIT binary patch
literal 64
zcmZo`_$|QBBEZ1Nz`(@7(7?dMz;S^A2oxWHNCpNT2LUi2#BOU~22l(SV3L7>8>k5Y
DD^Ld(

literal 0
HcmV?d00001

diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits
new file mode 100644
index 0000000..900731f
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits
@@ -0,0 +1 @@
+64
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device
new file mode 100644
index 0000000..9e4789e
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device
@@ -0,0 +1 @@
+0x10fb
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits
new file mode 100644
index 0000000..900731f
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits
@@ -0,0 +1 @@
+64
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable
@@ -0,0 +1 @@
+1
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq
@@ -0,0 +1 @@
+0
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias
new file mode 100644
index 0000000..f4c76ed
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias
@@ -0,0 +1 @@
+pci:v00008086d000010FBsv00008086sd00000003bc02sc00i00
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus
@@ -0,0 +1 @@
+1
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node
new file mode 100644
index 0000000..3a2e3f4
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node
@@ -0,0 +1 @@
+-1
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource
new file mode 100644
index 0000000..f388929
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource
@@ -0,0 +1,13 @@
+0x00000000d0080000 0x00000000d00fffff 0x000000000014220c
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x000000000000e020 0x000000000000e03f 0x0000000000040101
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x00000000d0104000 0x00000000d0107fff 0x000000000014220c
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x00000000ab000000 0x00000000ab0fffff 0x0000000000140204
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x00000000ab100000 0x00000000ab1fffff 0x0000000000140204
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x0000000000000000 0x0000000000000000 0x0000000000000000
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs
@@ -0,0 +1 @@
+0
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs
new file mode 100644
index 0000000..4b9026d
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs
@@ -0,0 +1 @@
+63
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device
new file mode 100644
index 0000000..89a932c
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device
@@ -0,0 +1 @@
+0x0003
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor
new file mode 100644
index 0000000..ce6dc4d
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor
@@ -0,0 +1 @@
+0x8086
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent
new file mode 100644
index 0000000..1dbe34d
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent
@@ -0,0 +1,6 @@
+DRIVER=ixgbe
+PCI_CLASS=20000
+PCI_ID=8086:10FB
+PCI_SUBSYS_ID=8086:0003
+PCI_SLOT_NAME=0000:01:00.0
+MODALIAS=pci:v00008086d000010FBsv00008086sd00000003bc02sc00i00
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor
new file mode 100644
index 0000000..ce6dc4d
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor
@@ -0,0 +1 @@
+0x8086
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 78c43b0..985a8d6 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1481,7 +1481,7 @@ rte_szedata2_eth_dev_init(struct rte_eth_dev *dev)
 		return -EINVAL;
 	}
 	snprintf(rsc_filename, PATH_MAX,
-		SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/resource%u",
+		"%s/" PCI_PRI_FMT "/resource%u", pci_get_sysfs_path(),
 		pci_addr->domain, pci_addr->bus,
 		pci_addr->devid, pci_addr->function, PCI_RESOURCE_NUMBER);
 	fd = open(rsc_filename, O_RDWR);
diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
index 9cdca06..845141b 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -179,7 +179,7 @@ legacy_virtio_has_msix(const struct rte_pci_addr *loc)
 	char dirname[PATH_MAX];
 
 	snprintf(dirname, sizeof(dirname),
-		     SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/msi_irqs",
+		     "%s/" PCI_PRI_FMT "/msi_irqs", pci_get_sysfs_path(),
 		     loc->domain, loc->bus, loc->devid, loc->function);
 
 	d = opendir(dirname);
diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
index 58c2951..f8c3dea 100644
--- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
@@ -151,3 +151,10 @@ DPDK_16.04 {
 	rte_eal_primary_proc_alive;
 
 } DPDK_2.2;
+
+DPDK_16.07 {
+	global:
+
+	pci_get_sysfs_path;
+
+} DPDK_16.04;
diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c
index 3cae4cb..0ec3b61 100644
--- a/lib/librte_eal/common/eal_common_pci.c
+++ b/lib/librte_eal/common/eal_common_pci.c
@@ -85,6 +85,19 @@
 struct pci_driver_list pci_driver_list;
 struct pci_device_list pci_device_list;
 
+#define SYSFS_PCI_DEVICES "/sys/bus/pci/devices"
+
+const char *pci_get_sysfs_path(void)
+{
+	const char *path = NULL;
+
+	path = getenv("SYSFS_PCI_DEVICES");
+	if (path == NULL)
+		return SYSFS_PCI_DEVICES;
+
+	return path;
+}
+
 static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev)
 {
 	struct rte_devargs *devargs;
diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index 8fa2712..7669fd7 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -91,7 +91,7 @@ extern struct pci_driver_list pci_driver_list; /**< Global list of PCI drivers.
 extern struct pci_device_list pci_device_list; /**< Global list of PCI devices. */
 
 /** Pathname of PCI devices directory. */
-#define SYSFS_PCI_DEVICES "/sys/bus/pci/devices"
+const char *pci_get_sysfs_path(void);
 
 /** Formatting string for PCI device identifier: Ex: 0000:00:01.0 */
 #define PCI_PRI_FMT "%.4" PRIx16 ":%.2" PRIx8 ":%.2" PRIx8 ".%" PRIx8
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index bdc08a0..5041228 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -66,8 +66,8 @@ pci_unbind_kernel_driver(struct rte_pci_device *dev)
 
 	/* open /sys/bus/pci/devices/AAAA:BB:CC.D/driver */
 	snprintf(filename, sizeof(filename),
-	         SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/driver/unbind",
-	         loc->domain, loc->bus, loc->devid, loc->function);
+		"%s/" PCI_PRI_FMT "/driver/unbind", pci_get_sysfs_path(),
+		loc->domain, loc->bus, loc->devid, loc->function);
 
 	f = fopen(filename, "w");
 	if (f == NULL) /* device was not bound */
@@ -453,7 +453,7 @@ rte_eal_pci_scan(void)
 	uint16_t domain;
 	uint8_t bus, devid, function;
 
-	dir = opendir(SYSFS_PCI_DEVICES);
+	dir = opendir(pci_get_sysfs_path());
 	if (dir == NULL) {
 		RTE_LOG(ERR, EAL, "%s(): opendir failed: %s\n",
 			__func__, strerror(errno));
@@ -468,8 +468,8 @@ rte_eal_pci_scan(void)
 				&bus, &devid, &function) != 0)
 			continue;
 
-		snprintf(dirname, sizeof(dirname), "%s/%s", SYSFS_PCI_DEVICES,
-			 e->d_name);
+		snprintf(dirname, sizeof(dirname), "%s/%s",
+				pci_get_sysfs_path(), e->d_name);
 		if (pci_scan_one(dirname, domain, bus, devid, function) < 0)
 			goto error;
 	}
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
index 068694d..b833244 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
@@ -161,14 +161,14 @@ pci_get_uio_dev(struct rte_pci_device *dev, char *dstbuf,
 	 * or uio:uioX */
 
 	snprintf(dirname, sizeof(dirname),
-			SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/uio",
+			"%s/" PCI_PRI_FMT "/uio", pci_get_sysfs_path(),
 			loc->domain, loc->bus, loc->devid, loc->function);
 
 	dir = opendir(dirname);
 	if (dir == NULL) {
 		/* retry with the parent directory */
 		snprintf(dirname, sizeof(dirname),
-				SYSFS_PCI_DEVICES "/" PCI_PRI_FMT,
+				"%s/" PCI_PRI_FMT, pci_get_sysfs_path(),
 				loc->domain, loc->bus, loc->devid, loc->function);
 		dir = opendir(dirname);
 
@@ -319,7 +319,8 @@ pci_uio_map_resource_by_index(struct rte_pci_device *dev, int res_idx,
 
 	/* update devname for mmap  */
 	snprintf(devname, sizeof(devname),
-			SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/resource%d",
+			"%s/" PCI_PRI_FMT "/resource%d",
+			pci_get_sysfs_path(),
 			loc->domain, loc->bus, loc->devid,
 			loc->function, res_idx);
 
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c b/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c
index 10266f8..f91b924 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c
@@ -602,7 +602,7 @@ pci_vfio_get_group_no(const char *pci_addr, int *iommu_group_no)
 
 	/* try to find out IOMMU group for this device */
 	snprintf(linkname, sizeof(linkname),
-			 SYSFS_PCI_DEVICES "/%s/iommu_group", pci_addr);
+			 "%s/%s/iommu_group", pci_get_sysfs_path(), pci_addr);
 
 	ret = readlink(linkname, filename, sizeof(filename));
 
diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
index 12503ef..3d0ff93 100644
--- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
@@ -154,3 +154,10 @@ DPDK_16.04 {
 	rte_eal_primary_proc_alive;
 
 } DPDK_2.2;
+
+DPDK_16.07 {
+	global:
+
+	pci_get_sysfs_path;
+
+} DPDK_16.04;
-- 
2.8.0

[PATCH v4 10/10] app/test: do not dump PCI devices in blacklist test

[Jan Viktorin]

Dumping of devices in a unittest is useless. Instead, test whether
the test has been set up well - i.e. there are no devices.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/test_pci.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 362ae3e..8051e53 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -238,7 +238,8 @@ test_pci_blacklist(void)
 	struct rte_devargs_list save_devargs_list;
 
 	printf("Dump all devices\n");
-	rte_eal_pci_dump(stdout);
+	TEST_ASSERT(TAILQ_EMPTY(&pci_driver_list),
+			"pci_driver_list not empty");
 
 	rte_eal_pci_register(&my_driver);
 	rte_eal_pci_register(&my_driver2);
-- 
2.8.0

[PATCH v3 01/11] app/test: introduce resources for tests

[Jan Viktorin]

Certain internal mechanisms of DPDK access different file system structures
(e.g. /sys/bus/pci/devices). It is difficult to test those cases automatically
by a unit test when such path is not hard-coded and there is no simple way how
to distribute fake ones with the current testing environment.

This patch adds a possibility to declare a resource embedded in the test binary
itself. The structure resource cover the generic situation - it provides a name
for lookup and pointers to the embedded data blob. A resource is registered
in a constructor by the macro REGISTER_RESOURCE.

Some initial tests of simple resources is included and added into the group_1.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v3:
* fixed doc comments
---
 app/test/Makefile         |  2 +
 app/test/autotest_data.py |  6 +++
 app/test/resource.c       | 66 +++++++++++++++++++++++++++++++
 app/test/resource.h       | 98 +++++++++++++++++++++++++++++++++++++++++++++++
 app/test/test_resource.c  | 75 ++++++++++++++++++++++++++++++++++++
 5 files changed, 247 insertions(+)
 create mode 100644 app/test/resource.c
 create mode 100644 app/test/resource.h
 create mode 100644 app/test/test_resource.c

diff --git a/app/test/Makefile b/app/test/Makefile
index a4907d5..7fbdd18 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -43,6 +43,8 @@ APP = test
 #
 SRCS-$(CONFIG_RTE_LIBRTE_CMDLINE) := commands.c
 SRCS-y += test.c
+SRCS-y += resource.c
+SRCS-y += test_resource.c
 SRCS-y += test_pci.c
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
diff --git a/app/test/autotest_data.py b/app/test/autotest_data.py
index dde4511..a80a105 100644
--- a/app/test/autotest_data.py
+++ b/app/test/autotest_data.py
@@ -88,6 +88,12 @@ parallel_test_group_list = [
 		 "Report" :	None,
 		},
 		{
+		 "Name" :	"Resource autotest",
+		 "Command" :	"resource_autotest",
+		 "Func" :	default_autotest,
+		 "Report" :	None,
+		},
+		{
 		 "Name" :	"Dump log history",
 		 "Command" :	"dump_log_history",
 		 "Func" :	dump_autotest,
diff --git a/app/test/resource.c b/app/test/resource.c
new file mode 100644
index 0000000..30513db
--- /dev/null
+++ b/app/test/resource.c
@@ -0,0 +1,66 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2016 RehiveTech. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of RehiveTech nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <string.h>
+#include <errno.h>
+#include <sys/queue.h>
+
+#include <rte_debug.h>
+
+#include "resource.h"
+
+struct resource_list resource_list = TAILQ_HEAD_INITIALIZER(resource_list);
+
+size_t resource_size(const struct resource *r)
+{
+	return r->end - r->begin;
+}
+
+const struct resource *resource_find(const char *name)
+{
+	struct resource *r;
+
+	TAILQ_FOREACH(r, &resource_list, next) {
+		RTE_VERIFY(r->name);
+
+		if (!strcmp(r->name, name))
+			return r;
+	}
+
+	return NULL;
+}
+
+void resource_register(struct resource *r)
+{
+	TAILQ_INSERT_TAIL(&resource_list, r, next);
+}
diff --git a/app/test/resource.h b/app/test/resource.h
new file mode 100644
index 0000000..37c589c
--- /dev/null
+++ b/app/test/resource.h
@@ -0,0 +1,98 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2016 RehiveTech. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of RehiveTech nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RESOURCE_H_
+#define _RESOURCE_H_
+
+/**
+ * @file
+ *
+ * Test Resource API
+ *
+ * Each test can require and use some external resources. Usually, an external
+ * resource is a file or a filesystem sub-hierarchy. A resource is included
+ * inside the test executable.
+ */
+
+#include <sys/queue.h>
+#include <stddef.h>
+
+#include <rte_eal.h>
+#include <rte_common.h>
+
+TAILQ_HEAD(resource_list, resource);
+extern struct resource_list resource_list;
+
+/**
+ * Representation of a resource. It points to the resource's binary data.
+ * The semantics of the binary data are defined by the target test.
+ */
+struct resource {
+	const char *name;  /**< Unique name of the resource */
+	const char *begin; /**< Start of resource data */
+	const char *end;   /**< End of resource data */
+	TAILQ_ENTRY(resource) next;
+};
+
+/**
+ * @return size of the given resource
+ */
+size_t resource_size(const struct resource *r);
+
+/**
+ * Find a resource by name in the global list of resources.
+ */
+const struct resource *resource_find(const char *name);
+
+/**
+ * Register a resource in the global list of resources.
+ * Not intended for direct use, please check the REGISTER_RESOURCE
+ * macro.
+ */
+void resource_register(struct resource *r);
+
+/**
+ * Definition of a resource described by its name, and pointers begin, end.
+ */
+#define REGISTER_RESOURCE(n, b, e) \
+static struct resource linkres_ ##n = {       \
+	.name = RTE_STR(n),     \
+	.begin = b,             \
+	.end = e,               \
+};                              \
+static void __attribute__((constructor,used)) resinitfn_ ##n(void) \
+{                               \
+	resource_register(&linkres_ ##n);  \
+}
+
+#endif
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
new file mode 100644
index 0000000..69391ad
--- /dev/null
+++ b/app/test/test_resource.c
@@ -0,0 +1,75 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2016 RehiveTech. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of RehiveTech nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <stdio.h>
+#include <string.h>
+
+#include "test.h"
+#include "resource.h"
+
+const char test_resource_dpdk_blob[] = {
+	'\x44', '\x50', '\x44', '\x4b', '\x00'
+};
+
+REGISTER_RESOURCE(test_resource_dpdk,
+		test_resource_dpdk_blob, test_resource_dpdk_blob + 4);
+
+static int test_resource_dpdk(void)
+{
+	const struct resource *r;
+
+	r = resource_find("test_resource_dpdk");
+	TEST_ASSERT_NOT_NULL(r, "Could not find test_resource_dpdk");
+	TEST_ASSERT(!strcmp(r->name, "test_resource_dpdk"),
+			"Found resource %s, expected test_resource_dpdk",
+			r->name);
+
+	TEST_ASSERT(!strncmp("DPDK", r->begin, 4),
+			"Unexpected payload: %.4s...", r->begin);
+
+	return 0;
+}
+
+static int test_resource(void)
+{
+	if (test_resource_dpdk())
+		return -1;
+
+	return 0;
+}
+
+static struct test_command resource_cmd = {
+	.command = "resource_autotest",
+	.callback = test_resource,
+};
+REGISTER_TEST_COMMAND(resource_cmd);
-- 
2.8.0

Re: [PATCH v3 01/11] app/test: introduce resources for tests

[Jan Viktorin]

I forgot to fix this:

Check patch error:
12817: 
ERROR: space required after that ',' (ctx:VxV)
#244: FILE: app/test/resource.h:93:
+static void __attribute__((constructor,used)) resinitfn_ ##n(void)                                        ^
   
   total: 1 errors, 0 warnings, 259 lines checked
   
will do for v4.

Jan

On Tue, 17 May 2016 20:34:51 +0200
Jan Viktorin <viktorin@rehivetech.com> wrote:

> Certain internal mechanisms of DPDK access different file system structures
> (e.g. /sys/bus/pci/devices). It is difficult to test those cases automatically
> by a unit test when such path is not hard-coded and there is no simple way how
> to distribute fake ones with the current testing environment.
> 
> This patch adds a possibility to declare a resource embedded in the test binary
> itself. The structure resource cover the generic situation - it provides a name
> for lookup and pointers to the embedded data blob. A resource is registered
> in a constructor by the macro REGISTER_RESOURCE.
> 
> Some initial tests of simple resources is included and added into the group_1.
> 
> Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
> ---
> v3:
> * fixed doc comments
> ---
>  app/test/Makefile         |  2 +
>  app/test/autotest_data.py |  6 +++
>  app/test/resource.c       | 66 +++++++++++++++++++++++++++++++
>  app/test/resource.h       | 98 +++++++++++++++++++++++++++++++++++++++++++++++
>  app/test/test_resource.c  | 75 ++++++++++++++++++++++++++++++++++++
>  5 files changed, 247 insertions(+)
>  create mode 100644 app/test/resource.c
>  create mode 100644 app/test/resource.h
>  create mode 100644 app/test/test_resource.c

[...]

[PATCH v3 02/11] mk: define objcopy-specific target and arch

[Jan Viktorin]

The program objcopy uses non-standard conventions to name the target and arch.
Define the values for supported architectures.

FIXME: tile and ppc_64 are not present.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 mk/arch/arm/rte.vars.mk     | 5 +++++
 mk/arch/arm64/rte.vars.mk   | 5 +++++
 mk/arch/i686/rte.vars.mk    | 5 +++++
 mk/arch/x86_64/rte.vars.mk  | 5 +++++
 mk/arch/x86_x32/rte.vars.mk | 5 +++++
 5 files changed, 25 insertions(+)

diff --git a/mk/arch/arm/rte.vars.mk b/mk/arch/arm/rte.vars.mk
index bd85140..2f8cf7c 100644
--- a/mk/arch/arm/rte.vars.mk
+++ b/mk/arch/arm/rte.vars.mk
@@ -37,3 +37,8 @@ CPU_LDFLAGS ?=
 CPU_ASFLAGS ?= -felf
 
 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf32-littlearm
+RTE_OBJCOPY_ARCH = arm
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
diff --git a/mk/arch/arm64/rte.vars.mk b/mk/arch/arm64/rte.vars.mk
index 32e3a5f..c168426 100644
--- a/mk/arch/arm64/rte.vars.mk
+++ b/mk/arch/arm64/rte.vars.mk
@@ -56,3 +56,8 @@ CPU_LDFLAGS ?=
 CPU_ASFLAGS ?= -felf
 
 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf64-littleaarch64
+RTE_OBJCOPY_ARCH = aarch64
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
diff --git a/mk/arch/i686/rte.vars.mk b/mk/arch/i686/rte.vars.mk
index 8ba9a23..6a25312 100644
--- a/mk/arch/i686/rte.vars.mk
+++ b/mk/arch/i686/rte.vars.mk
@@ -57,3 +57,8 @@ CPU_LDFLAGS ?= -melf_i386
 CPU_ASFLAGS ?= -felf
 
 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf32-i386
+RTE_OBJCOPY_ARCH = i386
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
diff --git a/mk/arch/x86_64/rte.vars.mk b/mk/arch/x86_64/rte.vars.mk
index b986f04..83723c8 100644
--- a/mk/arch/x86_64/rte.vars.mk
+++ b/mk/arch/x86_64/rte.vars.mk
@@ -57,3 +57,8 @@ CPU_LDFLAGS ?=
 CPU_ASFLAGS ?= -felf64
 
 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf64-x86-64
+RTE_OBJCOPY_ARCH = i386:x86-64
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
diff --git a/mk/arch/x86_x32/rte.vars.mk b/mk/arch/x86_x32/rte.vars.mk
index 3103dfc..676f316 100644
--- a/mk/arch/x86_x32/rte.vars.mk
+++ b/mk/arch/x86_x32/rte.vars.mk
@@ -61,3 +61,8 @@ ifneq ($(shell echo | $(CC) $(CPU_CFLAGS) -E - 2>/dev/null 1>/dev/null && echo 0
 endif
 
 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
+
+RTE_OBJCOPY_TARGET = elf32-x86-64
+RTE_OBJCOPY_ARCH = i386:x86-64
+
+export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH
-- 
2.8.0

[PATCH v3 03/11] app/test: support resources externally linked

[Jan Viktorin]

To include resources from other source that the C source code we can take
advantage of the objcopy behaviour, i.e. packing of an arbitrary file as an
object file that is linked to the target program.

A linked object file is always accessible as a pair

extern const char beg_<name>;
extern const char end_<name>;
(extern const char siz_<name>;)

A unit test that packs the resource.c source file is included.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/Makefile        | 19 +++++++++++++++++++
 app/test/resource.h      | 10 ++++++++++
 app/test/test_resource.c | 18 ++++++++++++++++++
 3 files changed, 47 insertions(+)

diff --git a/app/test/Makefile b/app/test/Makefile
index 7fbdd18..cfe68a6 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -33,6 +33,24 @@ include $(RTE_SDK)/mk/rte.vars.mk
 
 ifeq ($(CONFIG_RTE_APP_TEST),y)
 
+# Define an externally linked resource. A linked resource is an arbitrary
+# file that is linked into the test binary. The application refers to this
+# resource by name. The linked generates identifiers beg_<name> and end_<name>
+# for referencing by the C code.
+#
+# Parameters: <unique name>, <file to be linked>
+define linked_resource
+SRCS-y += $(1).res.o
+$(1).res.o: $(2)
+	$(OBJCOPY) -I binary -B $(RTE_OBJCOPY_ARCH) -O $(RTE_OBJCOPY_TARGET) \
+		--rename-section                                         \
+			.data=.rodata,alloc,load,data,contents,readonly  \
+		--redefine-sym _binary__dev_stdin_start=beg_$(1)         \
+		--redefine-sym _binary__dev_stdin_end=end_$(1)           \
+		--redefine-sym _binary__dev_stdin_size=siz_$(1)          \
+		/dev/stdin $$@ < $$<
+endef
+
 #
 # library name
 #
@@ -45,6 +63,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_CMDLINE) := commands.c
 SRCS-y += test.c
 SRCS-y += resource.c
 SRCS-y += test_resource.c
+$(eval $(call linked_resource,test_resource_c,resource.c))
 SRCS-y += test_pci.c
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
diff --git a/app/test/resource.h b/app/test/resource.h
index 37c589c..2ef817f 100644
--- a/app/test/resource.h
+++ b/app/test/resource.h
@@ -82,6 +82,16 @@ const struct resource *resource_find(const char *name);
 void resource_register(struct resource *r);
 
 /**
+ * Definition of a resource linked externally (by means of the used toolchain).
+ * Only the base name of the resource is expected. The name refers to the
+ * linked pointers beg_<name> and end_<name> provided externally.
+ */
+#define REGISTER_LINKED_RESOURCE(n) \
+extern const char beg_ ##n;         \
+extern const char end_ ##n;         \
+REGISTER_RESOURCE(n, &beg_ ##n, &end_ ##n); \
+
+/**
  * Definition of a resource described by its name, and pointers begin, end.
  */
 #define REGISTER_RESOURCE(n, b, e) \
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
index 69391ad..b397fa8 100644
--- a/app/test/test_resource.c
+++ b/app/test/test_resource.c
@@ -60,11 +60,29 @@ static int test_resource_dpdk(void)
 	return 0;
 }
 
+REGISTER_LINKED_RESOURCE(test_resource_c);
+
+static int test_resource_c(void)
+{
+	const struct resource *r;
+
+	r = resource_find("test_resource_c");
+	TEST_ASSERT_NOT_NULL(r, "No test_resource_c found");
+	TEST_ASSERT(!strcmp(r->name, "test_resource_c"),
+			"Found resource %s, expected test_resource_c",
+			r->name);
+
+	return 0;
+}
+
 static int test_resource(void)
 {
 	if (test_resource_dpdk())
 		return -1;
 
+	if (test_resource_c())
+		return -1;
+
 	return 0;
 }
 
-- 
2.8.0

[PATCH v3 04/11] app/test: add functions to create files from resources

[Jan Viktorin]

A resource can be written into the target filesystem by calling resource_fwrite
or resource_fwrite_file. Such file can be created before a test is started and
removed after the test finishes.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/resource.c      | 35 +++++++++++++++++++++++++++++++++++
 app/test/resource.h      | 14 ++++++++++++++
 app/test/test_resource.c | 10 ++++++++++
 3 files changed, 59 insertions(+)

diff --git a/app/test/resource.c b/app/test/resource.c
index 30513db..acb63c1 100644
--- a/app/test/resource.c
+++ b/app/test/resource.c
@@ -31,6 +31,7 @@
  *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+#include <stdio.h>
 #include <string.h>
 #include <errno.h>
 #include <sys/queue.h>
@@ -60,6 +61,40 @@ const struct resource *resource_find(const char *name)
 	return NULL;
 }
 
+int resource_fwrite(const struct resource *r, FILE *f)
+{
+	const size_t goal = resource_size(r);
+	size_t total = 0;
+
+	while (total < goal) {
+		size_t wlen = fwrite(r->begin + total, 1, goal - total, f);
+		if (wlen == 0) {
+			perror(__func__);
+			return -1;
+		}
+
+		total += wlen;
+	}
+
+	return 0;
+}
+
+int resource_fwrite_file(const struct resource *r, const char *fname)
+{
+	FILE *f;
+	int ret;
+
+	f = fopen(fname, "w");
+	if (f == NULL) {
+		perror(__func__);
+		return -1;
+	}
+
+	ret = resource_fwrite(r, f);
+	fclose(f);
+	return ret;
+}
+
 void resource_register(struct resource *r)
 {
 	TAILQ_INSERT_TAIL(&resource_list, r, next);
diff --git a/app/test/resource.h b/app/test/resource.h
index 2ef817f..849df05 100644
--- a/app/test/resource.h
+++ b/app/test/resource.h
@@ -45,6 +45,7 @@
  */
 
 #include <sys/queue.h>
+#include <stdio.h>
 #include <stddef.h>
 
 #include <rte_eal.h>
@@ -75,6 +76,19 @@ size_t resource_size(const struct resource *r);
 const struct resource *resource_find(const char *name);
 
 /**
+ * Write the raw data of the resource to the given file.
+ * @return 0 on success
+ */
+int resource_fwrite(const struct resource *r, FILE *f);
+
+/**
+ * Write the raw data of the resource to the given file given by name.
+ * The name is relative to the current working directory.
+ * @return 0 on success
+ */
+int resource_fwrite_file(const struct resource *r, const char *fname);
+
+/**
  * Register a resource in the global list of resources.
  * Not intended for direct use, please check the REGISTER_RESOURCE
  * macro.
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
index b397fa8..3d1bf00 100644
--- a/app/test/test_resource.c
+++ b/app/test/test_resource.c
@@ -65,6 +65,7 @@ REGISTER_LINKED_RESOURCE(test_resource_c);
 static int test_resource_c(void)
 {
 	const struct resource *r;
+	FILE *f;
 
 	r = resource_find("test_resource_c");
 	TEST_ASSERT_NOT_NULL(r, "No test_resource_c found");
@@ -72,6 +73,15 @@ static int test_resource_c(void)
 			"Found resource %s, expected test_resource_c",
 			r->name);
 
+	TEST_ASSERT_SUCCESS(resource_fwrite_file(r, "test_resource.c"),
+			"Failed to to write file %s", r->name);
+
+	f = fopen("test_resource.c", "r");
+	TEST_ASSERT_NOT_NULL(f,
+			"Missing extracted file resource.c");
+	fclose(f);
+	remove("test_resource.c");
+
 	return 0;
 }
 
-- 
2.8.0

[PATCH v3 05/11] app/test: support resources archived by tar

[Jan Viktorin]

When a more complex resource (a file hierarchy) is needed, packing every single
file as a single resource would be very ineffective. For that purpose, it is
possible to pack the files into a tar archive, extract it before test from the
resource and finally clean up all the created files.

This patch introduces functions resource_untar and resource_rm_by_tar to
perform those tasks. An example of using those functions is included as a test.

A new dependency is required to build the app/test: libarchive.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/Makefile        |   9 +++
 app/test/resource.c      | 195 +++++++++++++++++++++++++++++++++++++++++++++++
 app/test/resource.h      |  13 ++++
 app/test/test_resource.c |  29 +++++++
 4 files changed, 246 insertions(+)

diff --git a/app/test/Makefile b/app/test/Makefile
index cfe68a6..de5fa50 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -51,6 +51,13 @@ $(1).res.o: $(2)
 		/dev/stdin $$@ < $$<
 endef
 
+define linked_tar_resource
+$(1).tar: $(2)
+	tar -C $$(dir $$<) -cf $$@ $$(notdir $$<)
+
+$(call linked_resource,$(1),$(1).tar)
+endef
+
 #
 # library name
 #
@@ -64,6 +71,7 @@ SRCS-y += test.c
 SRCS-y += resource.c
 SRCS-y += test_resource.c
 $(eval $(call linked_resource,test_resource_c,resource.c))
+$(eval $(call linked_tar_resource,test_resource_tar,test_resource.c))
 SRCS-y += test_pci.c
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
@@ -183,6 +191,7 @@ CFLAGS += $(WERROR_FLAGS)
 CFLAGS += -D_GNU_SOURCE
 
 LDLIBS += -lm
+LDLIBS += -larchive
 
 # Disable VTA for memcpy test
 ifeq ($(CC), gcc)
diff --git a/app/test/resource.c b/app/test/resource.c
index acb63c1..668fc52 100644
--- a/app/test/resource.c
+++ b/app/test/resource.c
@@ -33,6 +33,8 @@
 
 #include <stdio.h>
 #include <string.h>
+#include <archive.h>
+#include <archive_entry.h>
 #include <errno.h>
 #include <sys/queue.h>
 
@@ -95,6 +97,199 @@ int resource_fwrite_file(const struct resource *r, const char *fname)
 	return ret;
 }
 
+static int do_copy(struct archive *r, struct archive *w)
+{
+	const void *buf;
+	size_t len;
+	off_t off;
+	int ret;
+
+	while (1) {
+		ret = archive_read_data_block(r, &buf, &len, &off);
+		if (ret == ARCHIVE_RETRY)
+			continue;
+
+		if (ret == ARCHIVE_EOF)
+			return 0;
+
+		if (ret != ARCHIVE_OK)
+			return ret;
+
+		do {
+			ret = archive_write_data_block(w, buf, len, off);
+			if (ret != ARCHIVE_OK && ret != ARCHIVE_RETRY)
+				return ret;
+		} while (ret != ARCHIVE_OK);
+	}
+}
+
+int resource_untar(const struct resource *res)
+{
+	struct archive *r;
+	struct archive *w;
+	struct archive_entry *e;
+	void *p;
+	int flags = 0;
+	int ret;
+
+	p = malloc(resource_size(res));
+	if (p == NULL)
+		rte_panic("Failed to malloc %zu B\n", resource_size(res));
+
+	memcpy(p, res->begin, resource_size(res));
+
+	r = archive_read_new();
+	if (r == NULL) {
+		free(p);
+		return -1;
+	}
+
+	archive_read_support_format_all(r);
+	archive_read_support_filter_all(r);
+
+	w = archive_write_disk_new();
+	if (w == NULL) {
+		archive_read_free(r);
+		free(p);
+		return -1;
+	}
+
+	flags |= ARCHIVE_EXTRACT_PERM;
+	flags |= ARCHIVE_EXTRACT_FFLAGS;
+	archive_write_disk_set_options(w, flags);
+	archive_write_disk_set_standard_lookup(w);
+
+	ret = archive_read_open_memory(r, p, resource_size(res));
+	if (ret != ARCHIVE_OK)
+		goto fail;
+
+	while (1) {
+		ret = archive_read_next_header(r, &e);
+		if (ret == ARCHIVE_EOF)
+			break;
+		if (ret != ARCHIVE_OK)
+			goto fail;
+
+		ret = archive_write_header(w, e);
+		if (ret == ARCHIVE_EOF)
+			break;
+		if (ret != ARCHIVE_OK)
+			goto fail;
+
+		if (archive_entry_size(e) == 0)
+			continue;
+
+		ret = do_copy(r, w);
+		if (ret != ARCHIVE_OK)
+			goto fail;
+
+		ret = archive_write_finish_entry(w);
+		if (ret != ARCHIVE_OK)
+			goto fail;
+	}
+
+	archive_write_free(w);
+	archive_read_free(r);
+	free(p);
+	return 0;
+
+fail:
+	archive_write_free(w);
+	archive_read_free(r);
+	free(p);
+	rte_panic("Failed: %s\n", archive_error_string(r));
+	return -1;
+}
+
+int resource_rm_by_tar(const struct resource *res)
+{
+	struct archive *r;
+	struct archive_entry *e;
+	void *p;
+	int try_again = 1;
+	int attempts = 0;
+	int ret;
+
+	p = malloc(resource_size(res));
+	if (p == NULL)
+		rte_panic("Failed to malloc %zu B\n", resource_size(res));
+
+	memcpy(p, res->begin, resource_size(res));
+
+	/* If somebody creates a file somewhere inside the extracted TAR
+	   hierarchy during a test the resource_rm_by_tar might loop
+	   infinitely. We prevent this by adding the attempts counter there.
+	   In normal case, max N iteration is done where N is the depth of
+	   the file-hierarchy.
+	 */
+	while (try_again && attempts < 10000) {
+		r = archive_read_new();
+		if (r == NULL) {
+			free(p);
+			return -1;
+		}
+
+		archive_read_support_format_all(r);
+		archive_read_support_filter_all(r);
+
+		ret = archive_read_open_memory(r, p, resource_size(res));
+		if (ret != ARCHIVE_OK) {
+			fprintf(stderr, "Failed: %s\n",
+					archive_error_string(r));
+			goto fail;
+		}
+
+		try_again = 0;
+
+		while (1) {
+			ret = archive_read_next_header(r, &e);
+			if (ret == ARCHIVE_EOF)
+				break;
+			if (ret != ARCHIVE_OK)
+				goto fail;
+
+			ret = remove(archive_entry_pathname(e));
+			if (ret < 0) {
+				switch (errno) {
+				case ENOTEMPTY:
+				case EEXIST:
+					try_again = 1;
+					break;
+
+				/* should not usually happen: */
+				case ENOENT:
+				case ENOTDIR:
+				case EROFS:
+					attempts += 1;
+					continue;
+				default:
+					perror("Failed to remove file");
+					goto fail;
+				}
+			}
+		}
+
+		archive_read_free(r);
+		attempts += 1;
+	}
+
+	if (attempts >= 10000) {
+		fprintf(stderr, "Failed to remove archive\n");
+		free(p);
+		return -1;
+	}
+
+	free(p);
+	return 0;
+
+fail:
+	archive_read_free(r);
+	free(p);
+
+	rte_panic("Failed: %s\n", archive_error_string(r));
+	return -1;
+}
+
 void resource_register(struct resource *r)
 {
 	TAILQ_INSERT_TAIL(&resource_list, r, next);
diff --git a/app/test/resource.h b/app/test/resource.h
index 849df05..5a9031a 100644
--- a/app/test/resource.h
+++ b/app/test/resource.h
@@ -89,6 +89,19 @@ int resource_fwrite(const struct resource *r, FILE *f);
 int resource_fwrite_file(const struct resource *r, const char *fname);
 
 /**
+ * Treat the given resource as a tar archive. Extract
+ * the archive to the current directory.
+ */
+int resource_untar(const struct resource *res);
+
+/**
+ * Treat the given resource as a tar archive. Remove
+ * all files (related to the current directory) listed
+ * in the tar archive.
+ */
+int resource_rm_by_tar(const struct resource *res);
+
+/**
  * Register a resource in the global list of resources.
  * Not intended for direct use, please check the REGISTER_RESOURCE
  * macro.
diff --git a/app/test/test_resource.c b/app/test/test_resource.c
index 3d1bf00..1e85040 100644
--- a/app/test/test_resource.c
+++ b/app/test/test_resource.c
@@ -85,6 +85,32 @@ static int test_resource_c(void)
 	return 0;
 }
 
+REGISTER_LINKED_RESOURCE(test_resource_tar);
+
+static int test_resource_tar(void)
+{
+	const struct resource *r;
+	FILE *f;
+
+	r = resource_find("test_resource_tar");
+	TEST_ASSERT_NOT_NULL(r, "No test_resource_tar found");
+	TEST_ASSERT(!strcmp(r->name, "test_resource_tar"),
+			"Found resource %s, expected test_resource_tar",
+			r->name);
+
+	TEST_ASSERT_SUCCESS(resource_untar(r),
+			"Failed to to untar %s", r->name);
+
+	f = fopen("test_resource.c", "r");
+	TEST_ASSERT_NOT_NULL(f,
+			"Missing extracted file test_resource.c");
+	fclose(f);
+
+	TEST_ASSERT_SUCCESS(resource_rm_by_tar(r),
+			"Failed to remove extracted contents of %s", r->name);
+	return 0;
+}
+
 static int test_resource(void)
 {
 	if (test_resource_dpdk())
@@ -93,6 +119,9 @@ static int test_resource(void)
 	if (test_resource_c())
 		return -1;
 
+	if (test_resource_tar())
+		return -1;
+
 	return 0;
 }
 
-- 
2.8.0

[PATCH v3 06/11] app/test: use linked list to store PCI drivers

[Jan Viktorin]

The test unregisters all drivers before start. The drivers were stored
into a fixed-sized array. This is inflexible. This patch change this to
utilize a linked list for the same purpose.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v3:
* fixed commit message
* used "backup" to describe the real_pci_driver_list
---
 app/test/test_pci.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 0ed357e..8b7c8bb 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -144,21 +144,24 @@ static void free_devargs_list(void)
 	}
 }
 
+/* backup real drivers (not used for testing) */
+struct pci_driver_list real_pci_driver_list =
+	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
+
 int
 test_pci(void)
 {
 	struct rte_devargs_list save_devargs_list;
 	struct rte_pci_driver *dr = NULL;
-	struct rte_pci_driver *save_pci_driver_list[NUM_MAX_DRIVERS];
-	unsigned i, num_drivers = 0;
 
 	printf("Dump all devices\n");
 	rte_eal_pci_dump(stdout);
 
 	/* Unregister all previous drivers */
-	TAILQ_FOREACH(dr, &pci_driver_list, next) {
+	while (!TAILQ_EMPTY(&pci_driver_list)) {
+		dr = TAILQ_FIRST(&pci_driver_list);
 		rte_eal_pci_unregister(dr);
-		save_pci_driver_list[num_drivers++] = dr;
+		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
 	}
 
 	rte_eal_pci_register(&my_driver);
@@ -197,8 +200,11 @@ test_pci(void)
 	rte_eal_pci_unregister(&my_driver2);
 
 	/* Restore original driver list */
-	for (i = 0; i < num_drivers; i++)
-		rte_eal_pci_register(save_pci_driver_list[i]);
+	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
+		dr = TAILQ_FIRST(&real_pci_driver_list);
+		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
+		rte_eal_pci_register(dr);
+	}
 
 	return 0;
 }
-- 
2.8.0

[PATCH v3 07/11] app/test: extract test_pci_setup and test_pci_cleanup

[Jan Viktorin]

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/test_pci.c | 47 ++++++++++++++++++++++++++++++++++-------------
 1 file changed, 34 insertions(+), 13 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 8b7c8bb..50078a0 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -148,21 +148,46 @@ static void free_devargs_list(void)
 struct pci_driver_list real_pci_driver_list =
 	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
 
+static int
+test_pci_setup(void)
+{
+	struct rte_pci_driver *dr;
+
+	/* Unregister original driver list */
+	while (!TAILQ_EMPTY(&pci_driver_list)) {
+		dr = TAILQ_FIRST(&pci_driver_list);
+		rte_eal_pci_unregister(dr);
+		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
+	}
+
+	return 0;
+}
+
+static int
+test_pci_cleanup(void)
+{
+	struct rte_pci_driver *dr;
+
+	/* Restore original driver list */
+	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
+		dr = TAILQ_FIRST(&real_pci_driver_list);
+		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
+		rte_eal_pci_register(dr);
+	}
+
+	return 0;
+}
+
 int
 test_pci(void)
 {
 	struct rte_devargs_list save_devargs_list;
-	struct rte_pci_driver *dr = NULL;
 
 	printf("Dump all devices\n");
 	rte_eal_pci_dump(stdout);
 
-	/* Unregister all previous drivers */
-	while (!TAILQ_EMPTY(&pci_driver_list)) {
-		dr = TAILQ_FIRST(&pci_driver_list);
-		rte_eal_pci_unregister(dr);
-		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
-	}
+	if (test_pci_setup())
+		return -1;
 
 	rte_eal_pci_register(&my_driver);
 	rte_eal_pci_register(&my_driver2);
@@ -199,12 +224,8 @@ test_pci(void)
 	rte_eal_pci_unregister(&my_driver);
 	rte_eal_pci_unregister(&my_driver2);
 
-	/* Restore original driver list */
-	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
-		dr = TAILQ_FIRST(&real_pci_driver_list);
-		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
-		rte_eal_pci_register(dr);
-	}
+	if (test_pci_cleanup())
+		return -1;
 
 	return 0;
 }
-- 
2.8.0

[PATCH v3 08/11] app/test: convert current pci_test into a single test case

[Jan Viktorin]

The current test_pci is just a single test case that tests the blacklisting
of devices. Rename it to test_pci_blacklist and call it from the test_pci.
The setup and cleanup are moved out of the test_pci_blacklist entirely
to cover all other tests.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v3:
* improved commit description
---
 app/test/test_pci.c | 85 +++++++++++++++++++++++++++++------------------------
 1 file changed, 47 insertions(+), 38 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 50078a0..6445ab0 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -144,51 +144,14 @@ static void free_devargs_list(void)
 	}
 }
 
-/* backup real drivers (not used for testing) */
-struct pci_driver_list real_pci_driver_list =
-	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
-
 static int
-test_pci_setup(void)
-{
-	struct rte_pci_driver *dr;
-
-	/* Unregister original driver list */
-	while (!TAILQ_EMPTY(&pci_driver_list)) {
-		dr = TAILQ_FIRST(&pci_driver_list);
-		rte_eal_pci_unregister(dr);
-		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
-	}
-
-	return 0;
-}
-
-static int
-test_pci_cleanup(void)
-{
-	struct rte_pci_driver *dr;
-
-	/* Restore original driver list */
-	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
-		dr = TAILQ_FIRST(&real_pci_driver_list);
-		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
-		rte_eal_pci_register(dr);
-	}
-
-	return 0;
-}
-
-int
-test_pci(void)
+test_pci_blacklist(void)
 {
 	struct rte_devargs_list save_devargs_list;
 
 	printf("Dump all devices\n");
 	rte_eal_pci_dump(stdout);
 
-	if (test_pci_setup())
-		return -1;
-
 	rte_eal_pci_register(&my_driver);
 	rte_eal_pci_register(&my_driver2);
 
@@ -224,6 +187,52 @@ test_pci(void)
 	rte_eal_pci_unregister(&my_driver);
 	rte_eal_pci_unregister(&my_driver2);
 
+	return 0;
+}
+
+/* backup real drivers (not used for testing) */
+struct pci_driver_list real_pci_driver_list =
+	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
+
+static int
+test_pci_setup(void)
+{
+	struct rte_pci_driver *dr;
+
+	/* Unregister original driver list */
+	while (!TAILQ_EMPTY(&pci_driver_list)) {
+		dr = TAILQ_FIRST(&pci_driver_list);
+		rte_eal_pci_unregister(dr);
+		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
+	}
+
+	return 0;
+}
+
+static int
+test_pci_cleanup(void)
+{
+	struct rte_pci_driver *dr;
+
+	/* Restore original driver list */
+	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
+		dr = TAILQ_FIRST(&real_pci_driver_list);
+		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
+		rte_eal_pci_register(dr);
+	}
+
+	return 0;
+}
+
+int
+test_pci(void)
+{
+	if (test_pci_setup())
+		return -1;
+
+	if (test_pci_blacklist())
+		return -1;
+
 	if (test_pci_cleanup())
 		return -1;
 
-- 
2.8.0

[PATCH v3 09/11] eal/pci: allow to override sysfs

[Jan Viktorin]

The SYSFS_PCI_DEVICES is a constant that makes the PCI testing difficult as
it points to an absolute path. We remove using this constant and introducing
a function pci_get_sysfs_path that gives the same value. However, the user can
pass a SYSFS_PCI_DEVICES env variable to override the path. It is now possible
to create a fake sysfs hierarchy for testing.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v3:
* changed subject
* test_pci_sysfs has been slightly modified to be more understandable
* fixed whitespace in *version.map files
---
 app/test/test_pci.c                             | 28 +++++++++++++++++++++++++
 drivers/net/szedata2/rte_eth_szedata2.c         |  2 +-
 drivers/net/virtio/virtio_pci.c                 |  2 +-
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  7 +++++++
 lib/librte_eal/common/eal_common_pci.c          | 13 ++++++++++++
 lib/librte_eal/common/include/rte_pci.h         |  2 +-
 lib/librte_eal/linuxapp/eal/eal_pci.c           |  6 +++---
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c       |  7 ++++---
 lib/librte_eal/linuxapp/eal/eal_pci_vfio.c      |  2 +-
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |  7 +++++++
 10 files changed, 66 insertions(+), 10 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 6445ab0..b8b729c 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -190,6 +190,31 @@ test_pci_blacklist(void)
 	return 0;
 }
 
+static int test_pci_sysfs(void)
+{
+	const char *orig;
+	const char *newpath;
+	int ret;
+
+	orig = pci_get_sysfs_path();
+	ret = setenv("SYSFS_PCI_DEVICES", "My Documents", 1);
+	TEST_ASSERT_SUCCESS(ret, "Failed setenv to My Documents");
+
+	newpath = pci_get_sysfs_path();
+	TEST_ASSERT(!strcmp(newpath, "My Documents"),
+			"pci_get_sysfs_path() should return 'My Documents' "
+			"but gives %s", newpath);
+
+	ret = setenv("SYSFS_PCI_DEVICES", orig, 1);
+	TEST_ASSERT_SUCCESS(ret, "Failed setenv back to '%s'", orig);
+
+	newpath = pci_get_sysfs_path();
+	TEST_ASSERT(!strcmp(orig, newpath),
+			"pci_get_sysfs_path returned unexpected path: "
+			"%s (expected: %s)", newpath, orig);
+	return 0;
+}
+
 /* backup real drivers (not used for testing) */
 struct pci_driver_list real_pci_driver_list =
 	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
@@ -227,6 +252,9 @@ test_pci_cleanup(void)
 int
 test_pci(void)
 {
+	if (test_pci_sysfs())
+		return -1;
+
 	if (test_pci_setup())
 		return -1;
 
diff --git a/drivers/net/szedata2/rte_eth_szedata2.c b/drivers/net/szedata2/rte_eth_szedata2.c
index 78c43b0..985a8d6 100644
--- a/drivers/net/szedata2/rte_eth_szedata2.c
+++ b/drivers/net/szedata2/rte_eth_szedata2.c
@@ -1481,7 +1481,7 @@ rte_szedata2_eth_dev_init(struct rte_eth_dev *dev)
 		return -EINVAL;
 	}
 	snprintf(rsc_filename, PATH_MAX,
-		SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/resource%u",
+		"%s/" PCI_PRI_FMT "/resource%u", pci_get_sysfs_path(),
 		pci_addr->domain, pci_addr->bus,
 		pci_addr->devid, pci_addr->function, PCI_RESOURCE_NUMBER);
 	fd = open(rsc_filename, O_RDWR);
diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
index 9cdca06..845141b 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -179,7 +179,7 @@ legacy_virtio_has_msix(const struct rte_pci_addr *loc)
 	char dirname[PATH_MAX];
 
 	snprintf(dirname, sizeof(dirname),
-		     SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/msi_irqs",
+		     "%s/" PCI_PRI_FMT "/msi_irqs", pci_get_sysfs_path(),
 		     loc->domain, loc->bus, loc->devid, loc->function);
 
 	d = opendir(dirname);
diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
index 58c2951..f8c3dea 100644
--- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
@@ -151,3 +151,10 @@ DPDK_16.04 {
 	rte_eal_primary_proc_alive;
 
 } DPDK_2.2;
+
+DPDK_16.07 {
+	global:
+
+	pci_get_sysfs_path;
+
+} DPDK_16.04;
diff --git a/lib/librte_eal/common/eal_common_pci.c b/lib/librte_eal/common/eal_common_pci.c
index 3cae4cb..0ec3b61 100644
--- a/lib/librte_eal/common/eal_common_pci.c
+++ b/lib/librte_eal/common/eal_common_pci.c
@@ -85,6 +85,19 @@
 struct pci_driver_list pci_driver_list;
 struct pci_device_list pci_device_list;
 
+#define SYSFS_PCI_DEVICES "/sys/bus/pci/devices"
+
+const char *pci_get_sysfs_path(void)
+{
+	const char *path = NULL;
+
+	path = getenv("SYSFS_PCI_DEVICES");
+	if (path == NULL)
+		return SYSFS_PCI_DEVICES;
+
+	return path;
+}
+
 static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev)
 {
 	struct rte_devargs *devargs;
diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index 8fa2712..7669fd7 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -91,7 +91,7 @@ extern struct pci_driver_list pci_driver_list; /**< Global list of PCI drivers.
 extern struct pci_device_list pci_device_list; /**< Global list of PCI devices. */
 
 /** Pathname of PCI devices directory. */
-#define SYSFS_PCI_DEVICES "/sys/bus/pci/devices"
+const char *pci_get_sysfs_path(void);
 
 /** Formatting string for PCI device identifier: Ex: 0000:00:01.0 */
 #define PCI_PRI_FMT "%.4" PRIx16 ":%.2" PRIx8 ":%.2" PRIx8 ".%" PRIx8
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index bdc08a0..8e130a2 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -66,7 +66,7 @@ pci_unbind_kernel_driver(struct rte_pci_device *dev)
 
 	/* open /sys/bus/pci/devices/AAAA:BB:CC.D/driver */
 	snprintf(filename, sizeof(filename),
-	         SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/driver/unbind",
+	         "%s/" PCI_PRI_FMT "/driver/unbind", pci_get_sysfs_path(),
 	         loc->domain, loc->bus, loc->devid, loc->function);
 
 	f = fopen(filename, "w");
@@ -453,7 +453,7 @@ rte_eal_pci_scan(void)
 	uint16_t domain;
 	uint8_t bus, devid, function;
 
-	dir = opendir(SYSFS_PCI_DEVICES);
+	dir = opendir(pci_get_sysfs_path());
 	if (dir == NULL) {
 		RTE_LOG(ERR, EAL, "%s(): opendir failed: %s\n",
 			__func__, strerror(errno));
@@ -468,7 +468,7 @@ rte_eal_pci_scan(void)
 				&bus, &devid, &function) != 0)
 			continue;
 
-		snprintf(dirname, sizeof(dirname), "%s/%s", SYSFS_PCI_DEVICES,
+		snprintf(dirname, sizeof(dirname), "%s/%s", pci_get_sysfs_path(),
 			 e->d_name);
 		if (pci_scan_one(dirname, domain, bus, devid, function) < 0)
 			goto error;
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
index 068694d..b833244 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
@@ -161,14 +161,14 @@ pci_get_uio_dev(struct rte_pci_device *dev, char *dstbuf,
 	 * or uio:uioX */
 
 	snprintf(dirname, sizeof(dirname),
-			SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/uio",
+			"%s/" PCI_PRI_FMT "/uio", pci_get_sysfs_path(),
 			loc->domain, loc->bus, loc->devid, loc->function);
 
 	dir = opendir(dirname);
 	if (dir == NULL) {
 		/* retry with the parent directory */
 		snprintf(dirname, sizeof(dirname),
-				SYSFS_PCI_DEVICES "/" PCI_PRI_FMT,
+				"%s/" PCI_PRI_FMT, pci_get_sysfs_path(),
 				loc->domain, loc->bus, loc->devid, loc->function);
 		dir = opendir(dirname);
 
@@ -319,7 +319,8 @@ pci_uio_map_resource_by_index(struct rte_pci_device *dev, int res_idx,
 
 	/* update devname for mmap  */
 	snprintf(devname, sizeof(devname),
-			SYSFS_PCI_DEVICES "/" PCI_PRI_FMT "/resource%d",
+			"%s/" PCI_PRI_FMT "/resource%d",
+			pci_get_sysfs_path(),
 			loc->domain, loc->bus, loc->devid,
 			loc->function, res_idx);
 
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c b/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c
index 10266f8..f91b924 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c
@@ -602,7 +602,7 @@ pci_vfio_get_group_no(const char *pci_addr, int *iommu_group_no)
 
 	/* try to find out IOMMU group for this device */
 	snprintf(linkname, sizeof(linkname),
-			 SYSFS_PCI_DEVICES "/%s/iommu_group", pci_addr);
+			 "%s/%s/iommu_group", pci_get_sysfs_path(), pci_addr);
 
 	ret = readlink(linkname, filename, sizeof(filename));
 
diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
index 12503ef..3d0ff93 100644
--- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
@@ -154,3 +154,10 @@ DPDK_16.04 {
 	rte_eal_primary_proc_alive;
 
 } DPDK_2.2;
+
+DPDK_16.07 {
+	global:
+
+	pci_get_sysfs_path;
+
+} DPDK_16.04;
-- 
2.8.0

Re: [PATCH v3 09/11] eal/pci: allow to override sysfs

[Jan Viktorin]

I forgot to fix those:

   12826: 
   ERROR: code indent should use tabs where possible
   #175: FILE: lib/librte_eal/linuxapp/eal/eal_pci.c:69:
   +^I         "%s/" PCI_PRI_FMT "/driver/unbind", pci_get_sysfs_path(),$
   
   WARNING: line over 80 characters
   #193: FILE: lib/librte_eal/linuxapp/eal/eal_pci.c:471:
   +snprintf(dirname, sizeof(dirname), "%s/%s", pci_get_sysfs_path(),
   
   total: 1 errors, 1 warnings, 160 lines checked
   
   NOTE: whitespace errors detected, you may wish to use scripts/cleanpatch or
         scripts/cleanfile

will do for v4.

Jan

On Tue, 17 May 2016 20:34:59 +0200
Jan Viktorin <viktorin@rehivetech.com> wrote:

> The SYSFS_PCI_DEVICES is a constant that makes the PCI testing difficult as
> it points to an absolute path. We remove using this constant and introducing
> a function pci_get_sysfs_path that gives the same value. However, the user can
> pass a SYSFS_PCI_DEVICES env variable to override the path. It is now possible
> to create a fake sysfs hierarchy for testing.
> 
> Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
> ---
> v3:
> * changed subject
> * test_pci_sysfs has been slightly modified to be more understandable
> * fixed whitespace in *version.map files
> ---
>  app/test/test_pci.c                             | 28 +++++++++++++++++++++++++
>  drivers/net/szedata2/rte_eth_szedata2.c         |  2 +-
>  drivers/net/virtio/virtio_pci.c                 |  2 +-
>  lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  7 +++++++
>  lib/librte_eal/common/eal_common_pci.c          | 13 ++++++++++++
>  lib/librte_eal/common/include/rte_pci.h         |  2 +-
>  lib/librte_eal/linuxapp/eal/eal_pci.c           |  6 +++---
>  lib/librte_eal/linuxapp/eal/eal_pci_uio.c       |  7 ++++---
>  lib/librte_eal/linuxapp/eal/eal_pci_vfio.c      |  2 +-
>  lib/librte_eal/linuxapp/eal/rte_eal_version.map |  7 +++++++
>  10 files changed, 66 insertions(+), 10 deletions(-)
> 

[...]

[PATCH v3 10/11] app/test: scan PCI bus using a fake sysfs

[Jan Viktorin]

Scan the PCI bus by providing a fake sysfs with a PCI device. The fake sysfs
is a packed file hierarchy linked into the test.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/Makefile                                  |   1 +
 app/test/test_pci.c                                |  58 +++++++++++++++++++--
 .../bus/pci/devices/0000:01:00.0/class             |   1 +
 .../bus/pci/devices/0000:01:00.0/config            | Bin 0 -> 64 bytes
 .../devices/0000:01:00.0/consistent_dma_mask_bits  |   1 +
 .../bus/pci/devices/0000:01:00.0/device            |   1 +
 .../bus/pci/devices/0000:01:00.0/dma_mask_bits     |   1 +
 .../bus/pci/devices/0000:01:00.0/enable            |   1 +
 .../bus/pci/devices/0000:01:00.0/irq               |   1 +
 .../bus/pci/devices/0000:01:00.0/modalias          |   1 +
 .../bus/pci/devices/0000:01:00.0/msi_bus           |   1 +
 .../bus/pci/devices/0000:01:00.0/numa_node         |   1 +
 .../bus/pci/devices/0000:01:00.0/resource          |  13 +++++
 .../bus/pci/devices/0000:01:00.0/sriov_numvfs      |   1 +
 .../bus/pci/devices/0000:01:00.0/sriov_totalvfs    |   1 +
 .../bus/pci/devices/0000:01:00.0/subsystem_device  |   1 +
 .../bus/pci/devices/0000:01:00.0/subsystem_vendor  |   1 +
 .../bus/pci/devices/0000:01:00.0/uevent            |   6 +++
 .../bus/pci/devices/0000:01:00.0/vendor            |   1 +
 19 files changed, 89 insertions(+), 3 deletions(-)
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/config
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent
 create mode 100644 app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor

diff --git a/app/test/Makefile b/app/test/Makefile
index de5fa50..1c6c29e 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -73,6 +73,7 @@ SRCS-y += test_resource.c
 $(eval $(call linked_resource,test_resource_c,resource.c))
 $(eval $(call linked_tar_resource,test_resource_tar,test_resource.c))
 SRCS-y += test_pci.c
+$(eval $(call linked_tar_resource,test_pci_sysfs,test_pci_sysfs))
 SRCS-y += test_prefetch.c
 SRCS-y += test_byteorder.c
 SRCS-y += test_per_lcore.c
diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index b8b729c..69066cb 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -43,6 +43,7 @@
 #include <rte_devargs.h>
 
 #include "test.h"
+#include "resource.h"
 
 /* Generic maximum number of drivers to have room to allocate all drivers */
 #define NUM_MAX_DRIVERS 256
@@ -215,37 +216,88 @@ static int test_pci_sysfs(void)
 	return 0;
 }
 
-/* backup real drivers (not used for testing) */
+/* backup real devices & drivers (not used for testing) */
 struct pci_driver_list real_pci_driver_list =
 	TAILQ_HEAD_INITIALIZER(real_pci_driver_list);
+struct pci_device_list real_pci_device_list =
+	TAILQ_HEAD_INITIALIZER(real_pci_device_list);
+
+REGISTER_LINKED_RESOURCE(test_pci_sysfs);
 
 static int
 test_pci_setup(void)
 {
+	struct rte_pci_device *dev;
 	struct rte_pci_driver *dr;
+	const struct resource *r;
+	int ret;
+
+	r = resource_find("test_pci_sysfs");
+	TEST_ASSERT_NOT_NULL(r, "missing resource test_pci_sysfs");
+
+	ret = resource_untar(r);
+	TEST_ASSERT_SUCCESS(ret, "failed to untar %s", r->name);
+
+	ret = setenv("SYSFS_PCI_DEVICES", "test_pci_sysfs/bus/pci/devices", 1);
+	TEST_ASSERT_SUCCESS(ret, "failed to setenv");
 
-	/* Unregister original driver list */
+	/* Unregister original devices & drivers lists */
 	while (!TAILQ_EMPTY(&pci_driver_list)) {
 		dr = TAILQ_FIRST(&pci_driver_list);
 		rte_eal_pci_unregister(dr);
 		TAILQ_INSERT_TAIL(&real_pci_driver_list, dr, next);
 	}
 
+	while (!TAILQ_EMPTY(&pci_device_list)) {
+		dev = TAILQ_FIRST(&pci_device_list);
+		TAILQ_REMOVE(&pci_device_list, dev, next);
+		TAILQ_INSERT_TAIL(&real_pci_device_list, dev, next);
+	}
+
+	ret = rte_eal_pci_scan();
+	TEST_ASSERT_SUCCESS(ret, "failed to scan PCI bus");
+	rte_eal_pci_dump(stdout);
+
 	return 0;
 }
 
 static int
 test_pci_cleanup(void)
 {
+	struct rte_pci_device *dev;
 	struct rte_pci_driver *dr;
+	const struct resource *r;
+	int ret;
+
+	unsetenv("SYSFS_PCI_DEVICES");
+
+	r = resource_find("test_pci_sysfs");
+	TEST_ASSERT_NOT_NULL(r, "missing resource test_pci_sysfs");
+
+	ret = resource_rm_by_tar(r);
+	TEST_ASSERT_SUCCESS(ret, "Failed to delete resource %s", r->name);
 
-	/* Restore original driver list */
+	/* FIXME: there is no API in DPDK to free a rte_pci_device so we
+	   cannot free the devices in the right way. Let's assume that we
+	   don't care for tests. */
+	while (!TAILQ_EMPTY(&pci_device_list)) {
+		dev = TAILQ_FIRST(&pci_device_list);
+		TAILQ_REMOVE(&pci_device_list, dev, next);
+	}
+
+	/* Restore original devices & drivers lists */
 	while (!TAILQ_EMPTY(&real_pci_driver_list)) {
 		dr = TAILQ_FIRST(&real_pci_driver_list);
 		TAILQ_REMOVE(&real_pci_driver_list, dr, next);
 		rte_eal_pci_register(dr);
 	}
 
+	while (!TAILQ_EMPTY(&real_pci_device_list)) {
+		dev = TAILQ_FIRST(&real_pci_device_list);
+		TAILQ_REMOVE(&real_pci_device_list, dev, next);
+		TAILQ_INSERT_TAIL(&pci_device_list, dev, next);
+	}
+
 	return 0;
 }
 
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class
new file mode 100644
index 0000000..2f9c1da
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/class
@@ -0,0 +1 @@
+0x020000
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/config b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/config
new file mode 100644
index 0000000000000000000000000000000000000000..7752421cf13a5aa00a28eaee02cac2add9ce5566
GIT binary patch
literal 64
zcmZo`_$|QBBEZ1Nz`(@7(7?dMz;S^A2oxWHNCpNT2LUi2#BOU~22l(SV3L7>8>k5Y
DD^Ld(

literal 0
HcmV?d00001

diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits
new file mode 100644
index 0000000..900731f
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/consistent_dma_mask_bits
@@ -0,0 +1 @@
+64
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device
new file mode 100644
index 0000000..9e4789e
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/device
@@ -0,0 +1 @@
+0x10fb
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits
new file mode 100644
index 0000000..900731f
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/dma_mask_bits
@@ -0,0 +1 @@
+64
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/enable
@@ -0,0 +1 @@
+1
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/irq
@@ -0,0 +1 @@
+0
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias
new file mode 100644
index 0000000..f4c76ed
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/modalias
@@ -0,0 +1 @@
+pci:v00008086d000010FBsv00008086sd00000003bc02sc00i00
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/msi_bus
@@ -0,0 +1 @@
+1
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node
new file mode 100644
index 0000000..3a2e3f4
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/numa_node
@@ -0,0 +1 @@
+-1
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource
new file mode 100644
index 0000000..f388929
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/resource
@@ -0,0 +1,13 @@
+0x00000000d0080000 0x00000000d00fffff 0x000000000014220c
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x000000000000e020 0x000000000000e03f 0x0000000000040101
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x00000000d0104000 0x00000000d0107fff 0x000000000014220c
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x00000000ab000000 0x00000000ab0fffff 0x0000000000140204
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x00000000ab100000 0x00000000ab1fffff 0x0000000000140204
+0x0000000000000000 0x0000000000000000 0x0000000000000000
+0x0000000000000000 0x0000000000000000 0x0000000000000000
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_numvfs
@@ -0,0 +1 @@
+0
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs
new file mode 100644
index 0000000..4b9026d
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/sriov_totalvfs
@@ -0,0 +1 @@
+63
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device
new file mode 100644
index 0000000..89a932c
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_device
@@ -0,0 +1 @@
+0x0003
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor
new file mode 100644
index 0000000..ce6dc4d
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/subsystem_vendor
@@ -0,0 +1 @@
+0x8086
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent
new file mode 100644
index 0000000..1dbe34d
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/uevent
@@ -0,0 +1,6 @@
+DRIVER=ixgbe
+PCI_CLASS=20000
+PCI_ID=8086:10FB
+PCI_SUBSYS_ID=8086:0003
+PCI_SLOT_NAME=0000:01:00.0
+MODALIAS=pci:v00008086d000010FBsv00008086sd00000003bc02sc00i00
diff --git a/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor
new file mode 100644
index 0000000..ce6dc4d
--- /dev/null
+++ b/app/test/test_pci_sysfs/bus/pci/devices/0000:01:00.0/vendor
@@ -0,0 +1 @@
+0x8086
-- 
2.8.0

[PATCH v3 11/11] app/test: do not dump PCI devices in blacklist test

[Jan Viktorin]

Dumping of devices in a unittest is useless. Instead, test whether the test
has been set up well - i.e. there are no devices.

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 app/test/test_pci.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/app/test/test_pci.c b/app/test/test_pci.c
index 69066cb..5506a11 100644
--- a/app/test/test_pci.c
+++ b/app/test/test_pci.c
@@ -150,8 +150,8 @@ test_pci_blacklist(void)
 {
 	struct rte_devargs_list save_devargs_list;
 
-	printf("Dump all devices\n");
-	rte_eal_pci_dump(stdout);
+	TEST_ASSERT(TAILQ_EMPTY(&pci_driver_list),
+			"pci_driver_list not empty");
 
 	rte_eal_pci_register(&my_driver);
 	rte_eal_pci_register(&my_driver2);
-- 
2.8.0