[Buildroot] [PATCH v3 0/3] dpdk: new package

[Buildroot] [PATCH v3 0/3] dpdk: new package

[Jan Viktorin]

Hello all,

this my thrid approach to upstream support of the DPDK into Buildroot. Briefly:

* The DPDK is a library (www.dpdk.org) that allows high-speed (10-100 Gbps)
  packet sending/receiving at very high packet rates while bypassing the Linux
  Kernel.
* I maintain the ARM(v7) part of the DPDK so I am interested in building DPDK
  by Buildroot.
* The DPDK tests require pexpect and ptyprocess python packages to run so they
  are included.

Changes for v3:

* Line overfull fixes (all patches).
* Hopefully solved issues reported by T. Petazzoni.
* As using a newer DPDK, the patch now utilizes the new installation infra.
* I've decided to include building of example applications, however, it is not
  built nor installed in a standardized way, so, it's done by dpdk.mk.

Jan Viktorin (3):
  python-ptyprocess: new package
  python-pexpect: new package
  dpdk: new package

 package/Config.in                              |   3 +
 package/dpdk/Config.in                         |  53 ++++++++++++
 package/dpdk/dpdk.hash                         |   2 +
 package/dpdk/dpdk.mk                           | 109 +++++++++++++++++++++++++
 package/python-pexpect/Config.in               |  11 +++
 package/python-pexpect/python-pexpect.mk       |  14 ++++
 package/python-ptyprocess/Config.in            |   7 ++
 package/python-ptyprocess/python-ptyprocess.mk |  13 +++
 8 files changed, 212 insertions(+)
 create mode 100644 package/dpdk/Config.in
 create mode 100644 package/dpdk/dpdk.hash
 create mode 100644 package/dpdk/dpdk.mk
 create mode 100644 package/python-pexpect/Config.in
 create mode 100644 package/python-pexpect/python-pexpect.mk
 create mode 100644 package/python-ptyprocess/Config.in
 create mode 100644 package/python-ptyprocess/python-ptyprocess.mk

-- 
2.7.0

[Buildroot] [PATCH v3 1/3] python-ptyprocess: new package

[Jan Viktorin]

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 package/Config.in                              |  1 +
 package/python-ptyprocess/Config.in            |  7 +++++++
 package/python-ptyprocess/python-ptyprocess.mk | 13 +++++++++++++
 3 files changed, 21 insertions(+)
 create mode 100644 package/python-ptyprocess/Config.in
 create mode 100644 package/python-ptyprocess/python-ptyprocess.mk

diff --git a/package/Config.in b/package/Config.in
index 27db639..df01e55 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -689,6 +689,7 @@ menu "External python modules"
 	source "package/python-posix-ipc/Config.in"
 	source "package/python-protobuf/Config.in"
 	source "package/python-psutil/Config.in"
+	source "package/python-ptyprocess/Config.in"
 	source "package/python-pyasn/Config.in"
 	source "package/python-pyasn-modules/Config.in"
 	source "package/python-pycli/Config.in"
diff --git a/package/python-ptyprocess/Config.in b/package/python-ptyprocess/Config.in
new file mode 100644
index 0000000..4378943
--- /dev/null
+++ b/package/python-ptyprocess/Config.in
@@ -0,0 +1,7 @@
+config BR2_PACKAGE_PYTHON_PTYPROCESS
+	bool "python-ptyprocess"
+	help
+	  Launch a subprocess in a pseudo terminal (pty), and interact with
+	  both the process and its pty.
+
+	  https://github.com/pexpect/ptyprocess
diff --git a/package/python-ptyprocess/python-ptyprocess.mk b/package/python-ptyprocess/python-ptyprocess.mk
new file mode 100644
index 0000000..9e6b762
--- /dev/null
+++ b/package/python-ptyprocess/python-ptyprocess.mk
@@ -0,0 +1,13 @@
+################################################################################
+#
+# python-ptyprocess
+#
+################################################################################
+
+PYTHON_PTYPROCESS_VERSION = 0.5
+PYTHON_PTYPROCESS_SITE = $(call github,pexpect,ptyprocess,$(PYTHON_PTYPROCESS_VERSION))
+PYTHON_PTYPROCESS_LICENSE = ISC
+PYTHON_PTYPROCESS_LICENSE_FILES = LICENSE
+PYTHON_PTYPROCESS_SETUP_TYPE = distutils
+
+$(eval $(python-package))
-- 
2.7.0

[Buildroot] [PATCH v3 2/3] python-pexpect: new package

[Jan Viktorin]

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 package/Config.in                        |  1 +
 package/python-pexpect/Config.in         | 11 +++++++++++
 package/python-pexpect/python-pexpect.mk | 14 ++++++++++++++
 3 files changed, 26 insertions(+)
 create mode 100644 package/python-pexpect/Config.in
 create mode 100644 package/python-pexpect/python-pexpect.mk

diff --git a/package/Config.in b/package/Config.in
index df01e55..6fc0466 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -686,6 +686,7 @@ menu "External python modules"
 	source "package/python-paho-mqtt/Config.in"
 	source "package/python-pam/Config.in"
 	source "package/python-paramiko/Config.in"
+	source "package/python-pexpect/Config.in"
 	source "package/python-posix-ipc/Config.in"
 	source "package/python-protobuf/Config.in"
 	source "package/python-psutil/Config.in"
diff --git a/package/python-pexpect/Config.in b/package/python-pexpect/Config.in
new file mode 100644
index 0000000..4d04c01
--- /dev/null
+++ b/package/python-pexpect/Config.in
@@ -0,0 +1,11 @@
+config BR2_PACKAGE_PYTHON_PEXPECT
+	bool "python-pexpect"
+	select BR2_PACKAGE_PYTHON_PTYPROCESS
+	help
+	  Pexpect is a pure Python module for spawning child applications;
+	  controlling them; and responding to expected patterns in their
+	  output. Pexpect works like Don Libes? Expect. Pexpect allows your
+	  script to spawn a child application and control it as if a human
+	  were typing commands.
+
+	  https://pexpect.readthedocs.org
diff --git a/package/python-pexpect/python-pexpect.mk b/package/python-pexpect/python-pexpect.mk
new file mode 100644
index 0000000..9de7849
--- /dev/null
+++ b/package/python-pexpect/python-pexpect.mk
@@ -0,0 +1,14 @@
+################################################################################
+#
+# python-pexpect
+#
+################################################################################
+
+PYTHON_PEXPECT_VERSION = 4.0.1
+PYTHON_PEXPECT_SITE = $(call github,pexpect,pexpect,$(PYTHON_PEXPECT_VERSION))
+PYTHON_PEXPECT_LICENSE = ISC
+PYTHON_PEXPECT_LICENSE_FILES = LICENSE
+PYTHON_PEXPECT_SETUP_TYPE = distutils
+PYTHON_PEXPECT_DEPENDENCIES = python-ptyprocess
+
+$(eval $(python-package))
-- 
2.7.0

[Buildroot] [PATCH v3 3/3] dpdk: new package

[Jan Viktorin]

This patch introduces support of the DPDK library (www.dpdk.org) into
Buildroot. DPDK is a library for high-speed packet sending/receiving
while bypassing the Linux Kernel. It allows to reach a high throughput
for 10-100 Gbps networks on the x86 platform.

The package compiles and installs DPDK libraries on the target and
staging and allows to compile other applications depending on the DPDK
library. It can also install some basic tools the DPDK provides
(testpmd, python scripts, test suite).

The patch assumes DPDK 16.04-rc1. This version contains support for the ARM
architecture. The ARM ports can be tested by

 qemu_aarch64_virt_defconfig
 qemu_arm_vexpress_defconfig

The included hash was calculated locally by downloading the tar.gz archives by
hand.

There are unfortunately some pitfalls:

* it may require to enable PCI, MSIX, UIO in the Linux Kernel
  (some defconfigs does not include as default and it is platform
  dependent as ARMv7 almost does not use PCI)

* when building PCAP PMD driver, the libpcap is required (partially
  fixed as suggested by Arnout)

* some tools the DPDK provides depend on Python(2) so the user has
  to enable it to install those

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v2: (mostly suggestions by Arnout)
* simplified Config.in - avoid version, source and config selection
* improved dependency on libpcap
* user python scripts are included if python package is enabled
* installing of tests includes autotest*.py scripts (we do not care
  about the python here, assuming the user will install it manually
  when testing)
* minor coding style fixes
* depends on python-pexpect package
* using version 2.2.0-rc3
v3
* bump to version 16.04-rc1
* utilizing the new install syntax of DPDK
* fixed line wrapping
* testpmd is now always installed (no option for this)
* fixed python dependencies (T. Petazzoni)
* supporting i686+ architectures (T. Petazzoni)
* dropped PPC support for now (T. Petazzoni)
* utilizing KCONFIG_ENABLE_OPT macro (T. Petazzoni)
* support for build & install of examples (ugly code)
* build with MAKE instead of MAKE1 seems OK now (T. Petazzoni)
* dropped COMBINE_LIBS (no more in DPDK) (T. Petazzoni)
* added HAS_SYNC_x
* few random fixes

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 package/Config.in      |   1 +
 package/dpdk/Config.in |  53 ++++++++++++++++++++++++
 package/dpdk/dpdk.hash |   2 +
 package/dpdk/dpdk.mk   | 109 +++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 165 insertions(+)
 create mode 100644 package/dpdk/Config.in
 create mode 100644 package/dpdk/dpdk.hash
 create mode 100644 package/dpdk/dpdk.mk

diff --git a/package/Config.in b/package/Config.in
index 6fc0466..ab38a76 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -1095,6 +1095,7 @@ menu "Networking"
 	source "package/cgic/Config.in"
 	source "package/cppzmq/Config.in"
 	source "package/czmq/Config.in"
+	source "package/dpdk/Config.in"
 	source "package/filemq/Config.in"
 	source "package/flickcurl/Config.in"
 	source "package/fmlib/Config.in"
diff --git a/package/dpdk/Config.in b/package/dpdk/Config.in
new file mode 100644
index 0000000..a42271e
--- /dev/null
+++ b/package/dpdk/Config.in
@@ -0,0 +1,53 @@
+config BR2_PACKAGE_DPDK
+       bool "dpdk"
+       depends on (BR2_i386 && !BR2_x86_i386 && !BR2_x86_i486 \
+		      && !BR2_x86_i586 && !BR2_x86_x1000) \
+                  || BR2_ARM_CPU_ARMV7A || BR2_aarch64 || BR2_aarch64_be
+       depends on BR2_TOOLCHAIN_USES_GLIBC
+       depends on BR2_TOOLCHAIN_HAS_SYNC_1
+       depends on BR2_TOOLCHAIN_HAS_SYNC_2
+       depends on BR2_TOOLCHAIN_HAS_SYNC_4
+       depends on BR2_TOOLCHAIN_HAS_SYNC_8
+       help
+	 DPDK is a set of libraries and drivers for fast packet processing. It
+	 was designed to run on any processors, however, Intel x86 has been the
+	 first CPU to be supported. Ports for other CPUs like IBM Power 8 and
+	 ARM are under progress. It runs mostly in Linux userland. A FreeBSD
+	 port is now available for a subset of DPDK features.
+
+	 Notes:
+	 * To build the included Linux Kernel drivers, it is necessary to
+	   enable CONFIG_PCI_MSI, CONFIG_UIO.
+	 * To build the PCAP PMD properly, you need to enable the libpcap
+	   manually.
+	 * You may need to install the python2 interpreter if you want to use
+	   scripts dpdk_nic_bind.py and cpu_layout.py
+
+         http://www.dpdk.org/
+
+if BR2_PACKAGE_DPDK
+
+config BR2_PACKAGE_DPDK_CONFIG
+	string "Configuration"
+	default "i686-native-linuxapp-gcc" \
+		if BR2_x86_i686
+	default "x86_64-native-linuxapp-gcc" \
+		if BR2_x86_64
+	default "arm-armv7a-linuxapp-gcc" \
+		if BR2_ARM_CPU_ARMV7A
+	default "arm64-armv8a-linuxapp-gcc" \
+		if BR2_aarch64 || BR2_aarch64_be
+
+config BR2_PACKAGE_DPDK_TEST
+	bool "Install tests suite"
+	select BR2_PACKAGE_PYTHON_PEXPECT if BR2_PACKAGE_PYTHON
+	help
+	  Install all DPDK tests. If you want to run the tests by the included
+	  autotest.py script you need to enable python manually.
+
+config BR2_PACKAGE_DPDK_EXAMPLES
+	bool "Build & install examples"
+	help
+	  Build and install all examples selected by the current configuration.
+
+endif
diff --git a/package/dpdk/dpdk.hash b/package/dpdk/dpdk.hash
new file mode 100644
index 0000000..1c47f19
--- /dev/null
+++ b/package/dpdk/dpdk.hash
@@ -0,0 +1,2 @@
+# Locally calculated
+sha256	18df4b6ac74a4383b65d7c79420b637d0c296636265388dbf4513d1ec5caa09f  dpdk-16.04-rc1.tar.gz
diff --git a/package/dpdk/dpdk.mk b/package/dpdk/dpdk.mk
new file mode 100644
index 0000000..02860fd
--- /dev/null
+++ b/package/dpdk/dpdk.mk
@@ -0,0 +1,109 @@
+################################################################################
+#
+# dpdk
+#
+################################################################################
+
+DPDK_VERSION = 16.04-rc1
+DPDK_SITE = http://dpdk.org/browse/dpdk/snapshot
+DPDK_SOURCE = dpdk-$(DPDK_VERSION).tar.gz
+
+DPDK_LICENSE = BSD (core), GPLv2+ (Linux drivers)
+DPDK_LICENSE_FILES = GNUmakefile LICENSE.GPL
+DPDK_INSTALL_STAGING = YES
+
+DPDK_DEPENDENCIES += linux
+
+ifeq ($(BR2_PACKAGE_LIBPCAP),y)
+DPDK_DEPENDENCIES += libpcap
+endif
+
+ifeq ($(BR2_SHARED_LIBS),y)
+define DPDK_ENABLE_SHARED_LIBS
+	$(call KCONFIG_ENABLE_OPT,CONFIG_RTE_BUILD_SHARED_LIB,\
+			$(@D)/build/.config)
+endef
+
+DPDK_POST_CONFIGURE_HOOKS += DPDK_ENABLE_SHARED_LIBS
+endif
+
+# We're building a kernel module without using the kernel-module infra,
+# so we need to tell we want module support in the kernel
+ifeq ($(BR2_PACKAGE_DPDK),y)
+LINUX_NEEDS_MODULES = y
+endif
+
+DPDK_CONFIG = $(call qstrip,$(BR2_PACKAGE_DPDK_CONFIG))
+
+ifeq ($(BR2_PACKAGE_DPDK_EXAMPLES),y)
+# Build of DPDK examples is not very straight-forward. It requires to have
+# the SDK and runtime installed on same place to reference it by RTE_SDK.
+# We place it locally in the build directory.
+define DPDK_BUILD_EXAMPLES
+	$(MAKE) -C $(@D) DESTDIR=$(@D)/examples-sdk \
+		CROSS=$(TARGET_CROSS) install-sdk install-runtime
+	$(MAKE) -C $(@D) RTE_KERNELDIR=$(LINUX_DIR) CROSS=$(TARGET_CROSS) \
+		RTE_SDK=$(@D)/examples-sdk/usr/local/share/dpdk \
+		T=$(DPDK_CONFIG) examples
+endef
+
+DPDK_EXAMPLES_PATH = $(@D)/examples-sdk/usr/local/share/dpdk/examples
+
+# Installation of examples is not supported in DPDK so we do it explicitly
+# here. As the binaries and libraries do not have a single or regular location
+# where to find them after build, we search for them by find.
+define DPDK_INSTALL_EXAMPLES
+	$(INSTALL) -m 0755 -d $(TARGET_DIR)/usr/local/bin
+	$(INSTALL) -m 0755 -d $(TARGET_DIR)/usr/local/lib
+	for f in `find $(DPDK_EXAMPLES_PATH) -executable -type f   \
+			-name '[a-z]*.so*' | grep '\/lib\/.*'`; do \
+		$(INSTALL) -m 0755 -D $$f                          \
+			$(TARGET_DIR)/usr/local/lib/`basename $$f`;\
+	done
+	for f in `find $(DPDK_EXAMPLES_PATH) -executable -type f   \
+			! -name '*.so*' | grep '\/app\/.*'`; do    \
+		$(INSTALL) -m 0755 -D $$f                          \
+			$(TARGET_DIR)/usr/local/bin/`basename $$f`;\
+	done
+endef
+
+# Build of the power example is broken (at least for 16.04).
+define DPDK_DISABLE_POWER
+	$(call KCONFIG_DISABLE_OPT,CONFIG_RTE_LIBRTE_POWER,\
+			$(@D)/build/.config)
+endef
+
+DPDK_POST_CONFIGURE_HOOKS += DPDK_DISABLE_POWER
+endif
+
+define DPDK_CONFIGURE_CMDS
+	$(MAKE) -C $(@D) T=$(DPDK_CONFIG) RTE_KERNELDIR=$(LINUX_DIR) \
+			   CROSS=$(TARGET_CROSS) config
+endef
+
+define DPDK_BUILD_CMDS
+	$(MAKE) -C $(@D) RTE_KERNELDIR=$(LINUX_DIR) CROSS=$(TARGET_CROSS)
+	$(DPDK_BUILD_EXAMPLES)
+endef
+
+define DPDK_INSTALL_STAGING_CMDS
+	$(MAKE) -C $(@D) DESTDIR=$(STAGING_DIR) prefix=/usr \
+		 CROSS=$(TARGET_CROSS) install-sdk
+endef
+
+ifeq ($(BR2_PACKAGE_DPDK_TEST),y)
+define DPDK_INSTALL_TARGET_TEST
+	$(INSTALL) -m 0755 -d $(TARGET_DIR)/usr/dpdk
+	$(INSTALL) -m 0755 -D $(@D)/build/app/test $(TARGET_DIR)/usr/dpdk
+	$(INSTALL) -m 0755 -D $(@D)/app/test/*.py $(TARGET_DIR)/usr/dpdk
+endef
+endif
+
+define DPDK_INSTALL_TARGET_CMDS
+	$(MAKE) -C $(@D) DESTDIR=$(TARGET_DIR) prefix=/usr \
+		CROSS=$(TARGET_CROSS) install-runtime
+	$(DPDK_INSTALL_TARGET_TEST)
+	$(DPDK_INSTALL_EXAMPLES)
+endef
+
+$(eval $(generic-package))
-- 
2.7.0

[Buildroot] [PATCH v3 1/3] python-ptyprocess: new package

[Yegor Yefremov]

Hi Jan,

the patch looks good to me. Below some comments:

On Tue, Mar 22, 2016 at 11:36 AM, Jan Viktorin <viktorin@rehivetech.com> wrote:
> Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
> ---
>  package/Config.in                              |  1 +
>  package/python-ptyprocess/Config.in            |  7 +++++++
>  package/python-ptyprocess/python-ptyprocess.mk | 13 +++++++++++++
>  3 files changed, 21 insertions(+)
>  create mode 100644 package/python-ptyprocess/Config.in
>  create mode 100644 package/python-ptyprocess/python-ptyprocess.mk
>
> diff --git a/package/Config.in b/package/Config.in
> index 27db639..df01e55 100644
> --- a/package/Config.in
> +++ b/package/Config.in
> @@ -689,6 +689,7 @@ menu "External python modules"
>         source "package/python-posix-ipc/Config.in"
>         source "package/python-protobuf/Config.in"
>         source "package/python-psutil/Config.in"
> +       source "package/python-ptyprocess/Config.in"
>         source "package/python-pyasn/Config.in"
>         source "package/python-pyasn-modules/Config.in"
>         source "package/python-pycli/Config.in"
> diff --git a/package/python-ptyprocess/Config.in b/package/python-ptyprocess/Config.in
> new file mode 100644
> index 0000000..4378943
> --- /dev/null
> +++ b/package/python-ptyprocess/Config.in
> @@ -0,0 +1,7 @@
> +config BR2_PACKAGE_PYTHON_PTYPROCESS
> +       bool "python-ptyprocess"
> +       help
> +         Launch a subprocess in a pseudo terminal (pty), and interact with
> +         both the process and its pty.
> +
> +         https://github.com/pexpect/ptyprocess
> diff --git a/package/python-ptyprocess/python-ptyprocess.mk b/package/python-ptyprocess/python-ptyprocess.mk
> new file mode 100644
> index 0000000..9e6b762
> --- /dev/null
> +++ b/package/python-ptyprocess/python-ptyprocess.mk
> @@ -0,0 +1,13 @@
> +################################################################################
> +#
> +# python-ptyprocess
> +#
> +################################################################################
> +
> +PYTHON_PTYPROCESS_VERSION = 0.5

There is newer version available 0.5.1

> +PYTHON_PTYPROCESS_SITE = $(call github,pexpect,ptyprocess,$(PYTHON_PTYPROCESS_VERSION))

Please get the package from https://pypi.python.org/pypi/ptyprocess
and add a hash file.

> +PYTHON_PTYPROCESS_LICENSE = ISC
> +PYTHON_PTYPROCESS_LICENSE_FILES = LICENSE
> +PYTHON_PTYPROCESS_SETUP_TYPE = distutils
> +
> +$(eval $(python-package))

Yegor

[Buildroot] [PATCH v3 2/3] python-pexpect: new package

[Yegor Yefremov]

Hi Jan,

On Tue, Mar 22, 2016 at 11:36 AM, Jan Viktorin <viktorin@rehivetech.com> wrote:
> Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
> ---
>  package/Config.in                        |  1 +
>  package/python-pexpect/Config.in         | 11 +++++++++++
>  package/python-pexpect/python-pexpect.mk | 14 ++++++++++++++
>  3 files changed, 26 insertions(+)
>  create mode 100644 package/python-pexpect/Config.in
>  create mode 100644 package/python-pexpect/python-pexpect.mk
>
> diff --git a/package/Config.in b/package/Config.in
> index df01e55..6fc0466 100644
> --- a/package/Config.in
> +++ b/package/Config.in
> @@ -686,6 +686,7 @@ menu "External python modules"
>         source "package/python-paho-mqtt/Config.in"
>         source "package/python-pam/Config.in"
>         source "package/python-paramiko/Config.in"
> +       source "package/python-pexpect/Config.in"
>         source "package/python-posix-ipc/Config.in"
>         source "package/python-protobuf/Config.in"
>         source "package/python-psutil/Config.in"
> diff --git a/package/python-pexpect/Config.in b/package/python-pexpect/Config.in
> new file mode 100644
> index 0000000..4d04c01
> --- /dev/null
> +++ b/package/python-pexpect/Config.in
> @@ -0,0 +1,11 @@
> +config BR2_PACKAGE_PYTHON_PEXPECT
> +       bool "python-pexpect"
> +       select BR2_PACKAGE_PYTHON_PTYPROCESS
> +       help
> +         Pexpect is a pure Python module for spawning child applications;
> +         controlling them; and responding to expected patterns in their
> +         output. Pexpect works like Don Libes? Expect. Pexpect allows your
> +         script to spawn a child application and control it as if a human
> +         were typing commands.
> +
> +         https://pexpect.readthedocs.org
> diff --git a/package/python-pexpect/python-pexpect.mk b/package/python-pexpect/python-pexpect.mk
> new file mode 100644
> index 0000000..9de7849
> --- /dev/null
> +++ b/package/python-pexpect/python-pexpect.mk
> @@ -0,0 +1,14 @@
> +################################################################################
> +#
> +# python-pexpect
> +#
> +################################################################################
> +
> +PYTHON_PEXPECT_VERSION = 4.0.1
> +PYTHON_PEXPECT_SITE = $(call github,pexpect,pexpect,$(PYTHON_PEXPECT_VERSION))

Please get the package from https://pypi.python.org/pypi/pexpect and
add a hash file.

> +PYTHON_PEXPECT_LICENSE = ISC
> +PYTHON_PEXPECT_LICENSE_FILES = LICENSE
> +PYTHON_PEXPECT_SETUP_TYPE = distutils
> +PYTHON_PEXPECT_DEPENDENCIES = python-ptyprocess

remove this build-time dependency

> +
> +$(eval $(python-package))
> --
> 2.7.0
>

[Buildroot] [PATCH v3 3/3] dpdk: new package

[Thomas Petazzoni]

Hello,

On Tue, 22 Mar 2016 11:36:26 +0100, Jan Viktorin wrote:

> diff --git a/package/dpdk/Config.in b/package/dpdk/Config.in
> new file mode 100644
> index 0000000..a42271e
> --- /dev/null
> +++ b/package/dpdk/Config.in
> @@ -0,0 +1,53 @@
> +config BR2_PACKAGE_DPDK
> +       bool "dpdk"

Indentation of properties should be done with one tab (ditto in the
following lines)

> +       depends on (BR2_i386 && !BR2_x86_i386 && !BR2_x86_i486 \
> +		      && !BR2_x86_i586 && !BR2_x86_x1000) \
> +                  || BR2_ARM_CPU_ARMV7A || BR2_aarch64 || BR2_aarch64_be

I'm wondering why you have this list of architecture dependencies. Is
it because there is really some architecture specific code, or is it a
left-over from the times we didn't had the BR2_TOOLCHAIN_HAS_x
options ?

If that's really needed, please add a blind option like this:

config BR2_PACKAGE_DPDK_ARCH_SUPPORTS
	bool
	default y if ...

That you can re-use as a dependency, both for the comment (see below)
and the main option.

> +       depends on BR2_TOOLCHAIN_USES_GLIBC

Then you need to have a comment like:

comment "dpdk needs a toolchain w/ glibc"
	depends on !BR2_TOOLCHAIN_USES_GLIBC
	depends on BR2_TOOLCHAIN_HAS_SYNC_...

> +       depends on BR2_TOOLCHAIN_HAS_SYNC_1
> +       depends on BR2_TOOLCHAIN_HAS_SYNC_2
> +       depends on BR2_TOOLCHAIN_HAS_SYNC_4
> +       depends on BR2_TOOLCHAIN_HAS_SYNC_8

Cosmetic nit, but I would prefer to see:

	depends on BR2_TOOLCHAIN_HAS_SYNC_1 && \
		BR2_TOOLCHAIN_HAS_SYNC_2 && \
		BR2_TOOLCHAIN_HAS_SYNC_4 && \
		BR2_TOOLCHAIN_HAS_SYNC_8

	

> +       help
> +	 DPDK is a set of libraries and drivers for fast packet processing. It
> +	 was designed to run on any processors, however, Intel x86 has been the
> +	 first CPU to be supported. Ports for other CPUs like IBM Power 8 and
> +	 ARM are under progress. It runs mostly in Linux userland. A FreeBSD
> +	 port is now available for a subset of DPDK features.

Indentation of the help text should be one tab + *two* spaces.

> +
> +	 Notes:
> +	 * To build the included Linux Kernel drivers, it is necessary to
> +	   enable CONFIG_PCI_MSI, CONFIG_UIO.
> +	 * To build the PCAP PMD properly, you need to enable the libpcap
> +	   manually.
> +	 * You may need to install the python2 interpreter if you want to use
> +	   scripts dpdk_nic_bind.py and cpu_layout.py
> +
> +         http://www.dpdk.org/
> +
> +if BR2_PACKAGE_DPDK
> +
> +config BR2_PACKAGE_DPDK_CONFIG
> +	string "Configuration"
> +	default "i686-native-linuxapp-gcc" \
> +		if BR2_x86_i686
> +	default "x86_64-native-linuxapp-gcc" \
> +		if BR2_x86_64
> +	default "arm-armv7a-linuxapp-gcc" \
> +		if BR2_ARM_CPU_ARMV7A
> +	default "arm64-armv8a-linuxapp-gcc" \
> +		if BR2_aarch64 || BR2_aarch64_be

I don't remember if we discussed this: is there any reason to make this
configurable? I.e is it necessary to make it a visible Config.in option
as opposed to a blind one?

> +
> +config BR2_PACKAGE_DPDK_TEST
> +	bool "Install tests suite"
> +	select BR2_PACKAGE_PYTHON_PEXPECT if BR2_PACKAGE_PYTHON
> +	help
> +	  Install all DPDK tests. If you want to run the tests by the included
> +	  autotest.py script you need to enable python manually.

Is the test suite usable without Python ?

> diff --git a/package/dpdk/dpdk.mk b/package/dpdk/dpdk.mk
> new file mode 100644
> index 0000000..02860fd
> --- /dev/null
> +++ b/package/dpdk/dpdk.mk
> @@ -0,0 +1,109 @@
> +################################################################################
> +#
> +# dpdk
> +#
> +################################################################################
> +
> +DPDK_VERSION = 16.04-rc1

We normally don't package release candidate / beta version. Are you
confident that the final 16.04 release will be done before Buildroot
ships its own release at the end of May ?

> +DPDK_SITE = http://dpdk.org/browse/dpdk/snapshot
> +DPDK_SOURCE = dpdk-$(DPDK_VERSION).tar.gz

Not needed, that's the default value.

> +
> +DPDK_LICENSE = BSD (core), GPLv2+ (Linux drivers)

BSD should be more specific, like BSD-2c, BSD-3c, etc.

> +DPDK_LICENSE_FILES = GNUmakefile LICENSE.GPL
> +DPDK_INSTALL_STAGING = YES
> +
> +DPDK_DEPENDENCIES += linux

If you need the Linux kernel to be built, then you need to also have:

	depends on BR2_LINUX_KERNEL

in your Config.in, as well as a corresponding comment:

comment "dpdk needs the Linux kernel to be built"
	depends on !BR2_LINUX_KERNEL

> +
> +ifeq ($(BR2_PACKAGE_LIBPCAP),y)
> +DPDK_DEPENDENCIES += libpcap
> +endif
> +
> +ifeq ($(BR2_SHARED_LIBS),y)
> +define DPDK_ENABLE_SHARED_LIBS
> +	$(call KCONFIG_ENABLE_OPT,CONFIG_RTE_BUILD_SHARED_LIB,\
> +			$(@D)/build/.config)
> +endef
> +
> +DPDK_POST_CONFIGURE_HOOKS += DPDK_ENABLE_SHARED_LIBS
> +endif
> +
> +# We're building a kernel module without using the kernel-module infra,
> +# so we need to tell we want module support in the kernel
> +ifeq ($(BR2_PACKAGE_DPDK),y)
> +LINUX_NEEDS_MODULES = y
> +endif

This is no longer the "right" way of doing this. Just select
BR2_LINUX_NEEDS_MODULES in Config.in.

> +
> +DPDK_CONFIG = $(call qstrip,$(BR2_PACKAGE_DPDK_CONFIG))
> +
> +ifeq ($(BR2_PACKAGE_DPDK_EXAMPLES),y)
> +# Build of DPDK examples is not very straight-forward. It requires to have
> +# the SDK and runtime installed on same place to reference it by RTE_SDK.
> +# We place it locally in the build directory.
> +define DPDK_BUILD_EXAMPLES
> +	$(MAKE) -C $(@D) DESTDIR=$(@D)/examples-sdk \
> +		CROSS=$(TARGET_CROSS) install-sdk install-runtime
> +	$(MAKE) -C $(@D) RTE_KERNELDIR=$(LINUX_DIR) CROSS=$(TARGET_CROSS) \
> +		RTE_SDK=$(@D)/examples-sdk/usr/local/share/dpdk \
> +		T=$(DPDK_CONFIG) examples
> +endef
> +
> +DPDK_EXAMPLES_PATH = $(@D)/examples-sdk/usr/local/share/dpdk/examples
> +
> +# Installation of examples is not supported in DPDK so we do it explicitly
> +# here. As the binaries and libraries do not have a single or regular location
> +# where to find them after build, we search for them by find.
> +define DPDK_INSTALL_EXAMPLES
> +	$(INSTALL) -m 0755 -d $(TARGET_DIR)/usr/local/bin
> +	$(INSTALL) -m 0755 -d $(TARGET_DIR)/usr/local/lib

In Buildroot, everything is installed in /usr, not /usr/local. Why
should this package be different? Also, to create directories, we
typically use mkdir -p.

> +	for f in `find $(DPDK_EXAMPLES_PATH) -executable -type f   \
> +			-name '[a-z]*.so*' | grep '\/lib\/.*'`; do \
> +		$(INSTALL) -m 0755 -D $$f                          \
> +			$(TARGET_DIR)/usr/local/lib/`basename $$f`;\
> +	done
> +	for f in `find $(DPDK_EXAMPLES_PATH) -executable -type f   \
> +			! -name '*.so*' | grep '\/app\/.*'`; do    \
> +		$(INSTALL) -m 0755 -D $$f                          \
> +			$(TARGET_DIR)/usr/local/bin/`basename $$f`;\
> +	done

It's a bit annoying to have this logic. Could it be simplified to use
-path instead of -name and avoid the grep ?

Also, you need to add || exit 1 after the $(INSTALL) to exit the for
loop in case of failure.

Ideally (but separately from this patch), you could contribute to
upstream dpdk a way to install examples.

> +endef
> +
> +# Build of the power example is broken (at least for 16.04).
> +define DPDK_DISABLE_POWER
> +	$(call KCONFIG_DISABLE_OPT,CONFIG_RTE_LIBRTE_POWER,\
> +			$(@D)/build/.config)
> +endef
> +
> +DPDK_POST_CONFIGURE_HOOKS += DPDK_DISABLE_POWER
> +endif
> +
> +define DPDK_CONFIGURE_CMDS
> +	$(MAKE) -C $(@D) T=$(DPDK_CONFIG) RTE_KERNELDIR=$(LINUX_DIR) \
> +			   CROSS=$(TARGET_CROSS) config
> +endef
> +
> +define DPDK_BUILD_CMDS
> +	$(MAKE) -C $(@D) RTE_KERNELDIR=$(LINUX_DIR) CROSS=$(TARGET_CROSS)
> +	$(DPDK_BUILD_EXAMPLES)
> +endef
> +
> +define DPDK_INSTALL_STAGING_CMDS
> +	$(MAKE) -C $(@D) DESTDIR=$(STAGING_DIR) prefix=/usr \
> +		 CROSS=$(TARGET_CROSS) install-sdk
> +endef
> +
> +ifeq ($(BR2_PACKAGE_DPDK_TEST),y)
> +define DPDK_INSTALL_TARGET_TEST
> +	$(INSTALL) -m 0755 -d $(TARGET_DIR)/usr/dpdk
> +	$(INSTALL) -m 0755 -D $(@D)/build/app/test $(TARGET_DIR)/usr/dpdk
> +	$(INSTALL) -m 0755 -D $(@D)/app/test/*.py $(TARGET_DIR)/usr/dpdk
> +endef
> +endif
> +
> +define DPDK_INSTALL_TARGET_CMDS
> +	$(MAKE) -C $(@D) DESTDIR=$(TARGET_DIR) prefix=/usr \
> +		CROSS=$(TARGET_CROSS) install-runtime
> +	$(DPDK_INSTALL_TARGET_TEST)
> +	$(DPDK_INSTALL_EXAMPLES)
> +endef
> +
> +$(eval $(generic-package))

Other than that, looks good to me.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH v3 3/3] dpdk: new package

[Jan Viktorin]

Hello Thomas,,

thank you for comments. You can find your previous review here:

[1] http://lists.busybox.net/pipermail/buildroot/2015-December/146614.html
[2] http://lists.busybox.net/pipermail/buildroot/2015-December/146630.html

On Tue, 22 Mar 2016 21:12:32 +0100
Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote:

> Hello,
> 
> On Tue, 22 Mar 2016 11:36:26 +0100, Jan Viktorin wrote:
> 
> > diff --git a/package/dpdk/Config.in b/package/dpdk/Config.in
> > new file mode 100644
> > index 0000000..a42271e
> > --- /dev/null
> > +++ b/package/dpdk/Config.in
> > @@ -0,0 +1,53 @@
> > +config BR2_PACKAGE_DPDK
> > +       bool "dpdk"  
> 
> Indentation of properties should be done with one tab (ditto in the
> following lines)
> 
> > +       depends on (BR2_i386 && !BR2_x86_i386 && !BR2_x86_i486 \
> > +		      && !BR2_x86_i586 && !BR2_x86_x1000) \
> > +                  || BR2_ARM_CPU_ARMV7A || BR2_aarch64 || BR2_aarch64_be  
> 
> I'm wondering why you have this list of architecture dependencies. Is
> it because there is really some architecture specific code, or is it a
> left-over from the times we didn't had the BR2_TOOLCHAIN_HAS_x
> options ?

Shortly: i686+ | ARMv7-a | AArch64, based on your suggestion [2]. Should
I put a comment there?

> 
> If that's really needed, please add a blind option like this:
> 
> config BR2_PACKAGE_DPDK_ARCH_SUPPORTS
> 	bool
> 	default y if ...

OK, seems reasonable.

> 
> That you can re-use as a dependency, both for the comment (see below)
> and the main option.
> 
> > +       depends on BR2_TOOLCHAIN_USES_GLIBC  
> 
> Then you need to have a comment like:
> 
> comment "dpdk needs a toolchain w/ glibc"
> 	depends on !BR2_TOOLCHAIN_USES_GLIBC
> 	depends on BR2_TOOLCHAIN_HAS_SYNC_...

OK, will fix.

> 
> > +       depends on BR2_TOOLCHAIN_HAS_SYNC_1
> > +       depends on BR2_TOOLCHAIN_HAS_SYNC_2
> > +       depends on BR2_TOOLCHAIN_HAS_SYNC_4
> > +       depends on BR2_TOOLCHAIN_HAS_SYNC_8  
> 
> Cosmetic nit, but I would prefer to see:
> 
> 	depends on BR2_TOOLCHAIN_HAS_SYNC_1 && \
> 		BR2_TOOLCHAIN_HAS_SYNC_2 && \
> 		BR2_TOOLCHAIN_HAS_SYNC_4 && \
> 		BR2_TOOLCHAIN_HAS_SYNC_8

Well, no problem, I just wonder why... as it's less readable and quite
ugly :).

> 
> 	
> 
> > +       help
> > +	 DPDK is a set of libraries and drivers for fast packet processing. It
> > +	 was designed to run on any processors, however, Intel x86 has been the
> > +	 first CPU to be supported. Ports for other CPUs like IBM Power 8 and
> > +	 ARM are under progress. It runs mostly in Linux userland. A FreeBSD
> > +	 port is now available for a subset of DPDK features.  
> 
> Indentation of the help text should be one tab + *two* spaces.

The help text is right tab+ss, but the options are accidently indented
by spaces. Will fix.

> 
> > +
> > +	 Notes:
> > +	 * To build the included Linux Kernel drivers, it is necessary to
> > +	   enable CONFIG_PCI_MSI, CONFIG_UIO.
> > +	 * To build the PCAP PMD properly, you need to enable the libpcap
> > +	   manually.
> > +	 * You may need to install the python2 interpreter if you want to use
> > +	   scripts dpdk_nic_bind.py and cpu_layout.py
> > +
> > +         http://www.dpdk.org/
> > +
> > +if BR2_PACKAGE_DPDK
> > +
> > +config BR2_PACKAGE_DPDK_CONFIG
> > +	string "Configuration"
> > +	default "i686-native-linuxapp-gcc" \
> > +		if BR2_x86_i686
> > +	default "x86_64-native-linuxapp-gcc" \
> > +		if BR2_x86_64
> > +	default "arm-armv7a-linuxapp-gcc" \
> > +		if BR2_ARM_CPU_ARMV7A
> > +	default "arm64-armv8a-linuxapp-gcc" \
> > +		if BR2_aarch64 || BR2_aarch64_be  
> 
> I don't remember if we discussed this: is there any reason to make this
> configurable? I.e is it necessary to make it a visible Config.in option
> as opposed to a blind one?

You've already agreed on this [2] and it seems to be a good compromise.
There are multiple configurations per CPU architecture and a user might
want to use a custom one.

> 
> > +
> > +config BR2_PACKAGE_DPDK_TEST
> > +	bool "Install tests suite"
> > +	select BR2_PACKAGE_PYTHON_PEXPECT if BR2_PACKAGE_PYTHON
> > +	help
> > +	  Install all DPDK tests. If you want to run the tests by the included
> > +	  autotest.py script you need to enable python manually.  
> 
> Is the test suite usable without Python ?

Yes, it's compiled as a single binary _test_. I use it that way. The
python scripts are helpful for automatization.

> 
> > diff --git a/package/dpdk/dpdk.mk b/package/dpdk/dpdk.mk
> > new file mode 100644
> > index 0000000..02860fd
> > --- /dev/null
> > +++ b/package/dpdk/dpdk.mk
> > @@ -0,0 +1,109 @@
> > +################################################################################
> > +#
> > +# dpdk
> > +#
> > +################################################################################
> > +
> > +DPDK_VERSION = 16.04-rc1  
> 
> We normally don't package release candidate / beta version. Are you
> confident that the final 16.04 release will be done before Buildroot
> ships its own release at the end of May ?

The release of 16.04 should be out early in April
(http://dpdk.org/dev/roadmap).

> 
> > +DPDK_SITE = http://dpdk.org/browse/dpdk/snapshot
> > +DPDK_SOURCE = dpdk-$(DPDK_VERSION).tar.gz  
> 
> Not needed, that's the default value.

OK.

> 
> > +
> > +DPDK_LICENSE = BSD (core), GPLv2+ (Linux drivers)  
> 
> BSD should be more specific, like BSD-2c, BSD-3c, etc.

BSD-3c, thanks.

> 
> > +DPDK_LICENSE_FILES = GNUmakefile LICENSE.GPL
> > +DPDK_INSTALL_STAGING = YES
> > +
> > +DPDK_DEPENDENCIES += linux  
> 
> If you need the Linux kernel to be built, then you need to also have:
> 
> 	depends on BR2_LINUX_KERNEL
> 
> in your Config.in, as well as a corresponding comment:
> 
> comment "dpdk needs the Linux kernel to be built"
> 	depends on !BR2_LINUX_KERNEL

OK.

> 
> > +
> > +ifeq ($(BR2_PACKAGE_LIBPCAP),y)
> > +DPDK_DEPENDENCIES += libpcap
> > +endif
> > +
> > +ifeq ($(BR2_SHARED_LIBS),y)
> > +define DPDK_ENABLE_SHARED_LIBS
> > +	$(call KCONFIG_ENABLE_OPT,CONFIG_RTE_BUILD_SHARED_LIB,\
> > +			$(@D)/build/.config)
> > +endef
> > +
> > +DPDK_POST_CONFIGURE_HOOKS += DPDK_ENABLE_SHARED_LIBS
> > +endif
> > +
> > +# We're building a kernel module without using the kernel-module infra,
> > +# so we need to tell we want module support in the kernel
> > +ifeq ($(BR2_PACKAGE_DPDK),y)
> > +LINUX_NEEDS_MODULES = y
> > +endif  
> 
> This is no longer the "right" way of doing this. Just select
> BR2_LINUX_NEEDS_MODULES in Config.in.

Cool, I didn't know that.

> 
> > +
> > +DPDK_CONFIG = $(call qstrip,$(BR2_PACKAGE_DPDK_CONFIG))
> > +
> > +ifeq ($(BR2_PACKAGE_DPDK_EXAMPLES),y)
> > +# Build of DPDK examples is not very straight-forward. It requires to have
> > +# the SDK and runtime installed on same place to reference it by RTE_SDK.
> > +# We place it locally in the build directory.
> > +define DPDK_BUILD_EXAMPLES
> > +	$(MAKE) -C $(@D) DESTDIR=$(@D)/examples-sdk \
> > +		CROSS=$(TARGET_CROSS) install-sdk install-runtime
> > +	$(MAKE) -C $(@D) RTE_KERNELDIR=$(LINUX_DIR) CROSS=$(TARGET_CROSS) \
> > +		RTE_SDK=$(@D)/examples-sdk/usr/local/share/dpdk \
> > +		T=$(DPDK_CONFIG) examples
> > +endef
> > +
> > +DPDK_EXAMPLES_PATH = $(@D)/examples-sdk/usr/local/share/dpdk/examples
> > +
> > +# Installation of examples is not supported in DPDK so we do it explicitly
> > +# here. As the binaries and libraries do not have a single or regular location
> > +# where to find them after build, we search for them by find.
> > +define DPDK_INSTALL_EXAMPLES
> > +	$(INSTALL) -m 0755 -d $(TARGET_DIR)/usr/local/bin
> > +	$(INSTALL) -m 0755 -d $(TARGET_DIR)/usr/local/lib  
> 
> In Buildroot, everything is installed in /usr, not /usr/local. Why

Just testing the reviewers here :). Well, at first I wanted to easily
check that it installs the right binaries and this is better checked in
an empty directory then in the /usr/bin with a great amount of other
things. I will put those into /usr/bin.

> should this package be different? Also, to create directories, we
> typically use mkdir -p.

Will use mkdir -p.

> 
> > +	for f in `find $(DPDK_EXAMPLES_PATH) -executable -type f   \
> > +			-name '[a-z]*.so*' | grep '\/lib\/.*'`; do \
> > +		$(INSTALL) -m 0755 -D $$f                          \
> > +			$(TARGET_DIR)/usr/local/lib/`basename $$f`;\
> > +	done
> > +	for f in `find $(DPDK_EXAMPLES_PATH) -executable -type f   \
> > +			! -name '*.so*' | grep '\/app\/.*'`; do    \
> > +		$(INSTALL) -m 0755 -D $$f                          \
> > +			$(TARGET_DIR)/usr/local/bin/`basename $$f`;\
> > +	done  
> 
> It's a bit annoying to have this logic. Could it be simplified to use
> -path instead of -name and avoid the grep ?

OK, will try.

> 
> Also, you need to add || exit 1 after the $(INSTALL) to exit the for
> loop in case of failure.

OK, will fix.

> 
> Ideally (but separately from this patch), you could contribute to
> upstream dpdk a way to install examples.

True. First of all, I am happy to do it automatically at the moment. Of
course, this is not the right place to solve it.

I'll try to patch this upstream and/or move this into a DPDK patch
temporarily.

> 
> > +endef
> > +
> > +# Build of the power example is broken (at least for 16.04).
> > +define DPDK_DISABLE_POWER
> > +	$(call KCONFIG_DISABLE_OPT,CONFIG_RTE_LIBRTE_POWER,\
> > +			$(@D)/build/.config)
> > +endef
> > +
> > +DPDK_POST_CONFIGURE_HOOKS += DPDK_DISABLE_POWER
> > +endif
> > +
> > +define DPDK_CONFIGURE_CMDS
> > +	$(MAKE) -C $(@D) T=$(DPDK_CONFIG) RTE_KERNELDIR=$(LINUX_DIR) \
> > +			   CROSS=$(TARGET_CROSS) config
> > +endef
> > +
> > +define DPDK_BUILD_CMDS
> > +	$(MAKE) -C $(@D) RTE_KERNELDIR=$(LINUX_DIR) CROSS=$(TARGET_CROSS)
> > +	$(DPDK_BUILD_EXAMPLES)
> > +endef
> > +
> > +define DPDK_INSTALL_STAGING_CMDS
> > +	$(MAKE) -C $(@D) DESTDIR=$(STAGING_DIR) prefix=/usr \
> > +		 CROSS=$(TARGET_CROSS) install-sdk
> > +endef
> > +
> > +ifeq ($(BR2_PACKAGE_DPDK_TEST),y)
> > +define DPDK_INSTALL_TARGET_TEST
> > +	$(INSTALL) -m 0755 -d $(TARGET_DIR)/usr/dpdk
> > +	$(INSTALL) -m 0755 -D $(@D)/build/app/test $(TARGET_DIR)/usr/dpdk
> > +	$(INSTALL) -m 0755 -D $(@D)/app/test/*.py $(TARGET_DIR)/usr/dpdk
> > +endef
> > +endif
> > +
> > +define DPDK_INSTALL_TARGET_CMDS
> > +	$(MAKE) -C $(@D) DESTDIR=$(TARGET_DIR) prefix=/usr \
> > +		CROSS=$(TARGET_CROSS) install-runtime
> > +	$(DPDK_INSTALL_TARGET_TEST)
> > +	$(DPDK_INSTALL_EXAMPLES)
> > +endef
> > +
> > +$(eval $(generic-package))  
> 
> Other than that, looks good to me.

Thank you!
Jan

> 
> Thomas



-- 
   Jan Viktorin                  E-mail: Viktorin at RehiveTech.com
   System Architect              Web:    www.RehiveTech.com
   RehiveTech
   Brno, Czech Republic

[Buildroot] [PATCH v3 3/3] dpdk: new package

[Jan Viktorin]

Hello Thomas,

I've got some questions about the Config.in...

On Tue, 22 Mar 2016 21:12:32 +0100
Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote:

> Hello,
> 
> On Tue, 22 Mar 2016 11:36:26 +0100, Jan Viktorin wrote:
> 
> > diff --git a/package/dpdk/Config.in b/package/dpdk/Config.in
> > new file mode 100644
> > index 0000000..a42271e
> > --- /dev/null
> > +++ b/package/dpdk/Config.in
> > @@ -0,0 +1,53 @@
> > +config BR2_PACKAGE_DPDK
> > +       bool "dpdk"  
> 
> Indentation of properties should be done with one tab (ditto in the
> following lines)
> 
> > +       depends on (BR2_i386 && !BR2_x86_i386 && !BR2_x86_i486 \
> > +		      && !BR2_x86_i586 && !BR2_x86_x1000) \
> > +                  || BR2_ARM_CPU_ARMV7A || BR2_aarch64 || BR2_aarch64_be  
> 
> I'm wondering why you have this list of architecture dependencies. Is
> it because there is really some architecture specific code, or is it a
> left-over from the times we didn't had the BR2_TOOLCHAIN_HAS_x
> options ?
> 
> If that's really needed, please add a blind option like this:
> 
> config BR2_PACKAGE_DPDK_ARCH_SUPPORTS
> 	bool
> 	default y if ...

So, should I move here also the HAS_SYNC checks?

> 
> That you can re-use as a dependency, both for the comment (see below)
> and the main option.
> 
> > +       depends on BR2_TOOLCHAIN_USES_GLIBC  
> 
> Then you need to have a comment like:
> 
> comment "dpdk needs a toolchain w/ glibc"
> 	depends on !BR2_TOOLCHAIN_USES_GLIBC
> 	depends on BR2_TOOLCHAIN_HAS_SYNC_...

The conditiion must be "if all dependencies but BR2_TOOLCHAIN_USES_GLIBC
are met", is it right? So, should I put here the HAS_SYNC_*,
ARCH_SUPPORTS, BR2_LINUX_KERNEL and BR2_LINUX_NEEDS_MODULES?

> 
> > +       depends on BR2_TOOLCHAIN_HAS_SYNC_1
> > +       depends on BR2_TOOLCHAIN_HAS_SYNC_2
> > +       depends on BR2_TOOLCHAIN_HAS_SYNC_4
> > +       depends on BR2_TOOLCHAIN_HAS_SYNC_8  
> 
> Cosmetic nit, but I would prefer to see:
> 
> 	depends on BR2_TOOLCHAIN_HAS_SYNC_1 && \
> 		BR2_TOOLCHAIN_HAS_SYNC_2 && \
> 		BR2_TOOLCHAIN_HAS_SYNC_4 && \
> 		BR2_TOOLCHAIN_HAS_SYNC_8

[...]

> 	depends on BR2_LINUX_KERNEL
> 
> in your Config.in, as well as a corresponding comment:
> 
> comment "dpdk needs the Linux kernel to be built"
> 	depends on !BR2_LINUX_KERNEL

The same situation here. Should I test "all && !BR2_LINUX_KERNEL"?

> 
> > +
> > +ifeq ($(BR2_PACKAGE_LIBPCAP),y)
> > +DPDK_DEPENDENCIES += libpcap
> > +endif
> > +
> > +ifeq ($(BR2_SHARED_LIBS),y)
> > +define DPDK_ENABLE_SHARED_LIBS
> > +	$(call KCONFIG_ENABLE_OPT,CONFIG_RTE_BUILD_SHARED_LIB,\
> > +			$(@D)/build/.config)
> > +endef
> > +
> > +DPDK_POST_CONFIGURE_HOOKS += DPDK_ENABLE_SHARED_LIBS
> > +endif
> > +
> > +# We're building a kernel module without using the kernel-module infra,
> > +# so we need to tell we want module support in the kernel
> > +ifeq ($(BR2_PACKAGE_DPDK),y)
> > +LINUX_NEEDS_MODULES = y
> > +endif  
> 
> This is no longer the "right" way of doing this. Just select
> BR2_LINUX_NEEDS_MODULES in Config.in.
> 

[...]

My current state is:

config BR2_PACKAGE_DPDK
	depends on BR2_PACKAGE_DPDK_ARCH_SUPPORTS                               
	depends on BR2_TOOLCHAIN_USES_GLIBC
	depends on BR2_LINUX_NEEDS_MODULES
        depends on BR2_TOOLCHAIN_HAS_SYNC_1 &&\
                 BR2_TOOLCHAIN_HAS_SYNC_2 &&\
                 BR2_TOOLCHAIN_HAS_SYNC_4 &&\
                 BR2_TOOLCHAIN_HAS_SYNC_8

config BR2_PACKAGE_DPDK_ARCH_SUPPORTS                                           
        bool                                                                    
        default y if (BR2_i386 && !BR2_x86_i386 && !BR2_x86_i486 \              
                         && !BR2_x86_i586 && !BR2_x86_x1000) \                   
                || BR2_ARM_CPU_ARMV7A || BR2_aarch64 || BR2_aarch64_be

comment "dpdk needs a toolchain w/ glibc"
	depends on !BR2_TOOLCHAIN_USES_GLIBC
	depends on ??

comment "dpdk needs the Linux kernel to be built"
	depends on !BR2_LINUX_KERNEL
	depends on ??


Jan

> > +$(eval $(generic-package))  
> 
> Other than that, looks good to me.
> 
> Thomas



-- 
  Jan Viktorin                E-mail: Viktorin at RehiveTech.com
  System Architect            Web:    www.RehiveTech.com
  RehiveTech
  Brno, Czech Republic

[Buildroot] [PATCH v3 3/3] dpdk: new package

[Thomas Petazzoni]

Hello Jan,

On Fri, 25 Mar 2016 13:32:40 +0100, Jan Viktorin wrote:

> > config BR2_PACKAGE_DPDK_ARCH_SUPPORTS
> > 	bool
> > 	default y if ...
> 
> So, should I move here also the HAS_SYNC checks?

Yes, correct. Even though the HAS_SYNC dependencies are not
completely/only architecture related, we handle them as such, so it
makes sense to have them in the foo_ARCH_SUPPORTS variable.

> > comment "dpdk needs a toolchain w/ glibc"
> > 	depends on !BR2_TOOLCHAIN_USES_GLIBC
> > 	depends on BR2_TOOLCHAIN_HAS_SYNC_...
> 
> The conditiion must be "if all dependencies but BR2_TOOLCHAIN_USES_GLIBC
> are met", is it right? So, should I put here the HAS_SYNC_*,
> ARCH_SUPPORTS, BR2_LINUX_KERNEL and BR2_LINUX_NEEDS_MODULES?

I think you could do like the on2-8170-libs package, and have a single
comment:

comment "dpdk needs a glibc toolchain and a Linux kernel to be built"
	depends on BR2_PACKAGE_DPDK_ARCH_SUPPORTS
	depends on !BR2_LINUX_KERNEL || !BR2_TOOLCHAIN_USES_GLIBC

The idea is that the comment should not be visible if we are on an
architecture that anyway does not support DPDK. Otherwise someone say
on MIPS will see the comment, enable glibc and Linux kernel, and then
will see the comment disappear but not the option to enable dpdk
appearing. Which would be confusing, to say the least.

> My current state is:
> 
> config BR2_PACKAGE_DPDK
> 	depends on BR2_PACKAGE_DPDK_ARCH_SUPPORTS                               
> 	depends on BR2_TOOLCHAIN_USES_GLIBC

	depends on BR2_LINUX_KERNEL

> 	depends on BR2_LINUX_NEEDS_MODULES

This should be a select

>         depends on BR2_TOOLCHAIN_HAS_SYNC_1 &&\
>                  BR2_TOOLCHAIN_HAS_SYNC_2 &&\
>                  BR2_TOOLCHAIN_HAS_SYNC_4 &&\
>                  BR2_TOOLCHAIN_HAS_SYNC_8

This should go in the BR2_PACKAGE_DPDK_ARCH_SUPPORTS

> config BR2_PACKAGE_DPDK_ARCH_SUPPORTS                                           
>         bool                                                                    
>         default y if (BR2_i386 && !BR2_x86_i386 && !BR2_x86_i486 \              
>                          && !BR2_x86_i586 && !BR2_x86_x1000) \                   
>                 || BR2_ARM_CPU_ARMV7A || BR2_aarch64 || BR2_aarch64_be

This can be written in a much more readable way as:

	default y if BR2_i386 && !BR2_x86... && !BR2_...
	default y if BR2_ARM_CPU_ARMV7A
	default y if BR2_aarch64
	...

> comment "dpdk needs a toolchain w/ glibc"
> 	depends on !BR2_TOOLCHAIN_USES_GLIBC
> 	depends on ??
> 
> comment "dpdk needs the Linux kernel to be built"
> 	depends on !BR2_LINUX_KERNEL
> 	depends on ??

As said above, only one comment is needed. So here is a proposal:

config BR2_PACKAGE_DPDK_ARCH_SUPPORTS
	bool
	depends on BR2_TOOLCHAIN_HAS_SYNC_1 && \
		BR2_TOOLCHAIN_HAS_SYNC_2 && \
		BR2_TOOLCHAIN_HAS_SYNC_4 && \
		BR2_TOOLCHAIN_HAS_SYNC_8
	default y if BR2_i386 && !BR2_x86_i386 && !BR2_x86_i486 \              
			&& !BR2_x86_i586 && !BR2_x86_x1000
	default y if BR2_ARM_CPU_ARMV7A
	default y if BR2_aarch64 || BR2_aarch64_be

config BR2_PACKAGE_DPDK
	bool "dpdk"
	depends on BR2_PACKAGE_DPDK_ARCH_SUPPORTS
	depends on BR2_TOOLCHAIN_USES_GLIBC
	depends on BR2_LINUX_KERNEL
	select BR2_LINUX_KERNEL_NEEDS_MODULES
	help
	  Blabla

comment "dpdk needs a glibc toolchain and a Linux kernel to be built"
	depends on BR2_PACKAGE_DPDK_ARCH_SUPPORTS
	depends on !BR2_TOOLCHAIN_USES_GLIBC || !BR2_LINUX_KERNEL


Hope this helps,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH v4 0/3] dpdk: new package

[Jan Viktorin]

Hello,

I've hopefly fixed all the issues. The installation of DPDK examples has been
fixed according to T. Petazzoni suggestions. The DPDK *can* install examples
but it does it in quite a wierd way (with built garbage and into directories
app/ and lib/). Moreover, by overriding the target installation directory, the
build fails as it cannot find some SDK libraries.

DPDK 16.04-rc2 is out. The final release should appear in the next 2 weeks.

Regards
Jan

Jan Viktorin (3):
  python-ptyprocess: new package
  python-pexpect: new package
  dpdk: new package

 package/Config.in                                |   3 +
 package/dpdk/Config.in                           |  64 ++++++++++++++
 package/dpdk/dpdk.hash                           |   2 +
 package/dpdk/dpdk.mk                             | 106 +++++++++++++++++++++++
 package/python-pexpect/Config.in                 |  11 +++
 package/python-pexpect/python-pexpect.hash       |   4 +
 package/python-pexpect/python-pexpect.mk         |  15 ++++
 package/python-ptyprocess/Config.in              |   7 ++
 package/python-ptyprocess/python-ptyprocess.hash |   4 +
 package/python-ptyprocess/python-ptyprocess.mk   |  14 +++
 10 files changed, 230 insertions(+)
 create mode 100644 package/dpdk/Config.in
 create mode 100644 package/dpdk/dpdk.hash
 create mode 100644 package/dpdk/dpdk.mk
 create mode 100644 package/python-pexpect/Config.in
 create mode 100644 package/python-pexpect/python-pexpect.hash
 create mode 100644 package/python-pexpect/python-pexpect.mk
 create mode 100644 package/python-ptyprocess/Config.in
 create mode 100644 package/python-ptyprocess/python-ptyprocess.hash
 create mode 100644 package/python-ptyprocess/python-ptyprocess.mk

-- 
2.7.0

[Buildroot] [PATCH v4 1/3] python-ptyprocess: new package

[Jan Viktorin]

v4
* bump to 0.5.1
* source https://pypi.python.org/pypi/ptyprocess
* added hash file

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 package/Config.in                                |  1 +
 package/python-ptyprocess/Config.in              |  7 +++++++
 package/python-ptyprocess/python-ptyprocess.hash |  4 ++++
 package/python-ptyprocess/python-ptyprocess.mk   | 14 ++++++++++++++
 4 files changed, 26 insertions(+)
 create mode 100644 package/python-ptyprocess/Config.in
 create mode 100644 package/python-ptyprocess/python-ptyprocess.hash
 create mode 100644 package/python-ptyprocess/python-ptyprocess.mk

diff --git a/package/Config.in b/package/Config.in
index 8208da8..fda5231 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -689,6 +689,7 @@ menu "External python modules"
 	source "package/python-posix-ipc/Config.in"
 	source "package/python-protobuf/Config.in"
 	source "package/python-psutil/Config.in"
+	source "package/python-ptyprocess/Config.in"
 	source "package/python-pyasn/Config.in"
 	source "package/python-pyasn-modules/Config.in"
 	source "package/python-pycli/Config.in"
diff --git a/package/python-ptyprocess/Config.in b/package/python-ptyprocess/Config.in
new file mode 100644
index 0000000..4378943
--- /dev/null
+++ b/package/python-ptyprocess/Config.in
@@ -0,0 +1,7 @@
+config BR2_PACKAGE_PYTHON_PTYPROCESS
+	bool "python-ptyprocess"
+	help
+	  Launch a subprocess in a pseudo terminal (pty), and interact with
+	  both the process and its pty.
+
+	  https://github.com/pexpect/ptyprocess
diff --git a/package/python-ptyprocess/python-ptyprocess.hash b/package/python-ptyprocess/python-ptyprocess.hash
new file mode 100644
index 0000000..3973414
--- /dev/null
+++ b/package/python-ptyprocess/python-ptyprocess.hash
@@ -0,0 +1,4 @@
+# got from https://pypi.python.org
+md5sum	94e537122914cc9ec9c1eadcd36e73a1 ptyprocess-0.5.1.tar.gz
+# computed manually
+sha256sum 0530ce63a9295bfae7bd06edc02b6aa935619f486f0f1dc0972f516265ee81a6  ptyprocess-0.5.1.tar.gz
diff --git a/package/python-ptyprocess/python-ptyprocess.mk b/package/python-ptyprocess/python-ptyprocess.mk
new file mode 100644
index 0000000..74b1217
--- /dev/null
+++ b/package/python-ptyprocess/python-ptyprocess.mk
@@ -0,0 +1,14 @@
+################################################################################
+#
+# python-ptyprocess
+#
+################################################################################
+
+PYTHON_PTYPROCESS_VERSION = 0.5.1
+PYTHON_PTYPROCESS_SITE = https://pypi.python.org/pypi/ptyprocess
+PYTHON_PTYPROCESS_SOURCE = ptyprocess-$(PYTHON_PTYPROCESS_VERSION).tar.gz
+PYTHON_PTYPROCESS_LICENSE = ISC
+PYTHON_PTYPROCESS_LICENSE_FILES = LICENSE
+PYTHON_PTYPROCESS_SETUP_TYPE = distutils
+
+$(eval $(python-package))
-- 
2.7.0

[Buildroot] [PATCH v4 2/3] python-pexpect: new package

[Jan Viktorin]

v4
* source https://pypi.python.org/pypi/pexpect
* added hash file

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 package/Config.in                          |  1 +
 package/python-pexpect/Config.in           | 11 +++++++++++
 package/python-pexpect/python-pexpect.hash |  4 ++++
 package/python-pexpect/python-pexpect.mk   | 15 +++++++++++++++
 4 files changed, 31 insertions(+)
 create mode 100644 package/python-pexpect/Config.in
 create mode 100644 package/python-pexpect/python-pexpect.hash
 create mode 100644 package/python-pexpect/python-pexpect.mk

diff --git a/package/Config.in b/package/Config.in
index fda5231..1c995ca 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -686,6 +686,7 @@ menu "External python modules"
 	source "package/python-paho-mqtt/Config.in"
 	source "package/python-pam/Config.in"
 	source "package/python-paramiko/Config.in"
+	source "package/python-pexpect/Config.in"
 	source "package/python-posix-ipc/Config.in"
 	source "package/python-protobuf/Config.in"
 	source "package/python-psutil/Config.in"
diff --git a/package/python-pexpect/Config.in b/package/python-pexpect/Config.in
new file mode 100644
index 0000000..4d04c01
--- /dev/null
+++ b/package/python-pexpect/Config.in
@@ -0,0 +1,11 @@
+config BR2_PACKAGE_PYTHON_PEXPECT
+	bool "python-pexpect"
+	select BR2_PACKAGE_PYTHON_PTYPROCESS
+	help
+	  Pexpect is a pure Python module for spawning child applications;
+	  controlling them; and responding to expected patterns in their
+	  output. Pexpect works like Don Libes? Expect. Pexpect allows your
+	  script to spawn a child application and control it as if a human
+	  were typing commands.
+
+	  https://pexpect.readthedocs.org
diff --git a/package/python-pexpect/python-pexpect.hash b/package/python-pexpect/python-pexpect.hash
new file mode 100644
index 0000000..0d1e1fe
--- /dev/null
+++ b/package/python-pexpect/python-pexpect.hash
@@ -0,0 +1,4 @@
+# got from https://pypi.python.org
+md5sum	056df81e6ca7081f1015b4b147b977b7
+# computed manually:
+sha256sum 232795ebcaaf2e120396dbbaa3a129eda51757eeaae1911558f4ef8ee414fc6c  pexpect-4.0.1.tar.gz
diff --git a/package/python-pexpect/python-pexpect.mk b/package/python-pexpect/python-pexpect.mk
new file mode 100644
index 0000000..8b2809b
--- /dev/null
+++ b/package/python-pexpect/python-pexpect.mk
@@ -0,0 +1,15 @@
+################################################################################
+#
+# python-pexpect
+#
+################################################################################
+
+PYTHON_PEXPECT_VERSION = 4.0.1
+PYTHON_PEXPECT_SITE = https://pypi.python.org/pypi/pexpect
+PYTHON_PEXPECT_SOURCE = pexpect-$(PYTHON_PEXPECT_VERSION).tar.gz
+PYTHON_PEXPECT_LICENSE = ISC
+PYTHON_PEXPECT_LICENSE_FILES = LICENSE
+PYTHON_PEXPECT_SETUP_TYPE = distutils
+PYTHON_PEXPECT_DEPENDENCIES = python-ptyprocess
+
+$(eval $(python-package))
-- 
2.7.0

[Buildroot] [PATCH v4 3/3] dpdk: new package

[Jan Viktorin]

This patch introduces support of the DPDK library (www.dpdk.org) into
Buildroot. DPDK is a library for high-speed packet sending/receiving
while bypassing the Linux Kernel. It allows to reach a high throughput
for 10-100 Gbps networks on the x86 platform.

The package compiles and installs DPDK libraries on the target and
staging and allows to compile other applications depending on the DPDK
library. It can also install some basic tools the DPDK provides
(testpmd, python scripts, test suite).

The patch assumes DPDK 16.04-rc2. This version contains support for the ARM
architecture. The ARM ports can be tested by

 qemu_aarch64_virt_defconfig
 qemu_arm_vexpress_defconfig

The included hash was calculated locally by downloading the tar.gz archives by
hand.

There are unfortunately some pitfalls:

* it may require to enable PCI, MSIX, UIO in the Linux Kernel
  (some defconfigs does not include as default and it is platform
  dependent as ARMv7 almost does not use PCI)

* when building PCAP PMD driver, the libpcap is required (partially
  fixed as suggested by Arnout)

* some tools the DPDK provides depend on Python(2) so the user has
  to enable it to install those

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v2: (mostly suggestions by Arnout)
* simplified Config.in - avoid version, source and config selection
* improved dependency on libpcap
* user python scripts are included if python package is enabled
* installing of tests includes autotest*.py scripts (we do not care
  about the python here, assuming the user will install it manually
  when testing)
* minor coding style fixes
* depends on python-pexpect package
* using version 2.2.0-rc3
v3
* bump to version 16.04-rc1
* utilizing the new install syntax of DPDK
* fixed line wrapping
* testpmd is now always installed (no option for this)
* fixed python dependencies (T. Petazzoni)
* supporting i686+ architectures (T. Petazzoni)
* dropped PPC support for now
* utilizing KCONFIG_ENABLE_OPT macro
* support for build & install of examples (ugly code)
* build with MAKE instead of MAKE1 seems OK now
* dropped COMBINE_LIBS (no more in DPDK)
* added HAS_SYNC_x
* few random fixes
v4 (suggestions by T. Petazzoni)
* bump to version 16.04-rc2
* fixed indentation in Config.in
* fixed depends in Config.in
* handling kernel modules by BR2_LINUX_NEEDS_MODULES
* simplified installation of examples

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
 package/Config.in      |   1 +
 package/dpdk/Config.in |  64 +++++++++++++++++++++++++++++
 package/dpdk/dpdk.hash |   2 +
 package/dpdk/dpdk.mk   | 106 +++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 173 insertions(+)
 create mode 100644 package/dpdk/Config.in
 create mode 100644 package/dpdk/dpdk.hash
 create mode 100644 package/dpdk/dpdk.mk

diff --git a/package/Config.in b/package/Config.in
index 1c995ca..bd058f7 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -1097,6 +1097,7 @@ menu "Networking"
 	source "package/cgic/Config.in"
 	source "package/cppzmq/Config.in"
 	source "package/czmq/Config.in"
+	source "package/dpdk/Config.in"
 	source "package/filemq/Config.in"
 	source "package/flickcurl/Config.in"
 	source "package/fmlib/Config.in"
diff --git a/package/dpdk/Config.in b/package/dpdk/Config.in
new file mode 100644
index 0000000..69bc56e8
--- /dev/null
+++ b/package/dpdk/Config.in
@@ -0,0 +1,64 @@
+config BR2_PACKAGE_DPDK
+	bool "dpdk"
+	depends on BR2_PACKAGE_DPDK_ARCH_SUPPORTS
+	depends on BR2_TOOLCHAIN_USES_GLIBC
+	depends on BR2_LINUX_KERNEL
+	select BR2_LINUX_NEEDS_MODULES
+	help
+	  DPDK is a set of libraries and drivers for fast packet processing. It
+	  was designed to run on any processors, however, Intel x86 has been
+	  the first CPU to be supported. Ports for other CPUs like IBM Power 8
+	  and ARM are under progress. It runs mostly in Linux userland. A
+	  FreeBSD port is now available for a subset of DPDK features.
+
+	  Notes:
+	  * To build the included Linux Kernel drivers, it is necessary to
+	    enable CONFIG_PCI_MSI, CONFIG_UIO.
+	  * To build the PCAP PMD properly, you need to enable the libpcap
+	    manually.
+	  * You may need to install the python2 interpreter if you want to use
+	    scripts dpdk_nic_bind.py and cpu_layout.py
+
+	  http://www.dpdk.org/
+
+config BR2_PACKAGE_DPDK_ARCH_SUPPORTS
+	bool
+	depends on BR2_TOOLCHAIN_HAS_SYNC_1 &&\
+		BR2_TOOLCHAIN_HAS_SYNC_2 &&\
+		BR2_TOOLCHAIN_HAS_SYNC_4 &&\
+		BR2_TOOLCHAIN_HAS_SYNC_8
+	default y if (BR2_i386 && !BR2_x86_i386 && !BR2_x86_i486 \
+			&& !BR2_x86_i586 && !BR2_x86_x1000)
+	default y if BR2_ARM_CPU_ARMV7A
+	default y if BR2_aarch64 || BR2_aarch64_be
+
+comment "dpdk needs a glibc toolchain and a Linux Kernel to be built"
+	depends on BR2_PACKAGE_DPDK_ARCH_SUPPORTS
+	depends on !BR2_TOOLCHAIN_USES_GLIBC || !BR2_LINUX_KERNEL
+
+if BR2_PACKAGE_DPDK
+
+config BR2_PACKAGE_DPDK_CONFIG
+	string "Configuration"
+	default "i686-native-linuxapp-gcc" \
+		if BR2_x86_i686
+	default "x86_64-native-linuxapp-gcc" \
+		if BR2_x86_64
+	default "arm-armv7a-linuxapp-gcc" \
+		if BR2_ARM_CPU_ARMV7A
+	default "arm64-armv8a-linuxapp-gcc" \
+		if BR2_aarch64 || BR2_aarch64_be
+
+config BR2_PACKAGE_DPDK_TEST
+	bool "Install tests suite"
+	select BR2_PACKAGE_PYTHON_PEXPECT if BR2_PACKAGE_PYTHON
+	help
+	  Install all DPDK tests. If you want to run the tests by the included
+	  autotest.py script you need to enable python manually.
+
+config BR2_PACKAGE_DPDK_EXAMPLES
+	bool "Build & install examples"
+	help
+	  Build and install all examples selected by the current configuration.
+
+endif
diff --git a/package/dpdk/dpdk.hash b/package/dpdk/dpdk.hash
new file mode 100644
index 0000000..cce15e1
--- /dev/null
+++ b/package/dpdk/dpdk.hash
@@ -0,0 +1,2 @@
+# Locally calculated
+sha256	fdd96e8f3ae9506642c8cd961422960360abbe9d1ba28323704c93a3c065ba6d  dpdk-16.04-rc2.tar.gz
diff --git a/package/dpdk/dpdk.mk b/package/dpdk/dpdk.mk
new file mode 100644
index 0000000..6a4824a
--- /dev/null
+++ b/package/dpdk/dpdk.mk
@@ -0,0 +1,106 @@
+################################################################################
+#
+# dpdk
+#
+################################################################################
+
+DPDK_VERSION = 16.04-rc2
+DPDK_SITE = http://dpdk.org/browse/dpdk/snapshot
+
+DPDK_LICENSE = BSD-2c (core), GPLv2+ (Linux drivers)
+DPDK_LICENSE_FILES = GNUmakefile LICENSE.GPL
+DPDK_INSTALL_STAGING = YES
+
+DPDK_DEPENDENCIES += linux
+
+ifeq ($(BR2_PACKAGE_LIBPCAP),y)
+DPDK_DEPENDENCIES += libpcap
+endif
+
+ifeq ($(BR2_SHARED_LIBS),y)
+define DPDK_ENABLE_SHARED_LIBS
+	$(call KCONFIG_ENABLE_OPT,CONFIG_RTE_BUILD_SHARED_LIB,\
+			$(@D)/build/.config)
+endef
+
+DPDK_POST_CONFIGURE_HOOKS += DPDK_ENABLE_SHARED_LIBS
+endif
+
+# We're building a kernel module without using the kernel-module infra,
+# so we need to tell we want module support in the kernel
+ifeq ($(BR2_PACKAGE_DPDK),y)
+LINUX_NEEDS_MODULES = y
+endif
+
+DPDK_CONFIG = $(call qstrip,$(BR2_PACKAGE_DPDK_CONFIG))
+
+ifeq ($(BR2_PACKAGE_DPDK_EXAMPLES),y)
+# Build of DPDK examples is not very straight-forward. It requires to have
+# the SDK and runtime installed on same place to reference it by RTE_SDK.
+# We place it locally in the build directory.
+define DPDK_BUILD_EXAMPLES
+	$(MAKE) -C $(@D) DESTDIR=$(@D)/examples-sdk \
+		CROSS=$(TARGET_CROSS) install-sdk install-runtime
+	$(MAKE) -C $(@D) RTE_KERNELDIR=$(LINUX_DIR) CROSS=$(TARGET_CROSS) \
+		RTE_SDK=$(@D)/examples-sdk/usr/local/share/dpdk \
+		T=$(DPDK_CONFIG) examples
+endef
+
+DPDK_EXAMPLES_PATH = $(@D)/examples-sdk/usr/local/share/dpdk/examples
+
+# Installation of examples is tricky in DPDK so we do it explicitly here.
+# As the binaries and libraries do not have a single or regular location
+# where to find them after build, we search for them by find.
+define DPDK_INSTALL_EXAMPLES
+	for f in `find $(DPDK_EXAMPLES_PATH) -executable -type f       \
+			-path '*/lib/*.so*'`; do                       \
+		$(INSTALL) -m 0755 -D $$f                              \
+			$(TARGET_DIR)/usr/lib/`basename $$f` || exit 1;\
+	done
+	for f in `find $(DPDK_EXAMPLES_PATH) -executable -type f       \
+			-path '*/app/*'`; do                           \
+		$(INSTALL) -m 0755 -D $$f                              \
+			$(TARGET_DIR)/usr/bin/`basename $$f` || exit 1;\
+	done
+endef
+
+# Build of the power example is broken (at least for 16.04).
+define DPDK_DISABLE_POWER
+	$(call KCONFIG_DISABLE_OPT,CONFIG_RTE_LIBRTE_POWER,\
+			$(@D)/build/.config)
+endef
+
+DPDK_POST_CONFIGURE_HOOKS += DPDK_DISABLE_POWER
+endif
+
+define DPDK_CONFIGURE_CMDS
+	$(MAKE) -C $(@D) T=$(DPDK_CONFIG) RTE_KERNELDIR=$(LINUX_DIR) \
+			   CROSS=$(TARGET_CROSS) config
+endef
+
+define DPDK_BUILD_CMDS
+	$(MAKE) -C $(@D) RTE_KERNELDIR=$(LINUX_DIR) CROSS=$(TARGET_CROSS)
+	$(DPDK_BUILD_EXAMPLES)
+endef
+
+define DPDK_INSTALL_STAGING_CMDS
+	$(MAKE) -C $(@D) DESTDIR=$(STAGING_DIR) prefix=/usr \
+		 CROSS=$(TARGET_CROSS) install-sdk
+endef
+
+ifeq ($(BR2_PACKAGE_DPDK_TEST),y)
+define DPDK_INSTALL_TARGET_TEST
+	$(INSTALL) -m 0755 -d $(TARGET_DIR)/usr/dpdk
+	$(INSTALL) -m 0755 -D $(@D)/build/app/test $(TARGET_DIR)/usr/dpdk
+	$(INSTALL) -m 0755 -D $(@D)/app/test/*.py $(TARGET_DIR)/usr/dpdk
+endef
+endif
+
+define DPDK_INSTALL_TARGET_CMDS
+	$(MAKE) -C $(@D) DESTDIR=$(TARGET_DIR) prefix=/usr \
+		CROSS=$(TARGET_CROSS) install-runtime
+	$(DPDK_INSTALL_TARGET_TEST)
+	$(DPDK_INSTALL_EXAMPLES)
+endef
+
+$(eval $(generic-package))
-- 
2.7.0

[Buildroot] [PATCH v4 3/3] dpdk: new package

[Jan Viktorin]

On Sun, 27 Mar 2016 03:31:24 +0200
Jan Viktorin <viktorin@rehivetech.com> wrote:

> This patch introduces support of the DPDK library (www.dpdk.org) into
> Buildroot. DPDK is a library for high-speed packet sending/receiving
> while bypassing the Linux Kernel. It allows to reach a high throughput
> for 10-100 Gbps networks on the x86 platform.
> 
[...]
> +
> +# We're building a kernel module without using the kernel-module infra,
> +# so we need to tell we want module support in the kernel
> +ifeq ($(BR2_PACKAGE_DPDK),y)
> +LINUX_NEEDS_MODULES = y
> +endif

Ups, I forgot to delete this LINUX_NEEDS_MODULES (moved to Config.in
already).

Jan

> +
> +DPDK_CONFIG = $(call qstrip,$(BR2_PACKAGE_DPDK_CONFIG))
[...]

[Buildroot] [PATCH v4 2/3] python-pexpect: new package

[Jan Viktorin]

On Sun, 27 Mar 2016 03:31:23 +0200
Jan Viktorin <viktorin@rehivetech.com> wrote:

> v4
> * source https://pypi.python.org/pypi/pexpect
> * added hash file
> 
> Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
> ---

[...]

> diff --git a/package/python-pexpect/python-pexpect.hash b/package/python-pexpect/python-pexpect.hash
> new file mode 100644
> index 0000000..0d1e1fe
> --- /dev/null
> +++ b/package/python-pexpect/python-pexpect.hash
> @@ -0,0 +1,4 @@
> +# got from https://pypi.python.org
> +md5sum	056df81e6ca7081f1015b4b147b977b7
> +# computed manually:
> +sha256sum 232795ebcaaf2e120396dbbaa3a129eda51757eeaae1911558f4ef8ee414fc6c  pexpect-4.0.1.tar.gz

Should be md5 and sha256 (without sum)...

> diff --git a/package/python-pexpect/python-pexpect.mk b/package/python-pexpect/python-pexpect.mk
> new file mode 100644
> index 0000000..8b2809b
> --- /dev/null
> +++ b/package/python-pexpect/python-pexpect.mk
> @@ -0,0 +1,15 @@
> +################################################################################
> +#
> +# python-pexpect
> +#
> +################################################################################
> +
> +PYTHON_PEXPECT_VERSION = 4.0.1
> +PYTHON_PEXPECT_SITE = https://pypi.python.org/pypi/pexpect

The URL should be https://pypi.python.org/packages/source/p/pexpect.

> +PYTHON_PEXPECT_SOURCE = pexpect-$(PYTHON_PEXPECT_VERSION).tar.gz
> +PYTHON_PEXPECT_LICENSE = ISC
> +PYTHON_PEXPECT_LICENSE_FILES = LICENSE
> +PYTHON_PEXPECT_SETUP_TYPE = distutils
> +PYTHON_PEXPECT_DEPENDENCIES = python-ptyprocess
> +
> +$(eval $(python-package))

[Buildroot] [PATCH v4 1/3] python-ptyprocess: new package

[Jan Viktorin]

On Sun, 27 Mar 2016 03:31:22 +0200
Jan Viktorin <viktorin@rehivetech.com> wrote:

> v4
> * bump to 0.5.1
> * source https://pypi.python.org/pypi/ptyprocess
> * added hash file
> 
> Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
> ---

[...]

> diff --git a/package/python-ptyprocess/python-ptyprocess.hash b/package/python-ptyprocess/python-ptyprocess.hash
> new file mode 100644
> index 0000000..3973414
> --- /dev/null
> +++ b/package/python-ptyprocess/python-ptyprocess.hash
> @@ -0,0 +1,4 @@
> +# got from https://pypi.python.org
> +md5sum	94e537122914cc9ec9c1eadcd36e73a1 ptyprocess-0.5.1.tar.gz
> +# computed manually
> +sha256sum 0530ce63a9295bfae7bd06edc02b6aa935619f486f0f1dc0972f516265ee81a6  ptyprocess-0.5.1.tar.gz

Should be md5 and sha256 (without sum)...

> diff --git a/package/python-ptyprocess/python-ptyprocess.mk b/package/python-ptyprocess/python-ptyprocess.mk
> new file mode 100644
> index 0000000..74b1217
> --- /dev/null
> +++ b/package/python-ptyprocess/python-ptyprocess.mk
> @@ -0,0 +1,14 @@
> +################################################################################
> +#
> +# python-ptyprocess
> +#
> +################################################################################
> +
> +PYTHON_PTYPROCESS_VERSION = 0.5.1
> +PYTHON_PTYPROCESS_SITE = https://pypi.python.org/pypi/ptyprocess

The URL should be https://pypi.python.org/packages/source/p/ptyprocess.

It seems I accidently didn't test the very last version, sorry. Now, it seems to work.

> +PYTHON_PTYPROCESS_SOURCE = ptyprocess-$(PYTHON_PTYPROCESS_VERSION).tar.gz
> +PYTHON_PTYPROCESS_LICENSE = ISC
> +PYTHON_PTYPROCESS_LICENSE_FILES = LICENSE
> +PYTHON_PTYPROCESS_SETUP_TYPE = distutils
> +
> +$(eval $(python-package))

[Buildroot] [PATCH v4 2/3] python-pexpect: new package

[Yegor Yefremov]

On Sun, Mar 27, 2016 at 3:50 AM, Jan Viktorin <viktorin@rehivetech.com> wrote:
> On Sun, 27 Mar 2016 03:31:23 +0200
> Jan Viktorin <viktorin@rehivetech.com> wrote:
>
>> v4
>> * source https://pypi.python.org/pypi/pexpect
>> * added hash file
>>
>> Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
>> ---
>
> [...]
>
>> diff --git a/package/python-pexpect/python-pexpect.hash b/package/python-pexpect/python-pexpect.hash
>> new file mode 100644
>> index 0000000..0d1e1fe
>> --- /dev/null
>> +++ b/package/python-pexpect/python-pexpect.hash
>> @@ -0,0 +1,4 @@
>> +# got from https://pypi.python.org
>> +md5sum       056df81e6ca7081f1015b4b147b977b7
>> +# computed manually:
>> +sha256sum 232795ebcaaf2e120396dbbaa3a129eda51757eeaae1911558f4ef8ee414fc6c  pexpect-4.0.1.tar.gz
>
> Should be md5 and sha256 (without sum)...
>
>> diff --git a/package/python-pexpect/python-pexpect.mk b/package/python-pexpect/python-pexpect.mk
>> new file mode 100644
>> index 0000000..8b2809b
>> --- /dev/null
>> +++ b/package/python-pexpect/python-pexpect.mk
>> @@ -0,0 +1,15 @@
>> +################################################################################
>> +#
>> +# python-pexpect
>> +#
>> +################################################################################
>> +
>> +PYTHON_PEXPECT_VERSION = 4.0.1
>> +PYTHON_PEXPECT_SITE = https://pypi.python.org/pypi/pexpect
>
> The URL should be https://pypi.python.org/packages/source/p/pexpect.
>
>> +PYTHON_PEXPECT_SOURCE = pexpect-$(PYTHON_PEXPECT_VERSION).tar.gz
>> +PYTHON_PEXPECT_LICENSE = ISC
>> +PYTHON_PEXPECT_LICENSE_FILES = LICENSE
>> +PYTHON_PEXPECT_SETUP_TYPE = distutils
>> +PYTHON_PEXPECT_DEPENDENCIES = python-ptyprocess

drop this build-time dependency

Resend v5 with all fixes and add my

Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>

to both python patches.

Thanks.

Yegor

>> +
>> +$(eval $(python-package))

[Buildroot] [PATCH v4 1/3] python-ptyprocess: new package

[Thomas Petazzoni]

Hello,

On Sun, 27 Mar 2016 03:31:22 +0200, Jan Viktorin wrote:
> v4
> * bump to 0.5.1
> * source https://pypi.python.org/pypi/ptyprocess
> * added hash file

The changelog should not be in the commit log, but...

> 
> Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
> ---

 ... here.


> diff --git a/package/python-ptyprocess/Config.in b/package/python-ptyprocess/Config.in
> new file mode 100644
> index 0000000..4378943
> --- /dev/null
> +++ b/package/python-ptyprocess/Config.in
> @@ -0,0 +1,7 @@
> +config BR2_PACKAGE_PYTHON_PTYPROCESS
> +	bool "python-ptyprocess"
> +	help
> +	  Launch a subprocess in a pseudo terminal (pty), and interact with

Line slightly too long, I've rewrapped.

> +	  both the process and its pty.
> +
> +	  https://github.com/pexpect/ptyprocess
> diff --git a/package/python-ptyprocess/python-ptyprocess.hash b/package/python-ptyprocess/python-ptyprocess.hash
> new file mode 100644
> index 0000000..3973414
> --- /dev/null
> +++ b/package/python-ptyprocess/python-ptyprocess.hash
> @@ -0,0 +1,4 @@
> +# got from https://pypi.python.org
> +md5sum	94e537122914cc9ec9c1eadcd36e73a1 ptyprocess-0.5.1.tar.gz
> +# computed manually
> +sha256sum 0530ce63a9295bfae7bd06edc02b6aa935619f486f0f1dc0972f516265ee81a6  ptyprocess-0.5.1.tar.gz

As you noticed yourself, this hash file was wrong.


> +PYTHON_PTYPROCESS_VERSION = 0.5.1
> +PYTHON_PTYPROCESS_SITE = https://pypi.python.org/pypi/ptyprocess

Same for the URL.

I've fixed up those issues and applied. Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH v4 2/3] python-pexpect: new package

[Thomas Petazzoni]

Hello,

On Sun, 27 Mar 2016 03:31:23 +0200, Jan Viktorin wrote:
> v4
> * source https://pypi.python.org/pypi/pexpect
> * added hash file

The changelog should not be part of the commit, but should be ...

> 
> Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
> ---

... here.


> diff --git a/package/python-pexpect/Config.in b/package/python-pexpect/Config.in
> new file mode 100644
> index 0000000..4d04c01
> --- /dev/null
> +++ b/package/python-pexpect/Config.in
> @@ -0,0 +1,11 @@
> +config BR2_PACKAGE_PYTHON_PEXPECT
> +	bool "python-pexpect"
> +	select BR2_PACKAGE_PYTHON_PTYPROCESS
> +	help
> +	  Pexpect is a pure Python module for spawning child applications;
> +	  controlling them; and responding to expected patterns in their
> +	  output. Pexpect works like Don Libes? Expect. Pexpect allows your
> +	  script to spawn a child application and control it as if a human
> +	  were typing commands.

Same thing: lines slightly too long.

> diff --git a/package/python-pexpect/python-pexpect.hash b/package/python-pexpect/python-pexpect.hash
> new file mode 100644
> index 0000000..0d1e1fe
> --- /dev/null
> +++ b/package/python-pexpect/python-pexpect.hash
> @@ -0,0 +1,4 @@
> +# got from https://pypi.python.org
> +md5sum	056df81e6ca7081f1015b4b147b977b7

Not even a file name here.

> +# computed manually:
> +sha256sum 232795ebcaaf2e120396dbbaa3a129eda51757eeaae1911558f4ef8ee414fc6c  pexpect-4.0.1.tar.gz

I've fixed up the hash file.

> +PYTHON_PEXPECT_VERSION = 4.0.1
> +PYTHON_PEXPECT_SITE = https://pypi.python.org/pypi/pexpect

And the _SITE variable.

> +PYTHON_PEXPECT_SOURCE = pexpect-$(PYTHON_PEXPECT_VERSION).tar.gz
> +PYTHON_PEXPECT_LICENSE = ISC
> +PYTHON_PEXPECT_LICENSE_FILES = LICENSE
> +PYTHON_PEXPECT_SETUP_TYPE = distutils
> +PYTHON_PEXPECT_DEPENDENCIES = python-ptyprocess

And removed this dependency: it's only a run-time dependency, so it
only needs to be in the Config.in file.

Applied with those issues fixed, thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH v4 3/3] dpdk: new package

[Thomas Petazzoni]

Hello,

On Sun, 27 Mar 2016 03:31:24 +0200, Jan Viktorin wrote:

> The included hash was calculated locally by downloading the tar.gz archives by
> hand.
> 
> There are unfortunately some pitfalls:
> 
> * it may require to enable PCI, MSIX, UIO in the Linux Kernel
>   (some defconfigs does not include as default and it is platform
>   dependent as ARMv7 almost does not use PCI)

I have enabled PCI_MSI=y and UIO=y in my kernel configuration, and
still, I don't have any .ko file installed in my target. Is this
expected?

> diff --git a/package/dpdk/Config.in b/package/dpdk/Config.in
> new file mode 100644
> index 0000000..69bc56e8
> --- /dev/null
> +++ b/package/dpdk/Config.in
> @@ -0,0 +1,64 @@
> +config BR2_PACKAGE_DPDK
> +	bool "dpdk"
> +	depends on BR2_PACKAGE_DPDK_ARCH_SUPPORTS
> +	depends on BR2_TOOLCHAIN_USES_GLIBC
> +	depends on BR2_LINUX_KERNEL
> +	select BR2_LINUX_NEEDS_MODULES
> +	help
> +	  DPDK is a set of libraries and drivers for fast packet processing. It
> +	  was designed to run on any processors, however, Intel x86 has been
> +	  the first CPU to be supported. Ports for other CPUs like IBM Power 8
> +	  and ARM are under progress. It runs mostly in Linux userland. A
> +	  FreeBSD port is now available for a subset of DPDK features.
> +
> +	  Notes:
> +	  * To build the included Linux Kernel drivers, it is necessary to
> +	    enable CONFIG_PCI_MSI, CONFIG_UIO.
> +	  * To build the PCAP PMD properly, you need to enable the libpcap
> +	    manually.
> +	  * You may need to install the python2 interpreter if you want to use
> +	    scripts dpdk_nic_bind.py and cpu_layout.py
> +
> +	  http://www.dpdk.org/
> +
> +config BR2_PACKAGE_DPDK_ARCH_SUPPORTS
> +	bool
> +	depends on BR2_TOOLCHAIN_HAS_SYNC_1 &&\
> +		BR2_TOOLCHAIN_HAS_SYNC_2 &&\
> +		BR2_TOOLCHAIN_HAS_SYNC_4 &&\
> +		BR2_TOOLCHAIN_HAS_SYNC_8
> +	default y if (BR2_i386 && !BR2_x86_i386 && !BR2_x86_i486 \
> +			&& !BR2_x86_i586 && !BR2_x86_x1000)
> +	default y if BR2_ARM_CPU_ARMV7A
> +	default y if BR2_aarch64 || BR2_aarch64_be

What about BR2_x86_64 ?

> +
> +comment "dpdk needs a glibc toolchain and a Linux Kernel to be built"
> +	depends on BR2_PACKAGE_DPDK_ARCH_SUPPORTS
> +	depends on !BR2_TOOLCHAIN_USES_GLIBC || !BR2_LINUX_KERNEL

The BR2_PACKAGE_DPDK_ARCH_SUPPORTS option and the comment should go
before the BR2_PACKAGE_DPDK so that the dpdk sub-options can be
properly indented by menuconfig.

> +if BR2_PACKAGE_DPDK
> +
> +config BR2_PACKAGE_DPDK_CONFIG
> +	string "Configuration"
> +	default "i686-native-linuxapp-gcc" \
> +		if BR2_x86_i686
> +	default "x86_64-native-linuxapp-gcc" \
> +		if BR2_x86_64
> +	default "arm-armv7a-linuxapp-gcc" \
> +		if BR2_ARM_CPU_ARMV7A
> +	default "arm64-armv8a-linuxapp-gcc" \
> +		if BR2_aarch64 || BR2_aarch64_be
> +
> +config BR2_PACKAGE_DPDK_TEST
> +	bool "Install tests suite"
> +	select BR2_PACKAGE_PYTHON_PEXPECT if BR2_PACKAGE_PYTHON
> +	help
> +	  Install all DPDK tests. If you want to run the tests by the included
> +	  autotest.py script you need to enable python manually.
> +
> +config BR2_PACKAGE_DPDK_EXAMPLES
> +	bool "Build & install examples"
> +	help
> +	  Build and install all examples selected by the current configuration.

Please remove this option and the example build/installation. It is too
complicated for now. Upstream should create a proper build/installation
process for these. In the mean time, let's merge a dpdk package that
does not bother building/installing the examples.

> diff --git a/package/dpdk/dpdk.mk b/package/dpdk/dpdk.mk
> new file mode 100644
> index 0000000..6a4824a
> --- /dev/null
> +++ b/package/dpdk/dpdk.mk
> @@ -0,0 +1,106 @@
> +################################################################################
> +#
> +# dpdk
> +#
> +################################################################################
> +
> +DPDK_VERSION = 16.04-rc2

Please update to the latest 16.04 version.

> +DPDK_SITE = http://dpdk.org/browse/dpdk/snapshot
> +
> +DPDK_LICENSE = BSD-2c (core), GPLv2+ (Linux drivers)
> +DPDK_LICENSE_FILES = GNUmakefile LICENSE.GPL
> +DPDK_INSTALL_STAGING = YES
> +
> +DPDK_DEPENDENCIES += linux
> +
> +ifeq ($(BR2_PACKAGE_LIBPCAP),y)
> +DPDK_DEPENDENCIES += libpcap
> +endif
> +
> +ifeq ($(BR2_SHARED_LIBS),y)
> +define DPDK_ENABLE_SHARED_LIBS
> +	$(call KCONFIG_ENABLE_OPT,CONFIG_RTE_BUILD_SHARED_LIB,\
> +			$(@D)/build/.config)
> +endef
> +
> +DPDK_POST_CONFIGURE_HOOKS += DPDK_ENABLE_SHARED_LIBS
> +endif
> +
> +# We're building a kernel module without using the kernel-module infra,
> +# so we need to tell we want module support in the kernel
> +ifeq ($(BR2_PACKAGE_DPDK),y)
> +LINUX_NEEDS_MODULES = y
> +endif

This is no longer needed since you are selecting
BR2_LINUX_NEEDS_MODULES.

> +DPDK_CONFIG = $(call qstrip,$(BR2_PACKAGE_DPDK_CONFIG))
> +
> +ifeq ($(BR2_PACKAGE_DPDK_EXAMPLES),y)
> +# Build of DPDK examples is not very straight-forward. It requires to have
> +# the SDK and runtime installed on same place to reference it by RTE_SDK.
> +# We place it locally in the build directory.
> +define DPDK_BUILD_EXAMPLES
> +	$(MAKE) -C $(@D) DESTDIR=$(@D)/examples-sdk \
> +		CROSS=$(TARGET_CROSS) install-sdk install-runtime
> +	$(MAKE) -C $(@D) RTE_KERNELDIR=$(LINUX_DIR) CROSS=$(TARGET_CROSS) \
> +		RTE_SDK=$(@D)/examples-sdk/usr/local/share/dpdk \
> +		T=$(DPDK_CONFIG) examples
> +endef
> +
> +DPDK_EXAMPLES_PATH = $(@D)/examples-sdk/usr/local/share/dpdk/examples
> +
> +# Installation of examples is tricky in DPDK so we do it explicitly here.
> +# As the binaries and libraries do not have a single or regular location
> +# where to find them after build, we search for them by find.
> +define DPDK_INSTALL_EXAMPLES
> +	for f in `find $(DPDK_EXAMPLES_PATH) -executable -type f       \
> +			-path '*/lib/*.so*'`; do                       \
> +		$(INSTALL) -m 0755 -D $$f                              \
> +			$(TARGET_DIR)/usr/lib/`basename $$f` || exit 1;\
> +	done
> +	for f in `find $(DPDK_EXAMPLES_PATH) -executable -type f       \
> +			-path '*/app/*'`; do                           \
> +		$(INSTALL) -m 0755 -D $$f                              \
> +			$(TARGET_DIR)/usr/bin/`basename $$f` || exit 1;\
> +	done
> +endef
> +
> +# Build of the power example is broken (at least for 16.04).
> +define DPDK_DISABLE_POWER
> +	$(call KCONFIG_DISABLE_OPT,CONFIG_RTE_LIBRTE_POWER,\
> +			$(@D)/build/.config)
> +endef
> +
> +DPDK_POST_CONFIGURE_HOOKS += DPDK_DISABLE_POWER
> +endif

Please remove this entire ifeq ... endif block.

> +
> +define DPDK_CONFIGURE_CMDS
> +	$(MAKE) -C $(@D) T=$(DPDK_CONFIG) RTE_KERNELDIR=$(LINUX_DIR) \
> +			   CROSS=$(TARGET_CROSS) config
> +endef
> +
> +define DPDK_BUILD_CMDS
> +	$(MAKE) -C $(@D) RTE_KERNELDIR=$(LINUX_DIR) CROSS=$(TARGET_CROSS)

When the architecture is ARMv7, please pass:

	CPU_FLAGS=

Otherwise, the dpdk build system passes -mfloat-abi=softfp, which makes
the build fail when the selected ABI is EABIhf. And please report the
bug upstream: the dpdk build system should not make assumptions on the
selected ARM ABI.

> +	$(DPDK_BUILD_EXAMPLES)

This can be removed.

> +endef
> +
> +define DPDK_INSTALL_STAGING_CMDS
> +	$(MAKE) -C $(@D) DESTDIR=$(STAGING_DIR) prefix=/usr \
> +		 CROSS=$(TARGET_CROSS) install-sdk
> +endef
> +
> +ifeq ($(BR2_PACKAGE_DPDK_TEST),y)
> +define DPDK_INSTALL_TARGET_TEST
> +	$(INSTALL) -m 0755 -d $(TARGET_DIR)/usr/dpdk

	mkdir -p

> +	$(INSTALL) -m 0755 -D $(@D)/build/app/test $(TARGET_DIR)/usr/dpdk

Full destination path:

	$(INSTALL) -m 0755 -D $(@D)/build/app/test $(TARGET_DIR)/usr/dpdk/test

> +	$(INSTALL) -m 0755 -D $(@D)/app/test/*.py $(TARGET_DIR)/usr/dpdk

To copy multiple files:

	cp -dpfr $(@D)/app/test/*.py $(TARGET_DIR)/usr/dpdk

> +define DPDK_INSTALL_TARGET_CMDS
> +	$(MAKE) -C $(@D) DESTDIR=$(TARGET_DIR) prefix=/usr \
> +		CROSS=$(TARGET_CROSS) install-runtime
> +	$(DPDK_INSTALL_TARGET_TEST)
> +	$(DPDK_INSTALL_EXAMPLES)

Remove this one.

Can you fix those issues and resend?

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH v4 3/3] dpdk: new package

[Jan Viktorin]

On Fri, 15 Apr 2016 23:52:03 +0200
Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote:

> Hello,

Dear Thomas,

thanks a lot for your help! I've fixed those...

> > 
> > * it may require to enable PCI, MSIX, UIO in the Linux Kernel
> >   (some defconfigs does not include as default and it is platform
> >   dependent as ARMv7 almost does not use PCI)  
> 
> I have enabled PCI_MSI=y and UIO=y in my kernel configuration, and
> still, I don't have any .ko file installed in my target. Is this
> expected?

Of course, you are right. They are built but not installed. Fixed.

> 
[...]
> > +
> > +define DPDK_BUILD_CMDS
> > +	$(MAKE) -C $(@D) RTE_KERNELDIR=$(LINUX_DIR) CROSS=$(TARGET_CROSS)  
> 
> When the architecture is ARMv7, please pass:
> 
> 	CPU_FLAGS=
> 
> Otherwise, the dpdk build system passes -mfloat-abi=softfp, which makes
> the build fail when the selected ABI is EABIhf. And please report the
> bug upstream: the dpdk build system should not make assumptions on the
> selected ARM ABI.

Well, this leads to some hardcoded check of architecture into the
dpdk.mk. I don't like such approach. Would you instead accept the
following (temporary) patch into Buildroot as a solution for this?

http://dpdk.org/ml/archives/dev/2016-April/037577.html

I will remove it with the next DPDK release (when it's in the
mainline). I've tested and it builds successfully for both ABI and
EABIhf.

> 
[...]
> 
> Can you fix those issues and resend?

Sure, I will resend after I solve the ARM ABI issue.

> 
> Thanks!
> 
> Thomas

Regards
Jan

-- 
  Jan Viktorin                E-mail: Viktorin at RehiveTech.com
  System Architect            Web:    www.RehiveTech.com
  RehiveTech
  Brno, Czech Republic

[Buildroot] [PATCH v4 3/3] dpdk: new package

[Thomas Petazzoni]

Hello,

On Sat, 16 Apr 2016 02:07:08 +0200, Jan Viktorin wrote:

> > I have enabled PCI_MSI=y and UIO=y in my kernel configuration, and
> > still, I don't have any .ko file installed in my target. Is this
> > expected?
> 
> Of course, you are right. They are built but not installed. Fixed.

Good, thanks!

> > When the architecture is ARMv7, please pass:
> > 
> > 	CPU_FLAGS=
> > 
> > Otherwise, the dpdk build system passes -mfloat-abi=softfp, which makes
> > the build fail when the selected ABI is EABIhf. And please report the
> > bug upstream: the dpdk build system should not make assumptions on the
> > selected ARM ABI.
> 
> Well, this leads to some hardcoded check of architecture into the
> dpdk.mk. I don't like such approach.

Well, I don't see what's the problem with that really, but...

> Would you instead accept the
> following (temporary) patch into Buildroot as a solution for this?
> 
> http://dpdk.org/ml/archives/dev/2016-April/037577.html

... if such a patch gets accepted upstream, then I'm fine with carrying
the patch in Buildroot in the mean time.

> > Can you fix those issues and resend?
> 
> Sure, I will resend after I solve the ARM ABI issue.

Well, you solved it it seems, right? :-)

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH v5] dpdk: new package

[Jan Viktorin]

This patch introduces support of the DPDK library (www.dpdk.org) into
Buildroot. DPDK is a library for high-speed packet sending/receiving
while bypassing the Linux Kernel. It allows to reach a high throughput
for 10-100 Gbps networks on the x86 platform.

The package compiles and installs DPDK libraries on the target and
staging and allows to compile other applications depending on the DPDK
library. It can also install some basic tools the DPDK provides
(testpmd, python scripts, test suite).

The patch assumes DPDK 16.04. This version contains support for the ARM
architecture. The ARM ports can be tested by

 qemu_aarch64_virt_defconfig
 qemu_arm_vexpress_defconfig

The included hash was calculated locally by downloading the tar.gz archives by
hand.

There are unfortunately some pitfalls:

* it may require to enable PCI, MSIX, UIO in the Linux Kernel
  (some defconfigs does not include as default and it is platform
  dependent as ARMv7 almost does not use PCI)

* when building PCAP PMD driver, the libpcap is required (partially
  fixed as suggested by Arnout)

* some tools the DPDK provides depend on Python(2) so the user has
  to enable it to install those

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v2: (mostly suggestions by Arnout)
* simplified Config.in - avoid version, source and config selection
* improved dependency on libpcap
* user python scripts are included if python package is enabled
* installing of tests includes autotest*.py scripts (we do not care
  about the python here, assuming the user will install it manually
  when testing)
* minor coding style fixes
* depends on python-pexpect package
* using version 2.2.0-rc3
v3
* bump to version 16.04-rc1
* utilizing the new install syntax of DPDK
* fixed line wrapping
* testpmd is now always installed (no option for this)
* fixed python dependencies (T. Petazzoni)
* supporting i686+ architectures (T. Petazzoni)
* dropped PPC support for now
* utilizing KCONFIG_ENABLE_OPT macro
* support for build & install of examples (ugly code)
* build with MAKE instead of MAKE1 seems OK now
* dropped COMBINE_LIBS (no more in DPDK)
* added HAS_SYNC_x
* few random fixes
v4 (suggestions by T. Petazzoni)
* bump to version 16.04-rc2
* fixed indentation in Config.in
* fixed depends in Config.in
* handling kernel modules by BR2_LINUX_NEEDS_MODULES
* simplified installation of examples
v5
* bump to version 16.04
* dropped (forgotten) LINUX_NEEDS_MODULES=y from mk
* reordered Config.in (T. Petazzoni)
* added forgotten BR2_x86_64 (T. Petazzoni)
* dropped support for examples (T. Petazzoni)
* fix install of kernel modules (T. Petazzoni)
* patch ARM ABI enforcing (T. Petazzoni)
* minor install fixes (T. Petazzoni)
---
 package/Config.in                                  |  1 +
 ...01-mk-do-not-enforce-any-specific-ARM-ABI.patch | 31 +++++++++++
 package/dpdk/Config.in                             | 60 ++++++++++++++++++++++
 package/dpdk/dpdk.hash                             |  2 +
 package/dpdk/dpdk.mk                               | 60 ++++++++++++++++++++++
 5 files changed, 154 insertions(+)
 create mode 100644 package/dpdk/0001-mk-do-not-enforce-any-specific-ARM-ABI.patch
 create mode 100644 package/dpdk/Config.in
 create mode 100644 package/dpdk/dpdk.hash
 create mode 100644 package/dpdk/dpdk.mk

diff --git a/package/Config.in b/package/Config.in
index bcfe13b..022d02f 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -1102,6 +1102,7 @@ menu "Networking"
 	source "package/cgic/Config.in"
 	source "package/cppzmq/Config.in"
 	source "package/czmq/Config.in"
+	source "package/dpdk/Config.in"
 	source "package/filemq/Config.in"
 	source "package/flickcurl/Config.in"
 	source "package/fmlib/Config.in"
diff --git a/package/dpdk/0001-mk-do-not-enforce-any-specific-ARM-ABI.patch b/package/dpdk/0001-mk-do-not-enforce-any-specific-ARM-ABI.patch
new file mode 100644
index 0000000..9ac18a5
--- /dev/null
+++ b/package/dpdk/0001-mk-do-not-enforce-any-specific-ARM-ABI.patch
@@ -0,0 +1,31 @@
+From 524dc6e45487737f1465e7edade840613f083c8f Mon Sep 17 00:00:00 2001
+From: Jan Viktorin <viktorin@rehivetech.com>
+Date: Sat, 16 Apr 2016 00:11:18 +0200
+Subject: [PATCH] mk: do not enforce any specific ARM ABI
+
+The dpdk build system passes -mfloat-abi=softfp, which makes the build fail
+when the selected ABI is EABIhf. The dpdk build system should not make
+assumptions on the selected ARM ABI.
+
+Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
+Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
+---
+ mk/machine/armv7a/rte.vars.mk | 2 --
+ 1 file changed, 2 deletions(-)
+
+diff --git a/mk/machine/armv7a/rte.vars.mk b/mk/machine/armv7a/rte.vars.mk
+index abdb15e..36fa3de 100644
+--- a/mk/machine/armv7a/rte.vars.mk
++++ b/mk/machine/armv7a/rte.vars.mk
+@@ -54,8 +54,6 @@
+ # CPU_LDFLAGS =
+ # CPU_ASFLAGS =
+ 
+-CPU_CFLAGS += -mfloat-abi=softfp
+-
+ MACHINE_CFLAGS += -march=armv7-a
+ 
+ ifdef CONFIG_RTE_ARCH_ARM_TUNE
+-- 
+2.8.0
+
diff --git a/package/dpdk/Config.in b/package/dpdk/Config.in
new file mode 100644
index 0000000..27bf106
--- /dev/null
+++ b/package/dpdk/Config.in
@@ -0,0 +1,60 @@
+config BR2_PACKAGE_DPDK_ARCH_SUPPORTS
+	bool
+	depends on BR2_TOOLCHAIN_HAS_SYNC_1 &&\
+		BR2_TOOLCHAIN_HAS_SYNC_2 &&\
+		BR2_TOOLCHAIN_HAS_SYNC_4 &&\
+		BR2_TOOLCHAIN_HAS_SYNC_8
+	default y if (BR2_i386 && !BR2_x86_i386 && !BR2_x86_i486 \
+			&& !BR2_x86_i586 && !BR2_x86_x1000)
+	default y if BR2_x86_64
+	default y if BR2_ARM_CPU_ARMV7A
+	default y if BR2_aarch64 || BR2_aarch64_be
+
+comment "dpdk needs a glibc toolchain and a Linux Kernel to be built"
+	depends on BR2_PACKAGE_DPDK_ARCH_SUPPORTS
+	depends on !BR2_TOOLCHAIN_USES_GLIBC || !BR2_LINUX_KERNEL
+
+config BR2_PACKAGE_DPDK
+	bool "dpdk"
+	depends on BR2_PACKAGE_DPDK_ARCH_SUPPORTS
+	depends on BR2_TOOLCHAIN_USES_GLIBC
+	depends on BR2_LINUX_KERNEL
+	select BR2_LINUX_NEEDS_MODULES
+	help
+	  DPDK is a set of libraries and drivers for fast packet processing. It
+	  was designed to run on any processors, however, Intel x86 has been
+	  the first CPU to be supported. Ports for other CPUs like IBM Power 8
+	  and ARM are under progress. It runs mostly in Linux userland. A
+	  FreeBSD port is now available for a subset of DPDK features.
+
+	  Notes:
+	  * To build the included Linux Kernel drivers, it is necessary to
+	    enable CONFIG_PCI_MSI, CONFIG_UIO.
+	  * To build the PCAP PMD properly, you need to enable the libpcap
+	    manually.
+	  * You may need to install the python2 interpreter if you want to use
+	    scripts dpdk_nic_bind.py and cpu_layout.py
+
+	  http://www.dpdk.org/
+
+if BR2_PACKAGE_DPDK
+
+config BR2_PACKAGE_DPDK_CONFIG
+	string "Configuration"
+	default "i686-native-linuxapp-gcc" \
+		if BR2_x86_i686
+	default "x86_64-native-linuxapp-gcc" \
+		if BR2_x86_64
+	default "arm-armv7a-linuxapp-gcc" \
+		if BR2_ARM_CPU_ARMV7A
+	default "arm64-armv8a-linuxapp-gcc" \
+		if BR2_aarch64 || BR2_aarch64_be
+
+config BR2_PACKAGE_DPDK_TEST
+	bool "Install tests suite"
+	select BR2_PACKAGE_PYTHON_PEXPECT if BR2_PACKAGE_PYTHON
+	help
+	  Install all DPDK tests. If you want to run the tests by the included
+	  autotest.py script you need to enable python manually.
+
+endif
diff --git a/package/dpdk/dpdk.hash b/package/dpdk/dpdk.hash
new file mode 100644
index 0000000..3780c66
--- /dev/null
+++ b/package/dpdk/dpdk.hash
@@ -0,0 +1,2 @@
+# Locally calculated
+sha256	d631495bc6e8d4c4aec72999ac03c3ce213bb996cb88f3bf14bb980dad1d3f7b  dpdk-16.04.tar.gz
diff --git a/package/dpdk/dpdk.mk b/package/dpdk/dpdk.mk
new file mode 100644
index 0000000..ea53417
--- /dev/null
+++ b/package/dpdk/dpdk.mk
@@ -0,0 +1,60 @@
+################################################################################
+#
+# dpdk
+#
+################################################################################
+
+DPDK_VERSION = 16.04
+DPDK_SITE = http://dpdk.org/browse/dpdk/snapshot
+
+DPDK_LICENSE = BSD-2c (core), GPLv2+ (Linux drivers)
+DPDK_LICENSE_FILES = GNUmakefile LICENSE.GPL
+DPDK_INSTALL_STAGING = YES
+
+DPDK_DEPENDENCIES += linux
+
+ifeq ($(BR2_PACKAGE_LIBPCAP),y)
+DPDK_DEPENDENCIES += libpcap
+endif
+
+ifeq ($(BR2_SHARED_LIBS),y)
+define DPDK_ENABLE_SHARED_LIBS
+	$(call KCONFIG_ENABLE_OPT,CONFIG_RTE_BUILD_SHARED_LIB,\
+			$(@D)/build/.config)
+endef
+
+DPDK_POST_CONFIGURE_HOOKS += DPDK_ENABLE_SHARED_LIBS
+endif
+
+DPDK_CONFIG = $(call qstrip,$(BR2_PACKAGE_DPDK_CONFIG))
+
+define DPDK_CONFIGURE_CMDS
+	$(MAKE) -C $(@D) T=$(DPDK_CONFIG) RTE_KERNELDIR=$(LINUX_DIR) \
+			   CROSS=$(TARGET_CROSS) config
+endef
+
+define DPDK_BUILD_CMDS
+	$(MAKE) -C $(@D) RTE_KERNELDIR=$(LINUX_DIR) CROSS=$(TARGET_CROSS)
+endef
+
+define DPDK_INSTALL_STAGING_CMDS
+	$(MAKE) -C $(@D) DESTDIR=$(STAGING_DIR) prefix=/usr \
+		 CROSS=$(TARGET_CROSS) install-sdk
+endef
+
+ifeq ($(BR2_PACKAGE_DPDK_TEST),y)
+define DPDK_INSTALL_TARGET_TEST
+	mkdir -p $(TARGET_DIR)/usr/dpdk
+	$(INSTALL) -m 0755 -D $(@D)/build/app/test $(TARGET_DIR)/usr/dpdk/test
+	cp -dpfr $(@D)/app/test/*.py $(TARGET_DIR)/usr/dpdk
+endef
+endif
+
+define DPDK_INSTALL_TARGET_CMDS
+	$(MAKE) -C $(@D) DESTDIR=$(TARGET_DIR) CROSS=$(TARGET_CROSS) \
+		kerneldir=/lib/modules/$(LINUX_VERSION_PROBED)/extra/dpdk \
+		prefix=/usr install-runtime install-kmod
+	$(DPDK_INSTALL_TARGET_TEST)
+endef
+
+$(eval $(generic-package))
-- 
2.8.0

[Buildroot] [PATCH v5] dpdk: new package

[Thomas Petazzoni]

Hello,

On Sat, 16 Apr 2016 19:08:10 +0200, Jan Viktorin wrote:
> This patch introduces support of the DPDK library (www.dpdk.org) into
> Buildroot. DPDK is a library for high-speed packet sending/receiving
> while bypassing the Linux Kernel. It allows to reach a high throughput
> for 10-100 Gbps networks on the x86 platform.
> 
> The package compiles and installs DPDK libraries on the target and
> staging and allows to compile other applications depending on the DPDK
> library. It can also install some basic tools the DPDK provides
> (testpmd, python scripts, test suite).
> 
> The patch assumes DPDK 16.04. This version contains support for the ARM
> architecture. The ARM ports can be tested by
> 
>  qemu_aarch64_virt_defconfig
>  qemu_arm_vexpress_defconfig
> 
> The included hash was calculated locally by downloading the tar.gz archives by
> hand.
> 
> There are unfortunately some pitfalls:
> 
> * it may require to enable PCI, MSIX, UIO in the Linux Kernel
>   (some defconfigs does not include as default and it is platform
>   dependent as ARMv7 almost does not use PCI)
> 
> * when building PCAP PMD driver, the libpcap is required (partially
>   fixed as suggested by Arnout)
> 
> * some tools the DPDK provides depend on Python(2) so the user has
>   to enable it to install those
> 
> Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>

Thanks for this new iteration. Unfortunately, it still doesn't install
any kernel module to my system. At install time, your addition of the
install-kmod target simply does:

make[3]: Nothing to be done for 'install-kmod'.

Moreover, are you sure having a Linux kernel already built is always
necessary? My understanding is that while DPDK has its own kernel
modules, it can also rely on the kernel generic uio_pci_generic module.
In this case, DPDK only needs to build userspace components, and no
kernel component, which would make the dependency on Linux optional.

From the DPDK documentation: "To run any DPDK application, a suitable
uio module can be loaded into the running kernel. In many cases, the
standard uio_pci_generic module included in the Linux kernel can
provide the uio capability.".

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH v5] dpdk: new package

[Jan Viktorin]

Hello Thomas,

what configuration do you use?

In general, I cannot say? we are always fine with the in-kernel modules. There are configurations of DPDK that build certain drivers. Of course, DPDK community would like to have no out-of-tree drivers. Some devices require a certain driver.

The x86 configuration builds drivers. The armv7 one doesn't (there are no useful drivers for this purpose).

A user may serve its own configuration of DPDK where a kernel driver is selected and thus, the user would expect it to be built and installed.

It seems that in the future the in-kernel vfio is the way to? go.

Jan?Viktorin
RehiveTech
Sent?from?a?mobile?device
? P?vodn? zpr?va ?
Od: Thomas Petazzoni
Odesl?no: ned?le, 17. dubna 2016 16:38
Komu: Jan Viktorin
Kopie: buildroot at buildroot.org
P?edm?t: Re: [Buildroot] [PATCH v5] dpdk: new package

Hello,

On Sat, 16 Apr 2016 19:08:10 +0200, Jan Viktorin wrote:
> This patch introduces support of the DPDK library (www.dpdk.org) into
> Buildroot. DPDK is a library for high-speed packet sending/receiving
> while bypassing the Linux Kernel. It allows to reach a high throughput
> for 10-100 Gbps networks on the x86 platform.
> 
> The package compiles and installs DPDK libraries on the target and
> staging and allows to compile other applications depending on the DPDK
> library. It can also install some basic tools the DPDK provides
> (testpmd, python scripts, test suite).
> 
> The patch assumes DPDK 16.04. This version contains support for the ARM
> architecture. The ARM ports can be tested by
> 
> qemu_aarch64_virt_defconfig
> qemu_arm_vexpress_defconfig
> 
> The included hash was calculated locally by downloading the tar.gz archives by
> hand.
> 
> There are unfortunately some pitfalls:
> 
> * it may require to enable PCI, MSIX, UIO in the Linux Kernel
> (some defconfigs does not include as default and it is platform
> dependent as ARMv7 almost does not use PCI)
> 
> * when building PCAP PMD driver, the libpcap is required (partially
> fixed as suggested by Arnout)
> 
> * some tools the DPDK provides depend on Python(2) so the user has
> to enable it to install those
> 
> Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>

Thanks for this new iteration. Unfortunately, it still doesn't install
any kernel module to my system. At install time, your addition of the
install-kmod target simply does:

make[3]: Nothing to be done for 'install-kmod'.

Moreover, are you sure having a Linux kernel already built is always
necessary? My understanding is that while DPDK has its own kernel
modules, it can also rely on the kernel generic uio_pci_generic module.
In this case, DPDK only needs to build userspace components, and no
kernel component, which would make the dependency on Linux optional.

From the DPDK documentation: "To run any DPDK application, a suitable
uio module can be loaded into the running kernel. In many cases, the
standard uio_pci_generic module included in the Linux kernel can
provide the uio capability.".

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH v5] dpdk: new package

[Thomas Petazzoni]

Hello,

On Sun, 17 Apr 2016 17:56:07 +0200, Jan Viktorin wrote:

> what configuration do you use?

BR2_arm=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arm-full-2016.02-3-g762b7c9.tar.bz2"
BR2_TOOLCHAIN_EXTERNAL_GCC_4_7=y
BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_10=y
BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
BR2_TOOLCHAIN_EXTERNAL_INET_RPC=y
BR2_TOOLCHAIN_EXTERNAL_CXX=y
BR2_INIT_NONE=y
BR2_SYSTEM_BIN_SH_NONE=y
BR2_LINUX_KERNEL=y
BR2_LINUX_KERNEL_CUSTOM_VERSION=y
BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.5"
BR2_LINUX_KERNEL_DEFCONFIG="mvebu_v7"
# BR2_PACKAGE_BUSYBOX is not set
BR2_PACKAGE_DPDK=y
# BR2_TARGET_ROOTFS_TAR is not set

Before starting the build, I do "make linux-menuconfig", and enable
CONFIG_UIO in the kernel configuration.

> In general, I cannot say? we are always fine with the in-kernel
> modules. There are configurations of DPDK that build certain drivers.
> Of course, DPDK community would like to have no out-of-tree drivers.
> Some devices require a certain driver.
> 
> The x86 configuration builds drivers. The armv7 one doesn't (there
> are no useful drivers for this purpose).

Then IMO the dependency on the Linux kernel should be optional.

> A user may serve its own configuration of DPDK where a kernel driver
> is selected and thus, the user would expect it to be built and
> installed.

Right. But if the default configuration doesn't require the kernel to
be built, we probably shouldn't require it as well.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH v5] dpdk: new package

[Jan Viktorin]

Well, your config is for ARMv7 so no kmods are built because all current drivers require PCI which is hardly to see on this platform (I know just about Armada).

Ok. I don't know what would be the use case for dropping dependency on the kernel. Building just the rootfs without the kernel?

That means to check whether the kernel is enabled. And if it is then to enable building of the DPDK kernel modules. Right?

This can be done but I am afraid I have to disable certain modules by name from the current configuration. This is not very generic (consider adding a new out-of-tree kernel module or removing an obsolete one from the DPDK, custom DPDK tree with third party modules) but there are only few of them so it might be a working solution. I will check if there is a more generic way but I don't think so.
?
Jan?Viktorin
RehiveTech?
Sent?from?a?mobile?device
? P?vodn? zpr?va ?
Od: Thomas Petazzoni
Odesl?no: ned?le, 17. dubna 2016 21:35
Komu: Jan Viktorin
Kopie: buildroot at buildroot.org
P?edm?t: Re: [Buildroot] [PATCH v5] dpdk: new package

Hello,

On Sun, 17 Apr 2016 17:56:07 +0200, Jan Viktorin wrote:

> what configuration do you use?

BR2_arm=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arm-full-2016.02-3-g762b7c9.tar.bz2"
BR2_TOOLCHAIN_EXTERNAL_GCC_4_7=y
BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_10=y
BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
BR2_TOOLCHAIN_EXTERNAL_INET_RPC=y
BR2_TOOLCHAIN_EXTERNAL_CXX=y
BR2_INIT_NONE=y
BR2_SYSTEM_BIN_SH_NONE=y
BR2_LINUX_KERNEL=y
BR2_LINUX_KERNEL_CUSTOM_VERSION=y
BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.5"
BR2_LINUX_KERNEL_DEFCONFIG="mvebu_v7"
# BR2_PACKAGE_BUSYBOX is not set
BR2_PACKAGE_DPDK=y
# BR2_TARGET_ROOTFS_TAR is not set

Before starting the build, I do "make linux-menuconfig", and enable
CONFIG_UIO in the kernel configuration.

> In general, I cannot say? we are always fine with the in-kernel
> modules. There are configurations of DPDK that build certain drivers.
> Of course, DPDK community would like to have no out-of-tree drivers.
> Some devices require a certain driver.
> 
> The x86 configuration builds drivers. The armv7 one doesn't (there
> are no useful drivers for this purpose).

Then IMO the dependency on the Linux kernel should be optional.

> A user may serve its own configuration of DPDK where a kernel driver
> is selected and thus, the user would expect it to be built and
> installed.

Right. But if the default configuration doesn't require the kernel to
be built, we probably shouldn't require it as well.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH v5] dpdk: new package

[Thomas Petazzoni]

Hello,

On Sun, 17 Apr 2016 22:56:07 +0200, Jan Viktorin wrote:

> Well, your config is for ARMv7 so no kmods are built because all
> current drivers require PCI which is hardly to see on this platform
> (I know just about Armada).

Indeed mvebu_v7 is for Marvell Armada. Is DPDK only for PCI NICs? Can't
it be used for the internal NICs of certain ARM SoCs ?

> Ok. I don't know what would be the use case for dropping dependency
> on the kernel. Building just the rootfs without the kernel?

For example, yes. Another benefit of not having the mandatory kernel
dependency is that the autobuilders would be able to test the DPDK
package.

> That means to check whether the kernel is enabled. And if it is then
> to enable building of the DPDK kernel modules. Right?

Correct.

> This can be done but I am afraid I have to disable certain modules by
> name from the current configuration. This is not very generic
> (consider adding a new out-of-tree kernel module or removing an
> obsolete one from the DPDK, custom DPDK tree with third party
> modules) but there are only few of them so it might be a working
> solution. I will check if there is a more generic way but I don't
> think so.

I'm not sure what you mean here. If the default configuration on a
given architecture builds kernel modules, then on this architecture,
DPDK should depend on the kernel to be built. If the default
configuration on another architecture does not build kernel modules,
then there is no need to depend on the kernel.

That being said, if only ARM does not build any kernel module, maybe we
can simply do as you do right now, and depend on the kernel
unconditionally, to keep things simple.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH v5] dpdk: new package

[Jan Viktorin]

On Sun, 17 Apr 2016 23:06:07 +0200
Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote:

> Hello,
> 
> On Sun, 17 Apr 2016 22:56:07 +0200, Jan Viktorin wrote:
> 
> > Well, your config is for ARMv7 so no kmods are built because all
> > current drivers require PCI which is hardly to see on this platform
> > (I know just about Armada).  
> 
> Indeed mvebu_v7 is for Marvell Armada. Is DPDK only for PCI NICs? Can't
> it be used for the internal NICs of certain ARM SoCs ?

Yes. DPDK up to 16.04 supports only PCI devices. We are going to change
this for 16.07 (the next release).

> 
> > Ok. I don't know what would be the use case for dropping dependency
> > on the kernel. Building just the rootfs without the kernel?  
> 
> For example, yes. Another benefit of not having the mandatory kernel
> dependency is that the autobuilders would be able to test the DPDK
> package.

I have no clue about this. Does kernel dependency prevent some
auto-testing?

> 
> > That means to check whether the kernel is enabled. And if it is then
> > to enable building of the DPDK kernel modules. Right?  
> 
> Correct.

I would implement this in Buildroot by a list of configuration keys
naming certain drivers. If the kernel is disabled, I'd disable all of
them during configure phase. Otherwise, I'd leave them untouched (to
have the current configuration as is). So, I'd put in dpdk.mk
something like:

ifeq ($(BR2_LINUX_KERNEL),y)
DPDK_DEPENDENCIES += linux
else
DPDK_KMODS = EAL_IGB_UIO KNI_KMOD LIBRTE_XEN_DOM0

define DPDK_DISABLE_KMODS
$(foreach m,$(DPDK_KMODS),\
	$(call KCONFIG_DISABLE_OPT,CONFIG_RTE_$(m),$(@D)/build/.config))
endef

DPDK_POST_CONFIGURE_HOOKS += DPDK_DISABLE_KMODS
endif

Probably, I can improve it by testing whether a module is present
in the DPDK config and then setting the dependency on linux.

Anyway, it seems to be quite complex...

> 
> > This can be done but I am afraid I have to disable certain modules by
> > name from the current configuration. This is not very generic
> > (consider adding a new out-of-tree kernel module or removing an
> > obsolete one from the DPDK, custom DPDK tree with third party
> > modules) but there are only few of them so it might be a working
> > solution. I will check if there is a more generic way but I don't
> > think so.  
> 
> I'm not sure what you mean here. If the default configuration on a
> given architecture builds kernel modules, then on this architecture,
> DPDK should depend on the kernel to be built. If the default
> configuration on another architecture does not build kernel modules,
> then there is no need to depend on the kernel.
> 
> That being said, if only ARM does not build any kernel module, maybe we
> can simply do as you do right now, and depend on the kernel
> unconditionally, to keep things simple.

Yes, this would work for the existing defconfigs. What about a custom
one? E.g. I want some DPDK kernel modules for the Armada with a
PCI-E NIC. So, I need a custom DPDK configuration at the moment.
Fortunately, this is a rare case.

There is also a Xen module in DPDK that is not built by default so you
have to create a custom configuration (or alter an existing one) to
enable it.

I understand that you'd like to keep it as simple as possible. I'd like
to keep it simple too but also providing enough flexibility for easy
customizations. That's why I don't consider the ARMv7 configuration to
be "without kernel modules" as it is possible to enable some if needed.

Regards
Jan

> 
> Thanks!
> 
> Thomas



-- 
  Jan Viktorin                E-mail: Viktorin at RehiveTech.com
  System Architect            Web:    www.RehiveTech.com
  RehiveTech
  Brno, Czech Republic

[Buildroot] [PATCH v5] dpdk: new package

[Arnout Vandecappelle]

On 04/18/16 10:23, Jan Viktorin wrote:
> On Sun, 17 Apr 2016 23:06:07 +0200
> Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote:
>
>> Hello,
>>
>> On Sun, 17 Apr 2016 22:56:07 +0200, Jan Viktorin wrote:
>>
[snip]
>>> Ok. I don't know what would be the use case for dropping dependency
>>> on the kernel. Building just the rootfs without the kernel?
>>
>> For example, yes. Another benefit of not having the mandatory kernel
>> dependency is that the autobuilders would be able to test the DPDK
>> package.
>
> I have no clue about this. Does kernel dependency prevent some
> auto-testing?

  The base configs used in the autobuilders don't include a kernel (or 
bootloader or rootfs). Therefore, any package depending on the kernel is never 
built.

  We _could_ actually add a couple of base configs that do include a kernel. But 
someone has to do it :-)

>
>>
>>> That means to check whether the kernel is enabled. And if it is then
>>> to enable building of the DPDK kernel modules. Right?
>>
>> Correct.
>
> I would implement this in Buildroot by a list of configuration keys
> naming certain drivers. If the kernel is disabled, I'd disable all of
> them during configure phase. Otherwise, I'd leave them untouched (to
> have the current configuration as is). So, I'd put in dpdk.mk
> something like:
>
> ifeq ($(BR2_LINUX_KERNEL),y)
> DPDK_DEPENDENCIES += linux
> else
> DPDK_KMODS = EAL_IGB_UIO KNI_KMOD LIBRTE_XEN_DOM0
>
> define DPDK_DISABLE_KMODS
> $(foreach m,$(DPDK_KMODS),\

  Small nit: indent with tab here.

> 	$(call KCONFIG_DISABLE_OPT,CONFIG_RTE_$(m),$(@D)/build/.config))
> endef
>
> DPDK_POST_CONFIGURE_HOOKS += DPDK_DISABLE_KMODS

  Is dpdk using Kconfig? In that case, please use the kconfig-package infra. It 
gives you simple primitives for disabling and enabling options. Consult the manual.

> endif
>
> Probably, I can improve it by testing whether a module is present
> in the DPDK config and then setting the dependency on linux.

  That's not possible: DEPENDENCIES must be declared when the makefiles are 
parsed, but the tarball containing .config is only extracted afterwards.


  Regards,
  Arnout

>
> Anyway, it seems to be quite complex...
>
>>
>>> This can be done but I am afraid I have to disable certain modules by
>>> name from the current configuration. This is not very generic
>>> (consider adding a new out-of-tree kernel module or removing an
>>> obsolete one from the DPDK, custom DPDK tree with third party
>>> modules) but there are only few of them so it might be a working
>>> solution. I will check if there is a more generic way but I don't
>>> think so.
>>
>> I'm not sure what you mean here. If the default configuration on a
>> given architecture builds kernel modules, then on this architecture,
>> DPDK should depend on the kernel to be built. If the default
>> configuration on another architecture does not build kernel modules,
>> then there is no need to depend on the kernel.
>>
>> That being said, if only ARM does not build any kernel module, maybe we
>> can simply do as you do right now, and depend on the kernel
>> unconditionally, to keep things simple.
>
> Yes, this would work for the existing defconfigs. What about a custom
> one? E.g. I want some DPDK kernel modules for the Armada with a
> PCI-E NIC. So, I need a custom DPDK configuration at the moment.
> Fortunately, this is a rare case.
>
> There is also a Xen module in DPDK that is not built by default so you
> have to create a custom configuration (or alter an existing one) to
> enable it.
>
> I understand that you'd like to keep it as simple as possible. I'd like
> to keep it simple too but also providing enough flexibility for easy
> customizations. That's why I don't consider the ARMv7 configuration to
> be "without kernel modules" as it is possible to enable some if needed.



-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH v5] dpdk: new package

[Jan Viktorin]

Hello Arnout,

thanks for your comments...

On Tue, 19 Apr 2016 00:50:27 +0200
Arnout Vandecappelle <arnout@mind.be> wrote:

> On 04/18/16 10:23, Jan Viktorin wrote:
> > On Sun, 17 Apr 2016 23:06:07 +0200
> > Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote:
> >  
> >> Hello,
> >>
> >> On Sun, 17 Apr 2016 22:56:07 +0200, Jan Viktorin wrote:
> >>  
> [snip]
> >>> Ok. I don't know what would be the use case for dropping dependency
> >>> on the kernel. Building just the rootfs without the kernel?  
> >>
> >> For example, yes. Another benefit of not having the mandatory kernel
> >> dependency is that the autobuilders would be able to test the DPDK
> >> package.  
> >
> > I have no clue about this. Does kernel dependency prevent some
> > auto-testing?  
> 
>   The base configs used in the autobuilders don't include a kernel (or 
> bootloader or rootfs). Therefore, any package depending on the kernel is never 
> built.
> 
>   We _could_ actually add a couple of base configs that do include a kernel. But 
> someone has to do it :-)

I have no idea how to access the autobuilder nor what kind of complexity
is connected with this...

> 
> >  
> >>  
> >>> That means to check whether the kernel is enabled. And if it is then
> >>> to enable building of the DPDK kernel modules. Right?  
> >>
> >> Correct.  
> >
> > I would implement this in Buildroot by a list of configuration keys
> > naming certain drivers. If the kernel is disabled, I'd disable all of
> > them during configure phase. Otherwise, I'd leave them untouched (to
> > have the current configuration as is). So, I'd put in dpdk.mk
> > something like:
> >
> > ifeq ($(BR2_LINUX_KERNEL),y)
> > DPDK_DEPENDENCIES += linux
> > else
> > DPDK_KMODS = EAL_IGB_UIO KNI_KMOD LIBRTE_XEN_DOM0
> >
> > define DPDK_DISABLE_KMODS
> > $(foreach m,$(DPDK_KMODS),\  
> 
>   Small nit: indent with tab here.

OK, it was just an ad hoc example. I'd rather know whether I can implement it this way.

> 
> > 	$(call KCONFIG_DISABLE_OPT,CONFIG_RTE_$(m),$(@D)/build/.config))
> > endef
> >
> > DPDK_POST_CONFIGURE_HOOKS += DPDK_DISABLE_KMODS  
> 
>   Is dpdk using Kconfig? In that case, please use the kconfig-package infra. It 
> gives you simple primitives for disabling and enabling options. Consult the manual.

Unfortunately, there is no kconfig. It just looks this way. The buildsystem is custom.

> 
> > endif
> >
> > Probably, I can improve it by testing whether a module is present
> > in the DPDK config and then setting the dependency on linux.  
> 
>   That's not possible: DEPENDENCIES must be declared when the makefiles are 
> parsed, but the tarball containing .config is only extracted afterwards.

OK, it was just an idea.

Regards
Jan

> 
> 
>   Regards,
>   Arnout
> 
> >
> > Anyway, it seems to be quite complex...
> >  
> >>  
> >>> This can be done but I am afraid I have to disable certain modules by
> >>> name from the current configuration. This is not very generic
> >>> (consider adding a new out-of-tree kernel module or removing an
> >>> obsolete one from the DPDK, custom DPDK tree with third party
> >>> modules) but there are only few of them so it might be a working
> >>> solution. I will check if there is a more generic way but I don't
> >>> think so.  
> >>
> >> I'm not sure what you mean here. If the default configuration on a
> >> given architecture builds kernel modules, then on this architecture,
> >> DPDK should depend on the kernel to be built. If the default
> >> configuration on another architecture does not build kernel modules,
> >> then there is no need to depend on the kernel.
> >>
> >> That being said, if only ARM does not build any kernel module, maybe we
> >> can simply do as you do right now, and depend on the kernel
> >> unconditionally, to keep things simple.  
> >
> > Yes, this would work for the existing defconfigs. What about a custom
> > one? E.g. I want some DPDK kernel modules for the Armada with a
> > PCI-E NIC. So, I need a custom DPDK configuration at the moment.
> > Fortunately, this is a rare case.
> >
> > There is also a Xen module in DPDK that is not built by default so you
> > have to create a custom configuration (or alter an existing one) to
> > enable it.
> >
> > I understand that you'd like to keep it as simple as possible. I'd like
> > to keep it simple too but also providing enough flexibility for easy
> > customizations. That's why I don't consider the ARMv7 configuration to
> > be "without kernel modules" as it is possible to enable some if needed.  
> 
> 
> 



-- 
   Jan Viktorin                  E-mail: Viktorin at RehiveTech.com
   System Architect              Web:    www.RehiveTech.com
   RehiveTech
   Brno, Czech Republic

[Buildroot] [PATCH v5] dpdk: new package

[Arnout Vandecappelle]

On 04/19/16 14:27, Jan Viktorin wrote:
> Hello Arnout,
>
> thanks for your comments...
>
> On Tue, 19 Apr 2016 00:50:27 +0200
> Arnout Vandecappelle <arnout@mind.be> wrote:
>
>> On 04/18/16 10:23, Jan Viktorin wrote:
>>> On Sun, 17 Apr 2016 23:06:07 +0200
>>> Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> On Sun, 17 Apr 2016 22:56:07 +0200, Jan Viktorin wrote:
>>>>
>> [snip]
>>>>> Ok. I don't know what would be the use case for dropping dependency
>>>>> on the kernel. Building just the rootfs without the kernel?
>>>>
>>>> For example, yes. Another benefit of not having the mandatory kernel
>>>> dependency is that the autobuilders would be able to test the DPDK
>>>> package.
>>>
>>> I have no clue about this. Does kernel dependency prevent some
>>> auto-testing?
>>
>>    The base configs used in the autobuilders don't include a kernel (or
>> bootloader or rootfs). Therefore, any package depending on the kernel is never
>> built.
>>
>>    We _could_ actually add a couple of base configs that do include a kernel. But
>> someone has to do it :-)
>
> I have no idea how to access the autobuilder nor what kind of complexity
> is connected with this...

  git://git.buildroot.net/buildroot-test/


>>> I would implement this in Buildroot by a list of configuration keys
>>> naming certain drivers. If the kernel is disabled, I'd disable all of
>>> them during configure phase. Otherwise, I'd leave them untouched (to
>>> have the current configuration as is). So, I'd put in dpdk.mk
>>> something like:
>>>
>>> ifeq ($(BR2_LINUX_KERNEL),y)
>>> DPDK_DEPENDENCIES += linux
>>> else
>>> DPDK_KMODS = EAL_IGB_UIO KNI_KMOD LIBRTE_XEN_DOM0
>>>
>>> define DPDK_DISABLE_KMODS
>>> $(foreach m,$(DPDK_KMODS),\
>>
>>    Small nit: indent with tab here.
>
> OK, it was just an ad hoc example. I'd rather know whether I can implement it this way.

  Yes it can.

  If you don't disable these options, what will the dpdk build system do? Error 
out? Or try to use the build system's kernel? Ideally there would be a way to 
globally tell it not to build modules. Manually maintaining a list of config 
options to disable is a bit cumbersome.


>
>>
>>> 	$(call KCONFIG_DISABLE_OPT,CONFIG_RTE_$(m),$(@D)/build/.config))
>>> endef
>>>
>>> DPDK_POST_CONFIGURE_HOOKS += DPDK_DISABLE_KMODS
>>
>>    Is dpdk using Kconfig? In that case, please use the kconfig-package infra. It
>> gives you simple primitives for disabling and enabling options. Consult the manual.
>
> Unfortunately, there is no kconfig. It just looks this way. The buildsystem is custom.

  So you have to manually create a config file with lines like
# CONFIG_FOO is not set
? How uncomfortable...

>
>>
>>> endif
>>>
>>> Probably, I can improve it by testing whether a module is present
>>> in the DPDK config and then setting the dependency on linux.
>>
>>    That's not possible: DEPENDENCIES must be declared when the makefiles are
>> parsed, but the tarball containing .config is only extracted afterwards.
>
> OK, it was just an idea.

  We can also still revert to depending on linux unconditionally like you have 
now. How likely is it that dpdk will be used in practice with no kernel modules? 
Or is this going to be more likely in the future, with heavier use of UIO and 
device tree?


  Regards,
  Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH v5] dpdk: new package

[Thomas Petazzoni]

Hello,

On Tue, 19 Apr 2016 21:30:10 +0200, Arnout Vandecappelle wrote:

>   We can also still revert to depending on linux unconditionally like you have 
> now. How likely is it that dpdk will be used in practice with no kernel modules? 
> Or is this going to be more likely in the future, with heavier use of UIO and 
> device tree?

My proposal would indeed to revert back to depend on Linux
unconditionally, and see what happens later, like if we have users who
use DPDK and complain that it depends on the Linux kernel package.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH v5] dpdk: new package

[Jan Viktorin]

On Tue, 19 Apr 2016 21:30:10 +0200
Arnout Vandecappelle <arnout@mind.be> wrote:

> On 04/19/16 14:27, Jan Viktorin wrote:
> > Hello Arnout,
> >
> > thanks for your comments...
> >
> > On Tue, 19 Apr 2016 00:50:27 +0200
> > Arnout Vandecappelle <arnout@mind.be> wrote:
> >  
> >> On 04/18/16 10:23, Jan Viktorin wrote:  
> >>> On Sun, 17 Apr 2016 23:06:07 +0200
> >>> Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote:
> >>>  
> >>>> Hello,
> >>>>
> >>>> On Sun, 17 Apr 2016 22:56:07 +0200, Jan Viktorin wrote:
> >>>>  
> >> [snip]  
> >>>>> Ok. I don't know what would be the use case for dropping dependency
> >>>>> on the kernel. Building just the rootfs without the kernel?  
> >>>>
> >>>> For example, yes. Another benefit of not having the mandatory kernel
> >>>> dependency is that the autobuilders would be able to test the DPDK
> >>>> package.  
> >>>
> >>> I have no clue about this. Does kernel dependency prevent some
> >>> auto-testing?  
> >>
> >>    The base configs used in the autobuilders don't include a kernel (or
> >> bootloader or rootfs). Therefore, any package depending on the kernel is never
> >> built.
> >>
> >>    We _could_ actually add a couple of base configs that do include a kernel. But
> >> someone has to do it :-)  
> >
> > I have no idea how to access the autobuilder nor what kind of complexity
> > is connected with this...  
> 
>   git://git.buildroot.net/buildroot-test/
> 
> 
> >>> I would implement this in Buildroot by a list of configuration keys
> >>> naming certain drivers. If the kernel is disabled, I'd disable all of
> >>> them during configure phase. Otherwise, I'd leave them untouched (to
> >>> have the current configuration as is). So, I'd put in dpdk.mk
> >>> something like:
> >>>
> >>> ifeq ($(BR2_LINUX_KERNEL),y)
> >>> DPDK_DEPENDENCIES += linux
> >>> else
> >>> DPDK_KMODS = EAL_IGB_UIO KNI_KMOD LIBRTE_XEN_DOM0
> >>>
> >>> define DPDK_DISABLE_KMODS
> >>> $(foreach m,$(DPDK_KMODS),\  
> >>
> >>    Small nit: indent with tab here.  
> >
> > OK, it was just an ad hoc example. I'd rather know whether I can implement it this way.  
> 
>   Yes it can.
> 
>   If you don't disable these options, what will the dpdk build system do? Error 
> out? Or try to use the build system's kernel? Ideally there would be a way to 

My experience is that if I don't pass the location of the kernel
sources it finds out some local one installed on the host system
or it errors out. Only the ARMv7 config builds as it disables all the
modules (unsupported).

It was not primarily developed with a cross-compilation support. The
target system is x86-something... The ARM support is something very
new.

> globally tell it not to build modules. Manually maintaining a list of config 
> options to disable is a bit cumbersome.

No, the modules are built based on the configuration, so an entry like
CONFIG_* something.

> 
> 
> >  
> >>  
> >>> 	$(call KCONFIG_DISABLE_OPT,CONFIG_RTE_$(m),$(@D)/build/.config))
> >>> endef
> >>>
> >>> DPDK_POST_CONFIGURE_HOOKS += DPDK_DISABLE_KMODS  
> >>
> >>    Is dpdk using Kconfig? In that case, please use the kconfig-package infra. It
> >> gives you simple primitives for disabling and enabling options. Consult the manual.  
> >
> > Unfortunately, there is no kconfig. It just looks this way. The buildsystem is custom.  
> 
>   So you have to manually create a config file with lines like
> # CONFIG_FOO is not set

Those lines are not used. There is no dependency check...

> ? How uncomfortable...

Not really. But yes ;). The new defconfigs include some core ones
(e.g. common_linuxapp) and just disable what they don't need or
vice-versa...

> 
> >  
> >>  
> >>> endif
> >>>
> >>> Probably, I can improve it by testing whether a module is present
> >>> in the DPDK config and then setting the dependency on linux.  
> >>
> >>    That's not possible: DEPENDENCIES must be declared when the makefiles are
> >> parsed, but the tarball containing .config is only extracted afterwards.  
> >
> > OK, it was just an idea.  
> 
>   We can also still revert to depending on linux unconditionally like you have 
> now. How likely is it that dpdk will be used in practice with no kernel modules? 
> Or is this going to be more likely in the future, with heavier use of UIO and 
> device tree?

I think that the DPDK community is going to use the mainline drivers as
much as possible (vfio). As far as I know, the UIO is considered
broken as it exposes too much into the userspace. Moreover, they
want to be supported by the major distros so anything very custom
is not very good. There are however two modules that are probably not to
be upstreamed: kni (connection between DPDK and the kernel network
stack) and xen (globally disabled by default). I've never used them
myself but others probably do...

Regards
Jan

> 
> 
>   Regards,
>   Arnout
> 

-- 
  Jan Viktorin                E-mail: Viktorin at RehiveTech.com
  System Architect            Web:    www.RehiveTech.com
  RehiveTech
  Brno, Czech Republic