[Buildroot] [PATCH next 1/1] package/rdma-core: new package

[Buildroot] [PATCH next 1/1] package/rdma-core: new package

[Julien Olivain]

This is the userspace components for the Linux Kernel's
drivers/infiniband subsystem.

https://github.com/linux-rdma/rdma-core

Signed-off-by: Julien Olivain <ju.o@free.fr>
---
Changes v2 -> v3:
- Update package version from 41.0 to 42.0
- Update kernel in test from 5.15.48 to 5.15.61

Changes v1 -> v2:
- Update package version from 39.0 to 41.0
- Add test_rdma_core.py entry to DEVELOPERS file
- Update kernel in test from 5.15.30 to 5.15.48
- Moved Config.in rdma comment in main pkg description
---
Tested rdma-core package in qemu_aarch64_virt_defconfig.
Also tested with commands:

    make check-package
    ...
    0 warnings generated

    utils/test-pkg -a -p rdma-core
    ...
    44 builds, 5 skipped, 0 build failed, 0 legal-info failed, 0 show-info failed

    support/testing/run-tests \
        -d dl \
        -o output_folder \
        tests.package.test_rdma_core.TestRdmaCore
    ...
    OK
---
 DEVELOPERS                                    |  2 +
 package/Config.in                             |  1 +
 package/rdma-core/Config.in                   | 20 +++++
 package/rdma-core/linux-rdma.fragment         |  9 +++
 package/rdma-core/rdma-core.hash              |  5 ++
 package/rdma-core/rdma-core.mk                | 24 ++++++
 package/rdma-core/readme.txt                  | 74 +++++++++++++++++++
 .../testing/tests/package/test_rdma_core.py   | 41 ++++++++++
 8 files changed, 176 insertions(+)
 create mode 100644 package/rdma-core/Config.in
 create mode 100644 package/rdma-core/linux-rdma.fragment
 create mode 100644 package/rdma-core/rdma-core.hash
 create mode 100644 package/rdma-core/rdma-core.mk
 create mode 100644 package/rdma-core/readme.txt
 create mode 100644 support/testing/tests/package/test_rdma_core.py

diff --git a/DEVELOPERS b/DEVELOPERS
index 4287fd9cf7..82e8a1fa6f 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -1678,6 +1678,7 @@ F:	package/ptm2human/
 F:	package/python-distro/
 F:	package/python-gnupg/
 F:	package/python-pyalsa/
+F:	package/rdma-core/
 F:	package/riscv-isa-sim/
 F:	package/zynaddsubfx/
 F:	support/testing/tests/package/sample_python_distro.py
@@ -1689,6 +1690,7 @@ F:	support/testing/tests/package/test_ola/
 F:	support/testing/tests/package/test_python_distro.py
 F:	support/testing/tests/package/test_python_gnupg.py
 F:	support/testing/tests/package/test_python_pyalsa.py
+F:	support/testing/tests/package/test_rdma_core.py
 
 N:	Julien Viard de Galbert <julien@vdg.name>
 F:	package/dieharder/
diff --git a/package/Config.in b/package/Config.in
index d1c098c48f..95e29e5abd 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -565,6 +565,7 @@ endmenu
 	source "package/pulseview/Config.in"
 	source "package/qoriq-cadence-dp-firmware/Config.in"
 	source "package/raspi-gpio/Config.in"
+	source "package/rdma-core/Config.in"
 	source "package/read-edid/Config.in"
 	source "package/rng-tools/Config.in"
 	source "package/rockchip-mali/Config.in"
diff --git a/package/rdma-core/Config.in b/package/rdma-core/Config.in
new file mode 100644
index 0000000000..82944b87a6
--- /dev/null
+++ b/package/rdma-core/Config.in
@@ -0,0 +1,20 @@
+config BR2_PACKAGE_RDMA_CORE
+	bool "rdma-core"
+	depends on BR2_USE_MMU # fork() used in rstream example
+	depends on BR2_TOOLCHAIN_HAS_SYNC_4
+	depends on !BR2_STATIC_LIBS # dlopen()
+	depends on BR2_TOOLCHAIN_HAS_THREADS
+	select BR2_PACKAGE_LIBNL
+	help
+	  This is the userspace components for the Linux Kernel's
+	  drivers/infiniband subsystem.
+
+	  Note: The 'rdma' utility program is provided by the
+	  'iproute2' package when compiled with 'libmnl'.
+
+	  https://github.com/linux-rdma/rdma-core
+
+comment "rdma-core needs a toolchain w/ threads, dynamic library"
+	depends on BR2_USE_MMU
+	depends on BR2_TOOLCHAIN_HAS_SYNC_4
+	depends on !BR2_TOOLCHAIN_HAS_THREADS || BR2_STATIC_LIBS
diff --git a/package/rdma-core/linux-rdma.fragment b/package/rdma-core/linux-rdma.fragment
new file mode 100644
index 0000000000..e305f27e62
--- /dev/null
+++ b/package/rdma-core/linux-rdma.fragment
@@ -0,0 +1,9 @@
+CONFIG_INFINIBAND=m
+CONFIG_INFINIBAND_USER_MAD=m
+CONFIG_INFINIBAND_USER_ACCESS=m
+CONFIG_RDMA_RXE=m
+CONFIG_INFINIBAND_IPOIB=m
+CONFIG_INFINIBAND_IPOIB_CM=y
+CONFIG_INFINIBAND_IPOIB_DEBUG_DATA=y
+CONFIG_INFINIBAND_RTRS_CLIENT=m
+CONFIG_INFINIBAND_RTRS_SERVER=m
diff --git a/package/rdma-core/rdma-core.hash b/package/rdma-core/rdma-core.hash
new file mode 100644
index 0000000000..65eec1ba8b
--- /dev/null
+++ b/package/rdma-core/rdma-core.hash
@@ -0,0 +1,5 @@
+# Locally calculated
+sha256  cedfa57258d3cdfadecb7c4ddef43c2d47b874e83350495e601bfa4ace2d73a1  rdma-core-42.0.tar.gz
+sha256  99e0df1d009a21d0dfb031600c550fd8f4efc0c6b2a4ef8b34a995aa6f79c9f4  COPYING.BSD_MIT
+sha256  8177f97513213526df2cf6184d8ff986c675afb514d4e68a404010521b880643  COPYING.GPL2
+sha256  c46a557f25b8ef9bec76526c4e593fc13e6cba27e7ba30d73b6497a689cf06f6  COPYING.md
diff --git a/package/rdma-core/rdma-core.mk b/package/rdma-core/rdma-core.mk
new file mode 100644
index 0000000000..6fd931960d
--- /dev/null
+++ b/package/rdma-core/rdma-core.mk
@@ -0,0 +1,24 @@
+################################################################################
+#
+# rdma-core
+#
+################################################################################
+
+RDMA_CORE_VERSION = 42.0
+RDMA_CORE_SITE = $(call github,linux-rdma,rdma-core,v$(RDMA_CORE_VERSION))
+RDMA_CORE_LICENSE = GPL-2.0 or BSD-2-Clause
+RDMA_CORE_LICENSE_FILES = COPYING.GPL2 COPYING.BSD_MIT COPYING.md
+RDMA_CORE_DEPENDENCIES = libnl
+RDMA_CORE_INSTALL_STAGING = YES
+
+RDMA_CORE_CONF_OPTS = \
+	-DNO_MAN_PAGES=1 \
+	-DNO_PYVERBS=1
+
+ifeq ($(BR2_TOOLCHAIN_HAS_LIBATOMIC),y)
+RDMA_CORE_CONF_OPTS += \
+	-DCMAKE_EXE_LINKER_FLAGS=-latomic \
+	-DCMAKE_SHARED_LINKER_FLAGS=-latomic
+endif
+
+$(eval $(cmake-package))
diff --git a/package/rdma-core/readme.txt b/package/rdma-core/readme.txt
new file mode 100644
index 0000000000..4f90720ca3
--- /dev/null
+++ b/package/rdma-core/readme.txt
@@ -0,0 +1,74 @@
+Testing rdma-core userspace tools
+=================================
+
+Testing rdma-core using Linux software RoCE implementation:
+https://en.wikipedia.org/wiki/RDMA_over_Converged_Ethernet
+
+Using two systems with working TCP/IP conficutation, for example:
+- Server IP: 192.168.123.10
+- Client IP: 192.168.123.20
+
+Make sure firewall configurations are appropriate. Routable RoCE v2
+uses udp/4791. ibv_rc_pingpong uses tcp/18515 for initial
+synchronization.
+
+Note: this test can be executed in two qemu virtual machines with
+bridged networking.
+
+
+Kernel configuration
+--------------------
+
+The Linux Kernel needs some Infiniband configuration. In this example
+the kernel "rdma_rxe" driver is needed (CONFIG_RDMA_RXE=y). See the
+example fragment file provided:
+
+    package/rdma-core/linux-rdma.fragment
+
+
+Buildroot package configuration
+-------------------------------
+
+For setting up a software RoCE link, the "rdma" program is needed. It
+is provided by the "iproute2" package, when "libmnl" is also
+selected. Make sure to have in your Buildroot configuration:
+
+    BR2_PACKAGE_IPROUTE2=y
+    BR2_PACKAGE_LIBMNL=y
+    BR2_PACKAGE_RDMA_CORE=y
+
+
+Setting up the rdma link
+------------------------
+
+On both server and client:
+
+    modprobe rdma_rxe
+    rdma link add rxe0 type rxe netdev eth0
+
+
+Testing with rping
+------------------
+
+On the server side, run the command:
+
+    rping -s -v
+
+On the client side, run the command:
+
+    rping -c -v -a 192.168.123.10
+
+
+Testing with ibv_rc_pingpong
+----------------------------
+
+To test with the pingpong example using the reliable connected (RC)
+transport:
+
+On the server side, run the command:
+
+    ibv_rc_pingpong -g rxe0 -g 1
+
+On the client side, run the command:
+
+    ibv_rc_pingpong -d rxe0 -g 1 192.168.123.10
diff --git a/support/testing/tests/package/test_rdma_core.py b/support/testing/tests/package/test_rdma_core.py
new file mode 100644
index 0000000000..c9ca2dc830
--- /dev/null
+++ b/support/testing/tests/package/test_rdma_core.py
@@ -0,0 +1,41 @@
+import os
+
+import infra.basetest
+
+
+class TestRdmaCore(infra.basetest.BRTest):
+
+    config = \
+        """
+        BR2_aarch64=y
+        BR2_TOOLCHAIN_EXTERNAL=y
+        BR2_TARGET_GENERIC_GETTY_PORT="ttyAMA0"
+        BR2_LINUX_KERNEL=y
+        BR2_LINUX_KERNEL_CUSTOM_VERSION=y
+        BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="5.15.61"
+        BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
+        BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/aarch64-virt/linux.config"
+        BR2_LINUX_KERNEL_CONFIG_FRAGMENT_FILES="package/rdma-core/linux-rdma.fragment"
+        BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL=y
+        BR2_TARGET_ROOTFS_CPIO=y
+        BR2_TARGET_ROOTFS_CPIO_GZIP=y
+        # BR2_TARGET_ROOTFS_TAR is not set
+        BR2_PACKAGE_IPROUTE2=y
+        BR2_PACKAGE_LIBMNL=y
+        BR2_PACKAGE_RDMA_CORE=y
+        """
+
+    def test_run(self):
+        img = os.path.join(self.builddir, "images", "rootfs.cpio.gz")
+        kern = os.path.join(self.builddir, "images", "Image")
+        self.emulator.boot(arch="aarch64",
+                           kernel=kern,
+                           kernel_cmdline=["console=ttyAMA0"],
+                           options=["-M", "virt", "-cpu", "cortex-a57", "-m", "512M", "-initrd", img])
+        self.emulator.login()
+
+        # Add the rxe0 interface
+        self.assertRunOk("rdma link add rxe0 type rxe netdev eth0")
+
+        # ibv_devinfo returns 255 if no devices are found
+        self.assertRunOk("ibv_devinfo -v")
-- 
2.37.2

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH next 1/1] package/rdma-core: new package

[Thomas Petazzoni]

Hello Julien,

On Sun, 21 Aug 2022 18:01:43 +0200
Julien Olivain <ju.o@free.fr> wrote:

> This is the userspace components for the Linux Kernel's
> drivers/infiniband subsystem.
> 
> https://github.com/linux-rdma/rdma-core
> 
> Signed-off-by: Julien Olivain <ju.o@free.fr>

Thanks for this patch. It looks really good. I have just a few
questions and one super minor comment, see below.

> diff --git a/package/rdma-core/readme.txt b/package/rdma-core/readme.txt
> new file mode 100644
> index 0000000000..4f90720ca3
> --- /dev/null
> +++ b/package/rdma-core/readme.txt
> @@ -0,0 +1,74 @@
> +Testing rdma-core userspace tools
> +=================================
> +
> +Testing rdma-core using Linux software RoCE implementation:
> +https://en.wikipedia.org/wiki/RDMA_over_Converged_Ethernet
> +
> +Using two systems with working TCP/IP conficutation, for example:

                                         ^^^^^^^ minor typo here

> +Kernel configuration
> +--------------------
> +
> +The Linux Kernel needs some Infiniband configuration. In this example
> +the kernel "rdma_rxe" driver is needed (CONFIG_RDMA_RXE=y). See the
> +example fragment file provided:
> +
> +    package/rdma-core/linux-rdma.fragment

Did you consider using the <pkg>_LINUX_CONFIG_FIXUPS variable instead?
This would make sure these configuration options are always enabled in
the kernel. Or is your fragment just an example?

> +Buildroot package configuration
> +-------------------------------
> +
> +For setting up a software RoCE link, the "rdma" program is needed. It
> +is provided by the "iproute2" package, when "libmnl" is also
> +selected. Make sure to have in your Buildroot configuration:
> +
> +    BR2_PACKAGE_IPROUTE2=y
> +    BR2_PACKAGE_LIBMNL=y
> +    BR2_PACKAGE_RDMA_CORE=y

Then is there a reason to not select iproute2 and libmnl? Or is it just
that they are needed only for specific use cases?

Thanks!

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH next 1/1] package/rdma-core: new package

[Julien Olivain]

Hi Thomas,

On 17/09/2022 22:49, Thomas Petazzoni wrote:
> Hello Julien,
> 
> On Sun, 21 Aug 2022 18:01:43 +0200
> Julien Olivain <ju.o@free.fr> wrote:
> 
>> This is the userspace components for the Linux Kernel's
>> drivers/infiniband subsystem.
>> 
>> https://github.com/linux-rdma/rdma-core
>> 
>> Signed-off-by: Julien Olivain <ju.o@free.fr>
> 
> Thanks for this patch. It looks really good. I have just a few
> questions and one super minor comment, see below.
> 
>> diff --git a/package/rdma-core/readme.txt 
>> b/package/rdma-core/readme.txt
>> new file mode 100644
>> index 0000000000..4f90720ca3
>> --- /dev/null
>> +++ b/package/rdma-core/readme.txt
>> @@ -0,0 +1,74 @@
>> +Testing rdma-core userspace tools
>> +=================================
>> +
>> +Testing rdma-core using Linux software RoCE implementation:
>> +https://en.wikipedia.org/wiki/RDMA_over_Converged_Ethernet
>> +
>> +Using two systems with working TCP/IP conficutation, for example:
> 
>                                          ^^^^^^^ minor typo here

Thanks.  I'll fix that in a new patch version.

>> +Kernel configuration
>> +--------------------
>> +
>> +The Linux Kernel needs some Infiniband configuration. In this example
>> +the kernel "rdma_rxe" driver is needed (CONFIG_RDMA_RXE=y). See the
>> +example fragment file provided:
>> +
>> +    package/rdma-core/linux-rdma.fragment
> 
> Did you consider using the <pkg>_LINUX_CONFIG_FIXUPS variable instead?
> This would make sure these configuration options are always enabled in
> the kernel. Or is your fragment just an example?

Yes, I considered <pkg>_LINUX_CONFIG_FIXUPS but decided not to use it.
This fragment is really an example with the rdma_rxe driver. The
rdma-core package could be used with any other Kernel InfiniBand
driver. The reason I picked this one, is it's a purely software
implementation and does not require any specific hardware to test.

The only Kernel config that could always be enabled would be
CONFIG_INFINIBAND to enable the core support. The value would be
limited as it would be an empty shell since no other driver/options
would be enabled.

I could then enable a bit more kernel configs, but I think this would
go against the Buildroot philosophy (which I could summarize as
"Buildroot only enables the mandatory things, the user enables only
what he/she needs/wants").

I used the word "example" for that purpose. If needed, I can clarify
even more in the text.

>> +Buildroot package configuration
>> +-------------------------------
>> +
>> +For setting up a software RoCE link, the "rdma" program is needed. It
>> +is provided by the "iproute2" package, when "libmnl" is also
>> +selected. Make sure to have in your Buildroot configuration:
>> +
>> +    BR2_PACKAGE_IPROUTE2=y
>> +    BR2_PACKAGE_LIBMNL=y
>> +    BR2_PACKAGE_RDMA_CORE=y
> 
> Then is there a reason to not select iproute2 and libmnl? Or is it just
> that they are needed only for specific use cases?

Yes, the reasons are:
- Those are indeed needed for a specific use cases,
- Those are not mandatory dependencies (neither for build nor runtime).

The high level purpose of the rdma-core package is to provides
user-space libraries (libibverbs.so, libibumad.so, librdmacm.so, ...)
that interfaces with kernel /dev/infiniband/* nodes. The package also
includes some test programs (e.g. "rping"...).

One caveat is that this "rdma" binary, likely needed when using
rdma-core is in fact in another package (iproute), and only enabled
when some other libraries is also present (libmnl). This what I tried
to capture and reflect here.

Any suggestion to clarify that in the patch?
Apart from the typo, I'll wait for your inputs before sending a new
patch v4.

> Thanks!
> 
> Thomas

Thanks,

Julien.
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH next 1/1] package/rdma-core: new package

[Thomas Petazzoni]

Hello Julien,

On Mon, 19 Sep 2022 00:38:27 +0200
Julien Olivain <ju.o@free.fr> wrote:


> >> +The Linux Kernel needs some Infiniband configuration. In this example
> >> +the kernel "rdma_rxe" driver is needed (CONFIG_RDMA_RXE=y). See the
> >> +example fragment file provided:
> >> +
> >> +    package/rdma-core/linux-rdma.fragment  
> > 
> > Did you consider using the <pkg>_LINUX_CONFIG_FIXUPS variable instead?
> > This would make sure these configuration options are always enabled in
> > the kernel. Or is your fragment just an example?  
> 
> Yes, I considered <pkg>_LINUX_CONFIG_FIXUPS but decided not to use it.
> This fragment is really an example with the rdma_rxe driver. The
> rdma-core package could be used with any other Kernel InfiniBand
> driver. The reason I picked this one, is it's a purely software
> implementation and does not require any specific hardware to test.
> 
> The only Kernel config that could always be enabled would be
> CONFIG_INFINIBAND to enable the core support. The value would be
> limited as it would be an empty shell since no other driver/options
> would be enabled.
> 
> I could then enable a bit more kernel configs, but I think this would
> go against the Buildroot philosophy (which I could summarize as
> "Buildroot only enables the mandatory things, the user enables only
> what he/she needs/wants").
> 
> I used the word "example" for that purpose. If needed, I can clarify
> even more in the text.

Then I think there is no need to provide this example, just put the
kernel config fragment as an artifact together with the runtime test.

> >> +Buildroot package configuration
> >> +-------------------------------
> >> +
> >> +For setting up a software RoCE link, the "rdma" program is needed. It
> >> +is provided by the "iproute2" package, when "libmnl" is also
> >> +selected. Make sure to have in your Buildroot configuration:
> >> +
> >> +    BR2_PACKAGE_IPROUTE2=y
> >> +    BR2_PACKAGE_LIBMNL=y
> >> +    BR2_PACKAGE_RDMA_CORE=y  
> > 
> > Then is there a reason to not select iproute2 and libmnl? Or is it just
> > that they are needed only for specific use cases?  
> 
> Yes, the reasons are:
> - Those are indeed needed for a specific use cases,
> - Those are not mandatory dependencies (neither for build nor runtime).

OK. Then indeed, suggesting that they can be relevant (and in which
case) in the Config.in help text would be useful.

Thanks!

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot