Re: [SPDK] Building spdk on CentOS6

[Shahar Salzman <shahar.salzman at kaminario.com>]

[-- Attachment #1: Type: text/plain, Size: 6374 bytes --]

Hi,


I attempted to push the gcc support patch to dpdk, and got rejected since they do not wish to support CentOS6.

Would it be possible to manually add the patch to the supported dpdk version (currently 18.01).

I would be happy to support CentOS6 issues.


Thanks,

Shahar


________________________________
From: Shahar Salzman
Sent: Tuesday, April 24, 2018 3:40:51 PM
To: Storage Performance Development Kit
Cc: Ilan Steinberg; Ido Benda; Yael Shavit
Subject: Re: Building spdk on CentOS6


OK, got everything to work.


For DPDK, I had to to do the following:

  *   add a patch to make the new __rte_experimental attribute to work
  *   remove a bunch of modules via config which had some gnarly compilation errors, and are not needed for spdk
  *   use -fno-strict-aliasing in the EXTRA_CFLAGS to avoid a ton of strict aliasing errors.


I tried spdk 18.01, and ran into some issues with the shared library build. As I saw that Daniel had already dealt with them, I tried cherry picking, but eventually used the master branch.

I got these type of warnings:


/usr/bin/ld: warning: librte_mbuf.so.3.1, needed by /khome/shahar.salzman/Kaminario/git/dpdk/x86_64-native-linuxapp-gcc/lib/librte_bus_pci.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: warning: librte_ethdev.so.8.1, needed by /khome/shahar.salzman/Kaminario/git/dpdk/x86_64-native-linuxapp-gcc/lib/librte_bus_pci.so, not found (try using -rpath or -rpath-link)

I added the following patch bellow to make them go away.

I'll approach the dpdk community on the mailing list in order to get the __rte_experimental stuff to work.
My final configuration is:

  *   CenrOS6.4
  *   gcc 4.4.7-17
  *   dpdk v18.02
  *   spdk master

What are the next steps?

Shahar

P.S. Here is the patch to remove the warnings above.
commit d4ef744de1cc4928972b7042d1c90aff12db7867
Author: shahar salzman <shahar.salzman(a)kaminario.com>
Date:   Tue Apr 24 14:31:52 2018 +0300

    lib/env_dpdk: add required libraries

diff --git a/lib/env_dpdk/env.mk b/lib/env_dpdk/env.mk
index 450043c..b46bfed 100644
--- a/lib/env_dpdk/env.mk
+++ b/lib/env_dpdk/env.mk
@@ -68,6 +68,18 @@ ifneq ($(wildcard $(DPDK_ABS_DIR)/lib/librte_malloc.*),)
 DPDK_LIB_LIST += rte_malloc
 endif

+ifneq (, $(wildcard $(DPDK_ABS_DIR)/lib/librte_mbuf.*))
+DPDK_LIB_LIST += rte_mbuf
+endif
+
+ifneq (, $(wildcard $(DPDK_ABS_DIR)/lib/librte_net.*))
+DPDK_LIB_LIST += rte_net
+endif
+
+ifneq (, $(wildcard $(DPDK_ABS_DIR)/lib/librte_ethdev.*))
+DPDK_LIB_LIST += rte_ethdev
+endif
+
 # librte_pci and librte_bus_pci were added in DPDK 17.11. Link these libraries conditionally
 # based on their existence to maintain backward compatibility.
 ifneq (, $(wildcard $(DPDK_ABS_DIR)/lib/librte_pci.*))



________________________________
From: SPDK <spdk-bounces(a)lists.01.org> on behalf of Shahar Salzman <shahar.salzman(a)kaminario.com>
Sent: Tuesday, April 24, 2018 9:57 AM
To: Storage Performance Development Kit
Subject: Re: [SPDK] Building spdk on CentOS6


Great, and thanks!


I have been running into issues compiling dpdk 17.11 on CentOS6 (there seems to be a gcc issue that does not exist in other versions), I'll focus my efforts on dpdk 18.02.

________________________________
From: SPDK <spdk-bounces(a)lists.01.org> on behalf of Verkamp, Daniel <daniel.verkamp(a)intel.com>
Sent: Monday, April 23, 2018 9:40:36 PM
To: Storage Performance Development Kit
Subject: Re: [SPDK] Building spdk on CentOS6


Hi Shahar,



I’ve pushed a patch for the SPDK master branch that should allow it to compile with DPDK 16.07: https://review.gerrithub.io/#/c/spdk/spdk/+/408743/

Gerrit Code Review<https://review.gerrithub.io/#/c/spdk/spdk/+/408743/>
review.gerrithub.io
Keep in touch. Copyright © 2017 | GerritForge Ltd. info(a)gerritforge.com www.gerritforge.com





However, DPDK 16.07 is no longer supported upstream; I highly recommend that you update to a newer version. SPDK is currently tested with DPDK 18.02.



The VFIO issue you mention has also been fixed on SPDK master, and it will be part of the SPDK v18.04 release.



Thanks,

-- Daniel



From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Shahar Salzman
Sent: Sunday, April 22, 2018 6:41 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: [SPDK] Building spdk on CentOS6



Hi,



Finally got to looking at support of spdk build on CentOS6, things look good, except for one issue.

spdk is latest 18.01.x version, dpdk is 16.07 (+3 dpdk patches to allow compilation) and some minor patches (mainly some memory configuration stuff), kernel is a patched 4.9.6.



build succeeded except for the usage of the dpdk function pci_vfio_is_enabled.

I had to apply the patch bellow, removing the usage of this function and then compilation completed without any issues.



It seems that I am missing some sort of dpdk configuration as I see that the function is built, but not packaged into the generated archive.



I went back to square one and ran the instructions in http://www.spdk.io/doc/getting_started.html, but I see no mention of dpdk there. Actually the ./configure requires it.



My next step is to use a more recent dpdk, but shouldn't this work with my version? Am I missing some dpdk configuration?

BTW, as we are not using vhost, on our 17.07 version we simply use CONFIG_VHOST=n in order to skip this, but I would be happier if we used a better solution.



Shahar



P.S. Here is the patch to remove use of this function:

diff --git a/lib/env_dpdk/vtophys.c b/lib/env_dpdk/vtophys.c

index 92aa256..f38929f 100644

--- a/lib/env_dpdk/vtophys.c

+++ b/lib/env_dpdk/vtophys.c

@@ -53,8 +53,10 @@

 #define SPDK_VFIO_ENABLED 1

 #include <linux/vfio.h>



+#if 0

 /* Internal DPDK function forward declaration */

 int pci_vfio_is_enabled(void);

+#endif



 struct spdk_vfio_dma_map {

        struct vfio_iommu_type1_dma_map map;

@@ -341,9 +343,11 @@ spdk_vtophys_iommu_init(void)

        DIR *dir;

        struct dirent *d;



+#if 0

        if (!pci_vfio_is_enabled()) {

                return;

        }

+#endif



        dir = opendir("/proc/self/fd");

        if (!dir) {





[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 20179 bytes --]