From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============2041277501384242847==" MIME-Version: 1.0 From: Walker, Benjamin Subject: Re: [SPDK] Building spdk on CentOS6 Date: Tue, 08 May 2018 16:49:44 +0000 Message-ID: <1525798183.2784.11.camel@intel.com> In-Reply-To: AM3PR04MB37036198E6BB7881B313FD8899A0@AM3PR04MB370.eurprd04.prod.outlook.com List-ID: To: spdk@lists.01.org --===============2041277501384242847== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable On Tue, 2018-05-08 at 14:34 +0000, Shahar Salzman wrote: > Got it. > = > I was tracking the dpdk project, and focused on the branch that Daniel to= ld me > to work on. > So would the correct process be to push the patch to the spdk/dpdk gerrit= hub > and discuss the commit, or the other way around? Yes - go ahead and push to spdk/dpdk on GerritHub so we can all review it. = The SPDK project has never written out a policy on which versions of DPDK are supported for any given SPDK release. The code is always tested using the l= atest released DPDK, and then attempts to support older versions of DPDK are done= in an ad hoc manner. Similarly, the project never explicitly states which oper= ating systems are supported. Instead, we indicate which operating systems the cod= e is regularly tested on (Fedora N and N-1, Ubuntu latest, Ubuntu LTS, CentOS 7, FreeBSD 11). I see that CentOS 6 is no longer receiving full updates and is in maintenan= ce only mode. It will stop receiving even maintenance updates in 2020. I think= that is indicative of the level of support SPDK should provide here. Given the above, if this patch can be applied to the DPDK fork and doesn't = have any additional impact, I'm not opposed to applying it. That wouldn't, in my mind, mean that SPDK is committed to supporting CentOS 6 indefinitely. But certainly that support could continue while it is relatively easy and convenient. > = > From: SPDK on behalf of Luse, Paul E intel.com> > Sent: Tuesday, May 8, 2018 5:14:10 PM > To: Storage Performance Development Kit > Subject: Re: [SPDK] Building spdk on CentOS6 > = > Sure, I understand that. SPDK maintains a fork of dpdk in github at http= s://g > ithub.com/spdk/dpdk for the purpose of temporarily putting things in that= we > need that can=E2=80=99t make it upstream yet for whatever reason =E2=80= =93 this is the > submodule in the SPDK repo. I=E2=80=99m not saying that it makes sense f= or your patch > to land there permanently but it does make sense to talk about it. > = > No problem on not being on the call, we=E2=80=99ll get some input from th= e maintainers > and others I=E2=80=99m sure=E2=80=A6 > = > Thx > Paul > = > From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Shahar Salzm= an > Sent: Tuesday, May 8, 2018 7:04 AM > To: Storage Performance Development Kit > Subject: Re: [SPDK] Building spdk on CentOS6 > = > Hey Paul, > = > The problem is that the patch is in dpdk and not in spdk, and this is the > reason I offered to support it. > The patch (bellow) is only a few lines long modifying the deprecated attr= ibute > definition, a modification to the config, and an extra compilation flag. > All of the above are required in order to build dpdk 18.01 on CentOS6, th= ere > are no problems building spdk on CentOS6. > = > I understood that you are already applying minor patches to dpdk, so ques= tion > is, whether this patch can be added? > = > Unfortunately due to some personal issues, I am unable to participate in > today's meeting. > = > Thanks, > Shahar > From: SPDK on behalf of Luse, Paul E intel.com> > Sent: Tuesday, May 8, 2018 4:25:29 PM > To: Storage Performance Development Kit > Subject: Re: [SPDK] Building spdk on CentOS6 > = > Hi Shahar, > = > Couple of quick things: > = > * thanks for putting all this time and effort into trying to make this wo= rk! > * in the community model, patches aren=E2=80=99t accepted based on one in= dividual=E2=80=99s > ability/desire to support them so appreciate the offer but in general any > patch that is accepted becomes the responsibly of the community (mostly t= he > maintainers) to support it long term. Otherwise we=E2=80=99d have what I= like to call > =E2=80=9Cthe flea market model=E2=80=9D where there=E2=80=99s a bunch of = separate individuals > supporting their own things with very little cohesiveness across everythi= ng J > = > So, wrt next steps, if the patch isn=E2=80=99t a tremendous amount of eff= ort I would > suggest you go ahead and submit it. That=E2=80=99s the best way to get ev= eryone on the > same page wrt exactly what we=E2=80=99re talking about. If it is some si= gnificant > effort then, as Pawel states, you can call into a community meeting (see = email > I sent out last night) or you can explain more details on this list. > = > Thanks again!! > Paul > = > From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Wodkowski, P= awelX > Sent: Tuesday, May 8, 2018 5:57 AM > To: Storage Performance Development Kit > Subject: Re: [SPDK] Building spdk on CentOS6 > = > I think someone will answer you on this mailing list but there is communi= ty > meeting today too. You can raise this if you wish. > = > Pawel > = > From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Shahar Salzm= an > Sent: Tuesday, May 8, 2018 2:23 PM > To: Storage Performance Development Kit > Subject: Re: [SPDK] Building spdk on CentOS6 > = > Hi, > = > I attempted to push the gcc support patch to dpdk, and got rejected since= they > do not wish to support CentOS6. > Would it be possible to manually add the patch to the supported dpdk vers= ion > (currently 18.01). > I would be happy to support CentOS6 issues. > = > Thanks, > Shahar > = > From: Shahar Salzman > Sent: Tuesday, April 24, 2018 3:40:51 PM > To: Storage Performance Development Kit > Cc: Ilan Steinberg; Ido Benda; Yael Shavit > Subject: Re: Building spdk on CentOS6 > = > OK, got everything to work. > = > For DPDK, I had to to do the following: > add a patch to make the new __rte_experimental attribute to work > remove a bunch of modules via config which had some gnarly compilation er= rors, > and are not needed for spdk > use -fno-strict-aliasing in the EXTRA_CFLAGS to avoid a ton of strict ali= asing > errors. > = > I tried spdk 18.01, and ran into some issues with the shared library buil= d. As > I saw that Daniel had already dealt with them, I tried cherry picking, but > eventually used the master branch. > I got these type of warnings: > = > /usr/bin/ld: warning: librte_mbuf.so.3.1, needed by > /khome/shahar.salzman/Kaminario/git/dpdk/x86_64-native-linuxapp- > gcc/lib/librte_bus_pci.so, not found (try using -rpath or -rpath-link) > /usr/bin/ld: warning: librte_ethdev.so.8.1, needed by > /khome/shahar.salzman/Kaminario/git/dpdk/x86_64-native-linuxapp- > gcc/lib/librte_bus_pci.so, not found (try using -rpath or -rpath-link) > = > I added the following patch bellow to make them go away. > = > I'll approach the dpdk community on the mailing list in order to get the > __rte_experimental stuff to work. > My final configuration is: > CenrOS6.4 > gcc 4.4.7-17 > dpdk v18.02 > spdk master > = > What are the next steps? > = > Shahar > = > P.S. Here is the patch to remove the warnings above. > commit d4ef744de1cc4928972b7042d1c90aff12db7867 > Author: shahar salzman > Date: Tue Apr 24 14:31:52 2018 +0300 > = > lib/env_dpdk: add required libraries > = > diff --git a/lib/env_dpdk/env.mk b/lib/env_dpdk/env.mk > index 450043c..b46bfed 100644 > --- a/lib/env_dpdk/env.mk > +++ b/lib/env_dpdk/env.mk > @@ -68,6 +68,18 @@ ifneq ($(wildcard $(DPDK_ABS_DIR)/lib/librte_malloc.*)= ,) > DPDK_LIB_LIST +=3D rte_malloc > endif > = > +ifneq (, $(wildcard $(DPDK_ABS_DIR)/lib/librte_mbuf.*)) > +DPDK_LIB_LIST +=3D rte_mbuf > +endif > + > +ifneq (, $(wildcard $(DPDK_ABS_DIR)/lib/librte_net.*)) > +DPDK_LIB_LIST +=3D rte_net > +endif > + > +ifneq (, $(wildcard $(DPDK_ABS_DIR)/lib/librte_ethdev.*)) > +DPDK_LIB_LIST +=3D rte_ethdev > +endif > + > # librte_pci and librte_bus_pci were added in DPDK 17.11. Link these > libraries conditionally > # based on their existence to maintain backward compatibility. > ifneq (, $(wildcard $(DPDK_ABS_DIR)/lib/librte_pci.*)) > = > = > = > From: SPDK on behalf of Shahar Salzman zman(a)kaminario.com> > Sent: Tuesday, April 24, 2018 9:57 AM > To: Storage Performance Development Kit > Subject: Re: [SPDK] Building spdk on CentOS6 > = > Great, and thanks! > = > I have been running into issues compiling dpdk 17.11 on CentOS6 (there se= ems > to be a gcc issue that does not exist in other versions), I'll focus my > efforts on dpdk 18.02. > From: SPDK on behalf of Verkamp, Daniel rkamp(a)intel.com> > Sent: Monday, April 23, 2018 9:40:36 PM > To: Storage Performance Development Kit > Subject: Re: [SPDK] Building spdk on CentOS6 > = > Hi Shahar, > = > I=E2=80=99ve pushed a patch for the SPDK master branch that should allow = it to compile > with DPDK 16.07: https://review.gerrithub.io/#/c/spdk/spdk/+/408743/ > = > Gerrit Code Review > = > review.gerrithub.io > = > Keep in touch. Copyright =C2=A9 2017 | GerritForge Ltd. info(a)gerritforg= e.com > www.gerritforge.com > = > = > However, DPDK 16.07 is no longer supported upstream; I highly recommend t= hat > you update to a newer version. SPDK is currently tested with DPDK 18.02. > = > The VFIO issue you mention has also been fixed on SPDK master, and it wil= l be > part of the SPDK v18.04 release. > = > Thanks, > -- Daniel > = > From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Shahar Salzm= an > Sent: Sunday, April 22, 2018 6:41 AM > To: Storage Performance Development Kit > Subject: [SPDK] Building spdk on CentOS6 > = > Hi, > = > Finally got to looking at support of spdk build on CentOS6, things look g= ood, > except for one issue. > spdk is latest 18.01.x version, dpdk is 16.07 (+3 dpdk patches to allow > compilation) and some minor patches (mainly some memory configuration stu= ff), > kernel is a patched 4.9.6. > = > build succeeded except for the usage of the dpdk function pci_vfio_is_ena= bled. > I had to apply the patch bellow, removing the usage of this function and = then > compilation completed without any issues. > = > It seems that I am missing some sort of dpdk configuration as I see that = the > function is built, but not packaged into the generated archive. > = > I went back to square one and ran the instructions in http://www.spdk.io/= doc/g > etting_started.html, but I see no mention of dpdk there. Actually the > ./configure requires it. > = > My next step is to use a more recent dpdk, but shouldn't this work with my > version? Am I missing some dpdk configuration? > BTW, as we are not using vhost, on our 17.07 version we simply use > CONFIG_VHOST=3Dn in order to skip this, but I would be happier if we used= a > better solution. > = > Shahar > = > P.S. Here is the patch to remove use of this function: > diff --git a/lib/env_dpdk/vtophys.c b/lib/env_dpdk/vtophys.c > index 92aa256..f38929f 100644 > --- a/lib/env_dpdk/vtophys.c > +++ b/lib/env_dpdk/vtophys.c > @@ -53,8 +53,10 @@ > #define SPDK_VFIO_ENABLED 1 > #include > = > +#if 0 > /* Internal DPDK function forward declaration */ > int pci_vfio_is_enabled(void); > +#endif > = > struct spdk_vfio_dma_map { > struct vfio_iommu_type1_dma_map map; > @@ -341,9 +343,11 @@ spdk_vtophys_iommu_init(void) > DIR *dir; > struct dirent *d; > = > +#if 0 > if (!pci_vfio_is_enabled()) { > return; > } > +#endif > = > dir =3D opendir("/proc/self/fd"); > if (!dir) { > = > = > _______________________________________________ > SPDK mailing list > SPDK(a)lists.01.org > https://lists.01.org/mailman/listinfo/spdk --===============2041277501384242847==--