[PATCH] mk: fix the combined library problems by replacing it with a linker script

[PATCH] mk: fix the combined library problems by replacing it with a linker script

[Panu Matilainen]

The physically linked-together combined library has been an increasing
source of problems, as was predicted when library and symbol versioning
was introduced. Replace the complex and fragile construction with a
simple linker script which achieves the same without all the problems,
remove the related kludges from eg mlx drivers.

Since creating the linker script is practically zero cost, remove the
config option and just create it always.

Based on a patch by Sergio Gonzales Monroy, linker script approach
initially suggested by Neil Horman.

Suggested-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
Suggested-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
---
 config/common_bsdapp      |   5 ---
 config/common_linuxapp    |   5 ---
 drivers/net/Makefile      |   1 -
 drivers/net/mlx4/Makefile |   6 ---
 drivers/net/mlx5/Makefile |   6 ---
 lib/Makefile              |   1 -
 mk/rte.app.mk             |  10 -----
 mk/rte.combinedlib.mk     |  57 ++++++++++++++++++++++++++
 mk/rte.lib.mk             |  16 --------
 mk/rte.sdkbuild.mk        |   4 +-
 mk/rte.sharelib.mk        | 101 ----------------------------------------------
 11 files changed, 59 insertions(+), 153 deletions(-)
 create mode 100644 mk/rte.combinedlib.mk
 delete mode 100644 mk/rte.sharelib.mk

diff --git a/config/common_bsdapp b/config/common_bsdapp
index bdf1fcd..f38cdc0 100644
--- a/config/common_bsdapp
+++ b/config/common_bsdapp
@@ -84,11 +84,6 @@ CONFIG_RTE_ARCH_STRICT_ALIGN=n
 CONFIG_RTE_BUILD_SHARED_LIB=n
 
 #
-# Combine to one single library
-#
-CONFIG_RTE_BUILD_COMBINE_LIBS=n
-
-#
 # Use newest code breaking previous ABI
 #
 CONFIG_RTE_NEXT_ABI=y
diff --git a/config/common_linuxapp b/config/common_linuxapp
index a565153..60f5d92 100644
--- a/config/common_linuxapp
+++ b/config/common_linuxapp
@@ -84,11 +84,6 @@ CONFIG_RTE_ARCH_STRICT_ALIGN=n
 CONFIG_RTE_BUILD_SHARED_LIB=n
 
 #
-# Combine to one single library
-#
-CONFIG_RTE_BUILD_COMBINE_LIBS=n
-
-#
 # Use newest code breaking previous ABI
 #
 CONFIG_RTE_NEXT_ABI=y
diff --git a/drivers/net/Makefile b/drivers/net/Makefile
index cddcd57..5d9c585 100644
--- a/drivers/net/Makefile
+++ b/drivers/net/Makefile
@@ -51,5 +51,4 @@ DIRS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio
 DIRS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD) += vmxnet3
 DIRS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT) += xenvirt
 
-include $(RTE_SDK)/mk/rte.sharelib.mk
 include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/drivers/net/mlx4/Makefile b/drivers/net/mlx4/Makefile
index 23b766d..d2f5692 100644
--- a/drivers/net/mlx4/Makefile
+++ b/drivers/net/mlx4/Makefile
@@ -31,12 +31,6 @@
 
 include $(RTE_SDK)/mk/rte.vars.mk
 
-ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS)$(CONFIG_RTE_BUILD_SHARED_LIB),yy)
-all:
-	@echo 'MLX4: Not supported in a combined shared library'
-	@false
-endif
-
 # Library name.
 LIB = librte_pmd_mlx4.a
 
diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
index ae568e6..736d205 100644
--- a/drivers/net/mlx5/Makefile
+++ b/drivers/net/mlx5/Makefile
@@ -31,12 +31,6 @@
 
 include $(RTE_SDK)/mk/rte.vars.mk
 
-ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS)$(CONFIG_RTE_BUILD_SHARED_LIB),yy)
-all:
-	@echo 'MLX5: Not supported in a combined shared library'
-	@false
-endif
-
 # Library name.
 LIB = librte_pmd_mlx5.a
 
diff --git a/lib/Makefile b/lib/Makefile
index 9727b83..a8fe4b2 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -62,5 +62,4 @@ DIRS-$(CONFIG_RTE_LIBRTE_KNI) += librte_kni
 DIRS-$(CONFIG_RTE_LIBRTE_IVSHMEM) += librte_ivshmem
 endif
 
-include $(RTE_SDK)/mk/rte.sharelib.mk
 include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 148653e..2a8fdf7 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -59,10 +59,6 @@ _LDLIBS-y += -L$(RTE_SDK_BIN)/lib
 
 _LDLIBS-y += --whole-archive
 
-_LDLIBS-$(CONFIG_RTE_BUILD_COMBINE_LIBS)    += -l$(RTE_LIBNAME)
-
-ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS),n)
-
 _LDLIBS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR)    += -lrte_distributor
 _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER)        += -lrte_reorder
 
@@ -88,8 +84,6 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED)          += -lrt
 
 _LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST)          += -lrte_vhost
 
-endif # ! CONFIG_RTE_BUILD_COMBINE_LIBS
-
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_PCAP)       += -lpcap
 
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2)   += -lsze2
@@ -114,8 +108,6 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_BNX2X_PMD)      += -lz
 
 _LDLIBS-y += --start-group
 
-ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS),n)
-
 _LDLIBS-$(CONFIG_RTE_LIBRTE_KVARGS)         += -lrte_kvargs
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MBUF)           += -lrte_mbuf
 _LDLIBS-$(CONFIG_RTE_LIBRTE_IP_FRAG)        += -lrte_ip_frag
@@ -153,8 +145,6 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL)       += -lrte_pmd_null
 
 endif # ! $(CONFIG_RTE_BUILD_SHARED_LIB)
 
-endif # ! CONFIG_RTE_BUILD_COMBINE_LIBS
-
 _LDLIBS-y += $(EXECENV_LDLIBS)
 _LDLIBS-y += --end-group
 _LDLIBS-y += --no-whole-archive
diff --git a/mk/rte.combinedlib.mk b/mk/rte.combinedlib.mk
new file mode 100644
index 0000000..033287c
--- /dev/null
+++ b/mk/rte.combinedlib.mk
@@ -0,0 +1,57 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+default: all
+
+ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y)
+EXT:=.so
+else
+EXT:=.a
+endif
+
+COMBINEDLIB := lib$(RTE_LIBNAME)$(EXT)
+
+LIBS := $(notdir $(wildcard $(RTE_OUTPUT)/lib/*$(EXT)))
+
+all: FORCE
+	$(Q)echo "GROUP ( $(LIBS) )" > $(RTE_OUTPUT)/lib/$(COMBINEDLIB)
+
+#
+# Clean all generated files
+#
+.PHONY: clean
+clean:
+	$(Q)rm -f $(RTE_OUTPUT)/lib/$(COMBINEDLIB)
+
+.PHONY: FORCE
+FORCE:
diff --git a/mk/rte.lib.mk b/mk/rte.lib.mk
index 06a1519..92c9d9e 100644
--- a/mk/rte.lib.mk
+++ b/mk/rte.lib.mk
@@ -134,14 +134,6 @@ endif
 		$(depfile_newer)),\
 		$(O_TO_S_DO))
 
-ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS),y)
-	$(if $(or \
-        $(file_missing),\
-        $(call cmdline_changed,$(O_TO_C_STR)),\
-        $(depfile_missing),\
-        $(depfile_newer)),\
-        $(O_TO_C_DO))
-endif
 else
 $(LIB): $(OBJS-y) $(DEP_$(LIB)) FORCE
 	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
@@ -157,14 +149,6 @@ $(LIB): $(OBJS-y) $(DEP_$(LIB)) FORCE
 	    $(depfile_missing),\
 	    $(depfile_newer)),\
 	    $(O_TO_A_DO))
-ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS),y)
-	$(if $(or \
-        $(file_missing),\
-        $(call cmdline_changed,$(O_TO_C_STR)),\
-        $(depfile_missing),\
-        $(depfile_newer)),\
-        $(O_TO_C_DO))
-endif
 endif
 
 #
diff --git a/mk/rte.sdkbuild.mk b/mk/rte.sdkbuild.mk
index 38ec7bd..319fe38 100644
--- a/mk/rte.sdkbuild.mk
+++ b/mk/rte.sdkbuild.mk
@@ -93,8 +93,8 @@ $(ROOTDIRS-y):
 	@[ -d $(BUILDDIR)/$@ ] || mkdir -p $(BUILDDIR)/$@
 	@echo "== Build $@"
 	$(Q)$(MAKE) S=$@ -f $(RTE_SRCDIR)/$@/Makefile -C $(BUILDDIR)/$@ all
-	@if [ $@ = drivers -a $(CONFIG_RTE_BUILD_COMBINE_LIBS) = y ]; then \
-		$(MAKE) -f $(RTE_SDK)/lib/Makefile sharelib; \
+	@if [ $@ = drivers ]; then \
+		$(MAKE) -f $(RTE_SDK)/mk/rte.combinedlib.mk; \
 	fi
 
 %_clean:
diff --git a/mk/rte.sharelib.mk b/mk/rte.sharelib.mk
deleted file mode 100644
index 7bb7219..0000000
--- a/mk/rte.sharelib.mk
+++ /dev/null
@@ -1,101 +0,0 @@
-#   BSD LICENSE
-#
-#   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
-#   All rights reserved.
-#
-#   Redistribution and use in source and binary forms, with or without
-#   modification, are permitted provided that the following conditions
-#   are met:
-#
-#     * Redistributions of source code must retain the above copyright
-#       notice, this list of conditions and the following disclaimer.
-#     * Redistributions in binary form must reproduce the above copyright
-#       notice, this list of conditions and the following disclaimer in
-#       the documentation and/or other materials provided with the
-#       distribution.
-#     * Neither the name of Intel Corporation nor the names of its
-#       contributors may be used to endorse or promote products derived
-#       from this software without specific prior written permission.
-#
-#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-include $(RTE_SDK)/mk/internal/rte.build-pre.mk
-
-# VPATH contains at least SRCDIR
-VPATH += $(SRCDIR)
-
-ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS),y)
-ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y)
-LIB_ONE := lib$(RTE_LIBNAME).so
-else
-LIB_ONE := lib$(RTE_LIBNAME).a
-endif
-endif
-
-.PHONY:sharelib
-sharelib: $(LIB_ONE) FORCE
-
-OBJS = $(wildcard $(RTE_OUTPUT)/build/lib/*.o)
-
-ifeq ($(LINK_USING_CC),1)
-# Override the definition of LD here, since we're linking with CC
-LD := $(CC) $(CPU_CFLAGS)
-O_TO_S = $(LD) $(call linkerprefix,$(CPU_LDFLAGS)) \
-	-shared $(OBJS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
-else
-O_TO_S = $(LD) $(CPU_LDFLAGS) \
-	-shared $(OBJS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
-endif
-
-O_TO_S_STR = $(subst ','\'',$(O_TO_S)) #'# fix syntax highlight
-O_TO_S_DISP = $(if $(V),"$(O_TO_S_STR)","  LD $(@)")
-O_TO_S_CMD = "cmd_$@ = $(O_TO_S_STR)"
-O_TO_S_DO = @set -e; \
-    echo $(O_TO_S_DISP); \
-    $(O_TO_S)
-
-O_TO_A =  $(AR) crus $(RTE_OUTPUT)/lib/$(LIB_ONE) $(OBJS)
-O_TO_A_STR = $(subst ','\'',$(O_TO_A)) #'# fix syntax highlight
-O_TO_A_DISP = $(if $(V),"$(O_TO_A_STR)","  LD $(@)")
-O_TO_A_CMD = "cmd_$@ = $(O_TO_A_STR)"
-O_TO_A_DO = @set -e; \
-    echo $(O_TO_A_DISP); \
-    $(O_TO_A)
-#
-# Archive objects to share library
-#
-
-ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS),y)
-ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y)
-$(LIB_ONE): FORCE
-	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
-	$(O_TO_S_DO)
-else
-$(LIB_ONE): FORCE
-	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
-	$(O_TO_A_DO)
-endif
-endif
-
-#
-# Clean all generated files
-#
-.PHONY: clean
-clean: _postclean
-
-.PHONY: doclean
-doclean:
-	$(Q)rm -rf $(LIB_ONE)
-
-.PHONY: FORCE
-FORCE:
-- 
2.5.0

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Neil Horman]

On Tue, Nov 24, 2015 at 04:31:17PM +0200, Panu Matilainen wrote:
> The physically linked-together combined library has been an increasing
> source of problems, as was predicted when library and symbol versioning
> was introduced. Replace the complex and fragile construction with a
> simple linker script which achieves the same without all the problems,
> remove the related kludges from eg mlx drivers.
> 
> Since creating the linker script is practically zero cost, remove the
> config option and just create it always.
> 
> Based on a patch by Sergio Gonzales Monroy, linker script approach
> initially suggested by Neil Horman.
> 
> Suggested-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
> Suggested-by: Neil Horman <nhorman@tuxdriver.com>
> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Nice work

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Aaron Conole]

Neil Horman <nhorman@tuxdriver.com> writes:
> On Tue, Nov 24, 2015 at 04:31:17PM +0200, Panu Matilainen wrote:
>> The physically linked-together combined library has been an increasing
>> source of problems, as was predicted when library and symbol versioning
>> was introduced. Replace the complex and fragile construction with a
>> simple linker script which achieves the same without all the problems,
>> remove the related kludges from eg mlx drivers.
>> 
>> Since creating the linker script is practically zero cost, remove the
>> config option and just create it always.
>> 
>> Based on a patch by Sergio Gonzales Monroy, linker script approach
>> initially suggested by Neil Horman.
>> 
>> Suggested-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
>> Suggested-by: Neil Horman <nhorman@tuxdriver.com>
>> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
> Acked-by: Neil Horman <nhorman@tuxdriver.com>
> Nice work

+1, and just for good measure:

Acked-by: Aaron Conole <aconole@redhat.com>

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Stephen Hemminger]

On Tue, 24 Nov 2015 16:31:17 +0200
Panu Matilainen <pmatilai@redhat.com> wrote:

> The physically linked-together combined library has been an increasing
> source of problems, as was predicted when library and symbol versioning
> was introduced. Replace the complex and fragile construction with a
> simple linker script which achieves the same without all the problems,
> remove the related kludges from eg mlx drivers.
> 
> Since creating the linker script is practically zero cost, remove the
> config option and just create it always.
> 
> Based on a patch by Sergio Gonzales Monroy, linker script approach
> initially suggested by Neil Horman.
> 
> Suggested-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
> Suggested-by: Neil Horman <nhorman@tuxdriver.com>
> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>

But it now means distros have to ship 20 libraries which seems like
a step back.

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Panu Matilainen]

On 11/25/2015 12:46 AM, Stephen Hemminger wrote:
> On Tue, 24 Nov 2015 16:31:17 +0200
> Panu Matilainen <pmatilai@redhat.com> wrote:
>
>> The physically linked-together combined library has been an increasing
>> source of problems, as was predicted when library and symbol versioning
>> was introduced. Replace the complex and fragile construction with a
>> simple linker script which achieves the same without all the problems,
>> remove the related kludges from eg mlx drivers.
>>
>> Since creating the linker script is practically zero cost, remove the
>> config option and just create it always.
>>
>> Based on a patch by Sergio Gonzales Monroy, linker script approach
>> initially suggested by Neil Horman.
>>
>> Suggested-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
>> Suggested-by: Neil Horman <nhorman@tuxdriver.com>
>> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
>
> But it now means distros have to ship 20 libraries which seems like
> a step back.

That's how Fedora and RHEL are shipping it already and nobody has so 
much as noticed anything strange, much less complained about it. 20 
libraries is but a drop in the ocean on a average distro. But more to 
the point, distros will prefer 50 working libraries over one that doesn't.

The combined library as it is simply is no longer a viable option. 
Besides just being broken (witness the strange hacks people are coming 
up with to work around issues in it) its ugly because it basically gives 
the middle finger to all the effort going into version compatibility, 
and its also big. Few projects will use every library in DPDK, but with 
the combined library they're forced to lug the 800 pound gorilla along 
needlessly.

	- Panu -

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Neil Horman]

On Wed, Nov 25, 2015 at 10:38:48AM +0200, Panu Matilainen wrote:
> On 11/25/2015 12:46 AM, Stephen Hemminger wrote:
> >On Tue, 24 Nov 2015 16:31:17 +0200
> >Panu Matilainen <pmatilai@redhat.com> wrote:
> >
> >>The physically linked-together combined library has been an increasing
> >>source of problems, as was predicted when library and symbol versioning
> >>was introduced. Replace the complex and fragile construction with a
> >>simple linker script which achieves the same without all the problems,
> >>remove the related kludges from eg mlx drivers.
> >>
> >>Since creating the linker script is practically zero cost, remove the
> >>config option and just create it always.
> >>
> >>Based on a patch by Sergio Gonzales Monroy, linker script approach
> >>initially suggested by Neil Horman.
> >>
> >>Suggested-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
> >>Suggested-by: Neil Horman <nhorman@tuxdriver.com>
> >>Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
> >
> >But it now means distros have to ship 20 libraries which seems like
> >a step back.
> 
> That's how Fedora and RHEL are shipping it already and nobody has so much as
> noticed anything strange, much less complained about it. 20 libraries is but
> a drop in the ocean on a average distro. But more to the point, distros will
> prefer 50 working libraries over one that doesn't.
> 
> The combined library as it is simply is no longer a viable option. Besides
> just being broken (witness the strange hacks people are coming up with to
> work around issues in it) its ugly because it basically gives the middle
> finger to all the effort going into version compatibility, and its also big.
> Few projects will use every library in DPDK, but with the combined library
> they're forced to lug the 800 pound gorilla along needlessly.
> 
Agreed,  This solves a ton of problems, and from a distro standpoint, no one
really cares how many libraries it is under the covers.  Its all just one
rpm/deb package anyway.

Neil

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Stephen Hemminger]

On Wed, 25 Nov 2015 10:38:48 +0200
Panu Matilainen <pmatilai@redhat.com> wrote:

> On 11/25/2015 12:46 AM, Stephen Hemminger wrote:
> > On Tue, 24 Nov 2015 16:31:17 +0200
> > Panu Matilainen <pmatilai@redhat.com> wrote:
> >
> >> The physically linked-together combined library has been an increasing
> >> source of problems, as was predicted when library and symbol versioning
> >> was introduced. Replace the complex and fragile construction with a
> >> simple linker script which achieves the same without all the problems,
> >> remove the related kludges from eg mlx drivers.
> >>
> >> Since creating the linker script is practically zero cost, remove the
> >> config option and just create it always.
> >>
> >> Based on a patch by Sergio Gonzales Monroy, linker script approach
> >> initially suggested by Neil Horman.
> >>
> >> Suggested-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
> >> Suggested-by: Neil Horman <nhorman@tuxdriver.com>
> >> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
> >
> > But it now means distros have to ship 20 libraries which seems like
> > a step back.
> 
> That's how Fedora and RHEL are shipping it already and nobody has so 
> much as noticed anything strange, much less complained about it. 20 
> libraries is but a drop in the ocean on a average distro. But more to 
> the point, distros will prefer 50 working libraries over one that doesn't.
> 
> The combined library as it is simply is no longer a viable option. 
> Besides just being broken (witness the strange hacks people are coming 
> up with to work around issues in it) its ugly because it basically gives 
> the middle finger to all the effort going into version compatibility, 
> and its also big. Few projects will use every library in DPDK, but with 
> the combined library they're forced to lug the 800 pound gorilla along 
> needlessly.
> 
> 	- Panu -
> 

Fixing the combined library took less than an hour for us.

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Panu Matilainen]

On 11/25/2015 06:08 PM, Stephen Hemminger wrote:
> On Wed, 25 Nov 2015 10:38:48 +0200
> Panu Matilainen <pmatilai@redhat.com> wrote:
>
>> On 11/25/2015 12:46 AM, Stephen Hemminger wrote:
>>> On Tue, 24 Nov 2015 16:31:17 +0200
>>> Panu Matilainen <pmatilai@redhat.com> wrote:
>>>
>>>> The physically linked-together combined library has been an increasing
>>>> source of problems, as was predicted when library and symbol versioning
>>>> was introduced. Replace the complex and fragile construction with a
>>>> simple linker script which achieves the same without all the problems,
>>>> remove the related kludges from eg mlx drivers.
>>>>
>>>> Since creating the linker script is practically zero cost, remove the
>>>> config option and just create it always.
>>>>
>>>> Based on a patch by Sergio Gonzales Monroy, linker script approach
>>>> initially suggested by Neil Horman.
>>>>
>>>> Suggested-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
>>>> Suggested-by: Neil Horman <nhorman@tuxdriver.com>
>>>> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
>>>
>>> But it now means distros have to ship 20 libraries which seems like
>>> a step back.
>>
>> That's how Fedora and RHEL are shipping it already and nobody has so
>> much as noticed anything strange, much less complained about it. 20
>> libraries is but a drop in the ocean on a average distro. But more to
>> the point, distros will prefer 50 working libraries over one that doesn't.
>>
>> The combined library as it is simply is no longer a viable option.
>> Besides just being broken (witness the strange hacks people are coming
>> up with to work around issues in it) its ugly because it basically gives
>> the middle finger to all the effort going into version compatibility,
>> and its also big. Few projects will use every library in DPDK, but with
>> the combined library they're forced to lug the 800 pound gorilla along
>> needlessly.
>>
>> 	- Panu -
>>
>
> Fixing the combined library took less than an hour for us.
>

An hour (and many more in times to come) that I bet you could've used on 
something more interesting than fighting that abomination.

	- Panu -

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Neil Horman]

On Wed, Nov 25, 2015 at 08:08:37AM -0800, Stephen Hemminger wrote:
> On Wed, 25 Nov 2015 10:38:48 +0200
> Panu Matilainen <pmatilai@redhat.com> wrote:
> 
> > On 11/25/2015 12:46 AM, Stephen Hemminger wrote:
> > > On Tue, 24 Nov 2015 16:31:17 +0200
> > > Panu Matilainen <pmatilai@redhat.com> wrote:
> > >
> > >> The physically linked-together combined library has been an increasing
> > >> source of problems, as was predicted when library and symbol versioning
> > >> was introduced. Replace the complex and fragile construction with a
> > >> simple linker script which achieves the same without all the problems,
> > >> remove the related kludges from eg mlx drivers.
> > >>
> > >> Since creating the linker script is practically zero cost, remove the
> > >> config option and just create it always.
> > >>
> > >> Based on a patch by Sergio Gonzales Monroy, linker script approach
> > >> initially suggested by Neil Horman.
> > >>
> > >> Suggested-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
> > >> Suggested-by: Neil Horman <nhorman@tuxdriver.com>
> > >> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
> > >
> > > But it now means distros have to ship 20 libraries which seems like
> > > a step back.
> > 
> > That's how Fedora and RHEL are shipping it already and nobody has so 
> > much as noticed anything strange, much less complained about it. 20 
> > libraries is but a drop in the ocean on a average distro. But more to 
> > the point, distros will prefer 50 working libraries over one that doesn't.
> > 
> > The combined library as it is simply is no longer a viable option. 
> > Besides just being broken (witness the strange hacks people are coming 
> > up with to work around issues in it) its ugly because it basically gives 
> > the middle finger to all the effort going into version compatibility, 
> > and its also big. Few projects will use every library in DPDK, but with 
> > the combined library they're forced to lug the 800 pound gorilla along 
> > needlessly.
> > 
> > 	- Panu -
> > 
> 
> Fixing the combined library took less than an hour for us.
How did you fix the versioning issue?

Neil

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Stephen Hemminger]

On Mon, 30 Nov 2015 10:03:43 -0500
Neil Horman <nhorman@tuxdriver.com> wrote:

> On Wed, Nov 25, 2015 at 08:08:37AM -0800, Stephen Hemminger wrote:
> > On Wed, 25 Nov 2015 10:38:48 +0200
> > Panu Matilainen <pmatilai@redhat.com> wrote:
> > 
> > > On 11/25/2015 12:46 AM, Stephen Hemminger wrote:
> > > > On Tue, 24 Nov 2015 16:31:17 +0200
> > > > Panu Matilainen <pmatilai@redhat.com> wrote:
> > > >
> > > >> The physically linked-together combined library has been an increasing
> > > >> source of problems, as was predicted when library and symbol versioning
> > > >> was introduced. Replace the complex and fragile construction with a
> > > >> simple linker script which achieves the same without all the problems,
> > > >> remove the related kludges from eg mlx drivers.
> > > >>
> > > >> Since creating the linker script is practically zero cost, remove the
> > > >> config option and just create it always.
> > > >>
> > > >> Based on a patch by Sergio Gonzales Monroy, linker script approach
> > > >> initially suggested by Neil Horman.
> > > >>
> > > >> Suggested-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
> > > >> Suggested-by: Neil Horman <nhorman@tuxdriver.com>
> > > >> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
> > > >
> > > > But it now means distros have to ship 20 libraries which seems like
> > > > a step back.
> > > 
> > > That's how Fedora and RHEL are shipping it already and nobody has so 
> > > much as noticed anything strange, much less complained about it. 20 
> > > libraries is but a drop in the ocean on a average distro. But more to 
> > > the point, distros will prefer 50 working libraries over one that doesn't.
> > > 
> > > The combined library as it is simply is no longer a viable option. 
> > > Besides just being broken (witness the strange hacks people are coming 
> > > up with to work around issues in it) its ugly because it basically gives 
> > > the middle finger to all the effort going into version compatibility, 
> > > and its also big. Few projects will use every library in DPDK, but with 
> > > the combined library they're forced to lug the 800 pound gorilla along 
> > > needlessly.
> > > 
> > > 	- Panu -
> > > 
> > 
> > Fixing the combined library took less than an hour for us.
> How did you fix the versioning issue?
> 
> Neil

This is what I did. 
Also decided to keep shared library version == major DPDK version
to avoid confusion.


mk: fix when building combined shared library

The DPDK mk file does not set shared object name or version
information as required by Debian.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>

--- a/mk/rte.sharelib.mk
+++ b/mk/rte.sharelib.mk
@@ -51,10 +51,10 @@ ifeq ($(LINK_USING_CC),1)
 # Override the definition of LD here, since we're linking with CC
 LD := $(CC) $(CPU_CFLAGS)
 O_TO_S = $(LD) $(call linkerprefix,$(CPU_LDFLAGS)) \
-	-shared $(OBJS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
+	-shared $(OBJS) -Wl,-soname,$(LIB_ONE).$(RTE_LIBVERS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
 else
 O_TO_S = $(LD) $(CPU_LDFLAGS) \
-	-shared $(OBJS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
+	-shared $(OBJS) -soname $(LIB_ONE).$(RTE_LIBVERS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
 endif
 
 O_TO_S_STR = $(subst ','\'',$(O_TO_S)) #'# fix syntax highlight
--- a/mk/rte.vars.mk
+++ b/mk/rte.vars.mk
@@ -74,8 +74,10 @@ ifneq ($(BUILDING_RTE_SDK),)
 endif
 
 RTE_LIBNAME := $(CONFIG_RTE_LIBNAME:"%"=%)
+RTE_LIBVERS := $(CONFIG_RTE_LIBVERS:"%"=%)
 ifeq ($(RTE_LIBNAME),)
 RTE_LIBNAME := intel_dpdk
+RTE_LIBVERS := 2
 endif
 
 # RTE_TARGET is deducted from config when we are building the SDK.

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Panu Matilainen]

On 11/30/2015 06:41 PM, Stephen Hemminger wrote:
> On Mon, 30 Nov 2015 10:03:43 -0500
> Neil Horman <nhorman@tuxdriver.com> wrote:
>
>> On Wed, Nov 25, 2015 at 08:08:37AM -0800, Stephen Hemminger wrote:
>>> On Wed, 25 Nov 2015 10:38:48 +0200
>>> Panu Matilainen <pmatilai@redhat.com> wrote:
>>>
>>>> On 11/25/2015 12:46 AM, Stephen Hemminger wrote:
>>>>> On Tue, 24 Nov 2015 16:31:17 +0200
>>>>> Panu Matilainen <pmatilai@redhat.com> wrote:
>>>>>
>>>>>> The physically linked-together combined library has been an increasing
>>>>>> source of problems, as was predicted when library and symbol versioning
>>>>>> was introduced. Replace the complex and fragile construction with a
>>>>>> simple linker script which achieves the same without all the problems,
>>>>>> remove the related kludges from eg mlx drivers.
>>>>>>
>>>>>> Since creating the linker script is practically zero cost, remove the
>>>>>> config option and just create it always.
>>>>>>
>>>>>> Based on a patch by Sergio Gonzales Monroy, linker script approach
>>>>>> initially suggested by Neil Horman.
>>>>>>
>>>>>> Suggested-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
>>>>>> Suggested-by: Neil Horman <nhorman@tuxdriver.com>
>>>>>> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
>>>>>
>>>>> But it now means distros have to ship 20 libraries which seems like
>>>>> a step back.
>>>>
>>>> That's how Fedora and RHEL are shipping it already and nobody has so
>>>> much as noticed anything strange, much less complained about it. 20
>>>> libraries is but a drop in the ocean on a average distro. But more to
>>>> the point, distros will prefer 50 working libraries over one that doesn't.
>>>>
>>>> The combined library as it is simply is no longer a viable option.
>>>> Besides just being broken (witness the strange hacks people are coming
>>>> up with to work around issues in it) its ugly because it basically gives
>>>> the middle finger to all the effort going into version compatibility,
>>>> and its also big. Few projects will use every library in DPDK, but with
>>>> the combined library they're forced to lug the 800 pound gorilla along
>>>> needlessly.
>>>>
>>>> 	- Panu -
>>>>
>>>
>>> Fixing the combined library took less than an hour for us.
>> How did you fix the versioning issue?
>>
>> Neil
>
> This is what I did.
> Also decided to keep shared library version == major DPDK version
> to avoid confusion.
>
>
> mk: fix when building combined shared library
>
> The DPDK mk file does not set shared object name or version
> information as required by Debian.
>
> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
>
> --- a/mk/rte.sharelib.mk
> +++ b/mk/rte.sharelib.mk
> @@ -51,10 +51,10 @@ ifeq ($(LINK_USING_CC),1)
>   # Override the definition of LD here, since we're linking with CC
>   LD := $(CC) $(CPU_CFLAGS)
>   O_TO_S = $(LD) $(call linkerprefix,$(CPU_LDFLAGS)) \
> -	-shared $(OBJS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
> +	-shared $(OBJS) -Wl,-soname,$(LIB_ONE).$(RTE_LIBVERS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
>   else
>   O_TO_S = $(LD) $(CPU_LDFLAGS) \
> -	-shared $(OBJS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
> +	-shared $(OBJS) -soname $(LIB_ONE).$(RTE_LIBVERS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
>   endif
>
>   O_TO_S_STR = $(subst ','\'',$(O_TO_S)) #'# fix syntax highlight
> --- a/mk/rte.vars.mk
> +++ b/mk/rte.vars.mk
> @@ -74,8 +74,10 @@ ifneq ($(BUILDING_RTE_SDK),)
>   endif
>
>   RTE_LIBNAME := $(CONFIG_RTE_LIBNAME:"%"=%)
> +RTE_LIBVERS := $(CONFIG_RTE_LIBVERS:"%"=%)
>   ifeq ($(RTE_LIBNAME),)
>   RTE_LIBNAME := intel_dpdk
> +RTE_LIBVERS := 2
>   endif
>
>   # RTE_TARGET is deducted from config when we are building the SDK.
>

Adding a soname and a semi-arbitrary version does not fix the 
fundamental problems:

Since the library lumps together everything in DPDK, you'd have to bump 
its version whenever any of the individual libraries bumps its version 
to have the version mean anything. DPDK 2.0 and 2.1 are supposedly 
binary compatible but 2.2 certainly is not, and beyond that who knows.

That in turn forces all apps to be rebuild whenever one of the libraries 
changes version, whether those apps use that particular library or not.

The combined library doesn't have symbol versioning, so besides the 
better version compatibility tracking it loses other benefits like 
limited symbol visibility.

Not to mention the extra complexity in makefiles to support it, the 
increasing amount of duct-tape required to hold it together. And still 
eg the MLX pmds declare the configuration not supported at all.

	- Panu -

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Robie Basak]

On Tue, Dec 01, 2015 at 02:21:02PM +0200, Panu Matilainen wrote:
> Adding a soname and a semi-arbitrary version does not fix the fundamental
> problems:
> 
> Since the library lumps together everything in DPDK, you'd have to bump its
> version whenever any of the individual libraries bumps its version to have
> the version mean anything. DPDK 2.0 and 2.1 are supposedly binary compatible
> but 2.2 certainly is not, and beyond that who knows.
> 
> That in turn forces all apps to be rebuild whenever one of the libraries
> changes version, whether those apps use that particular library or not.

If we bundle all the libraries together into one package, then in
distributions we have to rebuild anyway when any of the libraries
changes version since a dependent package can't just depend on any later
version, because we don't know in advance what ABI breaks might occur.
It's also trivial to do rebuilds in a distribution. I'd prefer to see
ABI versioning done right to avoid the pain that might occur there.
Rebuilding dependent packages is on the other hand straightforward.

> The combined library doesn't have symbol versioning, so besides the better
> version compatibility tracking it loses other benefits like limited symbol
> visibility.

The combined library *should* have symbol versioning, which I've brought
up before. This isn't a reason to not have a combined library; it is a
reason to fix the combined library.

Why is limited symbol visibility a benefit in this case?

> Not to mention the extra complexity in makefiles to support it, the
> increasing amount of duct-tape required to hold it together. And still eg
> the MLX pmds declare the configuration not supported at all.

I'd argue that this is because the build system is unnecessarily complex
currently. A library consumer should just be able to #include
<dpdk/foo.h> and link with -ldpdk. It should not have a build system or
custom flags imposed on it by one of the libraries it uses.

Robie

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Robie Basak]

Re-sending this unsigned since the ML rejected my signed email.

-1 from Ubuntu without further discussion since it will break us. Please
don't commit this patch yet.

I don't understand why we must have the complexity of so many shared
libraries. From a distribution packaging perspective, all I see is that
this multiplies potential work by twenty times and makes it awkward to
work with without special tooling (which then needs maintaining).

Before I go into details, it would be nice if someone could please
explain why DPDK has to be "special" in needing to do this? I don't
understand why DPDK must be different to every other userspace library
out there. If DPDK has a good need to be different then that's fair
enough. But I feel that if DPDK is deviating from the norm then we need
to frame the discussion from the perspective of "why DPDK must be
different", rather than having me trying to explain why the norm is the
right way to do it.

On Tue, Nov 24, 2015 at 04:31:17PM +0200, Panu Matilainen wrote:
> That's how Fedora and RHEL are shipping it already and nobody has so much
> as noticed anything strange, much less complained about it. 20 libraries
> is but a drop in the ocean on a average distro.

No, it is 20 times the work from the perspective of DPDK package
maintenance. Let me explain why.

In Debian and Ubuntu, we manage a library transition (an ABI bump in a
library together with all dependencies moving to use the new ABI) by
concurrently packaging both the old and new libraries at once. This
works well with the norm for libraries. We ship one binary package per
soname, with the major version as part of the package name. This allows
a system to have two (or more) ABIs installed simultaneously. For a
library transition, we just package the new version and then that can
land and work concurrently as we then individually update every
dependent (library-consuming) package.

This works because of conventions around sonames, which DPDK breaks
unless we treat it as twenty different libraries which changes our work
from easy to painful.

Usually a library transition is managed by hand by the package
maintainer. It's not taxing because it's straightforward and well
understood. Update and upload the new ABI source package, then find all
reverse dependencies and sort them out, recursively. But if the
maintainer must do it twenty times, then it becomes taxing and prone to
error. And if the reverse dependency tree differs depending on the split
library used by library consumers, then it gets far more complex to
follow.

Admittedly we could tool around this to make it easier, but that's extra
work (both initially and in maintenance) and prone to error (because
we'd only be doing it for DPDK).

Packaging a library is usually virtually a no-op in Debian and Ubuntu
nowadays. Our tooling does it all for us. But packaging DPDK is far from
this currently because of all this added complexity. From my perspective
this is unnecessary and makes no sense. We could do all kinds of things
to work around it (that's what packaging is about) but then we'd have to
maintain that specialness and I don't see why it must be awkward like
this instead of just doing it the same way as every other library.

> The combined library as it is simply is no longer a viable option.
> Besides just being broken (witness the strange hacks people are coming
> up with to work around issues in it) its ugly because it basically gives
> the middle finger to all the effort going into version compatibility,
> and its also big. Few projects will use every library in DPDK, but with
> the combined library they're forced to lug the 800 pound gorilla along
> needlessly.

It's broken because it's broken upstream, and that's what we should fix.
Why is it not viable? How does it give the middle finger to effort going
into version compatibility? Doing it the right way like every other
userspace library is what *gives us* version compatibility because then
distributions can straightforwardly install multiple ABI versions at
once.

Finally, I fail to see any "lug the 800 pound gorilla along" saving. We
(Ubuntu and Fedora) are both shipping all the libraries in one package,
whether split or combined, so they are all being lugged onto disk
anyway. Whether split or combined, there is no saving there. And memory
is hardly saved either because the kernel will just page in and out what
is needed in both cases. So how does this proposed change give us any
saving at all?

If distributions are expected to ship everything lumped together on one
package, then we don't get any benefit of having the library split up.

I did bring this up on this list[1] and my understanding of the outcome
then was that it would be fine for us to use the combined library, and
in time we could better define its ABI. Thus I'm not happy that you're
proposing to change tack on this, both because I'm far from convinced
it's a good idea for the project and wider ecosystem and also because it
creates more work for us.

I understand that in the embedded world you might want to build a subset
of the split libraries, and I'm not suggesting that we take away this
ability if there users who want it (though in that scenario I think
static builds probably make more sense). But in the distribution world,
for binaries we ship, I'd prefer to see things work the standard way
with standard tooling and nothing special going on when there is no need
for it.

Please can you explain why having a combined library is so much of a
problem?

Thanks,

Robie


[1] Message-ID: <20150902134900.GO467@mal.justgohome.co.uk>
    Archive: http://dpdk.org/ml/archives/dev/2015-September/023180.html

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Neil Horman]

On Mon, Nov 30, 2015 at 08:41:02AM -0800, Stephen Hemminger wrote:
> On Mon, 30 Nov 2015 10:03:43 -0500
> Neil Horman <nhorman@tuxdriver.com> wrote:
> 
> > On Wed, Nov 25, 2015 at 08:08:37AM -0800, Stephen Hemminger wrote:
> > > On Wed, 25 Nov 2015 10:38:48 +0200
> > > Panu Matilainen <pmatilai@redhat.com> wrote:
> > > 
> > > > On 11/25/2015 12:46 AM, Stephen Hemminger wrote:
> > > > > On Tue, 24 Nov 2015 16:31:17 +0200
> > > > > Panu Matilainen <pmatilai@redhat.com> wrote:
> > > > >
> > > > >> The physically linked-together combined library has been an increasing
> > > > >> source of problems, as was predicted when library and symbol versioning
> > > > >> was introduced. Replace the complex and fragile construction with a
> > > > >> simple linker script which achieves the same without all the problems,
> > > > >> remove the related kludges from eg mlx drivers.
> > > > >>
> > > > >> Since creating the linker script is practically zero cost, remove the
> > > > >> config option and just create it always.
> > > > >>
> > > > >> Based on a patch by Sergio Gonzales Monroy, linker script approach
> > > > >> initially suggested by Neil Horman.
> > > > >>
> > > > >> Suggested-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
> > > > >> Suggested-by: Neil Horman <nhorman@tuxdriver.com>
> > > > >> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
> > > > >
> > > > > But it now means distros have to ship 20 libraries which seems like
> > > > > a step back.
> > > > 
> > > > That's how Fedora and RHEL are shipping it already and nobody has so 
> > > > much as noticed anything strange, much less complained about it. 20 
> > > > libraries is but a drop in the ocean on a average distro. But more to 
> > > > the point, distros will prefer 50 working libraries over one that doesn't.
> > > > 
> > > > The combined library as it is simply is no longer a viable option. 
> > > > Besides just being broken (witness the strange hacks people are coming 
> > > > up with to work around issues in it) its ugly because it basically gives 
> > > > the middle finger to all the effort going into version compatibility, 
> > > > and its also big. Few projects will use every library in DPDK, but with 
> > > > the combined library they're forced to lug the 800 pound gorilla along 
> > > > needlessly.
> > > > 
> > > > 	- Panu -
> > > > 
> > > 
> > > Fixing the combined library took less than an hour for us.
> > How did you fix the versioning issue?
> > 
> > Neil
> 
> This is what I did. 
> Also decided to keep shared library version == major DPDK version
> to avoid confusion.
> 
Ok, thats the library versioning, which is great, but I was more concerned about
the symbol versioning - i.e. the collection of version linker scripts that get
applied when you build individual libraries, but completely ignored when you
build the combined library

Neil

> 
> mk: fix when building combined shared library
> 
> The DPDK mk file does not set shared object name or version
> information as required by Debian.
> 
> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> 
> --- a/mk/rte.sharelib.mk
> +++ b/mk/rte.sharelib.mk
> @@ -51,10 +51,10 @@ ifeq ($(LINK_USING_CC),1)
>  # Override the definition of LD here, since we're linking with CC
>  LD := $(CC) $(CPU_CFLAGS)
>  O_TO_S = $(LD) $(call linkerprefix,$(CPU_LDFLAGS)) \
> -	-shared $(OBJS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
> +	-shared $(OBJS) -Wl,-soname,$(LIB_ONE).$(RTE_LIBVERS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
>  else
>  O_TO_S = $(LD) $(CPU_LDFLAGS) \
> -	-shared $(OBJS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
> +	-shared $(OBJS) -soname $(LIB_ONE).$(RTE_LIBVERS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
>  endif
>  
>  O_TO_S_STR = $(subst ','\'',$(O_TO_S)) #'# fix syntax highlight
> --- a/mk/rte.vars.mk
> +++ b/mk/rte.vars.mk
> @@ -74,8 +74,10 @@ ifneq ($(BUILDING_RTE_SDK),)
>  endif
>  
>  RTE_LIBNAME := $(CONFIG_RTE_LIBNAME:"%"=%)
> +RTE_LIBVERS := $(CONFIG_RTE_LIBVERS:"%"=%)
>  ifeq ($(RTE_LIBNAME),)
>  RTE_LIBNAME := intel_dpdk
> +RTE_LIBVERS := 2
>  endif
>  
>  # RTE_TARGET is deducted from config when we are building the SDK.
> 

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Neil Horman]

On Tue, Dec 01, 2015 at 12:36:15PM +0000, Robie Basak wrote:
> On Tue, Dec 01, 2015 at 02:21:02PM +0200, Panu Matilainen wrote:
> > Adding a soname and a semi-arbitrary version does not fix the fundamental
> > problems:
> > 
> > Since the library lumps together everything in DPDK, you'd have to bump its
> > version whenever any of the individual libraries bumps its version to have
> > the version mean anything. DPDK 2.0 and 2.1 are supposedly binary compatible
> > but 2.2 certainly is not, and beyond that who knows.
> > 
> > That in turn forces all apps to be rebuild whenever one of the libraries
> > changes version, whether those apps use that particular library or not.
> 
> If we bundle all the libraries together into one package, then in
> distributions we have to rebuild anyway when any of the libraries
> changes version since a dependent package can't just depend on any later
> version, because we don't know in advance what ABI breaks might occur.
> It's also trivial to do rebuilds in a distribution. I'd prefer to see
> ABI versioning done right to avoid the pain that might occur there.
> Rebuilding dependent packages is on the other hand straightforward.
> 
> > The combined library doesn't have symbol versioning, so besides the better
> > version compatibility tracking it loses other benefits like limited symbol
> > visibility.
> 
> The combined library *should* have symbol versioning, which I've brought
> up before. This isn't a reason to not have a combined library; it is a
> reason to fix the combined library.
> 
Agreed, but using a linker script as the combined library gets you the proper
symbol versioning.

> Why is limited symbol visibility a benefit in this case?
> 
Because it prevents an application from inadvertently using symbols that would
otherwise appear in another library (i.e. if not using the combined library, you
know you've used a symbol in another library because you are then forced to add
that library to the build.

> > Not to mention the extra complexity in makefiles to support it, the
> > increasing amount of duct-tape required to hold it together. And still eg
> > the MLX pmds declare the configuration not supported at all.
> 
> I'd argue that this is because the build system is unnecessarily complex
> currently. A library consumer should just be able to #include
> <dpdk/foo.h> and link with -ldpdk. It should not have a build system or
> custom flags imposed on it by one of the libraries it uses.
> 
I don't disagree here, but again, modeling libdpdk.so as  a linker script gets
you to that point (or 99% of the way there at least).

Neil

> Robie
> 

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Neil Horman]

On Tue, Dec 01, 2015 at 12:37:37PM +0000, Robie Basak wrote:
> Re-sending this unsigned since the ML rejected my signed email.
> 
> -1 from Ubuntu without further discussion since it will break us. Please
> don't commit this patch yet.
> 
> I don't understand why we must have the complexity of so many shared
> libraries. From a distribution packaging perspective, all I see is that
> this multiplies potential work by twenty times and makes it awkward to
> work with without special tooling (which then needs maintaining).
> 
Theres nothing "complex" about the simple fact that a project builds lots of
libraries.  Its extreemely common. Any graphic window manager has exactly the
same situation, as do any number of tools that have multiple hardware backends
impelmented in user space (v4l, sane, iptables, to name just a few).

> Before I go into details, it would be nice if someone could please
> explain why DPDK has to be "special" in needing to do this? I don't
Its not special, see above.  Not saying the build environment cant be improved,
but the fact that there are multiple libraries is pretty straightforward.

> understand why DPDK must be different to every other userspace library
> out there. If DPDK has a good need to be different then that's fair
> enough. But I feel that if DPDK is deviating from the norm then we need
> to frame the discussion from the perspective of "why DPDK must be
> different", rather than having me trying to explain why the norm is the
> right way to do it.
> 
> On Tue, Nov 24, 2015 at 04:31:17PM +0200, Panu Matilainen wrote:
> > That's how Fedora and RHEL are shipping it already and nobody has so much
> > as noticed anything strange, much less complained about it. 20 libraries
> > is but a drop in the ocean on a average distro.
> 
> No, it is 20 times the work from the perspective of DPDK package
> maintenance. Let me explain why.
> 
> In Debian and Ubuntu, we manage a library transition (an ABI bump in a
> library together with all dependencies moving to use the new ABI) by
> concurrently packaging both the old and new libraries at once. This
> works well with the norm for libraries. We ship one binary package per
> soname, with the major version as part of the package name. This allows
> a system to have two (or more) ABIs installed simultaneously. For a
> library transition, we just package the new version and then that can
> land and work concurrently as we then individually update every
> dependent (library-consuming) package.
So thats, a distribution choice, not an upstream problem.  And it seems like a
problem you should have already solved (note the examples above).  If you feel
like you need to package multiple ABI versions in the same library, you can,
just update the LIBABIVER of all the libraries, instead of the ones that truly
change, so that each library is guaranteed a newer so version, to make the
library file name unique.  Yes you have to make a small change from upstream,
but thats part of the work that distribution maintainers do.

 
> This works because of conventions around sonames, which DPDK breaks
> unless we treat it as twenty different libraries which changes our work
> from easy to painful.
> 
> Usually a library transition is managed by hand by the package
> maintainer. It's not taxing because it's straightforward and well
> understood. Update and upload the new ABI source package, then find all
> reverse dependencies and sort them out, recursively. But if the
> maintainer must do it twenty times, then it becomes taxing and prone to
> error. And if the reverse dependency tree differs depending on the split
> library used by library consumers, then it gets far more complex to
> follow.
> 
> Admittedly we could tool around this to make it easier, but that's extra
> work (both initially and in maintenance) and prone to error (because
> we'd only be doing it for DPDK).
> 
You must already have a solution to this, I can't imagine you package all the
libraries for kde or gnome (or even pam) separately)

> Packaging a library is usually virtually a no-op in Debian and Ubuntu
> nowadays. Our tooling does it all for us. But packaging DPDK is far from
> this currently because of all this added complexity. From my perspective
> this is unnecessary and makes no sense. We could do all kinds of things
> to work around it (that's what packaging is about) but then we'd have to
> maintain that specialness and I don't see why it must be awkward like
> this instead of just doing it the same way as every other library.
> 
> > The combined library as it is simply is no longer a viable option.
> > Besides just being broken (witness the strange hacks people are coming
> > up with to work around issues in it) its ugly because it basically gives
> > the middle finger to all the effort going into version compatibility,
> > and its also big. Few projects will use every library in DPDK, but with
> > the combined library they're forced to lug the 800 pound gorilla along
> > needlessly.
> 
> It's broken because it's broken upstream, and that's what we should fix.
> Why is it not viable? How does it give the middle finger to effort going
> into version compatibility?
Because each individual library has a version script that gets applied during
link to version symbols properly.  Those scripts dont get applied when building
the combined library.


> Doing it the right way like every other
> userspace library is what *gives us* version compatibility because then
> distributions can straightforwardly install multiple ABI versions at
> once.
Again,  Not at all uncommon.  You're packaging methodology is the issue here,
not the fact that there are multiple libraries.

> Finally, I fail to see any "lug the 800 pound gorilla along" saving. We
> (Ubuntu and Fedora) are both shipping all the libraries in one package,
> whether split or combined, so they are all being lugged onto disk
> anyway. Whether split or combined, there is no saving there. And memory
> is hardly saved either because the kernel will just page in and out what
> is needed in both cases. So how does this proposed change give us any
> saving at all?
> 
Not true, initalization constructors for PMD's at the very least mean that every
pmd will get paged in weather you want it or not using the combined library.
Individual libraries let you dynamically load them (via dlopen).  I think the
same is true of several other facets of dpdk.

Neil

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Ferruh Yigit]

On Wed, Dec 02, 2015 at 06:44:19AM -0500, Neil Horman wrote:
> On Tue, Dec 01, 2015 at 12:37:37PM +0000, Robie Basak wrote:
> > Re-sending this unsigned since the ML rejected my signed email.
> > 
> > -1 from Ubuntu without further discussion since it will break us. Please
> > don't commit this patch yet.
> > 
> > I don't understand why we must have the complexity of so many shared
> > libraries. From a distribution packaging perspective, all I see is that
> > this multiplies potential work by twenty times and makes it awkward to
> > work with without special tooling (which then needs maintaining).
> > 
> Theres nothing "complex" about the simple fact that a project builds lots of
> libraries.  Its extreemely common. Any graphic window manager has exactly the
> same situation, as do any number of tools that have multiple hardware backends
> impelmented in user space (v4l, sane, iptables, to name just a few).
> 
> > Before I go into details, it would be nice if someone could please
> > explain why DPDK has to be "special" in needing to do this? I don't
> Its not special, see above.  Not saying the build environment cant be improved,
> but the fact that there are multiple libraries is pretty straightforward.
> 
> > understand why DPDK must be different to every other userspace library
> > out there. If DPDK has a good need to be different then that's fair
> > enough. But I feel that if DPDK is deviating from the norm then we need
> > to frame the discussion from the perspective of "why DPDK must be
> > different", rather than having me trying to explain why the norm is the
> > right way to do it.
> > 
> > On Tue, Nov 24, 2015 at 04:31:17PM +0200, Panu Matilainen wrote:
> > > That's how Fedora and RHEL are shipping it already and nobody has so much
> > > as noticed anything strange, much less complained about it. 20 libraries
> > > is but a drop in the ocean on a average distro.
> > 
> > No, it is 20 times the work from the perspective of DPDK package
> > maintenance. Let me explain why.
> > 
> > In Debian and Ubuntu, we manage a library transition (an ABI bump in a
> > library together with all dependencies moving to use the new ABI) by
> > concurrently packaging both the old and new libraries at once. This
> > works well with the norm for libraries. We ship one binary package per
> > soname, with the major version as part of the package name. This allows
> > a system to have two (or more) ABIs installed simultaneously. For a
> > library transition, we just package the new version and then that can
> > land and work concurrently as we then individually update every
> > dependent (library-consuming) package.
> So thats, a distribution choice, not an upstream problem.  And it seems like a
> problem you should have already solved (note the examples above).  If you feel
> like you need to package multiple ABI versions in the same library, you can,
> just update the LIBABIVER of all the libraries, instead of the ones that truly
> change, so that each library is guaranteed a newer so version, to make the
> library file name unique.  Yes you have to make a small change from upstream,
> but thats part of the work that distribution maintainers do.
> 
>  
> > This works because of conventions around sonames, which DPDK breaks
> > unless we treat it as twenty different libraries which changes our work
> > from easy to painful.
> > 
> > Usually a library transition is managed by hand by the package
> > maintainer. It's not taxing because it's straightforward and well
> > understood. Update and upload the new ABI source package, then find all
> > reverse dependencies and sort them out, recursively. But if the
> > maintainer must do it twenty times, then it becomes taxing and prone to
> > error. And if the reverse dependency tree differs depending on the split
> > library used by library consumers, then it gets far more complex to
> > follow.
> > 
> > Admittedly we could tool around this to make it easier, but that's extra
> > work (both initially and in maintenance) and prone to error (because
> > we'd only be doing it for DPDK).
> > 
> You must already have a solution to this, I can't imagine you package all the
> libraries for kde or gnome (or even pam) separately)
> 
> > Packaging a library is usually virtually a no-op in Debian and Ubuntu
> > nowadays. Our tooling does it all for us. But packaging DPDK is far from
> > this currently because of all this added complexity. From my perspective
> > this is unnecessary and makes no sense. We could do all kinds of things
> > to work around it (that's what packaging is about) but then we'd have to
> > maintain that specialness and I don't see why it must be awkward like
> > this instead of just doing it the same way as every other library.
> > 
> > > The combined library as it is simply is no longer a viable option.
> > > Besides just being broken (witness the strange hacks people are coming
> > > up with to work around issues in it) its ugly because it basically gives
> > > the middle finger to all the effort going into version compatibility,
> > > and its also big. Few projects will use every library in DPDK, but with
> > > the combined library they're forced to lug the 800 pound gorilla along
> > > needlessly.
> > 
> > It's broken because it's broken upstream, and that's what we should fix.
> > Why is it not viable? How does it give the middle finger to effort going
> > into version compatibility?
> Because each individual library has a version script that gets applied during
> link to version symbols properly.  Those scripts dont get applied when building
> the combined library.
> 
> 
> > Doing it the right way like every other
> > userspace library is what *gives us* version compatibility because then
> > distributions can straightforwardly install multiple ABI versions at
> > once.
> Again,  Not at all uncommon.  You're packaging methodology is the issue here,
> not the fact that there are multiple libraries.
> 
> > Finally, I fail to see any "lug the 800 pound gorilla along" saving. We
> > (Ubuntu and Fedora) are both shipping all the libraries in one package,
> > whether split or combined, so they are all being lugged onto disk
> > anyway. Whether split or combined, there is no saving there. And memory
> > is hardly saved either because the kernel will just page in and out what
> > is needed in both cases. So how does this proposed change give us any
> > saving at all?
> > 
> Not true, initalization constructors for PMD's at the very least mean that every
> pmd will get paged in weather you want it or not using the combined library.
> Individual libraries let you dynamically load them (via dlopen).  I think the
> same is true of several other facets of dpdk.
> 
> Neil
> 
I just sent a patch that fixes versioning in combined library, can you please check it: http://dpdk.org/dev/patchwork/patch/9259/

Thanks,
ferruh

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Christian Ehrhardt]

Hi Ferruh,
while not tackling the "soname for combined lib" which I felt to be
the center of all this discussion.
I like that with your patch the symbols in the combined lib are no
more anonymous, but versioned according to the maps the DPDK sub
libraries are maintaining anyway.
Some more technical feedback on your post.

I wonder if we would find a similar solution for the overall soname version.
Unfortunately all naive approaches that came to my mind so far
(summing up LIBABIVER, biggest of LIBABIVER, ...) all have all too
obvious faults.
And I didn't have time for a complex one yet.
I feel that - in favor of non-perfect, but simple solution - it might
be easier to carry one global LIBABIVER for the combined lib that is
increased on each DPDK release IF (very likely) one of the ABIs was
changed in the Release.
Something like a LIBABISUBVER could be increased on any release that
did change, but not break the old ABI.
Well, likely still too naive - I need more time to really get into all
that - like to better understand all the benefits and drawbacks of the
linker script approach.

Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd


On Thu, Dec 3, 2015 at 2:31 AM, Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> On Wed, Dec 02, 2015 at 06:44:19AM -0500, Neil Horman wrote:
>> On Tue, Dec 01, 2015 at 12:37:37PM +0000, Robie Basak wrote:
>> > Re-sending this unsigned since the ML rejected my signed email.
>> >
>> > -1 from Ubuntu without further discussion since it will break us. Please
>> > don't commit this patch yet.
>> >
>> > I don't understand why we must have the complexity of so many shared
>> > libraries. From a distribution packaging perspective, all I see is that
>> > this multiplies potential work by twenty times and makes it awkward to
>> > work with without special tooling (which then needs maintaining).
>> >
>> Theres nothing "complex" about the simple fact that a project builds lots of
>> libraries.  Its extreemely common. Any graphic window manager has exactly the
>> same situation, as do any number of tools that have multiple hardware backends
>> impelmented in user space (v4l, sane, iptables, to name just a few).
>>
>> > Before I go into details, it would be nice if someone could please
>> > explain why DPDK has to be "special" in needing to do this? I don't
>> Its not special, see above.  Not saying the build environment cant be improved,
>> but the fact that there are multiple libraries is pretty straightforward.
>>
>> > understand why DPDK must be different to every other userspace library
>> > out there. If DPDK has a good need to be different then that's fair
>> > enough. But I feel that if DPDK is deviating from the norm then we need
>> > to frame the discussion from the perspective of "why DPDK must be
>> > different", rather than having me trying to explain why the norm is the
>> > right way to do it.
>> >
>> > On Tue, Nov 24, 2015 at 04:31:17PM +0200, Panu Matilainen wrote:
>> > > That's how Fedora and RHEL are shipping it already and nobody has so much
>> > > as noticed anything strange, much less complained about it. 20 libraries
>> > > is but a drop in the ocean on a average distro.
>> >
>> > No, it is 20 times the work from the perspective of DPDK package
>> > maintenance. Let me explain why.
>> >
>> > In Debian and Ubuntu, we manage a library transition (an ABI bump in a
>> > library together with all dependencies moving to use the new ABI) by
>> > concurrently packaging both the old and new libraries at once. This
>> > works well with the norm for libraries. We ship one binary package per
>> > soname, with the major version as part of the package name. This allows
>> > a system to have two (or more) ABIs installed simultaneously. For a
>> > library transition, we just package the new version and then that can
>> > land and work concurrently as we then individually update every
>> > dependent (library-consuming) package.
>> So thats, a distribution choice, not an upstream problem.  And it seems like a
>> problem you should have already solved (note the examples above).  If you feel
>> like you need to package multiple ABI versions in the same library, you can,
>> just update the LIBABIVER of all the libraries, instead of the ones that truly
>> change, so that each library is guaranteed a newer so version, to make the
>> library file name unique.  Yes you have to make a small change from upstream,
>> but thats part of the work that distribution maintainers do.
>>
>>
>> > This works because of conventions around sonames, which DPDK breaks
>> > unless we treat it as twenty different libraries which changes our work
>> > from easy to painful.
>> >
>> > Usually a library transition is managed by hand by the package
>> > maintainer. It's not taxing because it's straightforward and well
>> > understood. Update and upload the new ABI source package, then find all
>> > reverse dependencies and sort them out, recursively. But if the
>> > maintainer must do it twenty times, then it becomes taxing and prone to
>> > error. And if the reverse dependency tree differs depending on the split
>> > library used by library consumers, then it gets far more complex to
>> > follow.
>> >
>> > Admittedly we could tool around this to make it easier, but that's extra
>> > work (both initially and in maintenance) and prone to error (because
>> > we'd only be doing it for DPDK).
>> >
>> You must already have a solution to this, I can't imagine you package all the
>> libraries for kde or gnome (or even pam) separately)
>>
>> > Packaging a library is usually virtually a no-op in Debian and Ubuntu
>> > nowadays. Our tooling does it all for us. But packaging DPDK is far from
>> > this currently because of all this added complexity. From my perspective
>> > this is unnecessary and makes no sense. We could do all kinds of things
>> > to work around it (that's what packaging is about) but then we'd have to
>> > maintain that specialness and I don't see why it must be awkward like
>> > this instead of just doing it the same way as every other library.
>> >
>> > > The combined library as it is simply is no longer a viable option.
>> > > Besides just being broken (witness the strange hacks people are coming
>> > > up with to work around issues in it) its ugly because it basically gives
>> > > the middle finger to all the effort going into version compatibility,
>> > > and its also big. Few projects will use every library in DPDK, but with
>> > > the combined library they're forced to lug the 800 pound gorilla along
>> > > needlessly.
>> >
>> > It's broken because it's broken upstream, and that's what we should fix.
>> > Why is it not viable? How does it give the middle finger to effort going
>> > into version compatibility?
>> Because each individual library has a version script that gets applied during
>> link to version symbols properly.  Those scripts dont get applied when building
>> the combined library.
>>
>>
>> > Doing it the right way like every other
>> > userspace library is what *gives us* version compatibility because then
>> > distributions can straightforwardly install multiple ABI versions at
>> > once.
>> Again,  Not at all uncommon.  You're packaging methodology is the issue here,
>> not the fact that there are multiple libraries.
>>
>> > Finally, I fail to see any "lug the 800 pound gorilla along" saving. We
>> > (Ubuntu and Fedora) are both shipping all the libraries in one package,
>> > whether split or combined, so they are all being lugged onto disk
>> > anyway. Whether split or combined, there is no saving there. And memory
>> > is hardly saved either because the kernel will just page in and out what
>> > is needed in both cases. So how does this proposed change give us any
>> > saving at all?
>> >
>> Not true, initalization constructors for PMD's at the very least mean that every
>> pmd will get paged in weather you want it or not using the combined library.
>> Individual libraries let you dynamically load them (via dlopen).  I think the
>> same is true of several other facets of dpdk.
>>
>> Neil
>>
> I just sent a patch that fixes versioning in combined library, can you please check it: http://dpdk.org/dev/patchwork/patch/9259/
>
> Thanks,
> ferruh
>

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Neil Horman]

On Thu, Dec 03, 2015 at 01:31:33AM +0000, Ferruh Yigit wrote:
> On Wed, Dec 02, 2015 at 06:44:19AM -0500, Neil Horman wrote:
> > On Tue, Dec 01, 2015 at 12:37:37PM +0000, Robie Basak wrote:
> > > Re-sending this unsigned since the ML rejected my signed email.
> > > 
> > > -1 from Ubuntu without further discussion since it will break us. Please
> > > don't commit this patch yet.
> > > 
> > > I don't understand why we must have the complexity of so many shared
> > > libraries. From a distribution packaging perspective, all I see is that
> > > this multiplies potential work by twenty times and makes it awkward to
> > > work with without special tooling (which then needs maintaining).
> > > 
> > Theres nothing "complex" about the simple fact that a project builds lots of
> > libraries.  Its extreemely common. Any graphic window manager has exactly the
> > same situation, as do any number of tools that have multiple hardware backends
> > impelmented in user space (v4l, sane, iptables, to name just a few).
> > 
> > > Before I go into details, it would be nice if someone could please
> > > explain why DPDK has to be "special" in needing to do this? I don't
> > Its not special, see above.  Not saying the build environment cant be improved,
> > but the fact that there are multiple libraries is pretty straightforward.
> > 
> > > understand why DPDK must be different to every other userspace library
> > > out there. If DPDK has a good need to be different then that's fair
> > > enough. But I feel that if DPDK is deviating from the norm then we need
> > > to frame the discussion from the perspective of "why DPDK must be
> > > different", rather than having me trying to explain why the norm is the
> > > right way to do it.
> > > 
> > > On Tue, Nov 24, 2015 at 04:31:17PM +0200, Panu Matilainen wrote:
> > > > That's how Fedora and RHEL are shipping it already and nobody has so much
> > > > as noticed anything strange, much less complained about it. 20 libraries
> > > > is but a drop in the ocean on a average distro.
> > > 
> > > No, it is 20 times the work from the perspective of DPDK package
> > > maintenance. Let me explain why.
> > > 
> > > In Debian and Ubuntu, we manage a library transition (an ABI bump in a
> > > library together with all dependencies moving to use the new ABI) by
> > > concurrently packaging both the old and new libraries at once. This
> > > works well with the norm for libraries. We ship one binary package per
> > > soname, with the major version as part of the package name. This allows
> > > a system to have two (or more) ABIs installed simultaneously. For a
> > > library transition, we just package the new version and then that can
> > > land and work concurrently as we then individually update every
> > > dependent (library-consuming) package.
> > So thats, a distribution choice, not an upstream problem.  And it seems like a
> > problem you should have already solved (note the examples above).  If you feel
> > like you need to package multiple ABI versions in the same library, you can,
> > just update the LIBABIVER of all the libraries, instead of the ones that truly
> > change, so that each library is guaranteed a newer so version, to make the
> > library file name unique.  Yes you have to make a small change from upstream,
> > but thats part of the work that distribution maintainers do.
> > 
> >  
> > > This works because of conventions around sonames, which DPDK breaks
> > > unless we treat it as twenty different libraries which changes our work
> > > from easy to painful.
> > > 
> > > Usually a library transition is managed by hand by the package
> > > maintainer. It's not taxing because it's straightforward and well
> > > understood. Update and upload the new ABI source package, then find all
> > > reverse dependencies and sort them out, recursively. But if the
> > > maintainer must do it twenty times, then it becomes taxing and prone to
> > > error. And if the reverse dependency tree differs depending on the split
> > > library used by library consumers, then it gets far more complex to
> > > follow.
> > > 
> > > Admittedly we could tool around this to make it easier, but that's extra
> > > work (both initially and in maintenance) and prone to error (because
> > > we'd only be doing it for DPDK).
> > > 
> > You must already have a solution to this, I can't imagine you package all the
> > libraries for kde or gnome (or even pam) separately)
> > 
> > > Packaging a library is usually virtually a no-op in Debian and Ubuntu
> > > nowadays. Our tooling does it all for us. But packaging DPDK is far from
> > > this currently because of all this added complexity. From my perspective
> > > this is unnecessary and makes no sense. We could do all kinds of things
> > > to work around it (that's what packaging is about) but then we'd have to
> > > maintain that specialness and I don't see why it must be awkward like
> > > this instead of just doing it the same way as every other library.
> > > 
> > > > The combined library as it is simply is no longer a viable option.
> > > > Besides just being broken (witness the strange hacks people are coming
> > > > up with to work around issues in it) its ugly because it basically gives
> > > > the middle finger to all the effort going into version compatibility,
> > > > and its also big. Few projects will use every library in DPDK, but with
> > > > the combined library they're forced to lug the 800 pound gorilla along
> > > > needlessly.
> > > 
> > > It's broken because it's broken upstream, and that's what we should fix.
> > > Why is it not viable? How does it give the middle finger to effort going
> > > into version compatibility?
> > Because each individual library has a version script that gets applied during
> > link to version symbols properly.  Those scripts dont get applied when building
> > the combined library.
> > 
> > 
> > > Doing it the right way like every other
> > > userspace library is what *gives us* version compatibility because then
> > > distributions can straightforwardly install multiple ABI versions at
> > > once.
> > Again,  Not at all uncommon.  You're packaging methodology is the issue here,
> > not the fact that there are multiple libraries.
> > 
> > > Finally, I fail to see any "lug the 800 pound gorilla along" saving. We
> > > (Ubuntu and Fedora) are both shipping all the libraries in one package,
> > > whether split or combined, so they are all being lugged onto disk
> > > anyway. Whether split or combined, there is no saving there. And memory
> > > is hardly saved either because the kernel will just page in and out what
> > > is needed in both cases. So how does this proposed change give us any
> > > saving at all?
> > > 
> > Not true, initalization constructors for PMD's at the very least mean that every
> > pmd will get paged in weather you want it or not using the combined library.
> > Individual libraries let you dynamically load them (via dlopen).  I think the
> > same is true of several other facets of dpdk.
> > 
> > Neil
> > 
> I just sent a patch that fixes versioning in combined library, can you please check it: http://dpdk.org/dev/patchwork/patch/9259/
> 
> Thanks,
> ferruh
> 
> 
I've seen the patch, and I appreciate the effort, but it really seems to me like
more of the same.  That is to say, its a good effort but it really creates
additional ifrastructure to allow a single library to be built, but the fact of
the matter is, a single library isn't needed.  The build system is setup to
crate multiple libraries, and a linker scripts allows for the combined library
functionality, without adding additional clutter to the Makefiles.  The argument
that its more work to support multiple libraries in some distributions simply
doesn't ring true with me, because that must be a problem which is already
solved for other popular projects which are architected in a simmilar fashion.

Regards
Neil

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Thomas Monjalon]

2015-11-24 16:31, Panu Matilainen:
> The physically linked-together combined library has been an increasing
> source of problems, as was predicted when library and symbol versioning
> was introduced. Replace the complex and fragile construction with a
> simple linker script which achieves the same without all the problems,
> remove the related kludges from eg mlx drivers.
> 
> Since creating the linker script is practically zero cost, remove the
> config option and just create it always.
> 
> Based on a patch by Sergio Gonzales Monroy, linker script approach
> initially suggested by Neil Horman.
> 
> Suggested-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
> Suggested-by: Neil Horman <nhorman@tuxdriver.com>
> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>

As it is a big change and discussion is not totally closed,
it is deferred to release 2.3.
The fix from Ferruh could be sufficient for 2.2.

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Christian Ehrhardt]

Hi,
FYI I kind of "gave up" (not as bad as it sounds) and started looking into
shipping it as individual libraries + linker script as well.
To me it seemed what was more accepted in all the former discussions.

It will surely cause more work for me in the short term, but I hope after
the initial hill I have to climb to make it happen it will be not too much
in future releases.

So if "we" were the only one causing this to be deferred consider it for
2.2.
That way distributions would become more similar which might help consumers
of the DPDK libraries.
In the worst case I can reverse apply it for 2.2 to get some more time to
get it to work properly for us later on.

Looking at the great changes to "make install" by Thomas being in 2.2 -
getting the linker script "official" in 2.2 as well would also help to not
get a major overhaul to packaging every version :-)

have a great week,
Christian



Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd

On Fri, Dec 4, 2015 at 6:19 PM, Thomas Monjalon <thomas.monjalon@6wind.com>
wrote:

> 2015-11-24 16:31, Panu Matilainen:
> > The physically linked-together combined library has been an increasing
> > source of problems, as was predicted when library and symbol versioning
> > was introduced. Replace the complex and fragile construction with a
> > simple linker script which achieves the same without all the problems,
> > remove the related kludges from eg mlx drivers.
> >
> > Since creating the linker script is practically zero cost, remove the
> > config option and just create it always.
> >
> > Based on a patch by Sergio Gonzales Monroy, linker script approach
> > initially suggested by Neil Horman.
> >
> > Suggested-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
> > Suggested-by: Neil Horman <nhorman@tuxdriver.com>
> > Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
>
> As it is a big change and discussion is not totally closed,
> it is deferred to release 2.3.
> The fix from Ferruh could be sufficient for 2.2.
>

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Martinx - ジェームズ]

Hi Christian,

 You can count on me to help testing DPDK for Ubuntu, I have plans for it!

 I have some experience with Debian packaging too... I'm currently
maintaining few Ubuntu PPAs, for fun...    =)

 Also, I have hardware available, with 10G, 40G and 100G NIC cards and
traffic generators.

 I would love to help! Specially when with DPDK on Xen (plans for in
on PVM, HVM, XenServer and Amazon EC2).

 Just a curiosity, I'm the designer/maintainer of "Xen LiveCD v2.0"
and I would like to build a new version of it, that will be based on
Xenial with DPDK.

 http://wiki.xenproject.org/wiki/Live_CD_(Xen_3.2_%2B_Debian_5.0)

 https://github.com/tmartinx/xenlivecd

 Hope to see DPDK compiling with Xen on 32-bit (it is broken now), so
we can enable it on Ubuntu!

Cheers!
Thiago

On 7 December 2015 at 06:27, Christian Ehrhardt
<christian.ehrhardt@canonical.com> wrote:
> Hi,
> FYI I kind of "gave up" (not as bad as it sounds) and started looking into
> shipping it as individual libraries + linker script as well.
> To me it seemed what was more accepted in all the former discussions.
>
> It will surely cause more work for me in the short term, but I hope after
> the initial hill I have to climb to make it happen it will be not too much
> in future releases.
>
> So if "we" were the only one causing this to be deferred consider it for
> 2.2.
> That way distributions would become more similar which might help consumers
> of the DPDK libraries.
> In the worst case I can reverse apply it for 2.2 to get some more time to
> get it to work properly for us later on.
>
> Looking at the great changes to "make install" by Thomas being in 2.2 -
> getting the linker script "official" in 2.2 as well would also help to not
> get a major overhaul to packaging every version :-)
>
> have a great week,
> Christian
>
>
>
> Christian Ehrhardt
> Software Engineer, Ubuntu Server
> Canonical Ltd
>
> On Fri, Dec 4, 2015 at 6:19 PM, Thomas Monjalon <thomas.monjalon@6wind.com>
> wrote:
>
>> 2015-11-24 16:31, Panu Matilainen:
>> > The physically linked-together combined library has been an increasing
>> > source of problems, as was predicted when library and symbol versioning
>> > was introduced. Replace the complex and fragile construction with a
>> > simple linker script which achieves the same without all the problems,
>> > remove the related kludges from eg mlx drivers.
>> >
>> > Since creating the linker script is practically zero cost, remove the
>> > config option and just create it always.
>> >
>> > Based on a patch by Sergio Gonzales Monroy, linker script approach
>> > initially suggested by Neil Horman.
>> >
>> > Suggested-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
>> > Suggested-by: Neil Horman <nhorman@tuxdriver.com>
>> > Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
>>
>> As it is a big change and discussion is not totally closed,
>> it is deferred to release 2.3.
>> The fix from Ferruh could be sufficient for 2.2.
>>

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Robie Basak]

On Wed, Dec 02, 2015 at 06:44:19AM -0500, Neil Horman wrote:
> Theres nothing "complex" about the simple fact that a project builds lots of
> libraries.  Its extreemely common. Any graphic window manager has exactly the
> same situation, as do any number of tools that have multiple hardware backends
> impelmented in user space (v4l, sane, iptables, to name just a few).
> 
> > Before I go into details, it would be nice if someone could please
> > explain why DPDK has to be "special" in needing to do this? I don't
> Its not special, see above.  Not saying the build environment cant be improved,
> but the fact that there are multiple libraries is pretty straightforward.

It's fine in principle for an upstream to ship multiple shared
libraries, but it is extra and unnecessary work unless there's a
*reason* to have multiple shared libraries. What are the reasons for
DPDK?

> > In Debian and Ubuntu, we manage a library transition (an ABI bump in a
> > library together with all dependencies moving to use the new ABI) by
> > concurrently packaging both the old and new libraries at once. This
> > works well with the norm for libraries. We ship one binary package per
> > soname, with the major version as part of the package name. This allows
> > a system to have two (or more) ABIs installed simultaneously. For a
> > library transition, we just package the new version and then that can
> > land and work concurrently as we then individually update every
> > dependent (library-consuming) package.

> So thats, a distribution choice, not an upstream problem.

No, that's how shared libraries work. By design, multiple ABI versions
can be co-installed. That's why sonames have the ABI major version
inside them and the filenames reflect the sonames.

It is a distribution choice to exploit this capability. But it is an
upstream problem if this capability is broken.

By shipping multiple shared libraries, DPDK isn't breaking this
capability per se. But if the upstream expectation is that it's no
additional work for distributions because the multiple libraries can
just be bundled together into a single distribution package, then _this_
is what breaks the capability.

Instead DPDK needs to acknowledge that splitting libraries _does_ cause
additional packaging work for any distribution that wants to use the
multiple co-installed ABI feature of shared libraries as they are
designed.

Then, it becomes for upstream a question of the trade-off: does the
benefit of split libraries outweigh the extra work this creates on
packagers? To understand this, first I need to understand the rationale
for shipping multiple shared libaries specifically in DPDK, and I feel
that you (well, Red Hat) have yet to present a case.

>                                                            And it seems like a
> problem you should have already solved (note the examples above).  If you feel
> like you need to package multiple ABI versions in the same library, you can,
> just update the LIBABIVER of all the libraries, instead of the ones that truly
> change, so that each library is guaranteed a newer so version, to make the
> library file name unique.  Yes you have to make a small change from upstream,
> but thats part of the work that distribution maintainers do.

If it makes sense for upstream, it would be better for all if the code
was maintained in once place rather than fragmented across distribution
patches. My argument here is that _does_ make sense for upstream, which
is why I took the question to this list before we uploaded our first
patched version to Ubuntu.

> You must already have a solution to this, I can't imagine you package all the
> libraries for kde or gnome (or even pam) separately)

PAM modules are unversioned, since they are dynamically loaded plugins
and nothing actually links to them (in the sense that there are
no executables that link to them at exec time). The ABI is defined by
the version of PAM installed, not the version of the plugin. So I don't
think we can really compare to PAM.

I'm less familiar with KDE and GNOME packaging since I specialise on
server. But taking GNOME, for example, I am unable to find any binary
packages where multiple versioned shared objects have been bundled.
Their shared library packaging matches my expectations. For
example (source -> binary -> filename):

  gdk-pixbuf -> libgdk-pixbuf2.0-0 -> /usr/lib/x86_64-linux-gnu/libgdk_pixbuf-2.0.so.0
  gconf -> libgconf-2-4 -> /usr/lib/x86_64-linux-gnu/libgconf-2.so.4
  gtk+3.0 -> libgtk-3-0 -> /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
  pango1.0 -> libpango-1.0-0 -> /usr/lib/x86_64-linux-gnu/libpango-1.0.so.0

Each binary package supplies no more than one soname. (Again I've ignored
unversioned pluggable modules for the same reason as PAM). If this isn't
what you mean, please can you find me a counter-example? Given a soname
you can find the binary package that provides it at
https://www.debian.org/distrib/packages under "Search the contents of
packages". I suggest you set the distribution to "testing" to find more
current sonames.

Christian points out to me that libc6 does ship multiple sonames in a
single package, but I think it's acceptable to consider this to be a
special case that DPDK cannot really look to as an example. We don't
normally co-install multiple ABI versions of libc because a major ABI
bump in libc is extremely rare, and when we do it's a very special case
that is handled as a major distribution-wide project.

In answer to "You must already have a solution to this", we do. Our
solution is to produce one binary package per soname. My point is that
in the case of DPDK, this creates extra unnecessary work. Alternatively,
we could treat DPDK packaging as the same sort of gargantuan task that
packaging GNOME and KDE are, but without a good reason to split
libraries this would be an artifical and unnecessary burden placed on
packagers by DPDK upstream, which is why I am against upstream doing
this.

> > Packaging a library is usually virtually a no-op in Debian and Ubuntu
> > nowadays. Our tooling does it all for us. But packaging DPDK is far from
> > this currently because of all this added complexity. From my perspective
> > this is unnecessary and makes no sense. We could do all kinds of things
> > to work around it (that's what packaging is about) but then we'd have to
> > maintain that specialness and I don't see why it must be awkward like
> > this instead of just doing it the same way as every other library.
> > 
> > > The combined library as it is simply is no longer a viable option.
> > > Besides just being broken (witness the strange hacks people are coming
> > > up with to work around issues in it) its ugly because it basically gives
> > > the middle finger to all the effort going into version compatibility,
> > > and its also big. Few projects will use every library in DPDK, but with
> > > the combined library they're forced to lug the 800 pound gorilla along
> > > needlessly.
> > 
> > It's broken because it's broken upstream, and that's what we should fix.
> > Why is it not viable? How does it give the middle finger to effort going
> > into version compatibility?
> Because each individual library has a version script that gets applied during
> link to version symbols properly.  Those scripts dont get applied when building
> the combined library.

So this is just an upstream bug that needs resolving in the combined
library case? Then I appreciate Ferruh Yigit's efforts in fixing this
bug upstream. Thank you Ferruh Yigit.

> > Doing it the right way like every other
> > userspace library is what *gives us* version compatibility because then
> > distributions can straightforwardly install multiple ABI versions at
> > once.
> Again,  Not at all uncommon.  You're packaging methodology is the issue here,
> not the fact that there are multiple libraries.

No, our packaging methdology is sound as I hope I've explained well
enough above. The real issue is the yet-to-be-justified decision to
split libraries creating unnecessary packaging work given that we wish
to shared libraries properly rather than bundling all the sonames
together (which defeats the point of split libraries in the first
place).

> > Finally, I fail to see any "lug the 800 pound gorilla along" saving. We
> > (Ubuntu and Fedora) are both shipping all the libraries in one package,
> > whether split or combined, so they are all being lugged onto disk
> > anyway. Whether split or combined, there is no saving there. And memory
> > is hardly saved either because the kernel will just page in and out what
> > is needed in both cases. So how does this proposed change give us any
> > saving at all?
> > 
> Not true, initalization constructors for PMD's at the very least mean that every
> pmd will get paged in weather you want it or not using the combined library.
> Individual libraries let you dynamically load them (via dlopen).  I think the
> same is true of several other facets of dpdk.

What's the objective impact of this? Can you quantify your claimed
saving? How does it compare to, say, the extra IOPS required in loading
multiple shared libraries and the extra pages that they could consume?
Are these things at all significant in an issue someone will face in the
real world?

On Tue, Dec 01, 2015 at 08:30:43AM -0500, Neil Horman wrote:
> On Tue, Dec 01, 2015 at 12:36:15PM +0000, Robie Basak wrote:
> > Why is limited symbol visibility a benefit in this case?
> > 
> Because it prevents an application from inadvertently using symbols that would
> otherwise appear in another library (i.e. if not using the combined library, you
> know you've used a symbol in another library because you are then forced to add
> that library to the build.

Does Ferruh Yigit's patch address this?

On Thu, Dec 03, 2015 at 09:59:24AM -0500, Neil Horman wrote:
> I've seen the patch, and I appreciate the effort, but it really seems to me like
> more of the same.  That is to say, its a good effort but it really creates
> additional ifrastructure to allow a single library to be built, but the fact of
> the matter is, a single library isn't needed.  The build system is setup to
> crate multiple libraries, and a linker scripts allows for the combined library
> functionality, without adding additional clutter to the Makefiles.  The argument
> that its more work to support multiple libraries in some distributions simply
> doesn't ring true with me, because that must be a problem which is already
> solved for other popular projects which are architected in a simmilar fashion.

I think I've rebutted all of this above. If you think there's any part
left here that I've failed to address, please let me know and I can go
into it.

Thanks,

Robie

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Neil Horman]

On Tue, Dec 08, 2015 at 05:03:26PM +0000, Robie Basak wrote:
> On Wed, Dec 02, 2015 at 06:44:19AM -0500, Neil Horman wrote:
> > Theres nothing "complex" about the simple fact that a project builds lots of
> > libraries.  Its extreemely common. Any graphic window manager has exactly the
> > same situation, as do any number of tools that have multiple hardware backends
> > impelmented in user space (v4l, sane, iptables, to name just a few).
> > 
> > > Before I go into details, it would be nice if someone could please
> > > explain why DPDK has to be "special" in needing to do this? I don't
> > Its not special, see above.  Not saying the build environment cant be improved,
> > but the fact that there are multiple libraries is pretty straightforward.
> 
> It's fine in principle for an upstream to ship multiple shared
> libraries, but it is extra and unnecessary work unless there's a
> *reason* to have multiple shared libraries. What are the reasons for
> DPDK?
> 
Separation of functionality.  There is no need to build a library that supports
10 different NICS when a given application is only using one.  Likewise with
other discrete functionality, e.g. you don't link against the ACL library if you
dont' want to use ACL's.

> > > In Debian and Ubuntu, we manage a library transition (an ABI bump in a
> > > library together with all dependencies moving to use the new ABI) by
> > > concurrently packaging both the old and new libraries at once. This
> > > works well with the norm for libraries. We ship one binary package per
> > > soname, with the major version as part of the package name. This allows
> > > a system to have two (or more) ABIs installed simultaneously. For a
> > > library transition, we just package the new version and then that can
> > > land and work concurrently as we then individually update every
> > > dependent (library-consuming) package.
> 
> > So thats, a distribution choice, not an upstream problem.
> 
> No, that's how shared libraries work. By design, multiple ABI versions
> can be co-installed. That's why sonames have the ABI major version
> inside them and the filenames reflect the sonames.
> 
We're talking about different things.  The DPDK support ABI versioning in their
sonames, if you would look at the makefiles and output, you would clearly see
that.  Theres no reason that multiple versions of DPDK can't be co-installed,
thats easy, the question here is weather it should support multiple discrete
libraries or only a single large library.  It seems from your comments that a
single monolithic library should be the only option, and thats clearly the less
flexible one.

> It is a distribution choice to exploit this capability. But it is an
> upstream problem if this capability is broken.
> 
Its not broken.  In what way do you think soname versioning is broken in DPDK?
And in what way is it broken that the only solution is to merge all the
libraries into a single monolithic one?

> By shipping multiple shared libraries, DPDK isn't breaking this
> capability per se. But if the upstream expectation is that it's no
> additional work for distributions because the multiple libraries can
> just be bundled together into a single distribution package, then _this_
> is what breaks the capability.
> 
> Instead DPDK needs to acknowledge that splitting libraries _does_ cause
> additional packaging work for any distribution that wants to use the
> multiple co-installed ABI feature of shared libraries as they are
> designed.
> 

Very well, allowing multiple separate libraries causes some extra work for
packaging.  Specifically it requires that distributions carry a patch that
instantiate a specific library ABI major version number that is incremented ni
lock step for every library in a given build (which is admittedly not what
upstream currently does).  I don't see that as overly hard to do, but to each
their own.

However, the solution there is to propose a patch that defines a single ABI
variable in the makefile structure that is applied to every library on a symbol
version bump.  The answer is _not_ to somehow entangle that with the idea that
we have to have a single monolithic library.  Their separate issues, and you
can solve the problem that you are complaining about without throwing the
proverbial baby out with the bathwater.

> Then, it becomes for upstream a question of the trade-off: does the
> benefit of split libraries outweigh the extra work this creates on
> packagers? To understand this, first I need to understand the rationale
> for shipping multiple shared libaries specifically in DPDK, and I feel
> that you (well, Red Hat) have yet to present a case.
>
Some of the reasons I've mentioned above.  Regardless however, the solution
your advocating to the problem you describe is orthogonal to the complaints
you're making there.  If your goal is to allow multiple ABI versions in a given
package, then propose that ABI versions be incremented monolithically in the
upstream build. Even if a given library hasn't changed, it doesn't hurt to bump
the version number - that is to say, as a distribution, if an application links
against library A version 2, it will also link against library B version 2
(assuming it uses library B), even if library B has no ABI changes in it (thats
simply an artifact of how packaging works, you dont' typically mix and match
libraries from separate builds). So...just bump the soname version number, and
while you're at it, make it a global build setting, not a per library build
setting.  Then you can use it to append a version number to the combined library
(script) as well

What you shouldn't do is assume that each library _has_ to have an independent
ABI version number, and use that to bootstrap an argument that the only solution
is a single combined library.

> >                                                            And it seems like a
> > problem you should have already solved (note the examples above).  If you feel
> > like you need to package multiple ABI versions in the same library, you can,
> > just update the LIBABIVER of all the libraries, instead of the ones that truly
> > change, so that each library is guaranteed a newer so version, to make the
> > library file name unique.  Yes you have to make a small change from upstream,
> > but thats part of the work that distribution maintainers do.
> 
Yes, this makes good sense.

> If it makes sense for upstream, it would be better for all if the code
> was maintained in once place rather than fragmented across distribution
> patches. My argument here is that _does_ make sense for upstream, which
> is why I took the question to this list before we uploaded our first
> patched version to Ubuntu.
> 
yes, thats fine.  So, it seems like perhaps we're talking about the same
solution here. If we have a single LIBABIVER variable that is applied to each
DSO, why do we then further need to only allow a single combined library?

Neil

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Thomas Monjalon]

Hi,

I'm reviving this old thread.
My understanding is that everybody prefer the linker script
than the current combined library which had neither symbol versioning
nor library dependency informations.

Comments below:

2015-11-24 16:31, Panu Matilainen:
> The physically linked-together combined library has been an increasing
> source of problems, as was predicted when library and symbol versioning
> was introduced. Replace the complex and fragile construction with a
> simple linker script which achieves the same without all the problems,
> remove the related kludges from eg mlx drivers.
> 
> Since creating the linker script is practically zero cost, remove the
> config option and just create it always.
[...]
> --- /dev/null
> +++ b/mk/rte.combinedlib.mk
> @@ -0,0 +1,57 @@
> +#   BSD LICENSE
> +#
> +#   Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
> +#   All rights reserved.
> +#
> +#   Redistribution and use in source and binary forms, with or without
> +#   modification, are permitted provided that the following conditions
> +#   are met:
> +#
> +#     * Redistributions of source code must retain the above copyright
> +#       notice, this list of conditions and the following disclaimer.
> +#     * Redistributions in binary form must reproduce the above copyright
> +#       notice, this list of conditions and the following disclaimer in
> +#       the documentation and/or other materials provided with the
> +#       distribution.
> +#     * Neither the name of Intel Corporation nor the names of its
> +#       contributors may be used to endorse or promote products derived
> +#       from this software without specific prior written permission.

Why this header, Panu?
I think you should write your own copyright, and assume the linker script ;)

It needs to be rebased and some docs comments must be removed or updated.
I'll send a v2.

[PATCH v2] mk: replace the combined library with a linker script

[Thomas Monjalon]

From: Panu Matilainen <pmatilai@redhat.com>

The physically linked-together combined library has been an increasing
source of problems, as was predicted when library and symbol versioning
was introduced. Replace the complex and fragile construction with a
simple linker script which achieves the same without all the problems,
remove the related kludges from eg mlx drivers.

Since creating the linker script is practically zero cost, remove the
config option and just create it always.

Based on a patch by Sergio Gonzales Monroy, linker script approach
initially suggested by Neil Horman.

Suggested-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
Suggested-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
---
v2:
- move RTE_LIBNAME assignment rte.vars.mk to rte.combinedlib.mk
- update crypto
- update doc
- update rte.app.mk
- update test-build.sh

 config/common_bsdapp                             |   5 --
 config/common_linuxapp                           |   5 --
 doc/guides/contributing/patches.rst              |  12 ++-
 doc/guides/nics/mlx4.rst                         |   5 --
 doc/guides/nics/mlx5.rst                         |   5 --
 drivers/crypto/Makefile                          |   3 +-
 drivers/net/Makefile                             |   1 -
 drivers/net/mlx4/Makefile                        |   6 --
 drivers/net/mlx5/Makefile                        |   6 --
 lib/Makefile                                     |   1 -
 mk/rte.app.mk                                    |  17 +---
 drivers/crypto/Makefile => mk/rte.combinedlib.mk |  28 +++++-
 mk/rte.lib.mk                                    |  16 ----
 mk/rte.sdkbuild.mk                               |   4 +-
 mk/rte.sharelib.mk                               | 105 -----------------------
 mk/rte.vars.mk                                   |   2 -
 scripts/test-build.sh                            |   7 +-
 17 files changed, 36 insertions(+), 192 deletions(-)
 copy drivers/crypto/Makefile => mk/rte.combinedlib.mk (81%)
 delete mode 100644 mk/rte.sharelib.mk

diff --git a/config/common_bsdapp b/config/common_bsdapp
index 696382c..7df5ac6 100644
--- a/config/common_bsdapp
+++ b/config/common_bsdapp
@@ -74,11 +74,6 @@ CONFIG_RTE_ARCH_STRICT_ALIGN=n
 CONFIG_RTE_BUILD_SHARED_LIB=n
 
 #
-# Combine to one single library
-#
-CONFIG_RTE_BUILD_COMBINE_LIBS=n
-
-#
 # Use newest code breaking previous ABI
 #
 CONFIG_RTE_NEXT_ABI=y
diff --git a/config/common_linuxapp b/config/common_linuxapp
index f1638db..26df137 100644
--- a/config/common_linuxapp
+++ b/config/common_linuxapp
@@ -74,11 +74,6 @@ CONFIG_RTE_ARCH_STRICT_ALIGN=n
 CONFIG_RTE_BUILD_SHARED_LIB=n
 
 #
-# Combine to one single library
-#
-CONFIG_RTE_BUILD_COMBINE_LIBS=n
-
-#
 # Use newest code breaking previous ABI
 #
 CONFIG_RTE_NEXT_ABI=y
diff --git a/doc/guides/contributing/patches.rst b/doc/guides/contributing/patches.rst
index 5dd8f79..3ebe95b 100644
--- a/doc/guides/contributing/patches.rst
+++ b/doc/guides/contributing/patches.rst
@@ -267,7 +267,7 @@ Checking Compilation
 Compilation of patches and changes should be tested using the the ``test-build.sh`` script in the ``scripts``
 directory of the DPDK repo::
 
-  scripts/test-build.sh x86_64-native-linuxapp-gcc+next+shared+combined
+  scripts/test-build.sh x86_64-native-linuxapp-gcc+next+shared
 
 The script usage is::
 
@@ -283,10 +283,8 @@ Where:
 Examples of configs are::
 
    x86_64-native-linuxapp-gcc
-   x86_64-native-linuxapp-gcc+next+shared+combined
-   x86_64-native-linuxapp-gcc+shared+next
-   x86_64-native-linuxapp-clang+shared+combined
-   i686-native-linuxapp-gcc+combined
+   x86_64-native-linuxapp-gcc+next+shared
+   x86_64-native-linuxapp-clang+shared
 
 The builds can be modifies via the following environmental variables:
 
@@ -302,8 +300,8 @@ These can be set from the command line or in the config files shown above in the
 The recommended configurations and options to test compilation prior to submitting patches are::
 
    x86_64-native-linuxapp-gcc+shared+next
-   x86_64-native-linuxapp-clang+shared+combined
-   i686-native-linuxapp-gcc+combined
+   x86_64-native-linuxapp-clang+shared
+   i686-native-linuxapp-gcc
 
    export DPDK_DEP_ZLIB=y
    export DPDK_DEP_PCAP=y
diff --git a/doc/guides/nics/mlx4.rst b/doc/guides/nics/mlx4.rst
index 7757013..49f4626 100644
--- a/doc/guides/nics/mlx4.rst
+++ b/doc/guides/nics/mlx4.rst
@@ -47,11 +47,6 @@ There is also a `section dedicated to this poll mode driver
    be enabled manually by setting ``CONFIG_RTE_LIBRTE_MLX4_PMD=y`` and
    recompiling DPDK.
 
-.. warning::
-
-   ``CONFIG_RTE_BUILD_COMBINE_LIBS`` with ``CONFIG_RTE_BUILD_SHARED_LIB``
-   is not supported and thus the compilation will fail with this configuration.
-
 Implementation details
 ----------------------
 
diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index b2a12ce..66794e6 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -48,11 +48,6 @@ There is also a `section dedicated to this poll mode driver
    be enabled manually by setting ``CONFIG_RTE_LIBRTE_MLX5_PMD=y`` and
    recompiling DPDK.
 
-.. warning::
-
-   ``CONFIG_RTE_BUILD_COMBINE_LIBS`` with ``CONFIG_RTE_BUILD_SHARED_LIB``
-   is not supported and thus the compilation will fail with this configuration.
-
 Implementation details
 ----------------------
 
diff --git a/drivers/crypto/Makefile b/drivers/crypto/Makefile
index d07ee96..d0258da 100644
--- a/drivers/crypto/Makefile
+++ b/drivers/crypto/Makefile
@@ -34,5 +34,4 @@ include $(RTE_SDK)/mk/rte.vars.mk
 DIRS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB) += aesni_mb
 DIRS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat
 
-include $(RTE_SDK)/mk/rte.sharelib.mk
-include $(RTE_SDK)/mk/rte.subdir.mk
\ No newline at end of file
+include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/drivers/net/Makefile b/drivers/net/Makefile
index 6e4497e..0c3393f 100644
--- a/drivers/net/Makefile
+++ b/drivers/net/Makefile
@@ -52,5 +52,4 @@ DIRS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio
 DIRS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD) += vmxnet3
 DIRS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT) += xenvirt
 
-include $(RTE_SDK)/mk/rte.sharelib.mk
 include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/drivers/net/mlx4/Makefile b/drivers/net/mlx4/Makefile
index 23b766d..d2f5692 100644
--- a/drivers/net/mlx4/Makefile
+++ b/drivers/net/mlx4/Makefile
@@ -31,12 +31,6 @@
 
 include $(RTE_SDK)/mk/rte.vars.mk
 
-ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS)$(CONFIG_RTE_BUILD_SHARED_LIB),yy)
-all:
-	@echo 'MLX4: Not supported in a combined shared library'
-	@false
-endif
-
 # Library name.
 LIB = librte_pmd_mlx4.a
 
diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
index ae568e6..736d205 100644
--- a/drivers/net/mlx5/Makefile
+++ b/drivers/net/mlx5/Makefile
@@ -31,12 +31,6 @@
 
 include $(RTE_SDK)/mk/rte.vars.mk
 
-ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS)$(CONFIG_RTE_BUILD_SHARED_LIB),yy)
-all:
-	@echo 'MLX5: Not supported in a combined shared library'
-	@false
-endif
-
 # Library name.
 LIB = librte_pmd_mlx5.a
 
diff --git a/lib/Makefile b/lib/Makefile
index ef172ea..6840f87 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -64,5 +64,4 @@ DIRS-$(CONFIG_RTE_LIBRTE_KNI) += librte_kni
 DIRS-$(CONFIG_RTE_LIBRTE_IVSHMEM) += librte_ivshmem
 endif
 
-include $(RTE_SDK)/mk/rte.sharelib.mk
 include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 8ecab41..daac09f 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -59,10 +59,6 @@ _LDLIBS-y += -L$(RTE_SDK_BIN)/lib
 
 _LDLIBS-y += --whole-archive
 
-_LDLIBS-$(CONFIG_RTE_BUILD_COMBINE_LIBS)    += -l$(RTE_LIBNAME)
-
-ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS),n)
-
 _LDLIBS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR)    += -lrte_distributor
 _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER)        += -lrte_reorder
 
@@ -88,8 +84,6 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED)          += -lrt
 
 _LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST)          += -lrte_vhost
 
-endif # ! CONFIG_RTE_BUILD_COMBINE_LIBS
-
 ifeq ($(CONFIG_RTE_LIBRTE_VHOST_NUMA),y)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST)          += -lnuma
 endif
@@ -99,9 +93,8 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST)          += -lfuse
 endif
 
 # The static libraries do not know their dependencies.
-# The combined library fails also to store this information.
-# So linking with static or combined library requires explicit dependencies.
-ifneq ($(CONFIG_RTE_BUILD_COMBINE_LIBS)$(CONFIG_RTE_BUILD_SHARED_LIB),ny)
+# So linking with static library requires explicit dependencies.
+ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_PCAP)       += -lpcap
 _LDLIBS-$(CONFIG_RTE_LIBRTE_BNX2X_PMD)      += -lz
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD)       += -libverbs
@@ -111,12 +104,10 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT)    += -lxenstore
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD)      += -lgxio
 # QAT PMD has a dependency on libcrypto (from openssl) for calculating HMAC precomputes
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_QAT)        += -lcrypto
-endif # CONFIG_RTE_BUILD_COMBINE_LIBS or not CONFIG_RTE_BUILD_SHARED_LIBS
+endif # !CONFIG_RTE_BUILD_SHARED_LIBS
 
 _LDLIBS-y += --start-group
 
-ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS),n)
-
 _LDLIBS-$(CONFIG_RTE_LIBRTE_KVARGS)         += -lrte_kvargs
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MBUF)           += -lrte_mbuf
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MBUF_OFFLOAD)   += -lrte_mbuf_offload
@@ -161,8 +152,6 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB)   += -L$(AESNI_MULTI_BUFFER_LIB_PATH)
 
 endif # ! $(CONFIG_RTE_BUILD_SHARED_LIB)
 
-endif # ! CONFIG_RTE_BUILD_COMBINE_LIBS
-
 _LDLIBS-y += $(EXECENV_LDLIBS)
 _LDLIBS-y += --end-group
 _LDLIBS-y += --no-whole-archive
diff --git a/drivers/crypto/Makefile b/mk/rte.combinedlib.mk
similarity index 81%
copy from drivers/crypto/Makefile
copy to mk/rte.combinedlib.mk
index d07ee96..fe4817b 100644
--- a/drivers/crypto/Makefile
+++ b/mk/rte.combinedlib.mk
@@ -31,8 +31,28 @@
 
 include $(RTE_SDK)/mk/rte.vars.mk
 
-DIRS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB) += aesni_mb
-DIRS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat
+default: all
 
-include $(RTE_SDK)/mk/rte.sharelib.mk
-include $(RTE_SDK)/mk/rte.subdir.mk
\ No newline at end of file
+ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y)
+EXT:=.so
+else
+EXT:=.a
+endif
+
+RTE_LIBNAME := dpdk
+COMBINEDLIB := lib$(RTE_LIBNAME)$(EXT)
+
+LIBS := $(notdir $(wildcard $(RTE_OUTPUT)/lib/*$(EXT)))
+
+all: FORCE
+	$(Q)echo "GROUP ( $(LIBS) )" > $(RTE_OUTPUT)/lib/$(COMBINEDLIB)
+
+#
+# Clean all generated files
+#
+.PHONY: clean
+clean:
+	$(Q)rm -f $(RTE_OUTPUT)/lib/$(COMBINEDLIB)
+
+.PHONY: FORCE
+FORCE:
diff --git a/mk/rte.lib.mk b/mk/rte.lib.mk
index 24c81e7..8f7e021 100644
--- a/mk/rte.lib.mk
+++ b/mk/rte.lib.mk
@@ -138,14 +138,6 @@ endif
 		$(depfile_newer)),\
 		$(O_TO_S_DO))
 
-ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS)$(EXTLIB_BUILD),yn)
-	$(if $(or \
-        $(file_missing),\
-        $(call cmdline_changed,$(O_TO_C_STR)),\
-        $(depfile_missing),\
-        $(depfile_newer)),\
-        $(O_TO_C_DO))
-endif
 else
 $(LIB): $(OBJS-y) $(DEP_$(LIB)) FORCE
 	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
@@ -161,14 +153,6 @@ $(LIB): $(OBJS-y) $(DEP_$(LIB)) FORCE
 	    $(depfile_missing),\
 	    $(depfile_newer)),\
 	    $(O_TO_A_DO))
-ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS)$(EXTLIB_BUILD),yn)
-	$(if $(or \
-        $(file_missing),\
-        $(call cmdline_changed,$(O_TO_C_STR)),\
-        $(depfile_missing),\
-        $(depfile_newer)),\
-        $(O_TO_C_DO))
-endif
 endif
 
 #
diff --git a/mk/rte.sdkbuild.mk b/mk/rte.sdkbuild.mk
index 85f603c..eec5241 100644
--- a/mk/rte.sdkbuild.mk
+++ b/mk/rte.sdkbuild.mk
@@ -77,8 +77,8 @@ $(ROOTDIRS-y):
 	@[ -d $(BUILDDIR)/$@ ] || mkdir -p $(BUILDDIR)/$@
 	@echo "== Build $@"
 	$(Q)$(MAKE) S=$@ -f $(RTE_SRCDIR)/$@/Makefile -C $(BUILDDIR)/$@ all
-	@if [ $@ = drivers -a $(CONFIG_RTE_BUILD_COMBINE_LIBS) = y ]; then \
-		$(MAKE) -f $(RTE_SDK)/lib/Makefile sharelib; \
+	@if [ $@ = drivers ]; then \
+		$(MAKE) -f $(RTE_SDK)/mk/rte.combinedlib.mk; \
 	fi
 
 %_clean:
diff --git a/mk/rte.sharelib.mk b/mk/rte.sharelib.mk
deleted file mode 100644
index 70c49a8..0000000
--- a/mk/rte.sharelib.mk
+++ /dev/null
@@ -1,105 +0,0 @@
-#   BSD LICENSE
-#
-#   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
-#   All rights reserved.
-#
-#   Redistribution and use in source and binary forms, with or without
-#   modification, are permitted provided that the following conditions
-#   are met:
-#
-#     * Redistributions of source code must retain the above copyright
-#       notice, this list of conditions and the following disclaimer.
-#     * Redistributions in binary form must reproduce the above copyright
-#       notice, this list of conditions and the following disclaimer in
-#       the documentation and/or other materials provided with the
-#       distribution.
-#     * Neither the name of Intel Corporation nor the names of its
-#       contributors may be used to endorse or promote products derived
-#       from this software without specific prior written permission.
-#
-#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-include $(RTE_SDK)/mk/internal/rte.build-pre.mk
-
-# VPATH contains at least SRCDIR
-VPATH += $(SRCDIR)
-
-ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS),y)
-ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y)
-LIB_ONE := lib$(RTE_LIBNAME).so
-else
-LIB_ONE := lib$(RTE_LIBNAME).a
-endif
-COMBINED_MAP=$(BUILDDIR)/lib/libdpdk.map
-COMBINED_LDFLAGS += --version-script=$(COMBINED_MAP)
-endif
-
-.PHONY:sharelib
-sharelib: $(LIB_ONE) FORCE
-
-OBJS = $(wildcard $(RTE_OUTPUT)/build/lib/*.o)
-
-ifeq ($(LINK_USING_CC),1)
-# Override the definition of LD here, since we're linking with CC
-LD := $(CC) $(CPU_CFLAGS)
-O_TO_S = $(LD) $(call linkerprefix,$(CPU_LDFLAGS)) \
-	 $(call linkerprefix,$(COMBINED_LDFLAGS)) \
-	-shared $(OBJS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
-else
-O_TO_S = $(LD) $(CPU_LDFLAGS) $(COMBINED_LDFLAGS) \
-	-shared $(OBJS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
-endif
-
-O_TO_S_STR = $(subst ','\'',$(O_TO_S)) #'# fix syntax highlight
-O_TO_S_DISP = $(if $(V),"$(O_TO_S_STR)","  LD $(@)")
-O_TO_S_CMD = "cmd_$@ = $(O_TO_S_STR)"
-O_TO_S_DO = @set -e; \
-    echo $(O_TO_S_DISP); \
-    $(O_TO_S)
-
-O_TO_A =  $(AR) crus $(RTE_OUTPUT)/lib/$(LIB_ONE) $(OBJS)
-O_TO_A_STR = $(subst ','\'',$(O_TO_A)) #'# fix syntax highlight
-O_TO_A_DISP = $(if $(V),"$(O_TO_A_STR)","  LD $(@)")
-O_TO_A_CMD = "cmd_$@ = $(O_TO_A_STR)"
-O_TO_A_DO = @set -e; \
-    echo $(O_TO_A_DISP); \
-    $(O_TO_A)
-#
-# Archive objects to share library
-#
-
-ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS),y)
-ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y)
-$(LIB_ONE): FORCE
-	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
-	@$(SRCDIR)/scripts/merge-maps.sh > $(COMBINED_MAP)
-	$(O_TO_S_DO)
-else
-$(LIB_ONE): FORCE
-	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
-	$(O_TO_A_DO)
-endif
-endif
-
-#
-# Clean all generated files
-#
-.PHONY: clean
-clean: _postclean
-
-.PHONY: doclean
-doclean:
-	$(Q)rm -rf $(LIB_ONE)
-
-.PHONY: FORCE
-FORCE:
diff --git a/mk/rte.vars.mk b/mk/rte.vars.mk
index 7e7ee14..2d734bd 100644
--- a/mk/rte.vars.mk
+++ b/mk/rte.vars.mk
@@ -66,8 +66,6 @@ endif
 
 RTE_TARGET ?= $(RTE_ARCH)-$(RTE_MACHINE)-$(RTE_EXEC_ENV)-$(RTE_TOOLCHAIN)
 
-RTE_LIBNAME := dpdk
-
 ifeq ($(BUILDING_RTE_SDK),)
 # if we are building an external app/lib, include internal/rte.extvars.mk that will
 # define RTE_OUTPUT, RTE_SRCDIR, RTE_EXTMK, RTE_SDK_BIN, (etc ...)
diff --git a/scripts/test-build.sh b/scripts/test-build.sh
index 6d28c5d..ead7b47 100755
--- a/scripts/test-build.sh
+++ b/scripts/test-build.sh
@@ -55,7 +55,7 @@ print_help () {
 	        -s    short test with only first config without examples/doc
 
 	config: defconfig name followed by switches delimited with "+" sign
-	        Example: x86_64-native-linuxapp-gcc+next+shared+combined
+	        Example: x86_64-native-linuxapp-gcc+next+shared
 	        Default is to enable most of the options.
 	        The external dependencies are setup with DPDK_DEP_* variables.
 	END_OF_HELP
@@ -101,8 +101,6 @@ config () # <directory> <target> <options>
 		sed -ri           's,(NEXT_ABI=)y,\1n,' $1/.config
 		! echo $3 | grep -q shared || \
 		sed -ri         's,(SHARED_LIB=)n,\1y,' $1/.config
-		! echo $3 | grep -q combined || \
-		sed -ri       's,(COMBINE_LIBS=)n,\1y,' $1/.config
 		echo $2 | grep -q '^i686' || \
 		sed -ri               's,(NUMA=)n,\1y,' $1/.config
 		sed -ri         's,(PCI_CONFIG=)n,\1y,' $1/.config
@@ -110,7 +108,6 @@ config () # <directory> <target> <options>
 		sed -ri             's,(BYPASS=)n,\1y,' $1/.config
 		test "$DPDK_DEP_MOFED" != y || \
 		echo $2 | grep -q '^clang$' || \
-		echo $3 | grep -q 'shared.*combined' || \
 		sed -ri           's,(MLX._PMD=)n,\1y,' $1/.config
 		test "$DPDK_DEP_SZE" != y || \
 		echo $2 | grep -q '^i686' || \
@@ -122,11 +119,9 @@ config () # <directory> <target> <options>
 		sed -ri               's,(PCAP=)n,\1y,' $1/.config
 		test -z "$AESNI_MULTI_BUFFER_LIB_PATH" || \
 		echo $2 | grep -q '^i686' || \
-		echo $3 | grep -q 'shared.*combined' || \
 		sed -ri       's,(PMD_AESNI_MB=)n,\1y,' $1/.config
 		test "$DPDK_DEP_SSL" != y || \
 		echo $2 | grep -q '^i686' || \
-		echo $3 | grep -q 'shared.*combined' || \
 		sed -ri            's,(PMD_QAT=)n,\1y,' $1/.config
 		sed -ri        's,(KNI_VHOST.*=)n,\1y,' $1/.config
 		sed -ri           's,(SCHED_.*=)n,\1y,' $1/.config
-- 
2.7.0

Re: [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Panu Matilainen]

On 02/23/2016 10:07 PM, Thomas Monjalon wrote:
> Hi,
>
> I'm reviving this old thread.

Thanks.

> My understanding is that everybody prefer the linker script
> than the current combined library which had neither symbol versioning
> nor library dependency informations.

Yeah it seemed to me most (if not everybody) had converged on the side 
of the linker script approach.

>
> Comments below:
>
> 2015-11-24 16:31, Panu Matilainen:
>> The physically linked-together combined library has been an increasing
>> source of problems, as was predicted when library and symbol versioning
>> was introduced. Replace the complex and fragile construction with a
>> simple linker script which achieves the same without all the problems,
>> remove the related kludges from eg mlx drivers.
>>
>> Since creating the linker script is practically zero cost, remove the
>> config option and just create it always.
> [...]
>> --- /dev/null
>> +++ b/mk/rte.combinedlib.mk
>> @@ -0,0 +1,57 @@
>> +#   BSD LICENSE
>> +#
>> +#   Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
>> +#   All rights reserved.
>> +#
>> +#   Redistribution and use in source and binary forms, with or without
>> +#   modification, are permitted provided that the following conditions
>> +#   are met:
>> +#
>> +#     * Redistributions of source code must retain the above copyright
>> +#       notice, this list of conditions and the following disclaimer.
>> +#     * Redistributions in binary form must reproduce the above copyright
>> +#       notice, this list of conditions and the following disclaimer in
>> +#       the documentation and/or other materials provided with the
>> +#       distribution.
>> +#     * Neither the name of Intel Corporation nor the names of its
>> +#       contributors may be used to endorse or promote products derived
>> +#       from this software without specific prior written permission.
>
> Why this header, Panu?
> I think you should write your own copyright, and assume the linker script ;)

Its just inherited from the original patch by Sergio. As he's the actual 
author here, it didn't seem appropriate for me to remove it.

>
> It needs to be rebased and some docs comments must be removed or updated.
> I'll send a v2.
>

Thanks,

	- Panu -

Re: [PATCH v2] mk: replace the combined library with a linker script

[Thomas Monjalon]

ping
I would like to be sure nothing is forgotten in this new revision.

2016-02-23 23:20, Thomas Monjalon:
> From: Panu Matilainen <pmatilai@redhat.com>
> 
> The physically linked-together combined library has been an increasing
> source of problems, as was predicted when library and symbol versioning
> was introduced. Replace the complex and fragile construction with a
> simple linker script which achieves the same without all the problems,
> remove the related kludges from eg mlx drivers.
> 
> Since creating the linker script is practically zero cost, remove the
> config option and just create it always.
> 
> Based on a patch by Sergio Gonzales Monroy, linker script approach
> initially suggested by Neil Horman.
> 
> Suggested-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
> Suggested-by: Neil Horman <nhorman@tuxdriver.com>
> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
> Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
> ---
> v2:
> - move RTE_LIBNAME assignment rte.vars.mk to rte.combinedlib.mk
> - update crypto
> - update doc
> - update rte.app.mk
> - update test-build.sh

Re: [PATCH v2] mk: replace the combined library with a linker script

[Panu Matilainen]

On 03/01/2016 03:40 PM, Thomas Monjalon wrote:
> ping
> I would like to be sure nothing is forgotten in this new revision.

Sorry, didn't realize you were waiting for input from me, it feels a bit 
strange to comment on something supposedly coming from myself :)

> 2016-02-23 23:20, Thomas Monjalon:
>> From: Panu Matilainen <pmatilai@redhat.com>
>>
>> The physically linked-together combined library has been an increasing
>> source of problems, as was predicted when library and symbol versioning
>> was introduced. Replace the complex and fragile construction with a
>> simple linker script which achieves the same without all the problems,
>> remove the related kludges from eg mlx drivers.
>>
>> Since creating the linker script is practically zero cost, remove the
>> config option and just create it always.
>>
>> Based on a patch by Sergio Gonzales Monroy, linker script approach
>> initially suggested by Neil Horman.
>>
>> Suggested-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
>> Suggested-by: Neil Horman <nhorman@tuxdriver.com>
>> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
>> Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
>> ---
>> v2:
>> - move RTE_LIBNAME assignment rte.vars.mk to rte.combinedlib.mk
>> - update crypto
>> - update doc
>> - update rte.app.mk
>> - update test-build.sh
>

Briefly tested, gets generated and installed as it should etc - looks 
good to me.

	- Panu -

Re: [PATCH v2] mk: replace the combined library with a linker script

[Panu Matilainen]

On 03/01/2016 04:48 PM, Panu Matilainen wrote:
> On 03/01/2016 03:40 PM, Thomas Monjalon wrote:
>> ping
>> I would like to be sure nothing is forgotten in this new revision.
>
> Sorry, didn't realize you were waiting for input from me, it feels a bit
> strange to comment on something supposedly coming from myself :)
>
>> 2016-02-23 23:20, Thomas Monjalon:
>>> From: Panu Matilainen <pmatilai@redhat.com>
>>>
>>> The physically linked-together combined library has been an increasing
>>> source of problems, as was predicted when library and symbol versioning
>>> was introduced. Replace the complex and fragile construction with a
>>> simple linker script which achieves the same without all the problems,
>>> remove the related kludges from eg mlx drivers.
>>>
>>> Since creating the linker script is practically zero cost, remove the
>>> config option and just create it always.
>>>
>>> Based on a patch by Sergio Gonzales Monroy, linker script approach
>>> initially suggested by Neil Horman.
>>>
>>> Suggested-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
>>> Suggested-by: Neil Horman <nhorman@tuxdriver.com>
>>> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
>>> Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
>>> ---
>>> v2:
>>> - move RTE_LIBNAME assignment rte.vars.mk to rte.combinedlib.mk
>>> - update crypto
>>> - update doc
>>> - update rte.app.mk
>>> - update test-build.sh
>>
>
> Briefly tested, gets generated and installed as it should etc - looks
> good to me.

Forgot to note that the patch doesn't apply anymore because of 
scripts/test-build.sh changes, so it needs a rebase. Want me to send a 
v3 or will you handle it when committing?

On a related note, if this is about to go in then I'd rather have it 
sooner than later because it also conflicts with the LDLIBS fixing 
that's been slowly going on for months and months but been on hold 
lately, partly because of this hangup.

	- Panu -

Re: [PATCH v2] mk: replace the combined library with a linker script

[Thomas Monjalon]

2016-03-02 14:30, Panu Matilainen:
> On 03/01/2016 04:48 PM, Panu Matilainen wrote:
> > On 03/01/2016 03:40 PM, Thomas Monjalon wrote:
> >> ping
> >> I would like to be sure nothing is forgotten in this new revision.
> >
> > Sorry, didn't realize you were waiting for input from me, it feels a bit
> > strange to comment on something supposedly coming from myself :)
> >
> >> 2016-02-23 23:20, Thomas Monjalon:
> >>> From: Panu Matilainen <pmatilai@redhat.com>
> >>>
> >>> The physically linked-together combined library has been an increasing
> >>> source of problems, as was predicted when library and symbol versioning
> >>> was introduced. Replace the complex and fragile construction with a
> >>> simple linker script which achieves the same without all the problems,
> >>> remove the related kludges from eg mlx drivers.
> >>>
> >>> Since creating the linker script is practically zero cost, remove the
> >>> config option and just create it always.
> >>>
> >>> Based on a patch by Sergio Gonzales Monroy, linker script approach
> >>> initially suggested by Neil Horman.
> >>>
> >>> Suggested-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
> >>> Suggested-by: Neil Horman <nhorman@tuxdriver.com>
> >>> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
> >>> Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
> >>> ---
> >>> v2:
> >>> - move RTE_LIBNAME assignment rte.vars.mk to rte.combinedlib.mk
> >>> - update crypto
> >>> - update doc
> >>> - update rte.app.mk
> >>> - update test-build.sh
> >>
> >
> > Briefly tested, gets generated and installed as it should etc - looks
> > good to me.
> 
> Forgot to note that the patch doesn't apply anymore because of 
> scripts/test-build.sh changes, so it needs a rebase. Want me to send a 
> v3 or will you handle it when committing?
> 
> On a related note, if this is about to go in then I'd rather have it 
> sooner than later because it also conflicts with the LDLIBS fixing 
> that's been slowly going on for months and months but been on hold 
> lately, partly because of this hangup.

Applied, thanks

Re: [PATCH v2] mk: replace the combined library with a linker script

[Panu Matilainen]

On 03/02/2016 02:40 PM, Thomas Monjalon wrote:
> 2016-03-02 14:30, Panu Matilainen:
>> On 03/01/2016 04:48 PM, Panu Matilainen wrote:
>>> On 03/01/2016 03:40 PM, Thomas Monjalon wrote:
>>>> ping
>>>> I would like to be sure nothing is forgotten in this new revision.
>>>
>>> Sorry, didn't realize you were waiting for input from me, it feels a bit
>>> strange to comment on something supposedly coming from myself :)
>>>
>>>> 2016-02-23 23:20, Thomas Monjalon:
>>>>> From: Panu Matilainen <pmatilai@redhat.com>
>>>>>
>>>>> The physically linked-together combined library has been an increasing
>>>>> source of problems, as was predicted when library and symbol versioning
>>>>> was introduced. Replace the complex and fragile construction with a
>>>>> simple linker script which achieves the same without all the problems,
>>>>> remove the related kludges from eg mlx drivers.
>>>>>
>>>>> Since creating the linker script is practically zero cost, remove the
>>>>> config option and just create it always.
>>>>>
>>>>> Based on a patch by Sergio Gonzales Monroy, linker script approach
>>>>> initially suggested by Neil Horman.
>>>>>
>>>>> Suggested-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
>>>>> Suggested-by: Neil Horman <nhorman@tuxdriver.com>
>>>>> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
>>>>> Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
>>>>> ---
>>>>> v2:
>>>>> - move RTE_LIBNAME assignment rte.vars.mk to rte.combinedlib.mk
>>>>> - update crypto
>>>>> - update doc
>>>>> - update rte.app.mk
>>>>> - update test-build.sh
>>>>
>>>
>>> Briefly tested, gets generated and installed as it should etc - looks
>>> good to me.
>>
>> Forgot to note that the patch doesn't apply anymore because of
>> scripts/test-build.sh changes, so it needs a rebase. Want me to send a
>> v3 or will you handle it when committing?
>>
>> On a related note, if this is about to go in then I'd rather have it
>> sooner than later because it also conflicts with the LDLIBS fixing
>> that's been slowly going on for months and months but been on hold
>> lately, partly because of this hangup.
>
> Applied, thanks
>

Awesome, thank you! :)

	- Panu -