[PATCH 0/4] Fix build issues with CONFIG_RTE_BUILD_COMBINE_LIBS=y

[PATCH 0/4] Fix build issues with CONFIG_RTE_BUILD_COMBINE_LIBS=y

[Sergio Gonzalez Monroy]

When building DPDK with CONFIG_RTE_BUILD_COMBINE_LIBS=y, the result is not
the expected behavior.

 - It does link the combine library using LD instead of CC which results
in application linking errors. 

 - It creates both individual libraries and combine library, then linking
applications against all of them.

This patch set aims to fix those issues.

The last patch 'cleanup', in my opinion, simplifies and removes duplication of
rules.
It is not required for fixing the issues mentioned above.

Sergio Gonzalez Monroy (4):
  Link combined shared library using CC
  Do not generate individual libs when configured with RTE_BUILD_COMBINE_LIBS=y
  Link apps only against combined lib or individual libs, not both
  Cleanup

 mk/rte.app.mk      | 13 +++++---
 mk/rte.lib.mk      | 90 +++++++++++++-----------------------------------------
 mk/rte.sharelib.mk | 47 ++++++++++++++--------------
 3 files changed, 54 insertions(+), 96 deletions(-)

Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

-- 
1.9.3

[PATCH 1/4] Link combined shared library using CC

[Sergio Gonzalez Monroy]

It adds override definition of LD when linking using CC.
Also it adds muldefs option to the linker when building shared libraries
(same as building individual libraries).

Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 mk/rte.sharelib.mk | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/mk/rte.sharelib.mk b/mk/rte.sharelib.mk
index c0a811a..942671d 100644
--- a/mk/rte.sharelib.mk
+++ b/mk/rte.sharelib.mk
@@ -40,12 +40,20 @@ LIB_ONE := lib$(RTE_LIBNAME).a
 endif
 endif
 
+ifeq ($(LINK_USING_CC),1)
+# Override the definition of LD here, since we're linking with CC
+LD := $(CC)
+LD_MULDEFS := $(call linkerprefix,-z$(comma)muldefs)
+CPU_LDFLAGS := $(call linkerprefix,$(CPU_LDFLAGS))
+endif
+
 .PHONY:sharelib
 sharelib: $(LIB_ONE) FORCE
 
 OBJS = $(wildcard $(RTE_OUTPUT)/build/lib/*.o)
 
-O_TO_S = $(LD) $(CPU_LDFLAGS) -shared $(OBJS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
+O_TO_S = $(LD) $(CPU_LDFLAGS) $(LD_MULDEFS) -shared $(OBJS)\
+	-o $(RTE_OUTPUT)/lib/$(LIB_ONE)
 O_TO_S_STR = $(subst ','\'',$(O_TO_S)) #'# fix syntax highlight
 O_TO_S_DISP = $(if $(V),"$(O_TO_S_STR)","  LD $(@)")
 O_TO_S_CMD = "cmd_$@ = $(O_TO_S_STR)"
-- 
1.9.3

[PATCH 2/4] Do not generate individual libs when configured with RTE_BUILD_COMBINE_LIBS=y

[Sergio Gonzalez Monroy]

When RTE_BUILD_COMBINE_LIBS=y is configured, there won't be individual shared
libraries to copy over.

Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 mk/rte.lib.mk | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/mk/rte.lib.mk b/mk/rte.lib.mk
index f458258..d594692 100644
--- a/mk/rte.lib.mk
+++ b/mk/rte.lib.mk
@@ -158,9 +158,11 @@ endif
 # install lib in $(RTE_OUTPUT)/lib
 #
 $(RTE_OUTPUT)/lib/$(LIB): $(LIB)
+ifneq ($(RTE_BUILD_COMBINE_LIBS),y)
 	@echo "  INSTALL-LIB $(LIB)"
 	@[ -d $(RTE_OUTPUT)/lib ] || mkdir -p $(RTE_OUTPUT)/lib
 	$(Q)cp -f $(LIB) $(RTE_OUTPUT)/lib
+endif
 
 #
 # Clean all generated files
-- 
1.9.3

Re: [PATCH 2/4] Do not generate individual libs when configured with RTE_BUILD_COMBINE_LIBS=y

[Matthew Hall]

On Thu, Oct 02, 2014 at 04:56:24PM +0100, Sergio Gonzalez Monroy wrote:
> When RTE_BUILD_COMBINE_LIBS=y is configured, there won't be individual shared
> libraries to copy over.

As a user of RTE_BUILD_COMBINE_LIBS, disabling the ability to use the 
individual libs after the option is enabled is not helpful. It would be more 
helpful if all libs are kept available, so when working with multiple DPDK 
apps which might prefer combined or uncombined, everything will still work.

Thanks,
Matthew.

[PATCH 3/4] Link apps only against combined lib or individual libs, not both

[Sergio Gonzalez Monroy]

Link only against combined library or individual libraries.

Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 mk/rte.app.mk | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 34dff2a..6f752dd 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -59,6 +59,13 @@ LDLIBS += -L$(RTE_SDK_BIN)/lib
 #
 ifeq ($(NO_AUTOLIBS),)
 
+ifeq ($(RTE_BUILD_COMBINE_LIBS),y)
+LDLIBS += --start-group
+LDLIBS += -l$(RTE_LIBNAME)
+LDLIBS += $(EXECENV_LDLIBS)
+LDLIBS += --end-group
+else
+
 LDLIBS += --whole-archive
 
 ifeq ($(CONFIG_RTE_LIBRTE_DISTRIBUTOR),y)
@@ -218,6 +225,8 @@ LDLIBS += --end-group
 
 LDLIBS += --no-whole-archive
 
+endif # ifeq ($(RTE_BUILD_COMBINE_LIBS),y)
+
 endif # ifeq ($(NO_AUTOLIBS),)
 
 LDLIBS += $(CPU_LDLIBS)
@@ -235,10 +244,6 @@ build: _postbuild
 
 exe2cmd = $(strip $(call dotfile,$(patsubst %,%.cmd,$(1))))
 
-ifeq ($(RTE_BUILD_COMBINE_LIBS),y)
-LDLIBS += -l$(RTE_LIBNAME)
-endif
-
 ifeq ($(LINK_USING_CC),1)
 LDLIBS := $(call linkerprefix,$(LDLIBS))
 LDFLAGS := $(call linkerprefix,$(LDFLAGS))
-- 
1.9.3

[PATCH 4/4] Cleanup

[Sergio Gonzalez Monroy]

Reduce code duplication.
This patch is not required for the patch set to work.

Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 mk/rte.lib.mk      | 88 +++++++++++++-----------------------------------------
 mk/rte.sharelib.mk | 39 ++++++++++--------------
 2 files changed, 35 insertions(+), 92 deletions(-)

diff --git a/mk/rte.lib.mk b/mk/rte.lib.mk
index d594692..84f5a64 100644
--- a/mk/rte.lib.mk
+++ b/mk/rte.lib.mk
@@ -66,93 +66,45 @@ LD_MULDEFS := $(call linkerprefix,-z$(comma)muldefs)
 CPU_LDFLAGS := $(call linkerprefix,$(CPU_LDFLAGS))
 endif
 
-O_TO_A = $(AR) crus $(LIB) $(OBJS-y)
-O_TO_A_STR = $(subst ','\'',$(O_TO_A)) #'# fix syntax highlight
-O_TO_A_DISP = $(if $(V),"$(O_TO_A_STR)","  AR $(@)")
-O_TO_A_CMD = "cmd_$@ = $(O_TO_A_STR)"
-O_TO_A_DO = @set -e; \
-	echo $(O_TO_A_DISP); \
-	$(O_TO_A) && \
-	echo $(O_TO_A_CMD) > $(call exe2cmd,$(@))
-
-O_TO_S = $(LD) $(CPU_LDFLAGS) $(LD_MULDEFS) -shared $(OBJS-y) -o $(LIB)
-O_TO_S_STR = $(subst ','\'',$(O_TO_S)) #'# fix syntax highlight
-O_TO_S_DISP = $(if $(V),"$(O_TO_S_STR)","  LD $(@)")
-O_TO_S_DO = @set -e; \
-	echo $(O_TO_S_DISP); \
-	$(O_TO_S) && \
-	echo $(O_TO_S_CMD) > $(call exe2cmd,$(@))
-
-ifeq ($(RTE_BUILD_SHARED_LIB),n)
-O_TO_C = $(AR) crus $(LIB_ONE) $(OBJS-y)
-O_TO_C_STR = $(subst ','\'',$(O_TO_C)) #'# fix syntax highlight
-O_TO_C_DISP = $(if $(V),"$(O_TO_C_STR)","  AR_C $(@)")
-O_TO_C_DO = @set -e; \
-	$(lib_dir) \
-	$(copy_obj)
+ifeq ($(RTE_BUILD_COMBINE_LIBS),y)
+O_TO_L_DO = @set -e; \
+	cp -f $(OBJS-y) $(RTE_OUTPUT)/build/lib;
+else
+ifeq ($(RTE_BUILD_SHARED_LIB),y)
+O_TO_L = $(LD) $(CPU_LDFLAGS) $(LD_MULDEFS) -shared $(OBJS-y) -o $(LIB)
+L_DISP=LD
 else
-O_TO_C = $(LD) $(LD_MULDEFS) -shared $(OBJS-y) -o $(LIB_ONE)
-O_TO_C_STR = $(subst ','\'',$(O_TO_C)) #'# fix syntax highlight
-O_TO_C_DISP = $(if $(V),"$(O_TO_C_STR)","  LD_C $(@)")
-O_TO_C_DO = @set -e; \
-	$(lib_dir) \
-	$(copy_obj)
+O_TO_L = $(AR) crus $(LIB) $(OBJS-y)
+L_DISP=AR
+endif
+O_TO_L_STR = $(subst ','\'',$(O_TO_L)) #') # fix syntax highlight
+O_TO_L_DISP = $(if $(V),"$(O_TO_L_STR)","  $(L_DISP) $(@)")
+O_TO_L_CMD = "cmd_$@ = $(O_TO_L_STR)"
+O_TO_L_DO = @set -e; \
+	echo $(O_TO_L_DISP); \
+	$(O_TO_L) && \
+	echo $(O_TO_L_CMD) > $(call exe2cmd,$(@))
 endif
 
-copy_obj = cp -f $(OBJS-y) $(RTE_OUTPUT)/build/lib;
-lib_dir = [ -d $(RTE_OUTPUT)/lib ] || mkdir -p $(RTE_OUTPUT)/lib;
 -include .$(LIB).cmd
 
 #
 # Archive objects in .a file if needed
 #
-ifeq ($(RTE_BUILD_SHARED_LIB),y)
 $(LIB): $(OBJS-y) $(DEP_$(LIB)) FORCE
 	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
 	$(if $(D),\
 		@echo -n "$< -> $@ " ; \
 		echo -n "file_missing=$(call boolean,$(file_missing)) " ; \
-		echo -n "cmdline_changed=$(call boolean,$(call cmdline_changed,$(O_TO_S_STR))) " ; \
+		echo -n "cmdline_changed=$(call boolean,$(call cmdline_changed,$(O_TO_L_STR))) " ; \
 		echo -n "depfile_missing=$(call boolean,$(depfile_missing)) " ; \
 		echo "depfile_newer=$(call boolean,$(depfile_newer)) ")
 	$(if $(or \
-		$(file_missing),\
-		$(call cmdline_changed,$(O_TO_S_STR)),\
-		$(depfile_missing),\
-		$(depfile_newer)),\
-		$(O_TO_S_DO))
-ifeq ($(RTE_BUILD_COMBINE_LIBS),y)
-	$(if $(or \
-        $(file_missing),\
-        $(call cmdline_changed,$(O_TO_C_STR)),\
-        $(depfile_missing),\
-        $(depfile_newer)),\
-        $(O_TO_C_DO))
-endif
-else
-$(LIB): $(OBJS-y) $(DEP_$(LIB)) FORCE
-	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
-	$(if $(D),\
-	    @echo -n "$< -> $@ " ; \
-	    echo -n "file_missing=$(call boolean,$(file_missing)) " ; \
-	    echo -n "cmdline_changed=$(call boolean,$(call cmdline_changed,$(O_TO_A_STR))) " ; \
-	    echo -n "depfile_missing=$(call boolean,$(depfile_missing)) " ; \
-	    echo "depfile_newer=$(call boolean,$(depfile_newer)) ")
-	$(if $(or \
 	    $(file_missing),\
-	    $(call cmdline_changed,$(O_TO_A_STR)),\
+	    $(call cmdline_changed,$(O_TO_L_STR)),\
 	    $(depfile_missing),\
 	    $(depfile_newer)),\
-	    $(O_TO_A_DO))
-ifeq ($(RTE_BUILD_COMBINE_LIBS),y)
-	$(if $(or \
-        $(file_missing),\
-        $(call cmdline_changed,$(O_TO_C_STR)),\
-        $(depfile_missing),\
-        $(depfile_newer)),\
-        $(O_TO_C_DO))
-endif
-endif
+	    $(O_TO_L_DO))
 
 #
 # install lib in $(RTE_OUTPUT)/lib
diff --git a/mk/rte.sharelib.mk b/mk/rte.sharelib.mk
index 942671d..0efa0b7 100644
--- a/mk/rte.sharelib.mk
+++ b/mk/rte.sharelib.mk
@@ -52,37 +52,28 @@ sharelib: $(LIB_ONE) FORCE
 
 OBJS = $(wildcard $(RTE_OUTPUT)/build/lib/*.o)
 
-O_TO_S = $(LD) $(CPU_LDFLAGS) $(LD_MULDEFS) -shared $(OBJS)\
+ifeq ($(RTE_BUILD_SHARED_LIB),y)
+O_TO_L = $(LD) $(CPU_LDFLAGS) $(LD_MULDEFS) -shared $(OBJS)\
 	-o $(RTE_OUTPUT)/lib/$(LIB_ONE)
-O_TO_S_STR = $(subst ','\'',$(O_TO_S)) #'# fix syntax highlight
-O_TO_S_DISP = $(if $(V),"$(O_TO_S_STR)","  LD $(@)")
-O_TO_S_CMD = "cmd_$@ = $(O_TO_S_STR)"
-O_TO_S_DO = @set -e; \
-    echo $(O_TO_S_DISP); \
-    $(O_TO_S)
+L_DISP=LD
+else
+O_TO_L = $(AR) crus $(RTE_OUTPUT)/lib/$(LIB_ONE) $(OBJS)
+L_DISP=AR
+endif
+O_TO_L_STR = $(subst ','\'',$(O_TO_L)) #')# fix syntax highlight
+O_TO_L_DISP = $(if $(V),"$(O_TO_L_STR)","  $(L_DISP) $(@)")
+O_TO_L_CMD = "cmd_$@ = $(O_TO_L_STR)"
+O_TO_L_DO = @set -e; \
+	mkdir -p $(RTE_OUTPUT)/lib; \
+	echo $(O_TO_L_DISP); \
+	$(O_TO_L)
 
-O_TO_A =  $(AR) crus $(RTE_OUTPUT)/lib/$(LIB_ONE) $(OBJS)
-O_TO_A_STR = $(subst ','\'',$(O_TO_A)) #'# fix syntax highlight
-O_TO_A_DISP = $(if $(V),"$(O_TO_A_STR)","  LD $(@)")
-O_TO_A_CMD = "cmd_$@ = $(O_TO_A_STR)"
-O_TO_A_DO = @set -e; \
-    echo $(O_TO_A_DISP); \
-    $(O_TO_A)
 #
 # Archive objects to share library
 #
 
-ifeq ($(RTE_BUILD_COMBINE_LIBS),y)
-ifeq ($(RTE_BUILD_SHARED_LIB),y)
 $(LIB_ONE): FORCE
-	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
-	$(O_TO_S_DO)
-else
-$(LIB_ONE): FORCE
-	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
-	$(O_TO_A_DO)
-endif
-endif
+	$(O_TO_L_DO)
 
 #
 # Clean all generated files
-- 
1.9.3

Re: [PATCH 0/4] Fix build issues with CONFIG_RTE_BUILD_COMBINE_LIBS=y

[Neil Horman]

On Thu, Oct 02, 2014 at 04:56:22PM +0100, Sergio Gonzalez Monroy wrote:
> When building DPDK with CONFIG_RTE_BUILD_COMBINE_LIBS=y, the result is not
> the expected behavior.
> 
>  - It does link the combine library using LD instead of CC which results
> in application linking errors. 
> 
>  - It creates both individual libraries and combine library, then linking
> applications against all of them.
> 
> This patch set aims to fix those issues.
> 
> The last patch 'cleanup', in my opinion, simplifies and removes duplication of
> rules.
> It is not required for fixing the issues mentioned above.
> 
> Sergio Gonzalez Monroy (4):
>   Link combined shared library using CC
>   Do not generate individual libs when configured with RTE_BUILD_COMBINE_LIBS=y
>   Link apps only against combined lib or individual libs, not both
>   Cleanup
> 
>  mk/rte.app.mk      | 13 +++++---
>  mk/rte.lib.mk      | 90 +++++++++++++-----------------------------------------
>  mk/rte.sharelib.mk | 47 ++++++++++++++--------------
>  3 files changed, 54 insertions(+), 96 deletions(-)
> 
> Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> 
> -- 
> 1.9.3
> 
> 

Just out of curiosity, whats the impetus behind a single shared library here?
Is it just to ease application linking operations?  If so, it almost seems to me
that we should abandon the individual linking method and just use this as the
default output (and do simmilarly for the static linking build)

Neil

Re: [PATCH 0/4] Fix build issues with CONFIG_RTE_BUILD_COMBINE_LIBS=y

[Matthew Hall]

On Thu, Oct 02, 2014 at 01:26:34PM -0400, Neil Horman wrote:
> Just out of curiosity, whats the impetus behind a single shared library here?
> Is it just to ease application linking operations?  If so, it almost seems to me
> that we should abandon the individual linking method and just use this as the
> default output (and do simmilarly for the static linking build)
> 
> Neil

Not clear if you wrote "single shared library" on purpose instead of "single 
static library". But for me the objective of COMBINE_LIBS usage would be 
getting a "single static library" for my app, which just works, and eliminates 
need of start-group, end-group, weird library ordering issues, etc. I'm not 
interested personally in a "shared library" because it'd run slower.

Personally my preference would be to do both the single libs and multiple libs 
in static format by default. Disk space is cheap, let's maximize user freedom 
and flexibility. But shared lib, since it performs less well, should be 
discouraged by default, although allowed if needed... some people prefer it 
because it's easier to patch security vulns if you can replace a buggy library 
for all the code on a system.

Matthew.

Re: [PATCH 0/4] Fix build issues with CONFIG_RTE_BUILD_COMBINE_LIBS=y

[Neil Horman]

On Thu, Oct 02, 2014 at 01:04:20PM -0700, Matthew Hall wrote:
> On Thu, Oct 02, 2014 at 01:26:34PM -0400, Neil Horman wrote:
> > Just out of curiosity, whats the impetus behind a single shared library here?
> > Is it just to ease application linking operations?  If so, it almost seems to me
> > that we should abandon the individual linking method and just use this as the
> > default output (and do simmilarly for the static linking build)
> > 
> > Neil
> 
> Not clear if you wrote "single shared library" on purpose instead of "single 
> static library". But for me the objective of COMBINE_LIBS usage would be 
> getting a "single static library" for my app, which just works, and eliminates 
> need of start-group, end-group, weird library ordering issues, etc. I'm not 
> interested personally in a "shared library" because it'd run slower.
> 
Actually I do need to revise my question, thank you.  you're right, doing a
single archive for static builds makes the most sense, because you wind up with
a static binary anyway, and as such, theres really no need for multiple dpdk
archives.  We should just create a single dpdk.a file and be done with it.

The shared libraries are a different story.  While at first it made sense to me
to merge them all, it actually doesn't because PMD's might be built
independently and shipped separate from the core library.  

> Personally my preference would be to do both the single libs and multiple libs 
> in static format by default. Disk space is cheap, let's maximize user freedom 
> and flexibility. But shared lib, since it performs less well, should be 
> discouraged by default, although allowed if needed... some people prefer it 
> because it's easier to patch security vulns if you can replace a buggy library 
> for all the code on a system.
> 
This seems somewhat irrelevant to the patch.  The default configuration is
already the way you want it to be, shared library performance is actually very
close to static performance, and yes, people can choose how they want to build.
Not sure what point your trying to make here.
Neil

> Matthew.
> 

Re: [PATCH 0/4] Fix build issues with CONFIG_RTE_BUILD_COMBINE_LIBS=y

[Matthew Hall]

On Thu, Oct 02, 2014 at 04:24:51PM -0400, Neil Horman wrote:
> This seems somewhat irrelevant to the patch.  The default configuration is
> already the way you want it to be, shared library performance is actually very
> close to static performance, and yes, people can choose how they want to build.
> Not sure what point your trying to make here.
> Neil

According to my reading, one of the subpatches threatened to disable the 
single static libs when the grouped one is built. If you are working with 
multiple DPDK apps, and one expects COMBINE_LIBS and the other does not, you 
can't use both with the same copy of DPDK anymore if that subpatch gets 
committed.

Matthew.

Re: [PATCH 0/4] Fix build issues with CONFIG_RTE_BUILD_COMBINE_LIBS=y

[Neil Horman]

On Thu, Oct 02, 2014 at 02:10:55PM -0700, Matthew Hall wrote:
> On Thu, Oct 02, 2014 at 04:24:51PM -0400, Neil Horman wrote:
> > This seems somewhat irrelevant to the patch.  The default configuration is
> > already the way you want it to be, shared library performance is actually very
> > close to static performance, and yes, people can choose how they want to build.
> > Not sure what point your trying to make here.
> > Neil
> 
> According to my reading, one of the subpatches threatened to disable the 
> single static libs when the grouped one is built. If you are working with 
> multiple DPDK apps, and one expects COMBINE_LIBS and the other does not, you 
> can't use both with the same copy of DPDK anymore if that subpatch gets 
> committed.
> 
Ah, sorry, I misread, and thought you were advoating for making static builds
the default over dynamic builds, which made no sense to me, as that was already
the case.

Neil

> Matthew.
> 

Re: [PATCH 0/4] Fix build issues with CONFIG_RTE_BUILD_COMBINE_LIBS=y

[Sergio Gonzalez Monroy]

On Thu, Oct 02, 2014 at 04:24:51PM -0400, Neil Horman wrote:
> On Thu, Oct 02, 2014 at 01:04:20PM -0700, Matthew Hall wrote:
> > On Thu, Oct 02, 2014 at 01:26:34PM -0400, Neil Horman wrote:
> > > Just out of curiosity, whats the impetus behind a single shared library here?
> > > Is it just to ease application linking operations?  If so, it almost seems to me
> > > that we should abandon the individual linking method and just use this as the
> > > default output (and do simmilarly for the static linking build)
> > > 
> > > Neil
> > 
> > Not clear if you wrote "single shared library" on purpose instead of "single 
> > static library". But for me the objective of COMBINE_LIBS usage would be 
> > getting a "single static library" for my app, which just works, and eliminates 
> > need of start-group, end-group, weird library ordering issues, etc. I'm not 
> > interested personally in a "shared library" because it'd run slower.
> > 
> Actually I do need to revise my question, thank you.  you're right, doing a
> single archive for static builds makes the most sense, because you wind up with
> a static binary anyway, and as such, theres really no need for multiple dpdk
> archives.  We should just create a single dpdk.a file and be done with it.
> 
> The shared libraries are a different story.  While at first it made sense to me
> to merge them all, it actually doesn't because PMD's might be built
> independently and shipped separate from the core library.  

Sorry Neil, could you elaborate a bit on why it would not make sense to have a 
single/combined shared library?

Sergio

> 
> > Personally my preference would be to do both the single libs and multiple libs 
> > in static format by default. Disk space is cheap, let's maximize user freedom 
> > and flexibility. But shared lib, since it performs less well, should be 
> > discouraged by default, although allowed if needed... some people prefer it 
> > because it's easier to patch security vulns if you can replace a buggy library 
> > for all the code on a system.
> > 
> This seems somewhat irrelevant to the patch.  The default configuration is
> already the way you want it to be, shared library performance is actually very
> close to static performance, and yes, people can choose how they want to build.
> Not sure what point your trying to make here.
> Neil
> 
> > Matthew.
> > 

Re: [PATCH 0/4] Fix build issues with CONFIG_RTE_BUILD_COMBINE_LIBS=y

[Neil Horman]

On Fri, Oct 03, 2014 at 11:31:10AM +0100, Sergio Gonzalez Monroy wrote:
> On Thu, Oct 02, 2014 at 04:24:51PM -0400, Neil Horman wrote:
> > On Thu, Oct 02, 2014 at 01:04:20PM -0700, Matthew Hall wrote:
> > > On Thu, Oct 02, 2014 at 01:26:34PM -0400, Neil Horman wrote:
> > > > Just out of curiosity, whats the impetus behind a single shared library here?
> > > > Is it just to ease application linking operations?  If so, it almost seems to me
> > > > that we should abandon the individual linking method and just use this as the
> > > > default output (and do simmilarly for the static linking build)
> > > > 
> > > > Neil
> > > 
> > > Not clear if you wrote "single shared library" on purpose instead of "single 
> > > static library". But for me the objective of COMBINE_LIBS usage would be 
> > > getting a "single static library" for my app, which just works, and eliminates 
> > > need of start-group, end-group, weird library ordering issues, etc. I'm not 
> > > interested personally in a "shared library" because it'd run slower.
> > > 
> > Actually I do need to revise my question, thank you.  you're right, doing a
> > single archive for static builds makes the most sense, because you wind up with
> > a static binary anyway, and as such, theres really no need for multiple dpdk
> > archives.  We should just create a single dpdk.a file and be done with it.
> > 
> > The shared libraries are a different story.  While at first it made sense to me
> > to merge them all, it actually doesn't because PMD's might be built
> > independently and shipped separate from the core library.  
> 
> Sorry Neil, could you elaborate a bit on why it would not make sense to have a 
> single/combined shared library?
> 
> Sergio
> 

Sorry, I wasn't trying to assert that it doesn't make sense to have a single
shared library, but rather I was trying to assert that we should only _ever_
build a single shared library.  I assert this because to create a single shared
library destroys part of the advantage that DSO's have. Namely that they allow
for selective hardware enablement in the field.  I.e. you can ship your pmd's
pacakged separately from your core, and not have to update your application (and
pull them in via the -d option on the command line).  If the dpdk builds
everything as a single binary, then you may end up needing to pull in pmds that
you don't yet need to support.

In short, my initial idea was dumb :)

Neil

> > 
> > > Personally my preference would be to do both the single libs and multiple libs 
> > > in static format by default. Disk space is cheap, let's maximize user freedom 
> > > and flexibility. But shared lib, since it performs less well, should be 
> > > discouraged by default, although allowed if needed... some people prefer it 
> > > because it's easier to patch security vulns if you can replace a buggy library 
> > > for all the code on a system.
> > > 
> > This seems somewhat irrelevant to the patch.  The default configuration is
> > already the way you want it to be, shared library performance is actually very
> > close to static performance, and yes, people can choose how they want to build.
> > Not sure what point your trying to make here.
> > Neil
> > 
> > > Matthew.
> > > 
> 

Re: [PATCH 0/4] Fix build issues with CONFIG_RTE_BUILD_COMBINE_LIBS=y

[Stephen Hemminger]

On Fri, 3 Oct 2014 07:28:33 -0400
Neil Horman <nhorman-2XuSBdqkA4R54TAoqtyWWQ@public.gmane.org> wrote:

>  I.e. you can ship your pmd's
> pacakged separately from your core

I was hoping only the application API would be "stable"
As we know from Linux kernel, internal API's will never remain stable.

Re: [PATCH 0/4] Fix build issues with CONFIG_RTE_BUILD_COMBINE_LIBS=y

[Neil Horman]

On Fri, Oct 03, 2014 at 04:52:40PM -0700, Stephen Hemminger wrote:
> On Fri, 3 Oct 2014 07:28:33 -0400
> Neil Horman <nhorman-2XuSBdqkA4R54TAoqtyWWQ@public.gmane.org> wrote:
> 
> >  I.e. you can ship your pmd's
> > pacakged separately from your core
> 
> I was hoping only the application API would be "stable"
> As we know from Linux kernel, internal API's will never remain stable.
> 

None of the API's are stable.  My only hope with the ABI series I've posted is
to keep the interfaces stable for a release beyond the next time they change, so
that application developers aren't consistently caught off guard if they don't
synchronize with the DPDK release schedule.

I know the kernel API's are constantly changing, but this isn't the kernel,
its a user space library.  Theres nothing that prevents a third party from
writing a pmd to interface to the ethdev library, which is no different from any
other user space library.  If the DPDK wants to get packaged like other
libraries in distributions, it needs to provide stability assurances like other
libraries.

Neil

Re: [PATCH 0/4] Fix build issues with CONFIG_RTE_BUILD_COMBINE_LIBS=y

[Thomas Monjalon]

2014-10-02 13:04, Matthew Hall:
> On Thu, Oct 02, 2014 at 01:26:34PM -0400, Neil Horman wrote:
> > Just out of curiosity, whats the impetus behind a single shared library here?
> > Is it just to ease application linking operations?  If so, it almost seems to me
> > that we should abandon the individual linking method and just use this as the
> > default output (and do simmilarly for the static linking build)
> 
> Not clear if you wrote "single shared library" on purpose instead of "single 
> static library". But for me the objective of COMBINE_LIBS usage would be 
> getting a "single static library" for my app, which just works, and eliminates 
> need of start-group, end-group, weird library ordering issues, etc. I'm not 
> interested personally in a "shared library" because it'd run slower.
> 
> Personally my preference would be to do both the single libs and multiple libs 
> in static format by default. Disk space is cheap, let's maximize user freedom 
> and flexibility. But shared lib, since it performs less well, should be 
> discouraged by default, although allowed if needed... some people prefer it 
> because it's easier to patch security vulns if you can replace a buggy library 
> for all the code on a system.

We need to simplify build options. So I'm fine to remove COMBINE_LIBS option
to always enable it.
About making only one single static library, I think it's a good idea if
it brings a real code simplification.

So the conclusion is to nack this patchset in favor of above changes.
Sergio, comments?

-- 
Thomas

Re: [PATCH 0/4] Fix build issues with CONFIG_RTE_BUILD_COMBINE_LIBS=y

[Sergio Gonzalez Monroy]

On Fri, Oct 03, 2014 at 09:15:20AM +0200, Thomas Monjalon wrote:
> 2014-10-02 13:04, Matthew Hall:
> > On Thu, Oct 02, 2014 at 01:26:34PM -0400, Neil Horman wrote:
> > > Just out of curiosity, whats the impetus behind a single shared library here?
> > > Is it just to ease application linking operations?  If so, it almost seems to me
> > > that we should abandon the individual linking method and just use this as the
> > > default output (and do simmilarly for the static linking build)
> > 
> > Not clear if you wrote "single shared library" on purpose instead of "single 
> > static library". But for me the objective of COMBINE_LIBS usage would be 
> > getting a "single static library" for my app, which just works, and eliminates 
> > need of start-group, end-group, weird library ordering issues, etc. I'm not 
> > interested personally in a "shared library" because it'd run slower.
> > 
> > Personally my preference would be to do both the single libs and multiple libs 
> > in static format by default. Disk space is cheap, let's maximize user freedom 
> > and flexibility. But shared lib, since it performs less well, should be 
> > discouraged by default, although allowed if needed... some people prefer it 
> > because it's easier to patch security vulns if you can replace a buggy library 
> > for all the code on a system.
> 
> We need to simplify build options. So I'm fine to remove COMBINE_LIBS option
> to always enable it.
> About making only one single static library, I think it's a good idea if
> it brings a real code simplification.
> 
> So the conclusion is to nack this patchset in favor of above changes.
> Sergio, comments?
> 

Frankly I did not think of users linking against single and combine lib for 
different apps.
I think If the goal is to simplify code then we should just provide one build
option, either single or combine. Personally, I do not have a preference.

So just to be clear, we would remove COMBINE_LIBS to always make a single combine
lib or to create both single and combine?
For the later option, would we be linking apps against single or combine libraries?

Sergio 

> -- 
> Thomas

Re: [PATCH 0/4] Fix build issues with CONFIG_RTE_BUILD_COMBINE_LIBS=y

[Thomas Monjalon]

2014-10-03 09:10, Sergio Gonzalez Monroy:
> On Fri, Oct 03, 2014 at 09:15:20AM +0200, Thomas Monjalon wrote:
> > 2014-10-02 13:04, Matthew Hall:
> > > On Thu, Oct 02, 2014 at 01:26:34PM -0400, Neil Horman wrote:
> > > > Just out of curiosity, whats the impetus behind a single shared library here?
> > > > Is it just to ease application linking operations?  If so, it almost seems to me
> > > > that we should abandon the individual linking method and just use this as the
> > > > default output (and do simmilarly for the static linking build)
> > > 
> > > Not clear if you wrote "single shared library" on purpose instead of "single 
> > > static library". But for me the objective of COMBINE_LIBS usage would be 
> > > getting a "single static library" for my app, which just works, and eliminates 
> > > need of start-group, end-group, weird library ordering issues, etc. I'm not 
> > > interested personally in a "shared library" because it'd run slower.
> > > 
> > > Personally my preference would be to do both the single libs and multiple libs 
> > > in static format by default. Disk space is cheap, let's maximize user freedom 
> > > and flexibility. But shared lib, since it performs less well, should be 
> > > discouraged by default, although allowed if needed... some people prefer it 
> > > because it's easier to patch security vulns if you can replace a buggy library 
> > > for all the code on a system.
> > 
> > We need to simplify build options. So I'm fine to remove COMBINE_LIBS option
> > to always enable it.
> > About making only one single static library, I think it's a good idea if
> > it brings a real code simplification.
> > 
> > So the conclusion is to nack this patchset in favor of above changes.
> > Sergio, comments?
> > 
> 
> Frankly I did not think of users linking against single and combine lib for 
> different apps.
> I think If the goal is to simplify code then we should just provide one build
> option, either single or combine. Personally, I do not have a preference.
> 
> So just to be clear, we would remove COMBINE_LIBS to always make a single combine
> lib or to create both single and combine?
> For the later option, would we be linking apps against single or combine libraries?

The proposal is to always build single (combined) lib AND to build separated
libs in case of shared libraries.
For static library: only one single (combined) static library.

-- 
Thomas

Re: [PATCH 0/4] Fix build issues with CONFIG_RTE_BUILD_COMBINE_LIBS=y

[Neil Horman]

On Fri, Oct 03, 2014 at 10:27:30AM +0200, Thomas Monjalon wrote:
> 2014-10-03 09:10, Sergio Gonzalez Monroy:
> > On Fri, Oct 03, 2014 at 09:15:20AM +0200, Thomas Monjalon wrote:
> > > 2014-10-02 13:04, Matthew Hall:
> > > > On Thu, Oct 02, 2014 at 01:26:34PM -0400, Neil Horman wrote:
> > > > > Just out of curiosity, whats the impetus behind a single shared library here?
> > > > > Is it just to ease application linking operations?  If so, it almost seems to me
> > > > > that we should abandon the individual linking method and just use this as the
> > > > > default output (and do simmilarly for the static linking build)
> > > > 
> > > > Not clear if you wrote "single shared library" on purpose instead of "single 
> > > > static library". But for me the objective of COMBINE_LIBS usage would be 
> > > > getting a "single static library" for my app, which just works, and eliminates 
> > > > need of start-group, end-group, weird library ordering issues, etc. I'm not 
> > > > interested personally in a "shared library" because it'd run slower.
> > > > 
> > > > Personally my preference would be to do both the single libs and multiple libs 
> > > > in static format by default. Disk space is cheap, let's maximize user freedom 
> > > > and flexibility. But shared lib, since it performs less well, should be 
> > > > discouraged by default, although allowed if needed... some people prefer it 
> > > > because it's easier to patch security vulns if you can replace a buggy library 
> > > > for all the code on a system.
> > > 
> > > We need to simplify build options. So I'm fine to remove COMBINE_LIBS option
> > > to always enable it.
> > > About making only one single static library, I think it's a good idea if
> > > it brings a real code simplification.
> > > 
> > > So the conclusion is to nack this patchset in favor of above changes.
> > > Sergio, comments?
> > > 
> > 
> > Frankly I did not think of users linking against single and combine lib for 
> > different apps.
> > I think If the goal is to simplify code then we should just provide one build
> > option, either single or combine. Personally, I do not have a preference.
> > 
> > So just to be clear, we would remove COMBINE_LIBS to always make a single combine
> > lib or to create both single and combine?
> > For the later option, would we be linking apps against single or combine libraries?
> 
> The proposal is to always build single (combined) lib AND to build separated
> libs in case of shared libraries.
> For static library: only one single (combined) static library.
> 
This makes good sense to me.  A single archive is just easier in the static  
case, since the resulting binary will strip out unused code anyway, and multiple
libraries are needed in the shared case so that we don't wind up having to load
more code than is needed at run time.

Neil

> -- 
> Thomas
> 

Re: [PATCH 0/4] Fix build issues with CONFIG_RTE_BUILD_COMBINE_LIBS=y

[Matthew Hall]

On Fri, Oct 03, 2014 at 07:32:34AM -0400, Neil Horman wrote:
> This makes good sense to me.  A single archive is just easier in the static  
> case, since the resulting binary will strip out unused code anyway, and multiple
> libraries are needed in the shared case so that we don't wind up having to load
> more code than is needed at run time.
> 
> Neil

This assertion is not true. Because the DPDK doesn't work if you don't specify 
the whole-archive link option. Which explicitly prevents stripping out any 
"unused code". Otherwise your PMD's will refuse to initialize.

This along with backward compatibility is why I was advising to build the full 
static library and the sublibraries so it will just work for everybody by 
default.

Matthew.

Re: [PATCH 0/4] Fix build issues with CONFIG_RTE_BUILD_COMBINE_LIBS=y

[Neil Horman]

On Fri, Oct 03, 2014 at 11:17:13AM -0700, Matthew Hall wrote:
> On Fri, Oct 03, 2014 at 07:32:34AM -0400, Neil Horman wrote:
> > This makes good sense to me.  A single archive is just easier in the static  
> > case, since the resulting binary will strip out unused code anyway, and multiple
> > libraries are needed in the shared case so that we don't wind up having to load
> > more code than is needed at run time.
> > 
> > Neil
> 
> This assertion is not true. Because the DPDK doesn't work if you don't specify 
> the whole-archive link option. Which explicitly prevents stripping out any 
> "unused code". Otherwise your PMD's will refuse to initialize.
> 
I'm not asserting that you remove the --whole-archive option, only that the
we assemble all objects into a single archive during the SDK build.  Its just a
convienience, because the application binary result will be the same, regardless
of weather you link several small archives, or a single large one (because of
the aforementioned --whole-archive option).  The only remaining issue is events
where an application doesn't need (for example) the acl library, or other
optional library.  With a single archive, you get everything you build even if
you don't need it.  But presumably if you're building a static binary, you're
likely building the dpdk as well and can configure optional libraries out of the
build.  Separate libraries are more a need for downstream
distributors/packagers, who use dynamic shared objects anyway.
 
> This along with backward compatibility is why I was advising to build the full 
> static library and the sublibraries so it will just work for everybody by 
> default.
> 
Backward compatibilty?  the DPDK doesn't yet provide run time compatibility
between releases (something I've been trying to change).  Nobody provides
compile time compatibility.  To do so would require fixing API's permenently.

Neil

Re: [PATCH 0/4] Fix build issues with CONFIG_RTE_BUILD_COMBINE_LIBS=y

[Matthew Hall]

On Fri, Oct 03, 2014 at 03:15:46PM -0400, Neil Horman wrote:
> With a single archive, you get everything you build even if you don't need 
> it.

Right, I was trying to avoid that for people who specifically didn't want it, 
if there are any... I'm not one of them.

> But presumably if you're building a static binary, you're
> likely building the dpdk as well and can configure optional libraries out of the
> build.  Separate libraries are more a need for downstream
> distributors/packagers, who use dynamic shared objects anyway.

Yeah, I was thinking it'd be nice if the downstream packagers could get a 
global '.a' and per-sublib '.a' as well. So that one dpdk package could be 
used by a client app which wanted everything, or only wanted portions.

> Backward compatibilty?  the DPDK doesn't yet provide run time compatibility
> between releases (something I've been trying to change).  Nobody provides
> compile time compatibility.  To do so would require fixing API's permenently.

Agreed. I was just advocating to avoid worsening the already existent issues. 
;)

Matthew.

Re: [PATCH 0/4] Fix build issues with CONFIG_RTE_BUILD_COMBINE_LIBS=y

[Neil Horman]

On Fri, Oct 03, 2014 at 02:21:50PM -0700, Matthew Hall wrote:
> On Fri, Oct 03, 2014 at 03:15:46PM -0400, Neil Horman wrote:
> > With a single archive, you get everything you build even if you don't need 
> > it.
> 
> Right, I was trying to avoid that for people who specifically didn't want it, 
> if there are any... I'm not one of them.
> 
> > But presumably if you're building a static binary, you're
> > likely building the dpdk as well and can configure optional libraries out of the
> > build.  Separate libraries are more a need for downstream
> > distributors/packagers, who use dynamic shared objects anyway.
> 
> Yeah, I was thinking it'd be nice if the downstream packagers could get a 
> global '.a' and per-sublib '.a' as well. So that one dpdk package could be 
> used by a client app which wanted everything, or only wanted portions.
> 
> > Backward compatibilty?  the DPDK doesn't yet provide run time compatibility
> > between releases (something I've been trying to change).  Nobody provides
> > compile time compatibility.  To do so would require fixing API's permenently.
> 
> Agreed. I was just advocating to avoid worsening the already existent issues. 
> ;)
> 
> Matthew.
> 
Fair enough
Neil

Re: [PATCH 0/4] Fix build issues with CONFIG_RTE_BUILD_COMBINE_LIBS=y

[Matthew Hall]

On Fri, Oct 03, 2014 at 10:27:30AM +0200, Thomas Monjalon wrote:
> The proposal is to always build single (combined) lib AND to build separated
> libs in case of shared libraries.
> For static library: only one single (combined) static library.

In the static case, this won't be backward compatible for people with existing 
linking, and won't allow people to pick which PMD's to activate without 
recompiling DPDK. If you're OK with that then I don't have an issue. But some 
others may not like it.

Matthew.

Re: [PATCH 0/4] Fix build issues with CONFIG_RTE_BUILD_COMBINE_LIBS=y

[Matthew Hall]

On Fri, Oct 03, 2014 at 09:15:20AM +0200, Thomas Monjalon wrote:
> We need to simplify build options. So I'm fine to remove COMBINE_LIBS option
> to always enable it.
> About making only one single static library, I think it's a good idea if
> it brings a real code simplification.
> 
> So the conclusion is to nack this patchset in favor of above changes.
> Sergio, comments?

It works for me... I love COMBINE_LIBS... but it won't be necessarily backward 
compatible with people already linking against existing DPDK. Hence why I 
proposed just always building the per-feature static libs and always building 
the combined lib rather than forcing people to do one or the other.

Also different people might have different configs where they link in greater 
or fewer or different PMD's or features in different configs of their build... 
they might be annoyed if their separate libs disappear.

Matthew.

[PATCH v2 0/4] Update build process

[Sergio Gonzalez Monroy]

As per the proposal, this patch set does:
 - Remove CONFIG_RTE_BUILD_COMBINE_LIBS as a configuration option.
 - For static library, build a single/combined library.
 - For shared libraries, build both individual/separated and single/combined
   libraries.
 - Link apps only against single/combined libs.


Sergio Gonzalez Monroy (4):
  Link combined shared library using CC
  Link apps only against single/combined library
  Update library build process
  Link apps/DSOs against EXECENV_LDLIBS with --as-needed

 config/common_bsdapp   |   3 +-
 config/common_linuxapp |   3 +-
 mk/rte.app.mk          | 164 ++-----------------------------------------------
 mk/rte.lib.mk          |  81 ++++++------------------
 mk/rte.sdkbuild.mk     |   2 +-
 mk/rte.sharelib.mk     |  54 ++++++++--------
 mk/rte.vars.mk         |   4 --
 7 files changed, 54 insertions(+), 257 deletions(-)

-- 
1.9.3

[PATCH v2 1/4] Link combined shared library using CC

[Sergio Gonzalez Monroy]

Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 mk/rte.sharelib.mk | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/mk/rte.sharelib.mk b/mk/rte.sharelib.mk
index c0a811a..73ce709 100644
--- a/mk/rte.sharelib.mk
+++ b/mk/rte.sharelib.mk
@@ -29,6 +29,8 @@
 #   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 #   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
+include $(RTE_SDK)/mk/internal/rte.build-pre.mk
+
 # VPATH contains at least SRCDIR
 VPATH += $(SRCDIR)
 
@@ -40,12 +42,20 @@ LIB_ONE := lib$(RTE_LIBNAME).a
 endif
 endif
 
+ifeq ($(LINK_USING_CC),1)
+# Override the definition of LD here, since we're linking with CC
+LD := $(CC)
+LD_MULDEFS := $(call linkerprefix,-z$(comma)muldefs)
+CPU_LDFLAGS := $(call linkerprefix,$(CPU_LDFLAGS))
+endif
+
 .PHONY:sharelib
 sharelib: $(LIB_ONE) FORCE
 
 OBJS = $(wildcard $(RTE_OUTPUT)/build/lib/*.o)
 
-O_TO_S = $(LD) $(CPU_LDFLAGS) -shared $(OBJS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
+O_TO_S = $(LD) $(CPU_LDFLAGS) $(LD_MULDEFS) -shared $(OBJS)\
+	-o $(RTE_OUTPUT)/lib/$(LIB_ONE)
 O_TO_S_STR = $(subst ','\'',$(O_TO_S)) #'# fix syntax highlight
 O_TO_S_DISP = $(if $(V),"$(O_TO_S_STR)","  LD $(@)")
 O_TO_S_CMD = "cmd_$@ = $(O_TO_S_STR)"
-- 
1.9.3

[PATCH v2 2/4] Link apps only against single/combined library

[Sergio Gonzalez Monroy]

Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 mk/rte.app.mk | 160 +---------------------------------------------------------
 1 file changed, 2 insertions(+), 158 deletions(-)

diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 34dff2a..5e00e67 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -60,164 +60,12 @@ LDLIBS += -L$(RTE_SDK_BIN)/lib
 ifeq ($(NO_AUTOLIBS),)
 
 LDLIBS += --whole-archive
-
-ifeq ($(CONFIG_RTE_LIBRTE_DISTRIBUTOR),y)
-LDLIBS += -lrte_distributor
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_KNI),y)
-ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),y)
-LDLIBS += -lrte_kni
-endif
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_IVSHMEM),y)
-ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),y)
-LDLIBS += -lrte_ivshmem
-endif
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_PIPELINE),y)
-LDLIBS += -lrte_pipeline
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_TABLE),y)
-LDLIBS += -lrte_table
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_PORT),y)
-LDLIBS += -lrte_port
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_TIMER),y)
-LDLIBS += -lrte_timer
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_HASH),y)
-LDLIBS += -lrte_hash
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_LPM),y)
-LDLIBS += -lrte_lpm
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_POWER),y)
-LDLIBS += -lrte_power
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_ACL),y)
-LDLIBS += -lrte_acl
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_METER),y)
-LDLIBS += -lrte_meter
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_SCHED),y)
-LDLIBS += -lrte_sched
-LDLIBS += -lm
-LDLIBS += -lrt
-endif
-
+LDLIBS += -l$(RTE_LIBNAME)
+LDLIBS += --no-whole-archive
 LDLIBS += --start-group
-
-ifeq ($(CONFIG_RTE_LIBRTE_KVARGS),y)
-LDLIBS += -lrte_kvargs
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_MBUF),y)
-LDLIBS += -lrte_mbuf
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_IP_FRAG),y)
-LDLIBS += -lrte_ip_frag
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_ETHER),y)
-LDLIBS += -lethdev
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_MALLOC),y)
-LDLIBS += -lrte_malloc
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_MEMPOOL),y)
-LDLIBS += -lrte_mempool
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_RING),y)
-LDLIBS += -lrte_ring
-endif
-
-ifeq ($(CONFIG_RTE_LIBC),y)
-LDLIBS += -lc
-LDLIBS += -lm
-endif
-
-ifeq ($(CONFIG_RTE_LIBGLOSS),y)
-LDLIBS += -lgloss
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_EAL),y)
-LDLIBS += -lrte_eal
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_CMDLINE),y)
-LDLIBS += -lrte_cmdline
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_CFGFILE),y)
-LDLIBS += -lrte_cfgfile
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_PMD_BOND),y)
-LDLIBS += -lrte_pmd_bond
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_PMD_XENVIRT),y)
-LDLIBS += -lrte_pmd_xenvirt
-LDLIBS += -lxenstore
-endif
-
-ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
-# plugins (link only if static libraries)
-
-ifeq ($(CONFIG_RTE_LIBRTE_VMXNET3_PMD),y)
-LDLIBS += -lrte_pmd_vmxnet3_uio
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_VIRTIO_PMD),y)
-LDLIBS += -lrte_pmd_virtio_uio
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_I40E_PMD),y)
-LDLIBS += -lrte_pmd_i40e
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_IXGBE_PMD),y)
-LDLIBS += -lrte_pmd_ixgbe
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_E1000_PMD),y)
-LDLIBS += -lrte_pmd_e1000
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_PMD_RING),y)
-LDLIBS += -lrte_pmd_ring
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_PMD_PCAP),y)
-LDLIBS += -lrte_pmd_pcap -lpcap
-endif
-
-endif # plugins
-
 LDLIBS += $(EXECENV_LDLIBS)
-
 LDLIBS += --end-group
 
-LDLIBS += --no-whole-archive
-
 endif # ifeq ($(NO_AUTOLIBS),)
 
 LDLIBS += $(CPU_LDLIBS)
@@ -235,10 +83,6 @@ build: _postbuild
 
 exe2cmd = $(strip $(call dotfile,$(patsubst %,%.cmd,$(1))))
 
-ifeq ($(RTE_BUILD_COMBINE_LIBS),y)
-LDLIBS += -l$(RTE_LIBNAME)
-endif
-
 ifeq ($(LINK_USING_CC),1)
 LDLIBS := $(call linkerprefix,$(LDLIBS))
 LDFLAGS := $(call linkerprefix,$(LDFLAGS))
-- 
1.9.3

[PATCH v2 3/4] Update library build process

[Sergio Gonzalez Monroy]

Remove COMBINE_LIBS option and by default build:
- CONFIG_RTE_BUILD_SHARED_LIB=y : both individual and combined libraries
- CONFIG_RTE_BUILD_SHARED_LIB=n : single combined library

Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 config/common_bsdapp   |  3 +--
 config/common_linuxapp |  3 +--
 mk/rte.lib.mk          | 73 ++++++++------------------------------------------
 mk/rte.sdkbuild.mk     |  2 +-
 mk/rte.sharelib.mk     | 41 +++++++++++-----------------
 mk/rte.vars.mk         |  4 ---
 6 files changed, 29 insertions(+), 97 deletions(-)

diff --git a/config/common_bsdapp b/config/common_bsdapp
index eebd05b..65d2ecc 100644
--- a/config/common_bsdapp
+++ b/config/common_bsdapp
@@ -79,9 +79,8 @@ CONFIG_RTE_FORCE_INTRINSICS=n
 CONFIG_RTE_BUILD_SHARED_LIB=n
 
 #
-# Combine to one single library
+# Library name
 #
-CONFIG_RTE_BUILD_COMBINE_LIBS=n
 CONFIG_RTE_LIBNAME=intel_dpdk
 
 #
diff --git a/config/common_linuxapp b/config/common_linuxapp
index 4713eb4..5bdcdf3 100644
--- a/config/common_linuxapp
+++ b/config/common_linuxapp
@@ -79,9 +79,8 @@ CONFIG_RTE_FORCE_INTRINSICS=n
 CONFIG_RTE_BUILD_SHARED_LIB=n
 
 #
-# Combine to one single library
+# Library name
 #
-CONFIG_RTE_BUILD_COMBINE_LIBS=n
 CONFIG_RTE_LIBNAME="intel_dpdk"
 
 #
diff --git a/mk/rte.lib.mk b/mk/rte.lib.mk
index f458258..e7420bf 100644
--- a/mk/rte.lib.mk
+++ b/mk/rte.lib.mk
@@ -66,48 +66,23 @@ LD_MULDEFS := $(call linkerprefix,-z$(comma)muldefs)
 CPU_LDFLAGS := $(call linkerprefix,$(CPU_LDFLAGS))
 endif
 
-O_TO_A = $(AR) crus $(LIB) $(OBJS-y)
-O_TO_A_STR = $(subst ','\'',$(O_TO_A)) #'# fix syntax highlight
-O_TO_A_DISP = $(if $(V),"$(O_TO_A_STR)","  AR $(@)")
-O_TO_A_CMD = "cmd_$@ = $(O_TO_A_STR)"
-O_TO_A_DO = @set -e; \
-	echo $(O_TO_A_DISP); \
-	$(O_TO_A) && \
-	echo $(O_TO_A_CMD) > $(call exe2cmd,$(@))
-
 O_TO_S = $(LD) $(CPU_LDFLAGS) $(LD_MULDEFS) -shared $(OBJS-y) -o $(LIB)
-O_TO_S_STR = $(subst ','\'',$(O_TO_S)) #'# fix syntax highlight
+O_TO_S_STR = $(subst ','\'',$(O_TO_S)) #') # fix syntax highlight
 O_TO_S_DISP = $(if $(V),"$(O_TO_S_STR)","  LD $(@)")
+O_TO_S_CMD = "cmd_$@ = $(O_TO_S_STR)"
 O_TO_S_DO = @set -e; \
+	cp -f $(OBJS-y) $(RTE_OUTPUT)/build/lib; \
 	echo $(O_TO_S_DISP); \
 	$(O_TO_S) && \
 	echo $(O_TO_S_CMD) > $(call exe2cmd,$(@))
 
-ifeq ($(RTE_BUILD_SHARED_LIB),n)
-O_TO_C = $(AR) crus $(LIB_ONE) $(OBJS-y)
-O_TO_C_STR = $(subst ','\'',$(O_TO_C)) #'# fix syntax highlight
-O_TO_C_DISP = $(if $(V),"$(O_TO_C_STR)","  AR_C $(@)")
-O_TO_C_DO = @set -e; \
-	$(lib_dir) \
-	$(copy_obj)
-else
-O_TO_C = $(LD) $(LD_MULDEFS) -shared $(OBJS-y) -o $(LIB_ONE)
-O_TO_C_STR = $(subst ','\'',$(O_TO_C)) #'# fix syntax highlight
-O_TO_C_DISP = $(if $(V),"$(O_TO_C_STR)","  LD_C $(@)")
-O_TO_C_DO = @set -e; \
-	$(lib_dir) \
-	$(copy_obj)
-endif
-
-copy_obj = cp -f $(OBJS-y) $(RTE_OUTPUT)/build/lib;
-lib_dir = [ -d $(RTE_OUTPUT)/lib ] || mkdir -p $(RTE_OUTPUT)/lib;
 -include .$(LIB).cmd
 
 #
 # Archive objects in .a file if needed
 #
-ifeq ($(RTE_BUILD_SHARED_LIB),y)
 $(LIB): $(OBJS-y) $(DEP_$(LIB)) FORCE
+ifeq ($(RTE_BUILD_SHARED_LIB),y)
 	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
 	$(if $(D),\
 		@echo -n "$< -> $@ " ; \
@@ -116,51 +91,25 @@ $(LIB): $(OBJS-y) $(DEP_$(LIB)) FORCE
 		echo -n "depfile_missing=$(call boolean,$(depfile_missing)) " ; \
 		echo "depfile_newer=$(call boolean,$(depfile_newer)) ")
 	$(if $(or \
-		$(file_missing),\
-		$(call cmdline_changed,$(O_TO_S_STR)),\
-		$(depfile_missing),\
-		$(depfile_newer)),\
-		$(O_TO_S_DO))
-ifeq ($(RTE_BUILD_COMBINE_LIBS),y)
-	$(if $(or \
-        $(file_missing),\
-        $(call cmdline_changed,$(O_TO_C_STR)),\
-        $(depfile_missing),\
-        $(depfile_newer)),\
-        $(O_TO_C_DO))
-endif
-else
-$(LIB): $(OBJS-y) $(DEP_$(LIB)) FORCE
-	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
-	$(if $(D),\
-	    @echo -n "$< -> $@ " ; \
-	    echo -n "file_missing=$(call boolean,$(file_missing)) " ; \
-	    echo -n "cmdline_changed=$(call boolean,$(call cmdline_changed,$(O_TO_A_STR))) " ; \
-	    echo -n "depfile_missing=$(call boolean,$(depfile_missing)) " ; \
-	    echo "depfile_newer=$(call boolean,$(depfile_newer)) ")
-	$(if $(or \
 	    $(file_missing),\
-	    $(call cmdline_changed,$(O_TO_A_STR)),\
+	    $(call cmdline_changed,$(O_TO_S_STR)),\
 	    $(depfile_missing),\
 	    $(depfile_newer)),\
-	    $(O_TO_A_DO))
-ifeq ($(RTE_BUILD_COMBINE_LIBS),y)
-	$(if $(or \
-        $(file_missing),\
-        $(call cmdline_changed,$(O_TO_C_STR)),\
-        $(depfile_missing),\
-        $(depfile_newer)),\
-        $(O_TO_C_DO))
-endif
+	    $(O_TO_S_DO))
+else
+	@set -e; \
+	cp -f $(OBJS-y) $(RTE_OUTPUT)/build/lib;
 endif
 
 #
 # install lib in $(RTE_OUTPUT)/lib
 #
 $(RTE_OUTPUT)/lib/$(LIB): $(LIB)
+ifeq ($(RTE_BUILD_SHARED_LIB),y)
 	@echo "  INSTALL-LIB $(LIB)"
 	@[ -d $(RTE_OUTPUT)/lib ] || mkdir -p $(RTE_OUTPUT)/lib
 	$(Q)cp -f $(LIB) $(RTE_OUTPUT)/lib
+endif
 
 #
 # Clean all generated files
diff --git a/mk/rte.sdkbuild.mk b/mk/rte.sdkbuild.mk
index 3154457..f20ec1e 100644
--- a/mk/rte.sdkbuild.mk
+++ b/mk/rte.sdkbuild.mk
@@ -93,7 +93,7 @@ $(ROOTDIRS-y):
 	@[ -d $(BUILDDIR)/$@ ] || mkdir -p $(BUILDDIR)/$@
 	@echo "== Build $@"
 	$(Q)$(MAKE) S=$@ -f $(RTE_SRCDIR)/$@/Makefile -C $(BUILDDIR)/$@ all
-	@if [ $@ = lib -a $(RTE_BUILD_COMBINE_LIBS) = y ]; then \
+	@if [ $@ = lib ]; then \
 		$(MAKE) -f $(RTE_SDK)/lib/Makefile sharelib; \
 	fi
 
diff --git a/mk/rte.sharelib.mk b/mk/rte.sharelib.mk
index 73ce709..8fc6548 100644
--- a/mk/rte.sharelib.mk
+++ b/mk/rte.sharelib.mk
@@ -34,13 +34,11 @@ include $(RTE_SDK)/mk/internal/rte.build-pre.mk
 # VPATH contains at least SRCDIR
 VPATH += $(SRCDIR)
 
-ifeq ($(RTE_BUILD_COMBINE_LIBS),y)
 ifeq ($(RTE_BUILD_SHARED_LIB),y)
 LIB_ONE := lib$(RTE_LIBNAME).so
 else
 LIB_ONE := lib$(RTE_LIBNAME).a
 endif
-endif
 
 ifeq ($(LINK_USING_CC),1)
 # Override the definition of LD here, since we're linking with CC
@@ -54,37 +52,28 @@ sharelib: $(LIB_ONE) FORCE
 
 OBJS = $(wildcard $(RTE_OUTPUT)/build/lib/*.o)
 
-O_TO_S = $(LD) $(CPU_LDFLAGS) $(LD_MULDEFS) -shared $(OBJS)\
+ifeq ($(RTE_BUILD_SHARED_LIB),y)
+O_TO_L = $(LD) $(CPU_LDFLAGS) $(LD_MULDEFS) -shared $(OBJS)\
 	-o $(RTE_OUTPUT)/lib/$(LIB_ONE)
-O_TO_S_STR = $(subst ','\'',$(O_TO_S)) #'# fix syntax highlight
-O_TO_S_DISP = $(if $(V),"$(O_TO_S_STR)","  LD $(@)")
-O_TO_S_CMD = "cmd_$@ = $(O_TO_S_STR)"
-O_TO_S_DO = @set -e; \
-    echo $(O_TO_S_DISP); \
-    $(O_TO_S)
+L_DISP=LD
+else
+O_TO_L = $(AR) crus $(RTE_OUTPUT)/lib/$(LIB_ONE) $(OBJS)
+L_DISP=AR
+endif
+O_TO_L_STR = $(subst ','\'',$(O_TO_L)) #')# fix syntax highlight
+O_TO_L_DISP = $(if $(V),"$(O_TO_L_STR)","  $(L_DISP) $(@)")
+O_TO_L_CMD = "cmd_$@ = $(O_TO_L_STR)"
+O_TO_L_DO = @set -e; \
+	mkdir -p $(RTE_OUTPUT)/lib; \
+	echo $(O_TO_L_DISP); \
+	$(O_TO_L)
 
-O_TO_A =  $(AR) crus $(RTE_OUTPUT)/lib/$(LIB_ONE) $(OBJS)
-O_TO_A_STR = $(subst ','\'',$(O_TO_A)) #'# fix syntax highlight
-O_TO_A_DISP = $(if $(V),"$(O_TO_A_STR)","  LD $(@)")
-O_TO_A_CMD = "cmd_$@ = $(O_TO_A_STR)"
-O_TO_A_DO = @set -e; \
-    echo $(O_TO_A_DISP); \
-    $(O_TO_A)
 #
 # Archive objects to share library
 #
 
-ifeq ($(RTE_BUILD_COMBINE_LIBS),y)
-ifeq ($(RTE_BUILD_SHARED_LIB),y)
 $(LIB_ONE): FORCE
-	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
-	$(O_TO_S_DO)
-else
-$(LIB_ONE): FORCE
-	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
-	$(O_TO_A_DO)
-endif
-endif
+	$(O_TO_L_DO)
 
 #
 # Clean all generated files
diff --git a/mk/rte.vars.mk b/mk/rte.vars.mk
index 1e874ee..5bcc4a5 100644
--- a/mk/rte.vars.mk
+++ b/mk/rte.vars.mk
@@ -67,10 +67,6 @@ ifneq ($(BUILDING_RTE_SDK),)
   ifeq ($(RTE_BUILD_SHARED_LIB),)
     RTE_BUILD_SHARED_LIB := n
   endif
-  RTE_BUILD_COMBINE_LIBS := $(CONFIG_RTE_BUILD_COMBINE_LIBS:"%"=%)
-  ifeq ($(RTE_BUILD_COMBINE_LIBS),)
-    RTE_BUILD_COMBINE_LIBS := n
-  endif
   RTE_LIBNAME := $(CONFIG_RTE_LIBNAME:"%"=%)
   ifeq ($(RTE_LIBNAME),)
     RTE_LIBNAME := intel_dpdk
-- 
1.9.3

Re: [PATCH v2 3/4] Update library build process

[Matthew Hall]

On Mon, Oct 06, 2014 at 11:52:34AM +0100, Sergio Gonzalez Monroy wrote:
> Remove COMBINE_LIBS option and by default build:
> - CONFIG_RTE_BUILD_SHARED_LIB=y : both individual and combined libraries
> - CONFIG_RTE_BUILD_SHARED_LIB=n : single combined library

As previously discussed.,It would be better for backward-compatibility of 
people linking against DPDK, if the static lib could come out as both a 
combined library and separate individual libraries by default.

Otherwise everybody linking against DPDK has to change their code, and it 
won't be easy for them to move forward and backward against different DPDKs 
before and after this change.

Thanks,
Matthew.

Re: [PATCH v2 3/4] Update library build process

[Sergio Gonzalez Monroy]

On Mon, Oct 06, 2014 at 01:46:07PM -0700, Matthew Hall wrote:
> On Mon, Oct 06, 2014 at 11:52:34AM +0100, Sergio Gonzalez Monroy wrote:
> > Remove COMBINE_LIBS option and by default build:
> > - CONFIG_RTE_BUILD_SHARED_LIB=y : both individual and combined libraries
> > - CONFIG_RTE_BUILD_SHARED_LIB=n : single combined library
> 
> As previously discussed.,It would be better for backward-compatibility of 
> people linking against DPDK, if the static lib could come out as both a 
> combined library and separate individual libraries by default.
> 
> Otherwise everybody linking against DPDK has to change their code, and it 
> won't be easy for them to move forward and backward against different DPDKs 
> before and after this change.
> 
> Thanks,
> Matthew.

Hi Matthew,

My understanding from previous conversations is that backward-compatibility
is not being provided for the next release, as such it is unlikely that
backward-compatibility for compile/linking will be provided.

I don't see how this particular patch would force people to change their code,
though in all likelihood they will have to as a result of ABI changes in the
following release.
The only difference now would be when they link their applications against a
single library instead of multiple separated libraries.

Thanks,
Sergio

Re: [PATCH v2 3/4] Update library build process

[Matthew Hall]

On Tue, Oct 07, 2014 at 10:55:11AM +0100, Sergio Gonzalez Monroy wrote:
> I don't see how this particular patch would force people to change their code,
> though in all likelihood they will have to as a result of ABI changes in the
> following release.
>
> The only difference now would be when they link their applications against a
> single library instead of multiple separated libraries.
> 
> Thanks,
> Sergio

Hi Sergio,

Changing all of your Makefiles does sound like changing code to me.

Matthew.

Re: [PATCH v2 3/4] Update library build process

[Sergio Gonzalez Monroy]

On Wed, Oct 08, 2014 at 03:36:55PM -0700, Matthew Hall wrote:
> On Tue, Oct 07, 2014 at 10:55:11AM +0100, Sergio Gonzalez Monroy wrote:
> > I don't see how this particular patch would force people to change their code,
> > though in all likelihood they will have to as a result of ABI changes in the
> > following release.
> >
> > The only difference now would be when they link their applications against a
> > single library instead of multiple separated libraries.
> > 
> > Thanks,
> > Sergio
> 
> Hi Sergio,
> 
> Changing all of your Makefiles does sound like changing code to me.
> 

Hi Matthew,

You are right, it is changing code.
What I was trying to say is that the impact of this patch is probrably going to
be one of the lowest as far as changing code for the next release.

Thanks,
Sergio

> Matthew.

Re: [PATCH v2 3/4] Update library build process

[Thomas Monjalon]

Hi Sergio,

2014-10-06 11:52, Sergio Gonzalez Monroy:
> Remove COMBINE_LIBS option and by default build:
> - CONFIG_RTE_BUILD_SHARED_LIB=y : both individual and combined libraries
> - CONFIG_RTE_BUILD_SHARED_LIB=n : single combined library
> 
> Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

I'd appreciate this hairy patch to be splitted and commented if possible.
It's not easy to review as is.

Thanks
-- 
Thomas

[PATCH v2 4/4] Link apps/DSOs against EXECENV_LDLIBS with --as-needed

[Sergio Gonzalez Monroy]

Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 mk/rte.app.mk      | 4 ++--
 mk/rte.lib.mk      | 8 +++++++-
 mk/rte.sharelib.mk | 7 ++++++-
 3 files changed, 15 insertions(+), 4 deletions(-)

diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 5e00e67..e775ad7 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -62,9 +62,9 @@ ifeq ($(NO_AUTOLIBS),)
 LDLIBS += --whole-archive
 LDLIBS += -l$(RTE_LIBNAME)
 LDLIBS += --no-whole-archive
-LDLIBS += --start-group
+LDLIBS += --as-needed
 LDLIBS += $(EXECENV_LDLIBS)
-LDLIBS += --end-group
+LDLIBS += --no-as-needed
 
 endif # ifeq ($(NO_AUTOLIBS),)
 
diff --git a/mk/rte.lib.mk b/mk/rte.lib.mk
index e7420bf..947e17d 100644
--- a/mk/rte.lib.mk
+++ b/mk/rte.lib.mk
@@ -59,14 +59,20 @@ build: _postbuild
 
 exe2cmd = $(strip $(call dotfile,$(patsubst %,%.cmd,$(1))))
 
+O_TO_S_LDLIBS := --as-needed
+O_TO_S_LDLIBS += $(EXECENV_LDLIBS)
+O_TO_S_LDLIBS += --no-as-needed
+
 ifeq ($(LINK_USING_CC),1)
 # Override the definition of LD here, since we're linking with CC
 LD := $(CC)
 LD_MULDEFS := $(call linkerprefix,-z$(comma)muldefs)
 CPU_LDFLAGS := $(call linkerprefix,$(CPU_LDFLAGS))
+O_TO_S_LDLIBS := $(call linkerprefix,$(O_TO_S_LDLIBS))
 endif
 
-O_TO_S = $(LD) $(CPU_LDFLAGS) $(LD_MULDEFS) -shared $(OBJS-y) -o $(LIB)
+O_TO_S = $(LD) $(CPU_LDFLAGS) $(LD_MULDEFS) -shared $(OBJS-y) \
+	$(O_TO_S_LDLIBS) -o $(LIB)
 O_TO_S_STR = $(subst ','\'',$(O_TO_S)) #') # fix syntax highlight
 O_TO_S_DISP = $(if $(V),"$(O_TO_S_STR)","  LD $(@)")
 O_TO_S_CMD = "cmd_$@ = $(O_TO_S_STR)"
diff --git a/mk/rte.sharelib.mk b/mk/rte.sharelib.mk
index 8fc6548..380aa7d 100644
--- a/mk/rte.sharelib.mk
+++ b/mk/rte.sharelib.mk
@@ -40,11 +40,16 @@ else
 LIB_ONE := lib$(RTE_LIBNAME).a
 endif
 
+O_TO_L_LDLIBS := --as-needed
+O_TO_L_LDLIBS += $(EXECENV_LDLIBS)
+O_TO_L_LDLIBS += --no-as-needed
+
 ifeq ($(LINK_USING_CC),1)
 # Override the definition of LD here, since we're linking with CC
 LD := $(CC)
 LD_MULDEFS := $(call linkerprefix,-z$(comma)muldefs)
 CPU_LDFLAGS := $(call linkerprefix,$(CPU_LDFLAGS))
+O_TO_L_LDLIBS := $(call linkerprefix,$(O_TO_L_LDLIBS))
 endif
 
 .PHONY:sharelib
@@ -54,7 +59,7 @@ OBJS = $(wildcard $(RTE_OUTPUT)/build/lib/*.o)
 
 ifeq ($(RTE_BUILD_SHARED_LIB),y)
 O_TO_L = $(LD) $(CPU_LDFLAGS) $(LD_MULDEFS) -shared $(OBJS)\
-	-o $(RTE_OUTPUT)/lib/$(LIB_ONE)
+	$(O_TO_L_LDLIBS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
 L_DISP=LD
 else
 O_TO_L = $(AR) crus $(RTE_OUTPUT)/lib/$(LIB_ONE) $(OBJS)
-- 
1.9.3

Re: [PATCH v2 4/4] Link apps/DSOs against EXECENV_LDLIBS with --as-needed

[Thomas Monjalon]

Please could you explain why patch is needed?

-- 
Thomas

Re: [PATCH v2 4/4] Link apps/DSOs against EXECENV_LDLIBS with --as-needed

[Sergio Gonzalez Monroy]

On Wed, Oct 08, 2014 at 05:38:52PM +0200, Thomas Monjalon wrote:
> Please could you explain why patch is needed?
> 
Hi Thomas,

Basically this patch just adds missing dependencies to DPDK DSOs. so the 
DL knows about them.

As an example, if we were to build an application against DPDK without
adding the libs defined in EXECENV_LDLIBS:

-- pre-patch:
LD test
test_red.o: In function `ovfl_test1':
test_red.c:(.text+0x1ac6): undefined reference to `log'
test_red.c:(.text+0x1ad3): undefined reference to `ceil'
test_red.o: In function `perf2_test':
test_red.c:(.text+0x2314): undefined reference to `pow'
test_red.o: In function `func_test3':
test_red.c:(.text+0x28d7): undefined reference to `pow'
test_red.o: In function `func_test6':
test_red.c:(.text+0x324c): undefined reference to `pow'
dpdk/x86_64-native-linuxapp-gcc/lib/libintel_dpdk.so: undefined reference to `dlopen'
dpdk/x86_64-native-linuxapp-gcc/lib/libintel_dpdk.so: undefined reference to `log2'
dpdk/x86_64-native-linuxapp-gcc/lib/libintel_dpdk.so: undefined reference to `dlerror'
dpdk/x86_64-native-linuxapp-gcc/lib/libintel_dpdk.so: undefined reference to `round'

-- post-patch:
LD test
/usr/bin/ld: test_red.o: undefined reference to symbol 'log@@GLIBC_2.2.5'
/usr/bin/ld: note: 'log@@GLIBC_2.2.5' is defined in DSO /lib64/libm.so.6 so try adding it to the linker command line
/lib64/libm.so.6: could not read symbols: Invalid operation

Thanks,
Sergio

> -- 
> Thomas

Re: [PATCH v2 0/4] Update build process

[Neil Horman]

On Mon, Oct 06, 2014 at 11:52:31AM +0100, Sergio Gonzalez Monroy wrote:
> As per the proposal, this patch set does:
>  - Remove CONFIG_RTE_BUILD_COMBINE_LIBS as a configuration option.
>  - For static library, build a single/combined library.
>  - For shared libraries, build both individual/separated and single/combined
>    libraries.
>  - Link apps only against single/combined libs.
> 
> 
> Sergio Gonzalez Monroy (4):
>   Link combined shared library using CC
>   Link apps only against single/combined library
>   Update library build process
>   Link apps/DSOs against EXECENV_LDLIBS with --as-needed
> 
>  config/common_bsdapp   |   3 +-
>  config/common_linuxapp |   3 +-
>  mk/rte.app.mk          | 164 ++-----------------------------------------------
>  mk/rte.lib.mk          |  81 ++++++------------------
>  mk/rte.sdkbuild.mk     |   2 +-
>  mk/rte.sharelib.mk     |  54 ++++++++--------
>  mk/rte.vars.mk         |   4 --
>  7 files changed, 54 insertions(+), 257 deletions(-)
> 
> -- 
> 1.9.3
> 
> 

I see you removed the --whole-archive option when building the single library
here.  Have you checked to make sure that all the constructors haven't been
stripped out?
Neil

Re: [PATCH v2 0/4] Update build process

[Sergio Gonzalez Monroy]

On Mon, Oct 06, 2014 at 10:49:46AM -0400, Neil Horman wrote:
> On Mon, Oct 06, 2014 at 11:52:31AM +0100, Sergio Gonzalez Monroy wrote:
> > As per the proposal, this patch set does:
> >  - Remove CONFIG_RTE_BUILD_COMBINE_LIBS as a configuration option.
> >  - For static library, build a single/combined library.
> >  - For shared libraries, build both individual/separated and single/combined
> >    libraries.
> >  - Link apps only against single/combined libs.
> > 
> > 
> > Sergio Gonzalez Monroy (4):
> >   Link combined shared library using CC
> >   Link apps only against single/combined library
> >   Update library build process
> >   Link apps/DSOs against EXECENV_LDLIBS with --as-needed
> > 
> >  config/common_bsdapp   |   3 +-
> >  config/common_linuxapp |   3 +-
> >  mk/rte.app.mk          | 164 ++-----------------------------------------------
> >  mk/rte.lib.mk          |  81 ++++++------------------
> >  mk/rte.sdkbuild.mk     |   2 +-
> >  mk/rte.sharelib.mk     |  54 ++++++++--------
> >  mk/rte.vars.mk         |   4 --
> >  7 files changed, 54 insertions(+), 257 deletions(-)
> > 
> > -- 
> > 1.9.3
> > 
> > 
> 
> I see you removed the --whole-archive option when building the single library
> here.  Have you checked to make sure that all the constructors haven't been
> stripped out?

I am not entirely sure I follow. There is no --whole-archive when building libraries,
at least not in my sources.
The flag is used when linking apps and I have not removed it as you can see on patch 2/4.

Sergio

> Neil
> 

Re: [PATCH v2 0/4] Update build process

[Neil Horman]

On Mon, Oct 06, 2014 at 04:01:33PM +0100, Sergio Gonzalez Monroy wrote:
> On Mon, Oct 06, 2014 at 10:49:46AM -0400, Neil Horman wrote:
> > On Mon, Oct 06, 2014 at 11:52:31AM +0100, Sergio Gonzalez Monroy wrote:
> > > As per the proposal, this patch set does:
> > >  - Remove CONFIG_RTE_BUILD_COMBINE_LIBS as a configuration option.
> > >  - For static library, build a single/combined library.
> > >  - For shared libraries, build both individual/separated and single/combined
> > >    libraries.
> > >  - Link apps only against single/combined libs.
> > > 
> > > 
> > > Sergio Gonzalez Monroy (4):
> > >   Link combined shared library using CC
> > >   Link apps only against single/combined library
> > >   Update library build process
> > >   Link apps/DSOs against EXECENV_LDLIBS with --as-needed
> > > 
> > >  config/common_bsdapp   |   3 +-
> > >  config/common_linuxapp |   3 +-
> > >  mk/rte.app.mk          | 164 ++-----------------------------------------------
> > >  mk/rte.lib.mk          |  81 ++++++------------------
> > >  mk/rte.sdkbuild.mk     |   2 +-
> > >  mk/rte.sharelib.mk     |  54 ++++++++--------
> > >  mk/rte.vars.mk         |   4 --
> > >  7 files changed, 54 insertions(+), 257 deletions(-)
> > > 
> > > -- 
> > > 1.9.3
> > > 
> > > 
> > 
> > I see you removed the --whole-archive option when building the single library
> > here.  Have you checked to make sure that all the constructors haven't been
> > stripped out?
> 
> I am not entirely sure I follow. There is no --whole-archive when building libraries,
> at least not in my sources.
> The flag is used when linking apps and I have not removed it as you can see on patch 2/4.
> 
> Sergio
You're right, I saw in one of your patches --whole-archive, but it was context,
not actual change, sorry
Neil

> 
> > Neil
> > 
> 

[PATCH v3 0/6] Update libs build process

[Sergio Gonzalez Monroy]

As per the proposal, this patch set does:
 - Remove CONFIG_RTE_BUILD_COMBINE_LIBS as a configuration option.
 - For static library, build a single/combined library.
 - For shared libraries, build both individual/separated and single/combined
   libraries.
 - Link apps only against single/combined libs.
 - Include external shared libs dependencies when building shared libraries.

v3:
 - Split some of the patches for easier review
 - Improve patches descriptions

Sergio Gonzalez Monroy (6):
  Link combined shared library using CC
  Link apps only against single/combined library
  Remove CONFIG_RTE_BUILD_COMBINE_LIBS and related
  Update library build process
  Avoid duplicated code
  Link apps/DSOs against EXECENV_LDLIBS with --as-needed

 config/common_bsdapp   |   3 +-
 config/common_linuxapp |   3 +-
 mk/rte.app.mk          | 164 ++-----------------------------------------------
 mk/rte.lib.mk          |  81 ++++++------------------
 mk/rte.sdkbuild.mk     |   2 +-
 mk/rte.sharelib.mk     |  54 ++++++++--------
 mk/rte.vars.mk         |   4 --
 7 files changed, 54 insertions(+), 257 deletions(-)

-- 
1.9.3

[PATCH v3 1/6] Link combined shared library using CC

[Sergio Gonzalez Monroy]

This patch just enables to link libs with CC as we currently do for apps.

Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 mk/rte.sharelib.mk | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/mk/rte.sharelib.mk b/mk/rte.sharelib.mk
index c0a811a..73ce709 100644
--- a/mk/rte.sharelib.mk
+++ b/mk/rte.sharelib.mk
@@ -29,6 +29,8 @@
 #   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 #   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
+include $(RTE_SDK)/mk/internal/rte.build-pre.mk
+
 # VPATH contains at least SRCDIR
 VPATH += $(SRCDIR)
 
@@ -40,12 +42,20 @@ LIB_ONE := lib$(RTE_LIBNAME).a
 endif
 endif
 
+ifeq ($(LINK_USING_CC),1)
+# Override the definition of LD here, since we're linking with CC
+LD := $(CC)
+LD_MULDEFS := $(call linkerprefix,-z$(comma)muldefs)
+CPU_LDFLAGS := $(call linkerprefix,$(CPU_LDFLAGS))
+endif
+
 .PHONY:sharelib
 sharelib: $(LIB_ONE) FORCE
 
 OBJS = $(wildcard $(RTE_OUTPUT)/build/lib/*.o)
 
-O_TO_S = $(LD) $(CPU_LDFLAGS) -shared $(OBJS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
+O_TO_S = $(LD) $(CPU_LDFLAGS) $(LD_MULDEFS) -shared $(OBJS)\
+	-o $(RTE_OUTPUT)/lib/$(LIB_ONE)
 O_TO_S_STR = $(subst ','\'',$(O_TO_S)) #'# fix syntax highlight
 O_TO_S_DISP = $(if $(V),"$(O_TO_S_STR)","  LD $(@)")
 O_TO_S_CMD = "cmd_$@ = $(O_TO_S_STR)"
-- 
1.9.3

[PATCH v3 2/6] Link apps only against single/combined library

[Sergio Gonzalez Monroy]

Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 mk/rte.app.mk | 160 +---------------------------------------------------------
 1 file changed, 2 insertions(+), 158 deletions(-)

diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 34dff2a..5e00e67 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -60,164 +60,12 @@ LDLIBS += -L$(RTE_SDK_BIN)/lib
 ifeq ($(NO_AUTOLIBS),)
 
 LDLIBS += --whole-archive
-
-ifeq ($(CONFIG_RTE_LIBRTE_DISTRIBUTOR),y)
-LDLIBS += -lrte_distributor
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_KNI),y)
-ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),y)
-LDLIBS += -lrte_kni
-endif
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_IVSHMEM),y)
-ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),y)
-LDLIBS += -lrte_ivshmem
-endif
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_PIPELINE),y)
-LDLIBS += -lrte_pipeline
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_TABLE),y)
-LDLIBS += -lrte_table
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_PORT),y)
-LDLIBS += -lrte_port
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_TIMER),y)
-LDLIBS += -lrte_timer
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_HASH),y)
-LDLIBS += -lrte_hash
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_LPM),y)
-LDLIBS += -lrte_lpm
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_POWER),y)
-LDLIBS += -lrte_power
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_ACL),y)
-LDLIBS += -lrte_acl
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_METER),y)
-LDLIBS += -lrte_meter
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_SCHED),y)
-LDLIBS += -lrte_sched
-LDLIBS += -lm
-LDLIBS += -lrt
-endif
-
+LDLIBS += -l$(RTE_LIBNAME)
+LDLIBS += --no-whole-archive
 LDLIBS += --start-group
-
-ifeq ($(CONFIG_RTE_LIBRTE_KVARGS),y)
-LDLIBS += -lrte_kvargs
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_MBUF),y)
-LDLIBS += -lrte_mbuf
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_IP_FRAG),y)
-LDLIBS += -lrte_ip_frag
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_ETHER),y)
-LDLIBS += -lethdev
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_MALLOC),y)
-LDLIBS += -lrte_malloc
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_MEMPOOL),y)
-LDLIBS += -lrte_mempool
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_RING),y)
-LDLIBS += -lrte_ring
-endif
-
-ifeq ($(CONFIG_RTE_LIBC),y)
-LDLIBS += -lc
-LDLIBS += -lm
-endif
-
-ifeq ($(CONFIG_RTE_LIBGLOSS),y)
-LDLIBS += -lgloss
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_EAL),y)
-LDLIBS += -lrte_eal
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_CMDLINE),y)
-LDLIBS += -lrte_cmdline
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_CFGFILE),y)
-LDLIBS += -lrte_cfgfile
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_PMD_BOND),y)
-LDLIBS += -lrte_pmd_bond
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_PMD_XENVIRT),y)
-LDLIBS += -lrte_pmd_xenvirt
-LDLIBS += -lxenstore
-endif
-
-ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
-# plugins (link only if static libraries)
-
-ifeq ($(CONFIG_RTE_LIBRTE_VMXNET3_PMD),y)
-LDLIBS += -lrte_pmd_vmxnet3_uio
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_VIRTIO_PMD),y)
-LDLIBS += -lrte_pmd_virtio_uio
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_I40E_PMD),y)
-LDLIBS += -lrte_pmd_i40e
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_IXGBE_PMD),y)
-LDLIBS += -lrte_pmd_ixgbe
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_E1000_PMD),y)
-LDLIBS += -lrte_pmd_e1000
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_PMD_RING),y)
-LDLIBS += -lrte_pmd_ring
-endif
-
-ifeq ($(CONFIG_RTE_LIBRTE_PMD_PCAP),y)
-LDLIBS += -lrte_pmd_pcap -lpcap
-endif
-
-endif # plugins
-
 LDLIBS += $(EXECENV_LDLIBS)
-
 LDLIBS += --end-group
 
-LDLIBS += --no-whole-archive
-
 endif # ifeq ($(NO_AUTOLIBS),)
 
 LDLIBS += $(CPU_LDLIBS)
@@ -235,10 +83,6 @@ build: _postbuild
 
 exe2cmd = $(strip $(call dotfile,$(patsubst %,%.cmd,$(1))))
 
-ifeq ($(RTE_BUILD_COMBINE_LIBS),y)
-LDLIBS += -l$(RTE_LIBNAME)
-endif
-
 ifeq ($(LINK_USING_CC),1)
 LDLIBS := $(call linkerprefix,$(LDLIBS))
 LDFLAGS := $(call linkerprefix,$(LDFLAGS))
-- 
1.9.3

[PATCH v3 3/6] Remove CONFIG_RTE_BUILD_COMBINE_LIBS and related

[Sergio Gonzalez Monroy]

Remove the configuration variable CONFIG_RTE_BUILD_COMBINE_LIBS and any
code related to it.

Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 config/common_bsdapp   |  3 +--
 config/common_linuxapp |  3 +--
 mk/rte.lib.mk          | 34 ----------------------------------
 mk/rte.sdkbuild.mk     |  2 +-
 mk/rte.sharelib.mk     |  4 ----
 mk/rte.vars.mk         |  4 ----
 6 files changed, 3 insertions(+), 47 deletions(-)

diff --git a/config/common_bsdapp b/config/common_bsdapp
index eebd05b..65d2ecc 100644
--- a/config/common_bsdapp
+++ b/config/common_bsdapp
@@ -79,9 +79,8 @@ CONFIG_RTE_FORCE_INTRINSICS=n
 CONFIG_RTE_BUILD_SHARED_LIB=n
 
 #
-# Combine to one single library
+# Library name
 #
-CONFIG_RTE_BUILD_COMBINE_LIBS=n
 CONFIG_RTE_LIBNAME=intel_dpdk
 
 #
diff --git a/config/common_linuxapp b/config/common_linuxapp
index 4713eb4..5bdcdf3 100644
--- a/config/common_linuxapp
+++ b/config/common_linuxapp
@@ -79,9 +79,8 @@ CONFIG_RTE_FORCE_INTRINSICS=n
 CONFIG_RTE_BUILD_SHARED_LIB=n
 
 #
-# Combine to one single library
+# Library name
 #
-CONFIG_RTE_BUILD_COMBINE_LIBS=n
 CONFIG_RTE_LIBNAME="intel_dpdk"
 
 #
diff --git a/mk/rte.lib.mk b/mk/rte.lib.mk
index f458258..53a149d 100644
--- a/mk/rte.lib.mk
+++ b/mk/rte.lib.mk
@@ -83,24 +83,6 @@ O_TO_S_DO = @set -e; \
 	$(O_TO_S) && \
 	echo $(O_TO_S_CMD) > $(call exe2cmd,$(@))
 
-ifeq ($(RTE_BUILD_SHARED_LIB),n)
-O_TO_C = $(AR) crus $(LIB_ONE) $(OBJS-y)
-O_TO_C_STR = $(subst ','\'',$(O_TO_C)) #'# fix syntax highlight
-O_TO_C_DISP = $(if $(V),"$(O_TO_C_STR)","  AR_C $(@)")
-O_TO_C_DO = @set -e; \
-	$(lib_dir) \
-	$(copy_obj)
-else
-O_TO_C = $(LD) $(LD_MULDEFS) -shared $(OBJS-y) -o $(LIB_ONE)
-O_TO_C_STR = $(subst ','\'',$(O_TO_C)) #'# fix syntax highlight
-O_TO_C_DISP = $(if $(V),"$(O_TO_C_STR)","  LD_C $(@)")
-O_TO_C_DO = @set -e; \
-	$(lib_dir) \
-	$(copy_obj)
-endif
-
-copy_obj = cp -f $(OBJS-y) $(RTE_OUTPUT)/build/lib;
-lib_dir = [ -d $(RTE_OUTPUT)/lib ] || mkdir -p $(RTE_OUTPUT)/lib;
 -include .$(LIB).cmd
 
 #
@@ -121,14 +103,6 @@ $(LIB): $(OBJS-y) $(DEP_$(LIB)) FORCE
 		$(depfile_missing),\
 		$(depfile_newer)),\
 		$(O_TO_S_DO))
-ifeq ($(RTE_BUILD_COMBINE_LIBS),y)
-	$(if $(or \
-        $(file_missing),\
-        $(call cmdline_changed,$(O_TO_C_STR)),\
-        $(depfile_missing),\
-        $(depfile_newer)),\
-        $(O_TO_C_DO))
-endif
 else
 $(LIB): $(OBJS-y) $(DEP_$(LIB)) FORCE
 	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
@@ -144,14 +118,6 @@ $(LIB): $(OBJS-y) $(DEP_$(LIB)) FORCE
 	    $(depfile_missing),\
 	    $(depfile_newer)),\
 	    $(O_TO_A_DO))
-ifeq ($(RTE_BUILD_COMBINE_LIBS),y)
-	$(if $(or \
-        $(file_missing),\
-        $(call cmdline_changed,$(O_TO_C_STR)),\
-        $(depfile_missing),\
-        $(depfile_newer)),\
-        $(O_TO_C_DO))
-endif
 endif
 
 #
diff --git a/mk/rte.sdkbuild.mk b/mk/rte.sdkbuild.mk
index 3154457..f20ec1e 100644
--- a/mk/rte.sdkbuild.mk
+++ b/mk/rte.sdkbuild.mk
@@ -93,7 +93,7 @@ $(ROOTDIRS-y):
 	@[ -d $(BUILDDIR)/$@ ] || mkdir -p $(BUILDDIR)/$@
 	@echo "== Build $@"
 	$(Q)$(MAKE) S=$@ -f $(RTE_SRCDIR)/$@/Makefile -C $(BUILDDIR)/$@ all
-	@if [ $@ = lib -a $(RTE_BUILD_COMBINE_LIBS) = y ]; then \
+	@if [ $@ = lib ]; then \
 		$(MAKE) -f $(RTE_SDK)/lib/Makefile sharelib; \
 	fi
 
diff --git a/mk/rte.sharelib.mk b/mk/rte.sharelib.mk
index 73ce709..b32bd35 100644
--- a/mk/rte.sharelib.mk
+++ b/mk/rte.sharelib.mk
@@ -34,13 +34,11 @@ include $(RTE_SDK)/mk/internal/rte.build-pre.mk
 # VPATH contains at least SRCDIR
 VPATH += $(SRCDIR)
 
-ifeq ($(RTE_BUILD_COMBINE_LIBS),y)
 ifeq ($(RTE_BUILD_SHARED_LIB),y)
 LIB_ONE := lib$(RTE_LIBNAME).so
 else
 LIB_ONE := lib$(RTE_LIBNAME).a
 endif
-endif
 
 ifeq ($(LINK_USING_CC),1)
 # Override the definition of LD here, since we're linking with CC
@@ -74,7 +72,6 @@ O_TO_A_DO = @set -e; \
 # Archive objects to share library
 #
 
-ifeq ($(RTE_BUILD_COMBINE_LIBS),y)
 ifeq ($(RTE_BUILD_SHARED_LIB),y)
 $(LIB_ONE): FORCE
 	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
@@ -84,7 +81,6 @@ $(LIB_ONE): FORCE
 	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
 	$(O_TO_A_DO)
 endif
-endif
 
 #
 # Clean all generated files
diff --git a/mk/rte.vars.mk b/mk/rte.vars.mk
index 1e874ee..5bcc4a5 100644
--- a/mk/rte.vars.mk
+++ b/mk/rte.vars.mk
@@ -67,10 +67,6 @@ ifneq ($(BUILDING_RTE_SDK),)
   ifeq ($(RTE_BUILD_SHARED_LIB),)
     RTE_BUILD_SHARED_LIB := n
   endif
-  RTE_BUILD_COMBINE_LIBS := $(CONFIG_RTE_BUILD_COMBINE_LIBS:"%"=%)
-  ifeq ($(RTE_BUILD_COMBINE_LIBS),)
-    RTE_BUILD_COMBINE_LIBS := n
-  endif
   RTE_LIBNAME := $(CONFIG_RTE_LIBNAME:"%"=%)
   ifeq ($(RTE_LIBNAME),)
     RTE_LIBNAME := intel_dpdk
-- 
1.9.3

[PATCH v3 4/6] Update library build process

[Sergio Gonzalez Monroy]

Remove all code related to building a separated static libs, and by default
only build a single/combined library when building static libs.
Build both separated and combined libraries when building shared libs.

Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 mk/rte.lib.mk | 39 +++++++++++----------------------------
 1 file changed, 11 insertions(+), 28 deletions(-)

diff --git a/mk/rte.lib.mk b/mk/rte.lib.mk
index 53a149d..e7420bf 100644
--- a/mk/rte.lib.mk
+++ b/mk/rte.lib.mk
@@ -66,19 +66,12 @@ LD_MULDEFS := $(call linkerprefix,-z$(comma)muldefs)
 CPU_LDFLAGS := $(call linkerprefix,$(CPU_LDFLAGS))
 endif
 
-O_TO_A = $(AR) crus $(LIB) $(OBJS-y)
-O_TO_A_STR = $(subst ','\'',$(O_TO_A)) #'# fix syntax highlight
-O_TO_A_DISP = $(if $(V),"$(O_TO_A_STR)","  AR $(@)")
-O_TO_A_CMD = "cmd_$@ = $(O_TO_A_STR)"
-O_TO_A_DO = @set -e; \
-	echo $(O_TO_A_DISP); \
-	$(O_TO_A) && \
-	echo $(O_TO_A_CMD) > $(call exe2cmd,$(@))
-
 O_TO_S = $(LD) $(CPU_LDFLAGS) $(LD_MULDEFS) -shared $(OBJS-y) -o $(LIB)
-O_TO_S_STR = $(subst ','\'',$(O_TO_S)) #'# fix syntax highlight
+O_TO_S_STR = $(subst ','\'',$(O_TO_S)) #') # fix syntax highlight
 O_TO_S_DISP = $(if $(V),"$(O_TO_S_STR)","  LD $(@)")
+O_TO_S_CMD = "cmd_$@ = $(O_TO_S_STR)"
 O_TO_S_DO = @set -e; \
+	cp -f $(OBJS-y) $(RTE_OUTPUT)/build/lib; \
 	echo $(O_TO_S_DISP); \
 	$(O_TO_S) && \
 	echo $(O_TO_S_CMD) > $(call exe2cmd,$(@))
@@ -88,8 +81,8 @@ O_TO_S_DO = @set -e; \
 #
 # Archive objects in .a file if needed
 #
-ifeq ($(RTE_BUILD_SHARED_LIB),y)
 $(LIB): $(OBJS-y) $(DEP_$(LIB)) FORCE
+ifeq ($(RTE_BUILD_SHARED_LIB),y)
 	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
 	$(if $(D),\
 		@echo -n "$< -> $@ " ; \
@@ -98,35 +91,25 @@ $(LIB): $(OBJS-y) $(DEP_$(LIB)) FORCE
 		echo -n "depfile_missing=$(call boolean,$(depfile_missing)) " ; \
 		echo "depfile_newer=$(call boolean,$(depfile_newer)) ")
 	$(if $(or \
-		$(file_missing),\
-		$(call cmdline_changed,$(O_TO_S_STR)),\
-		$(depfile_missing),\
-		$(depfile_newer)),\
-		$(O_TO_S_DO))
-else
-$(LIB): $(OBJS-y) $(DEP_$(LIB)) FORCE
-	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
-	$(if $(D),\
-	    @echo -n "$< -> $@ " ; \
-	    echo -n "file_missing=$(call boolean,$(file_missing)) " ; \
-	    echo -n "cmdline_changed=$(call boolean,$(call cmdline_changed,$(O_TO_A_STR))) " ; \
-	    echo -n "depfile_missing=$(call boolean,$(depfile_missing)) " ; \
-	    echo "depfile_newer=$(call boolean,$(depfile_newer)) ")
-	$(if $(or \
 	    $(file_missing),\
-	    $(call cmdline_changed,$(O_TO_A_STR)),\
+	    $(call cmdline_changed,$(O_TO_S_STR)),\
 	    $(depfile_missing),\
 	    $(depfile_newer)),\
-	    $(O_TO_A_DO))
+	    $(O_TO_S_DO))
+else
+	@set -e; \
+	cp -f $(OBJS-y) $(RTE_OUTPUT)/build/lib;
 endif
 
 #
 # install lib in $(RTE_OUTPUT)/lib
 #
 $(RTE_OUTPUT)/lib/$(LIB): $(LIB)
+ifeq ($(RTE_BUILD_SHARED_LIB),y)
 	@echo "  INSTALL-LIB $(LIB)"
 	@[ -d $(RTE_OUTPUT)/lib ] || mkdir -p $(RTE_OUTPUT)/lib
 	$(Q)cp -f $(LIB) $(RTE_OUTPUT)/lib
+endif
 
 #
 # Clean all generated files
-- 
1.9.3

[PATCH v3 5/6] Avoid duplicated code

[Sergio Gonzalez Monroy]

Rewrite rules to avoid duplicated code.

Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 mk/rte.sharelib.mk | 37 +++++++++++++++----------------------
 1 file changed, 15 insertions(+), 22 deletions(-)

diff --git a/mk/rte.sharelib.mk b/mk/rte.sharelib.mk
index b32bd35..8fc6548 100644
--- a/mk/rte.sharelib.mk
+++ b/mk/rte.sharelib.mk
@@ -52,35 +52,28 @@ sharelib: $(LIB_ONE) FORCE
 
 OBJS = $(wildcard $(RTE_OUTPUT)/build/lib/*.o)
 
-O_TO_S = $(LD) $(CPU_LDFLAGS) $(LD_MULDEFS) -shared $(OBJS)\
+ifeq ($(RTE_BUILD_SHARED_LIB),y)
+O_TO_L = $(LD) $(CPU_LDFLAGS) $(LD_MULDEFS) -shared $(OBJS)\
 	-o $(RTE_OUTPUT)/lib/$(LIB_ONE)
-O_TO_S_STR = $(subst ','\'',$(O_TO_S)) #'# fix syntax highlight
-O_TO_S_DISP = $(if $(V),"$(O_TO_S_STR)","  LD $(@)")
-O_TO_S_CMD = "cmd_$@ = $(O_TO_S_STR)"
-O_TO_S_DO = @set -e; \
-    echo $(O_TO_S_DISP); \
-    $(O_TO_S)
+L_DISP=LD
+else
+O_TO_L = $(AR) crus $(RTE_OUTPUT)/lib/$(LIB_ONE) $(OBJS)
+L_DISP=AR
+endif
+O_TO_L_STR = $(subst ','\'',$(O_TO_L)) #')# fix syntax highlight
+O_TO_L_DISP = $(if $(V),"$(O_TO_L_STR)","  $(L_DISP) $(@)")
+O_TO_L_CMD = "cmd_$@ = $(O_TO_L_STR)"
+O_TO_L_DO = @set -e; \
+	mkdir -p $(RTE_OUTPUT)/lib; \
+	echo $(O_TO_L_DISP); \
+	$(O_TO_L)
 
-O_TO_A =  $(AR) crus $(RTE_OUTPUT)/lib/$(LIB_ONE) $(OBJS)
-O_TO_A_STR = $(subst ','\'',$(O_TO_A)) #'# fix syntax highlight
-O_TO_A_DISP = $(if $(V),"$(O_TO_A_STR)","  LD $(@)")
-O_TO_A_CMD = "cmd_$@ = $(O_TO_A_STR)"
-O_TO_A_DO = @set -e; \
-    echo $(O_TO_A_DISP); \
-    $(O_TO_A)
 #
 # Archive objects to share library
 #
 
-ifeq ($(RTE_BUILD_SHARED_LIB),y)
 $(LIB_ONE): FORCE
-	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
-	$(O_TO_S_DO)
-else
-$(LIB_ONE): FORCE
-	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
-	$(O_TO_A_DO)
-endif
+	$(O_TO_L_DO)
 
 #
 # Clean all generated files
-- 
1.9.3

[PATCH v3 6/6] Link apps/DSOs against EXECENV_LDLIBS with --as-needed

[Sergio Gonzalez Monroy]

Include external shared libs dependencies when building shared
libraries.

Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 mk/rte.app.mk      | 4 ++--
 mk/rte.lib.mk      | 8 +++++++-
 mk/rte.sharelib.mk | 7 ++++++-
 3 files changed, 15 insertions(+), 4 deletions(-)

diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 5e00e67..e775ad7 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -62,9 +62,9 @@ ifeq ($(NO_AUTOLIBS),)
 LDLIBS += --whole-archive
 LDLIBS += -l$(RTE_LIBNAME)
 LDLIBS += --no-whole-archive
-LDLIBS += --start-group
+LDLIBS += --as-needed
 LDLIBS += $(EXECENV_LDLIBS)
-LDLIBS += --end-group
+LDLIBS += --no-as-needed
 
 endif # ifeq ($(NO_AUTOLIBS),)
 
diff --git a/mk/rte.lib.mk b/mk/rte.lib.mk
index e7420bf..947e17d 100644
--- a/mk/rte.lib.mk
+++ b/mk/rte.lib.mk
@@ -59,14 +59,20 @@ build: _postbuild
 
 exe2cmd = $(strip $(call dotfile,$(patsubst %,%.cmd,$(1))))
 
+O_TO_S_LDLIBS := --as-needed
+O_TO_S_LDLIBS += $(EXECENV_LDLIBS)
+O_TO_S_LDLIBS += --no-as-needed
+
 ifeq ($(LINK_USING_CC),1)
 # Override the definition of LD here, since we're linking with CC
 LD := $(CC)
 LD_MULDEFS := $(call linkerprefix,-z$(comma)muldefs)
 CPU_LDFLAGS := $(call linkerprefix,$(CPU_LDFLAGS))
+O_TO_S_LDLIBS := $(call linkerprefix,$(O_TO_S_LDLIBS))
 endif
 
-O_TO_S = $(LD) $(CPU_LDFLAGS) $(LD_MULDEFS) -shared $(OBJS-y) -o $(LIB)
+O_TO_S = $(LD) $(CPU_LDFLAGS) $(LD_MULDEFS) -shared $(OBJS-y) \
+	$(O_TO_S_LDLIBS) -o $(LIB)
 O_TO_S_STR = $(subst ','\'',$(O_TO_S)) #') # fix syntax highlight
 O_TO_S_DISP = $(if $(V),"$(O_TO_S_STR)","  LD $(@)")
 O_TO_S_CMD = "cmd_$@ = $(O_TO_S_STR)"
diff --git a/mk/rte.sharelib.mk b/mk/rte.sharelib.mk
index 8fc6548..380aa7d 100644
--- a/mk/rte.sharelib.mk
+++ b/mk/rte.sharelib.mk
@@ -40,11 +40,16 @@ else
 LIB_ONE := lib$(RTE_LIBNAME).a
 endif
 
+O_TO_L_LDLIBS := --as-needed
+O_TO_L_LDLIBS += $(EXECENV_LDLIBS)
+O_TO_L_LDLIBS += --no-as-needed
+
 ifeq ($(LINK_USING_CC),1)
 # Override the definition of LD here, since we're linking with CC
 LD := $(CC)
 LD_MULDEFS := $(call linkerprefix,-z$(comma)muldefs)
 CPU_LDFLAGS := $(call linkerprefix,$(CPU_LDFLAGS))
+O_TO_L_LDLIBS := $(call linkerprefix,$(O_TO_L_LDLIBS))
 endif
 
 .PHONY:sharelib
@@ -54,7 +59,7 @@ OBJS = $(wildcard $(RTE_OUTPUT)/build/lib/*.o)
 
 ifeq ($(RTE_BUILD_SHARED_LIB),y)
 O_TO_L = $(LD) $(CPU_LDFLAGS) $(LD_MULDEFS) -shared $(OBJS)\
-	-o $(RTE_OUTPUT)/lib/$(LIB_ONE)
+	$(O_TO_L_LDLIBS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
 L_DISP=LD
 else
 O_TO_L = $(AR) crus $(RTE_OUTPUT)/lib/$(LIB_ONE) $(OBJS)
-- 
1.9.3

Re: [PATCH v3 0/6] Update libs build process

[Gonzalez Monroy, Sergio]

Are there any more comments on this patch set?

Thanks,
Sergio


> -----Original Message-----
> From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Sergio Gonzalez
> Monroy
> Sent: Thursday, October 9, 2014 2:05 PM
> To: dev-VfR2kkLFssw@public.gmane.org
> Subject: [dpdk-dev] [PATCH v3 0/6] Update libs build process
> 
> As per the proposal, this patch set does:
>  - Remove CONFIG_RTE_BUILD_COMBINE_LIBS as a configuration option.
>  - For static library, build a single/combined library.
>  - For shared libraries, build both individual/separated and single/combined
>    libraries.
>  - Link apps only against single/combined libs.
>  - Include external shared libs dependencies when building shared libraries.
> 
> v3:
>  - Split some of the patches for easier review
>  - Improve patches descriptions
> 
> Sergio Gonzalez Monroy (6):
>   Link combined shared library using CC
>   Link apps only against single/combined library
>   Remove CONFIG_RTE_BUILD_COMBINE_LIBS and related
>   Update library build process
>   Avoid duplicated code
>   Link apps/DSOs against EXECENV_LDLIBS with --as-needed
> 
>  config/common_bsdapp   |   3 +-
>  config/common_linuxapp |   3 +-
>  mk/rte.app.mk          | 164 ++-----------------------------------------------
>  mk/rte.lib.mk          |  81 ++++++------------------
>  mk/rte.sdkbuild.mk     |   2 +-
>  mk/rte.sharelib.mk     |  54 ++++++++--------
>  mk/rte.vars.mk         |   4 --
>  7 files changed, 54 insertions(+), 257 deletions(-)
> 
> --
> 1.9.3

Re: [PATCH v3 0/6] Update libs build process

[Gonzalez Monroy, Sergio]

Hi Thomas,

Given that most of the comments/discussion for this patch set revolved 
around the removal of COMBINE_LIBS and what libs to build by default, 
I am inclined to drop this patch set, submit minimal patch to fix 
compiler errors (initial and main purpose of this patch set) and then 
submit an RFC regarding the use/removal of COMBINE_LIBS and other outstanding issues in the build system.

Does that sound like a better approach?

Thanks,
Sergio

> -----Original Message-----
> From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Gonzalez Monroy,
> Sergio
> Sent: Monday, October 13, 2014 5:02 PM
> To: dev-VfR2kkLFssw@public.gmane.org
> Subject: Re: [dpdk-dev] [PATCH v3 0/6] Update libs build process
> 
> Are there any more comments on this patch set?
> 
> Thanks,
> Sergio
> 
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Sergio Gonzalez
> > Monroy
> > Sent: Thursday, October 9, 2014 2:05 PM
> > To: dev-VfR2kkLFssw@public.gmane.org
> > Subject: [dpdk-dev] [PATCH v3 0/6] Update libs build process
> >
> > As per the proposal, this patch set does:
> >  - Remove CONFIG_RTE_BUILD_COMBINE_LIBS as a configuration option.
> >  - For static library, build a single/combined library.
> >  - For shared libraries, build both individual/separated and single/combined
> >    libraries.
> >  - Link apps only against single/combined libs.
> >  - Include external shared libs dependencies when building shared libraries.
> >
> > v3:
> >  - Split some of the patches for easier review
> >  - Improve patches descriptions
> >
> > Sergio Gonzalez Monroy (6):
> >   Link combined shared library using CC
> >   Link apps only against single/combined library
> >   Remove CONFIG_RTE_BUILD_COMBINE_LIBS and related
> >   Update library build process
> >   Avoid duplicated code
> >   Link apps/DSOs against EXECENV_LDLIBS with --as-needed
> >
> >  config/common_bsdapp   |   3 +-
> >  config/common_linuxapp |   3 +-
> >  mk/rte.app.mk          | 164 ++-----------------------------------------------
> >  mk/rte.lib.mk          |  81 ++++++------------------
> >  mk/rte.sdkbuild.mk     |   2 +-
> >  mk/rte.sharelib.mk     |  54 ++++++++--------
> >  mk/rte.vars.mk         |   4 --
> >  7 files changed, 54 insertions(+), 257 deletions(-)
> >
> > --
> > 1.9.3

Re: [PATCH v3 0/6] Update libs build process

[Gonzalez Monroy, Sergio]

Dropping patch set.

Thanks,
Sergio

> -----Original Message-----
> From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Gonzalez Monroy,
> Sergio
> Sent: Tuesday, October 21, 2014 10:44 AM
> To: Thomas Monjalon
> Cc: dev-VfR2kkLFssw@public.gmane.org
> Subject: Re: [dpdk-dev] [PATCH v3 0/6] Update libs build process
> 
> Hi Thomas,
> 
> Given that most of the comments/discussion for this patch set revolved
> around the removal of COMBINE_LIBS and what libs to build by default, I am
> inclined to drop this patch set, submit minimal patch to fix compiler errors
> (initial and main purpose of this patch set) and then submit an RFC regarding
> the use/removal of COMBINE_LIBS and other outstanding issues in the build
> system.
> 
> Does that sound like a better approach?
> 
> Thanks,
> Sergio
>