Underlinked libs and overlinked applications - an issue?

Underlinked libs and overlinked applications - an issue?

[Christian Ehrhardt]

Hi,
I was working on the new 16.04 build system to adapt deb packaging to it.
I remember somewhen back in the DPDK 2.2 and shared+combined library days I
had some issues with over/underlinking - but it seems those are still
existent or came back.

After a build in almost default config (just disabled the kernel modules)
and set RTE_MACHINE to default I find the following.

#1
The libraries are all only linked against external things - even clearly
using internal structures:
ldd usr/lib/x86_64-linux-gnu/librte_lpm.so.2
       linux-vdso.so.1 =>  (0x00007fff7e7a5000)
       libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f175d4dd000)
       /lib64/ld-linux-x86-64.so.2 (0x0000558d3afbf000)

#2
The Application then seem to try to make up for that by realizing all that
is missing.
But looking at the app alone it seems overlinked by that - it is not using
all of these on its own.
ldd usr/bin/cmdline_test
       linux-vdso.so.1 =>  (0x00007ffeec9ea000)
       librte_distributor.so.1 => not found
       librte_reorder.so.1 => not found
[...]
       librte_jobstats.so.1 => not found
[...]
And for example none of the librte_jobstats.so.1 symbols are used
"directly" in there.


I'm still digging into that concept of using a linker script for all of
that and some of the new implications by that. And eventually thing "work",
but this linking at least feels wrong to me.


So I wanted to ask - is that intentional - or should that be fixed?
If it should be fixed are there obvious suggestions where/how?
And if it is intentional - could one be so nice to elaborate it a bit for
me - thanks in advance.


Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd

Re: Underlinked libs and overlinked applications - an issue?

[Thomas Monjalon]

2016-05-19 17:38, Christian Ehrhardt:
> Hi,
> I was working on the new 16.04 build system to adapt deb packaging to it.
> I remember somewhen back in the DPDK 2.2 and shared+combined library days I
> had some issues with over/underlinking - but it seems those are still
> existent or came back.

I would say it has always been like that.
Thanks for raising the issue.

> After a build in almost default config (just disabled the kernel modules)
> and set RTE_MACHINE to default I find the following.
> 
> #1
> The libraries are all only linked against external things - even clearly
> using internal structures:
> ldd usr/lib/x86_64-linux-gnu/librte_lpm.so.2
>        linux-vdso.so.1 =>  (0x00007fff7e7a5000)
>        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f175d4dd000)
>        /lib64/ld-linux-x86-64.so.2 (0x0000558d3afbf000)

The DT_NEEDED entries are created only for external dependencies.
Probably we should create ones for internal dependencies based on the
variable DEPDIRS-y.

> #2
> The Application then seem to try to make up for that by realizing all that
> is missing.
> But looking at the app alone it seems overlinked by that - it is not using
> all of these on its own.
> ldd usr/bin/cmdline_test
>        linux-vdso.so.1 =>  (0x00007ffeec9ea000)
>        librte_distributor.so.1 => not found
>        librte_reorder.so.1 => not found
> [...]
>        librte_jobstats.so.1 => not found
> [...]
> And for example none of the librte_jobstats.so.1 symbols are used
> "directly" in there.

Yes every libraries are put for every apps in rte.app.mk.
Probably that we could better use DEPDIRS-y for apps.

Re: Underlinked libs and overlinked applications - an issue?

[Panu Matilainen]

On 05/19/2016 09:17 PM, Thomas Monjalon wrote:
> 2016-05-19 17:38, Christian Ehrhardt:
>> Hi,
>> I was working on the new 16.04 build system to adapt deb packaging to it.
>> I remember somewhen back in the DPDK 2.2 and shared+combined library days I
>> had some issues with over/underlinking - but it seems those are still
>> existent or came back.
>
> I would say it has always been like that.
> Thanks for raising the issue.
>
>> After a build in almost default config (just disabled the kernel modules)
>> and set RTE_MACHINE to default I find the following.
>>
>> #1
>> The libraries are all only linked against external things - even clearly
>> using internal structures:
>> ldd usr/lib/x86_64-linux-gnu/librte_lpm.so.2
>>        linux-vdso.so.1 =>  (0x00007fff7e7a5000)
>>        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f175d4dd000)
>>        /lib64/ld-linux-x86-64.so.2 (0x0000558d3afbf000)
>
> The DT_NEEDED entries are created only for external dependencies.
> Probably we should create ones for internal dependencies based on the
> variable DEPDIRS-y.

Yes, DT_NEEDED for internal dependencies are needed too, see eg recent 
commits c6417ce61f83 and 8180554d82b3. That was part of Sergios' build 
system improvement patch series last spring or so (but since then 
abandoned), I picked up the work and been doing it in smaller pieces. 
Phase 1 was the external dependencies, phase 2 is internal dependencies 
which I'm working on. Unfortunately health issues have stalled my work a 
lot recently :-/

>> #2
>> The Application then seem to try to make up for that by realizing all that
>> is missing.
>> But looking at the app alone it seems overlinked by that - it is not using
>> all of these on its own.
>> ldd usr/bin/cmdline_test
>>        linux-vdso.so.1 =>  (0x00007ffeec9ea000)
>>        librte_distributor.so.1 => not found
>>        librte_reorder.so.1 => not found
>> [...]
>>        librte_jobstats.so.1 => not found
>> [...]
>> And for example none of the librte_jobstats.so.1 symbols are used
>> "directly" in there.
>
> Yes every libraries are put for every apps in rte.app.mk.
> Probably that we could better use DEPDIRS-y for apps.
>

Another trick from Sergios' original patch series is to utilize 
AS_NEEDED() in the linker script, so it ends up looking something like 
this for shared library config (static would remain as it is now):

INPUT ( librte_eal librte_mempool librte_ring
         AS_NEEDED ( librte_acl librte_timer librte_vhost <...> )
)

...and then use the linker script for linking the apps, which should 
simplify rte.app.mk considerably. Or so the theory goes. Anyway, the 
above requires DT_NEEDED for the internal libraries to be present first, 
but once all these pieces are in place, dynamically linked apps will 
only link to the libraries they actually need.

	- Panu -

	- Panu -

Re: Underlinked libs and overlinked applications - an issue?

[Christian Ehrhardt]

On Fri, May 20, 2016 at 1:01 PM, Panu Matilainen <pmatilai@redhat.com>
wrote:

> On 05/19/2016 09:17 PM, Thomas Monjalon wrote:
>
>> 2016-05-19 17:38, Christian Ehrhardt:
>>
>>> Hi,
>>> I was working on the new 16.04 build system to adapt deb packaging to it.
>>> I remember somewhen back in the DPDK 2.2 and shared+combined library
>>> days I
>>> had some issues with over/underlinking - but it seems those are still
>>> existent or came back.
>>>
>>
>> I would say it has always been like that.
>> Thanks for raising the issue.
>>
>> After a build in almost default config (just disabled the kernel modules)
>>> and set RTE_MACHINE to default I find the following.
>>>
>>> #1
>>> The libraries are all only linked against external things - even clearly
>>> using internal structures:
>>> ldd usr/lib/x86_64-linux-gnu/librte_lpm.so.2
>>>        linux-vdso.so.1 =>  (0x00007fff7e7a5000)
>>>        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f175d4dd000)
>>>        /lib64/ld-linux-x86-64.so.2 (0x0000558d3afbf000)
>>>
>>
>> The DT_NEEDED entries are created only for external dependencies.
>> Probably we should create ones for internal dependencies based on the
>> variable DEPDIRS-y.
>>
>
> Yes, DT_NEEDED for internal dependencies are needed too, see eg recent
> commits c6417ce61f83 and 8180554d82b3. That was part of Sergios' build
> system improvement patch series last spring or so (but since then
> abandoned), I picked up the work and been doing it in smaller pieces. Phase
> 1 was the external dependencies, phase 2 is internal dependencies which I'm
> working on. Unfortunately health issues have stalled my work a lot recently
> :-/


Since I work on 16.04 as released currently I almost missed those, but I at
least remember seeing the first on the list.
I had almost the same change - adopting yours since it is accepted.
Maybe I can provide a few more baby steps on that path - or at least a few
more pieces for discussion and review.
I hope you get well soon Panu!


> #2
>>> The Application then seem to try to make up for that by realizing all
>>> that
>>> is missing.
>>> But looking at the app alone it seems overlinked by that - it is not
>>> using
>>> all of these on its own.
>>> ldd usr/bin/cmdline_test
>>>        linux-vdso.so.1 =>  (0x00007ffeec9ea000)
>>>        librte_distributor.so.1 => not found
>>>        librte_reorder.so.1 => not found
>>> [...]
>>>        librte_jobstats.so.1 => not found
>>> [...]
>>> And for example none of the librte_jobstats.so.1 symbols are used
>>> "directly" in there.
>>>
>>
>> Yes every libraries are put for every apps in rte.app.mk.
>> Probably that we could better use DEPDIRS-y for apps.
>>
>>
> Another trick from Sergios' original patch series is to utilize
> AS_NEEDED() in the linker script, so it ends up looking something like this
> for shared library config (static would remain as it is now):
>
> INPUT ( librte_eal librte_mempool librte_ring
>         AS_NEEDED ( librte_acl librte_timer librte_vhost <...> )
> )
>
> ...and then use the linker script for linking the apps, which should
> simplify rte.app.mk considerably. Or so the theory goes. Anyway, the
> above requires DT_NEEDED for the internal libraries to be present first,
> but once all these pieces are in place, dynamically linked apps will only
> link to the libraries they actually need.
>

Interesting approach, I wasn't even working on DPDK back then - that could
really simplify the mk there.
I was experimenting with just dropping the "no-as-needed" in
mk/exec-env/linuxapp/rte.vars.mk, but that seems to shoot too far - haven't
had the time to look deeper.
Interested if your approach could help.

But the other part of it - getting the proper DT_NEEDED into the libs (I
also consider underlinking worse than overlinking) seems to be just missing
a bunch of -l - my last test build looked fine.
There is the underlinking issue of librte_eal left that Sergio mentioned
back in 6d25d90c, but it seems to be the next baby step to solve most else
of the underlinking.
I'll adapt it to git level and at least throw it out for discussion later
on.

Re: Underlinked libs and overlinked applications - an issue?

[Christian Ehrhardt]

ok,
- little baby steps first
  I'll submit a trivial patch in reply to this fixing the underlinking in
all .so's, except librte_eal.

- discussion on the remaining open issue second.
  For the remaining issue I think that needs a special understanding of
linkers I still need to grasp.
  But since I failed to find a good explanation for the exact case we face
here I put it in a generalized form on StackOverflow so eventually more
than just us can benefit from the answer once/if we find an answer.

So to all of you - if you are not scared of the latter pages of ld/gcc man
pages :-) feel free to take a look at
http://stackoverflow.com/questions/37351699/how-to-create-both-so-files-for-two-circular-depending-libraries

For us, but not important for the question in general:
 liba = librte_eal, libb = librte_mempool
(and ring, ivshmem, but once it is solved once it is solved)

Time to stip for now, have a great Weekend
Christian



Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd

On Fri, May 20, 2016 at 2:41 PM, Christian Ehrhardt <
christian.ehrhardt@canonical.com> wrote:

> On Fri, May 20, 2016 at 1:01 PM, Panu Matilainen <pmatilai@redhat.com>
> wrote:
>
>> On 05/19/2016 09:17 PM, Thomas Monjalon wrote:
>>
>>> 2016-05-19 17:38, Christian Ehrhardt:
>>>
>>>> Hi,
>>>> I was working on the new 16.04 build system to adapt deb packaging to
>>>> it.
>>>> I remember somewhen back in the DPDK 2.2 and shared+combined library
>>>> days I
>>>> had some issues with over/underlinking - but it seems those are still
>>>> existent or came back.
>>>>
>>>
>>> I would say it has always been like that.
>>> Thanks for raising the issue.
>>>
>>> After a build in almost default config (just disabled the kernel modules)
>>>> and set RTE_MACHINE to default I find the following.
>>>>
>>>> #1
>>>> The libraries are all only linked against external things - even clearly
>>>> using internal structures:
>>>> ldd usr/lib/x86_64-linux-gnu/librte_lpm.so.2
>>>>        linux-vdso.so.1 =>  (0x00007fff7e7a5000)
>>>>        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f175d4dd000)
>>>>        /lib64/ld-linux-x86-64.so.2 (0x0000558d3afbf000)
>>>>
>>>
>>> The DT_NEEDED entries are created only for external dependencies.
>>> Probably we should create ones for internal dependencies based on the
>>> variable DEPDIRS-y.
>>>
>>
>> Yes, DT_NEEDED for internal dependencies are needed too, see eg recent
>> commits c6417ce61f83 and 8180554d82b3. That was part of Sergios' build
>> system improvement patch series last spring or so (but since then
>> abandoned), I picked up the work and been doing it in smaller pieces. Phase
>> 1 was the external dependencies, phase 2 is internal dependencies which I'm
>> working on. Unfortunately health issues have stalled my work a lot recently
>> :-/
>
>
> Since I work on 16.04 as released currently I almost missed those, but I
> at least remember seeing the first on the list.
> I had almost the same change - adopting yours since it is accepted.
> Maybe I can provide a few more baby steps on that path - or at least a few
> more pieces for discussion and review.
> I hope you get well soon Panu!
>
>
>> #2
>>>> The Application then seem to try to make up for that by realizing all
>>>> that
>>>> is missing.
>>>> But looking at the app alone it seems overlinked by that - it is not
>>>> using
>>>> all of these on its own.
>>>> ldd usr/bin/cmdline_test
>>>>        linux-vdso.so.1 =>  (0x00007ffeec9ea000)
>>>>        librte_distributor.so.1 => not found
>>>>        librte_reorder.so.1 => not found
>>>> [...]
>>>>        librte_jobstats.so.1 => not found
>>>> [...]
>>>> And for example none of the librte_jobstats.so.1 symbols are used
>>>> "directly" in there.
>>>>
>>>
>>> Yes every libraries are put for every apps in rte.app.mk.
>>> Probably that we could better use DEPDIRS-y for apps.
>>>
>>>
>> Another trick from Sergios' original patch series is to utilize
>> AS_NEEDED() in the linker script, so it ends up looking something like this
>> for shared library config (static would remain as it is now):
>>
>> INPUT ( librte_eal librte_mempool librte_ring
>>         AS_NEEDED ( librte_acl librte_timer librte_vhost <...> )
>> )
>>
>> ...and then use the linker script for linking the apps, which should
>> simplify rte.app.mk considerably. Or so the theory goes. Anyway, the
>> above requires DT_NEEDED for the internal libraries to be present first,
>> but once all these pieces are in place, dynamically linked apps will only
>> link to the libraries they actually need.
>>
>
> Interesting approach, I wasn't even working on DPDK back then - that could
> really simplify the mk there.
> I was experimenting with just dropping the "no-as-needed" in
> mk/exec-env/linuxapp/rte.vars.mk, but that seems to shoot too far -
> haven't had the time to look deeper.
> Interested if your approach could help.
>
> But the other part of it - getting the proper DT_NEEDED into the libs (I
> also consider underlinking worse than overlinking) seems to be just missing
> a bunch of -l - my last test build looked fine.
> There is the underlinking issue of librte_eal left that Sergio mentioned
> back in 6d25d90c, but it seems to be the next baby step to solve most else
> of the underlinking.
> I'll adapt it to git level and at least throw it out for discussion later
> on.
>
>

[PATCH] mk: fix underlinking issues of most librte libraries

[Christian Ehrhardt]

The individual libraries have various cross dependencies.
This is already refelcted in the DEPDIR dependency, but not yet in
proper DT_NEEDED flags in the .so's.
This adds the -l flags so that is properly stored in the .so's ELF
headers.

Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
---
 lib/librte_acl/Makefile         | 1 +
 lib/librte_cfgfile/Makefile     | 1 +
 lib/librte_cmdline/Makefile     | 1 +
 lib/librte_cryptodev/Makefile   | 1 +
 lib/librte_distributor/Makefile | 1 +
 lib/librte_ether/Makefile       | 1 +
 lib/librte_hash/Makefile        | 1 +
 lib/librte_ip_frag/Makefile     | 1 +
 lib/librte_ivshmem/Makefile     | 1 +
 lib/librte_jobstats/Makefile    | 1 +
 lib/librte_kni/Makefile         | 1 +
 lib/librte_kvargs/Makefile      | 1 +
 lib/librte_lpm/Makefile         | 1 +
 lib/librte_mbuf/Makefile        | 1 +
 lib/librte_mempool/Makefile     | 1 +
 lib/librte_meter/Makefile       | 1 +
 lib/librte_pipeline/Makefile    | 1 +
 lib/librte_port/Makefile        | 1 +
 lib/librte_power/Makefile       | 1 +
 lib/librte_reorder/Makefile     | 1 +
 lib/librte_ring/Makefile        | 1 +
 lib/librte_sched/Makefile       | 1 +
 lib/librte_table/Makefile       | 3 +++
 lib/librte_timer/Makefile       | 1 +
 lib/librte_vhost/Makefile       | 1 +
 25 files changed, 27 insertions(+)

diff --git a/lib/librte_acl/Makefile b/lib/librte_acl/Makefile
index 2e394c9..7ca0da4 100644
--- a/lib/librte_acl/Makefile
+++ b/lib/librte_acl/Makefile
@@ -91,6 +91,7 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_ACL)-include := rte_acl_osdep.h
 SYMLINK-$(CONFIG_RTE_LIBRTE_ACL)-include += rte_acl.h
 
 # this lib needs eal
+LDLIBS += -lrte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_ACL) += lib/librte_eal
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_cfgfile/Makefile b/lib/librte_cfgfile/Makefile
index 616aef0..668aadc 100644
--- a/lib/librte_cfgfile/Makefile
+++ b/lib/librte_cfgfile/Makefile
@@ -52,6 +52,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_CFGFILE) += rte_cfgfile.c
 SYMLINK-$(CONFIG_RTE_LIBRTE_CFGFILE)-include += rte_cfgfile.h
 
 # this lib needs eal
+LDLIBS += -lrte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_CFGFILE) += lib/librte_eal
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_cmdline/Makefile b/lib/librte_cmdline/Makefile
index 7d2d148..d134e18 100644
--- a/lib/librte_cmdline/Makefile
+++ b/lib/librte_cmdline/Makefile
@@ -62,6 +62,7 @@ INCS += cmdline_vt100.h cmdline_socket.h cmdline_cirbuf.h cmdline_parse_portlist
 SYMLINK-$(CONFIG_RTE_LIBRTE_CMDLINE)-include := $(INCS)
 
 # this lib needs eal
+LDLIBS += -lrte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_CMDLINE) += lib/librte_eal
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_cryptodev/Makefile b/lib/librte_cryptodev/Makefile
index 314a046..af69d37 100644
--- a/lib/librte_cryptodev/Makefile
+++ b/lib/librte_cryptodev/Makefile
@@ -53,6 +53,7 @@ SYMLINK-y-include += rte_cryptodev_pmd.h
 EXPORT_MAP := rte_cryptodev_version.map
 
 # library dependencies
+LDLIBS += -lrte_eal -lrte_mempool -lrte_ring -lrte_mbuf -lrte_kvargs
 DEPDIRS-y += lib/librte_eal
 DEPDIRS-y += lib/librte_mempool
 DEPDIRS-y += lib/librte_ring
diff --git a/lib/librte_distributor/Makefile b/lib/librte_distributor/Makefile
index 4c9af17..fb651e1 100644
--- a/lib/librte_distributor/Makefile
+++ b/lib/librte_distributor/Makefile
@@ -48,6 +48,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR) := rte_distributor.c
 SYMLINK-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR)-include := rte_distributor.h
 
 # this lib needs eal
+LDLIBS += -lrte_eal -lrte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR) += lib/librte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR) += lib/librte_mbuf
 
diff --git a/lib/librte_ether/Makefile b/lib/librte_ether/Makefile
index 0bb5dc9..31766e2 100644
--- a/lib/librte_ether/Makefile
+++ b/lib/librte_ether/Makefile
@@ -54,6 +54,7 @@ SYMLINK-y-include += rte_eth_ctrl.h
 SYMLINK-y-include += rte_dev_info.h
 
 # this lib depends upon:
+LDLIBS += -lrte_eal -lrte_mempool -lrte_ring -lrte_mbuf
 DEPDIRS-y += lib/librte_eal lib/librte_mempool lib/librte_ring lib/librte_mbuf
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_hash/Makefile b/lib/librte_hash/Makefile
index bb1ea99..6d94fe0 100644
--- a/lib/librte_hash/Makefile
+++ b/lib/librte_hash/Makefile
@@ -56,6 +56,7 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_HASH)-include += rte_thash.h
 SYMLINK-$(CONFIG_RTE_LIBRTE_HASH)-include += rte_fbk_hash.h
 
 # this lib needs eal and ring
+LDLIBS += -lrte_eal -lrte_ring
 DEPDIRS-$(CONFIG_RTE_LIBRTE_HASH) += lib/librte_eal lib/librte_ring
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_ip_frag/Makefile b/lib/librte_ip_frag/Makefile
index 9d06780..42a7052 100644
--- a/lib/librte_ip_frag/Makefile
+++ b/lib/librte_ip_frag/Makefile
@@ -54,6 +54,7 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_IP_FRAG)-include += rte_ip_frag.h
 
 
 # this library depends on rte_ether
+LDLIBS += -lrte_mempool -lethdev
 DEPDIRS-$(CONFIG_RTE_LIBRTE_IP_FRAG) += lib/librte_mempool lib/librte_ether
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_ivshmem/Makefile b/lib/librte_ivshmem/Makefile
index 16defdb..611a676 100644
--- a/lib/librte_ivshmem/Makefile
+++ b/lib/librte_ivshmem/Makefile
@@ -47,6 +47,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_IVSHMEM) := rte_ivshmem.c
 SYMLINK-$(CONFIG_RTE_LIBRTE_IVSHMEM)-include := rte_ivshmem.h
 
 # this lib needs eal
+LDLIBS += -lrte_mempool
 DEPDIRS-$(CONFIG_RTE_LIBRTE_IVSHMEM) += lib/librte_mempool
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_jobstats/Makefile b/lib/librte_jobstats/Makefile
index 136a448..dddf886 100644
--- a/lib/librte_jobstats/Makefile
+++ b/lib/librte_jobstats/Makefile
@@ -48,6 +48,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_JOBSTATS) := rte_jobstats.c
 SYMLINK-$(CONFIG_RTE_LIBRTE_JOBSTATS)-include := rte_jobstats.h
 
 # this lib needs eal
+LDLIBS += -lrte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += lib/librte_eal
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_kni/Makefile b/lib/librte_kni/Makefile
index 1398164..f2beb49 100644
--- a/lib/librte_kni/Makefile
+++ b/lib/librte_kni/Makefile
@@ -47,6 +47,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_KNI) := rte_kni.c
 SYMLINK-$(CONFIG_RTE_LIBRTE_KNI)-include := rte_kni.h
 
 # this lib needs eal
+LDLIBS += -lrte_eal -lrte_mbuf -lethdev
 DEPDIRS-$(CONFIG_RTE_LIBRTE_KNI) += lib/librte_eal lib/librte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_KNI) += lib/librte_ether
 
diff --git a/lib/librte_kvargs/Makefile b/lib/librte_kvargs/Makefile
index 87b09f2..ca2679b 100644
--- a/lib/librte_kvargs/Makefile
+++ b/lib/librte_kvargs/Makefile
@@ -50,6 +50,7 @@ INCS := rte_kvargs.h
 SYMLINK-$(CONFIG_RTE_LIBRTE_KVARGS)-include := $(INCS)
 
 # this lib needs eal
+LDLIBS += -lrte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_KVARGS) += lib/librte_eal
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_lpm/Makefile b/lib/librte_lpm/Makefile
index 656ade2..b60b98d 100644
--- a/lib/librte_lpm/Makefile
+++ b/lib/librte_lpm/Makefile
@@ -54,6 +54,7 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_LPM)-include += rte_lpm_sse.h
 endif
 
 # this lib needs eal
+LDLIBS += -lrte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_LPM) += lib/librte_eal
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_mbuf/Makefile b/lib/librte_mbuf/Makefile
index 8d62b0d..2cad00c 100644
--- a/lib/librte_mbuf/Makefile
+++ b/lib/librte_mbuf/Makefile
@@ -47,6 +47,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_MBUF) := rte_mbuf.c
 SYMLINK-$(CONFIG_RTE_LIBRTE_MBUF)-include := rte_mbuf.h
 
 # this lib needs eal
+LDLIBS += -lrte_eal -lrte_mempool
 DEPDIRS-$(CONFIG_RTE_LIBRTE_MBUF) += lib/librte_eal lib/librte_mempool
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_mempool/Makefile b/lib/librte_mempool/Makefile
index 43423e0..9776114 100644
--- a/lib/librte_mempool/Makefile
+++ b/lib/librte_mempool/Makefile
@@ -45,6 +45,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_MEMPOOL) +=  rte_mempool.c
 # install includes
 SYMLINK-$(CONFIG_RTE_LIBRTE_MEMPOOL)-include := rte_mempool.h
 
+LDLIBS += -lrte_eal -lrte_ring
 DEPDIRS-$(CONFIG_RTE_LIBRTE_MEMPOOL) += lib/librte_eal lib/librte_ring
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_meter/Makefile b/lib/librte_meter/Makefile
index f07fced..f978aa2 100644
--- a/lib/librte_meter/Makefile
+++ b/lib/librte_meter/Makefile
@@ -54,6 +54,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_METER) := rte_meter.c
 SYMLINK-$(CONFIG_RTE_LIBRTE_METER)-include := rte_meter.h
 
 # this lib depends upon:
+LDLIBS += -lrte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_METER) += lib/librte_eal
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_pipeline/Makefile b/lib/librte_pipeline/Makefile
index 822fd41..a301981 100644
--- a/lib/librte_pipeline/Makefile
+++ b/lib/librte_pipeline/Makefile
@@ -52,6 +52,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) := rte_pipeline.c
 SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_pipeline.h
 
 # this lib depends upon:
+LDLIBS += -lrte_table -lrte_port
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) := lib/librte_table
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += lib/librte_port
 
diff --git a/lib/librte_port/Makefile b/lib/librte_port/Makefile
index d4de5af..532f86c 100644
--- a/lib/librte_port/Makefile
+++ b/lib/librte_port/Makefile
@@ -70,6 +70,7 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_sched.h
 SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_source_sink.h
 
 # this lib depends upon:
+LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lethdev -lrte_ip_frag
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PORT) := lib/librte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PORT) += lib/librte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PORT) += lib/librte_mempool
diff --git a/lib/librte_power/Makefile b/lib/librte_power/Makefile
index cee95cd..d5c1be5 100644
--- a/lib/librte_power/Makefile
+++ b/lib/librte_power/Makefile
@@ -48,6 +48,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_POWER) += rte_power_kvm_vm.c guest_channel.c
 SYMLINK-$(CONFIG_RTE_LIBRTE_POWER)-include := rte_power.h
 
 # this lib needs eal
+LDLIBS += -lrte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_POWER) += lib/librte_eal
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_reorder/Makefile b/lib/librte_reorder/Makefile
index 0c01de1..1280da7 100644
--- a/lib/librte_reorder/Makefile
+++ b/lib/librte_reorder/Makefile
@@ -48,6 +48,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_REORDER) := rte_reorder.c
 SYMLINK-$(CONFIG_RTE_LIBRTE_REORDER)-include := rte_reorder.h
 
 # this lib depends upon:
+LDLIBS += -lrte_eal -lrte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_REORDER) += lib/librte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_REORDER) += lib/librte_eal
 
diff --git a/lib/librte_ring/Makefile b/lib/librte_ring/Makefile
index 4b1112e..c3a942f 100644
--- a/lib/librte_ring/Makefile
+++ b/lib/librte_ring/Makefile
@@ -46,6 +46,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_RING) := rte_ring.c
 # install includes
 SYMLINK-$(CONFIG_RTE_LIBRTE_RING)-include := rte_ring.h
 
+LDLIBS += -lrte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_RING) += lib/librte_eal
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_sched/Makefile b/lib/librte_sched/Makefile
index a782cd1..dc4ebd0 100644
--- a/lib/librte_sched/Makefile
+++ b/lib/librte_sched/Makefile
@@ -59,6 +59,7 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_SCHED)-include := rte_sched.h rte_bitmap.h rte_sched
 SYMLINK-$(CONFIG_RTE_LIBRTE_SCHED)-include += rte_reciprocal.h
 
 # this lib depends upon:
+LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_timer
 DEPDIRS-$(CONFIG_RTE_LIBRTE_SCHED) += lib/librte_mempool lib/librte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_SCHED) += lib/librte_net lib/librte_timer
 
diff --git a/lib/librte_table/Makefile b/lib/librte_table/Makefile
index 7f02af3..44e5584 100644
--- a/lib/librte_table/Makefile
+++ b/lib/librte_table/Makefile
@@ -72,14 +72,17 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_array.h
 SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_stub.h
 
 # this lib depends upon:
+LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_port -lrte_lpm
 DEPDIRS-$(CONFIG_RTE_LIBRTE_TABLE) := lib/librte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_TABLE) += lib/librte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_TABLE) += lib/librte_mempool
 DEPDIRS-$(CONFIG_RTE_LIBRTE_TABLE) += lib/librte_port
 DEPDIRS-$(CONFIG_RTE_LIBRTE_TABLE) += lib/librte_lpm
 ifeq ($(CONFIG_RTE_LIBRTE_ACL),y)
+LDLIBS += -lrte_acl
 DEPDIRS-$(CONFIG_RTE_LIBRTE_TABLE) += lib/librte_acl
 endif
+LDLIBS += -lrte_hash
 DEPDIRS-$(CONFIG_RTE_LIBRTE_TABLE) += lib/librte_hash
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_timer/Makefile b/lib/librte_timer/Makefile
index 2aabef8..3e96832 100644
--- a/lib/librte_timer/Makefile
+++ b/lib/librte_timer/Makefile
@@ -47,6 +47,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_TIMER) := rte_timer.c
 SYMLINK-$(CONFIG_RTE_LIBRTE_TIMER)-include := rte_timer.h
 
 # this lib needs eal
+LDLIBS += -lrte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_TIMER) += lib/librte_eal
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_vhost/Makefile b/lib/librte_vhost/Makefile
index e33ff53..756bb07 100644
--- a/lib/librte_vhost/Makefile
+++ b/lib/librte_vhost/Makefile
@@ -63,6 +63,7 @@ endif
 SYMLINK-$(CONFIG_RTE_LIBRTE_VHOST)-include += rte_virtio_net.h
 
 # dependencies
+LDLIBS += -lrte_eal -lethdev -lrte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_VHOST) += lib/librte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_VHOST) += lib/librte_ether
 DEPDIRS-$(CONFIG_RTE_LIBRTE_VHOST) += lib/librte_mbuf
-- 
2.7.4

Re: [PATCH] mk: fix underlinking issues of most librte libraries

[Thomas Monjalon]

2016-05-20 18:50, Christian Ehrhardt:
> The individual libraries have various cross dependencies.
> This is already refelcted in the DEPDIR dependency, but not yet in
> proper DT_NEEDED flags in the .so's.
> This adds the -l flags so that is properly stored in the .so's ELF
> headers.

Why not filling LDLIBS by parsing DEPDIRS-y in rte.lib.mk?

Re: [PATCH] mk: fix underlinking issues of most librte libraries

[Panu Matilainen]

On 05/20/2016 08:08 PM, Thomas Monjalon wrote:
> 2016-05-20 18:50, Christian Ehrhardt:
>> The individual libraries have various cross dependencies.
>> This is already refelcted in the DEPDIR dependency, but not yet in
>> proper DT_NEEDED flags in the .so's.
>> This adds the -l flags so that is properly stored in the .so's ELF
>> headers.
>
> Why not filling LDLIBS by parsing DEPDIRS-y in rte.lib.mk?
>

A fair question :) The thought has passed my mind too, but I thought 
it'd be too messy with differing library vs directory names and other 
exceptions. But in reality manually maintained separate LDLIBS is only 
going to get out of sync sooner or later, and turns out its not that bad 
anyway:

diff --git a/mk/rte.lib.mk b/mk/rte.lib.mk
index b420280..88b4e98 100644
--- a/mk/rte.lib.mk
+++ b/mk/rte.lib.mk
@@ -77,6 +77,12 @@ else
  _CPU_LDFLAGS := $(CPU_LDFLAGS)
  endif

+# Translate DEPDIRS-y into LDLIBS
+IGNORE_DEPS = -lrte_eal/% -lrte_net -lrte_compat
+_LDDIRS = $(subst librte_ether,libethdev,$(DEPDIRS-y))
+_LDDEPS = $(subst lib/lib,-l,$(_LDDIRS))
+LDLIBS += $(filter-out $(IGNORE_DEPS), $(_LDDEPS))
+
  O_TO_A = $(AR) crDs $(LIB) $(OBJS-y)
  O_TO_A_STR = $(subst ','\'',$(O_TO_A)) #'# fix syntax highlight
  O_TO_A_DISP = $(if $(V),"$(O_TO_A_STR)","  AR $(@)")

I'm also sure somebody more familiar with gmake could improve it, but it 
seems to work quite fine here. I'll wait a bit to see if there are other 
suggestions before sending it as a proper patch.

	- Panu -

Re: [PATCH] mk: fix underlinking issues of most librte libraries

[Christian Ehrhardt]

Hi Panu,
I already agreed to Thomas on IRC but won't have time until next week.
Thanks for making a patch that does that already - I'll give it a look and
some test on my end next week then.



Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd

On Tue, May 24, 2016 at 11:56 AM, Panu Matilainen <pmatilai@redhat.com>
wrote:

> On 05/20/2016 08:08 PM, Thomas Monjalon wrote:
>
>> 2016-05-20 18:50, Christian Ehrhardt:
>>
>>> The individual libraries have various cross dependencies.
>>> This is already refelcted in the DEPDIR dependency, but not yet in
>>> proper DT_NEEDED flags in the .so's.
>>> This adds the -l flags so that is properly stored in the .so's ELF
>>> headers.
>>>
>>
>> Why not filling LDLIBS by parsing DEPDIRS-y in rte.lib.mk?
>>
>>
> A fair question :) The thought has passed my mind too, but I thought it'd
> be too messy with differing library vs directory names and other
> exceptions. But in reality manually maintained separate LDLIBS is only
> going to get out of sync sooner or later, and turns out its not that bad
> anyway:
>
> diff --git a/mk/rte.lib.mk b/mk/rte.lib.mk
> index b420280..88b4e98 100644
> --- a/mk/rte.lib.mk
> +++ b/mk/rte.lib.mk
> @@ -77,6 +77,12 @@ else
>  _CPU_LDFLAGS := $(CPU_LDFLAGS)
>  endif
>
> +# Translate DEPDIRS-y into LDLIBS
> +IGNORE_DEPS = -lrte_eal/% -lrte_net -lrte_compat
> +_LDDIRS = $(subst librte_ether,libethdev,$(DEPDIRS-y))
> +_LDDEPS = $(subst lib/lib,-l,$(_LDDIRS))
> +LDLIBS += $(filter-out $(IGNORE_DEPS), $(_LDDEPS))
> +
>  O_TO_A = $(AR) crDs $(LIB) $(OBJS-y)
>  O_TO_A_STR = $(subst ','\'',$(O_TO_A)) #'# fix syntax highlight
>  O_TO_A_DISP = $(if $(V),"$(O_TO_A_STR)","  AR $(@)")
>
> I'm also sure somebody more familiar with gmake could improve it, but it
> seems to work quite fine here. I'll wait a bit to see if there are other
> suggestions before sending it as a proper patch.
>
>         - Panu -
>

Re: [PATCH] mk: fix underlinking issues of most librte libraries

[Ferruh Yigit]

On 5/24/2016 11:11 AM, Christian Ehrhardt wrote:
> Hi Panu,
> I already agreed to Thomas on IRC but won't have time until next week.
> Thanks for making a patch that does that already - I'll give it a look and
> some test on my end next week then.
> 
> 
> 
> Christian Ehrhardt
> Software Engineer, Ubuntu Server
> Canonical Ltd
> 
> On Tue, May 24, 2016 at 11:56 AM, Panu Matilainen <pmatilai@redhat.com>
> wrote:
> 
>> On 05/20/2016 08:08 PM, Thomas Monjalon wrote:
>>
>>> 2016-05-20 18:50, Christian Ehrhardt:
>>>
>>>> The individual libraries have various cross dependencies.
>>>> This is already refelcted in the DEPDIR dependency, but not yet in
>>>> proper DT_NEEDED flags in the .so's.
>>>> This adds the -l flags so that is properly stored in the .so's ELF
>>>> headers.
>>>>
>>>
>>> Why not filling LDLIBS by parsing DEPDIRS-y in rte.lib.mk?
>>>
>>>
>> A fair question :) The thought has passed my mind too, but I thought it'd
>> be too messy with differing library vs directory names and other
>> exceptions. But in reality manually maintained separate LDLIBS is only
>> going to get out of sync sooner or later, and turns out its not that bad
>> anyway:
>>
>> diff --git a/mk/rte.lib.mk b/mk/rte.lib.mk
>> index b420280..88b4e98 100644
>> --- a/mk/rte.lib.mk
>> +++ b/mk/rte.lib.mk
>> @@ -77,6 +77,12 @@ else
>>  _CPU_LDFLAGS := $(CPU_LDFLAGS)
>>  endif
>>
>> +# Translate DEPDIRS-y into LDLIBS
>> +IGNORE_DEPS = -lrte_eal/% -lrte_net -lrte_compat
>> +_LDDIRS = $(subst librte_ether,libethdev,$(DEPDIRS-y))
>> +_LDDEPS = $(subst lib/lib,-l,$(_LDDIRS))
>> +LDLIBS += $(filter-out $(IGNORE_DEPS), $(_LDDEPS))
>> +
>>  O_TO_A = $(AR) crDs $(LIB) $(OBJS-y)
>>  O_TO_A_STR = $(subst ','\'',$(O_TO_A)) #'# fix syntax highlight
>>  O_TO_A_DISP = $(if $(V),"$(O_TO_A_STR)","  AR $(@)")
>>
>> I'm also sure somebody more familiar with gmake could improve it, but it
>> seems to work quite fine here. I'll wait a bit to see if there are other
>> suggestions before sending it as a proper patch.
>>
>>         - Panu -
>>

I did test the patch with minor diff in rte.vars.mk [1], otherwise all
libraries kept needed in final library.

Patch itself looks good, overlinking problem in final binary seems fixed
[2] [3].

But this cause a compilation error for cmdline_test [4]. This is because
of cyclic dependency between librte_eal <-> librte_mempool.

Other applications don't have this compilation error because they use
librte_mempool, and since it is linked against application,  this error
not observed. cmdline_test itself doesn't have any direct call to
librte_mempool.

It is possible to add workaround to cmdline_test [5], (introduce
overlinking back), but should we attack on cyclic dependency instead?


Thanks,
ferruh

---

[1]
diff --git a/mk/exec-env/linuxapp/rte.vars.mk
b/mk/exec-env/linuxapp/rte.vars.mk
index d51bd17..10d37d5 100644
--- a/mk/exec-env/linuxapp/rte.vars.mk
+++ b/mk/exec-env/linuxapp/rte.vars.mk
@@ -46,7 +46,7 @@ EXECENV_CFLAGS  = -pthread
 endif

 # Workaround lack of DT_NEEDED entry
-EXECENV_LDFLAGS = --no-as-needed
+EXECENV_LDFLAGS = --as-needed

 EXECENV_LDLIBS  =
 EXECENV_ASFLAGS =


[2]
# ldd build/app/testpmd
        linux-vdso.so.1 (0x00007ffcff92e000)
        librte_mbuf.so.2.1 => .../build/lib/librte_mbuf.so.2.1
        libethdev.so.4.1 => .../build/lib/libethdev.so.4.1
        librte_mempool.so.2.1 => .../build/lib/librte_mempool.so.2.1
        librte_ring.so.1.1 => .../build/lib/librte_ring.so.1.1
        librte_eal.so.2.1 => .../build/lib/librte_eal.so.2.1
        librte_cmdline.so.2.1 => .../build/lib/librte_cmdline.so.2.1
        librte_pmd_bond.so.1.1 => .../build/lib/librte_pmd_bond.so.1.1
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f6879ff1000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f6879c30000)
        /lib64/ld-linux-x86-64.so.2 (0x0000564998dbd000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f6879a2c000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f6879814000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f687960c000)
        librte_kvargs.so.1.1 => .../build/lib/librte_kvargs.so.1.1

[3]
# ldd -u -r build/app/testpmd
#


[4]
== Build app/cmdline_test
  CC cmdline_test.o
  CC commands.o
  LD cmdline_test
.../build/lib/librte_eal.so: undefined reference to `rte_mempool_lookup'
.../build/lib/librte_eal.so: undefined reference to `rte_mempool_create'
collect2: error: ld returned 1 exit status
.../mk/rte.app.mk:243: recipe for target 'cmdline_test' failed
make[3]: *** [cmdline_test] Error 1


[5]
diff --git a/app/cmdline_test/Makefile b/app/cmdline_test/Makefile
index c6169f5..69ebfb2 100644
--- a/app/cmdline_test/Makefile
+++ b/app/cmdline_test/Makefile
@@ -46,6 +46,7 @@ SRCS-y += commands.c

 CFLAGS += -O3
 CFLAGS += $(WERROR_FLAGS)
+LDFLAGS += --no-as-needed

 # this application needs libraries first
 DEPDIRS-y += lib drivers

Re: [PATCH] mk: fix underlinking issues of most librte libraries

[Panu Matilainen]

On 05/26/2016 04:13 PM, Ferruh Yigit wrote:
> On 5/24/2016 11:11 AM, Christian Ehrhardt wrote:
>> Hi Panu,
>> I already agreed to Thomas on IRC but won't have time until next week.
>> Thanks for making a patch that does that already - I'll give it a look and
>> some test on my end next week then.
>>
>>
>>
>> Christian Ehrhardt
>> Software Engineer, Ubuntu Server
>> Canonical Ltd
>>
>> On Tue, May 24, 2016 at 11:56 AM, Panu Matilainen <pmatilai@redhat.com>
>> wrote:
>>
>>> On 05/20/2016 08:08 PM, Thomas Monjalon wrote:
>>>
>>>> 2016-05-20 18:50, Christian Ehrhardt:
>>>>
>>>>> The individual libraries have various cross dependencies.
>>>>> This is already refelcted in the DEPDIR dependency, but not yet in
>>>>> proper DT_NEEDED flags in the .so's.
>>>>> This adds the -l flags so that is properly stored in the .so's ELF
>>>>> headers.
>>>>>
>>>>
>>>> Why not filling LDLIBS by parsing DEPDIRS-y in rte.lib.mk?
>>>>
>>>>
>>> A fair question :) The thought has passed my mind too, but I thought it'd
>>> be too messy with differing library vs directory names and other
>>> exceptions. But in reality manually maintained separate LDLIBS is only
>>> going to get out of sync sooner or later, and turns out its not that bad
>>> anyway:
>>>
>>> diff --git a/mk/rte.lib.mk b/mk/rte.lib.mk
>>> index b420280..88b4e98 100644
>>> --- a/mk/rte.lib.mk
>>> +++ b/mk/rte.lib.mk
>>> @@ -77,6 +77,12 @@ else
>>>  _CPU_LDFLAGS := $(CPU_LDFLAGS)
>>>  endif
>>>
>>> +# Translate DEPDIRS-y into LDLIBS
>>> +IGNORE_DEPS = -lrte_eal/% -lrte_net -lrte_compat
>>> +_LDDIRS = $(subst librte_ether,libethdev,$(DEPDIRS-y))
>>> +_LDDEPS = $(subst lib/lib,-l,$(_LDDIRS))
>>> +LDLIBS += $(filter-out $(IGNORE_DEPS), $(_LDDEPS))
>>> +
>>>  O_TO_A = $(AR) crDs $(LIB) $(OBJS-y)
>>>  O_TO_A_STR = $(subst ','\'',$(O_TO_A)) #'# fix syntax highlight
>>>  O_TO_A_DISP = $(if $(V),"$(O_TO_A_STR)","  AR $(@)")
>>>
>>> I'm also sure somebody more familiar with gmake could improve it, but it
>>> seems to work quite fine here. I'll wait a bit to see if there are other
>>> suggestions before sending it as a proper patch.
>>>
>>>         - Panu -
>>>
>
> I did test the patch with minor diff in rte.vars.mk [1], otherwise all
> libraries kept needed in final library.
>
> Patch itself looks good, overlinking problem in final binary seems fixed
> [2] [3].
>
> But this cause a compilation error for cmdline_test [4]. This is because
> of cyclic dependency between librte_eal <-> librte_mempool.
>
> Other applications don't have this compilation error because they use
> librte_mempool, and since it is linked against application,  this error
> not observed. cmdline_test itself doesn't have any direct call to
> librte_mempool.
>
> It is possible to add workaround to cmdline_test [5], (introduce
> overlinking back), but should we attack on cyclic dependency instead?

Cyclic dependencies are nasty, eliminating it would be good.

But in the meanwhile it can be worked around in a generic manner by 
handling it in the libdpdk.so linker script, see 
http://dpdk.org/ml/archives/dev/2016-May/039413.html

	- Panu -

>
> Thanks,
> ferruh
>
> ---
>
> [1]
> diff --git a/mk/exec-env/linuxapp/rte.vars.mk
> b/mk/exec-env/linuxapp/rte.vars.mk
> index d51bd17..10d37d5 100644
> --- a/mk/exec-env/linuxapp/rte.vars.mk
> +++ b/mk/exec-env/linuxapp/rte.vars.mk
> @@ -46,7 +46,7 @@ EXECENV_CFLAGS  = -pthread
>  endif
>
>  # Workaround lack of DT_NEEDED entry
> -EXECENV_LDFLAGS = --no-as-needed
> +EXECENV_LDFLAGS = --as-needed
>
>  EXECENV_LDLIBS  =
>  EXECENV_ASFLAGS =
>
>
> [2]
> # ldd build/app/testpmd
>         linux-vdso.so.1 (0x00007ffcff92e000)
>         librte_mbuf.so.2.1 => .../build/lib/librte_mbuf.so.2.1
>         libethdev.so.4.1 => .../build/lib/libethdev.so.4.1
>         librte_mempool.so.2.1 => .../build/lib/librte_mempool.so.2.1
>         librte_ring.so.1.1 => .../build/lib/librte_ring.so.1.1
>         librte_eal.so.2.1 => .../build/lib/librte_eal.so.2.1
>         librte_cmdline.so.2.1 => .../build/lib/librte_cmdline.so.2.1
>         librte_pmd_bond.so.1.1 => .../build/lib/librte_pmd_bond.so.1.1
>         libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f6879ff1000)
>         libc.so.6 => /lib64/libc.so.6 (0x00007f6879c30000)
>         /lib64/ld-linux-x86-64.so.2 (0x0000564998dbd000)
>         libdl.so.2 => /lib64/libdl.so.2 (0x00007f6879a2c000)
>         libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f6879814000)
>         librt.so.1 => /lib64/librt.so.1 (0x00007f687960c000)
>         librte_kvargs.so.1.1 => .../build/lib/librte_kvargs.so.1.1
>
> [3]
> # ldd -u -r build/app/testpmd
> #
>
>
> [4]
> == Build app/cmdline_test
>   CC cmdline_test.o
>   CC commands.o
>   LD cmdline_test
> .../build/lib/librte_eal.so: undefined reference to `rte_mempool_lookup'
> .../build/lib/librte_eal.so: undefined reference to `rte_mempool_create'
> collect2: error: ld returned 1 exit status
> .../mk/rte.app.mk:243: recipe for target 'cmdline_test' failed
> make[3]: *** [cmdline_test] Error 1
>
>
> [5]
> diff --git a/app/cmdline_test/Makefile b/app/cmdline_test/Makefile
> index c6169f5..69ebfb2 100644
> --- a/app/cmdline_test/Makefile
> +++ b/app/cmdline_test/Makefile
> @@ -46,6 +46,7 @@ SRCS-y += commands.c
>
>  CFLAGS += -O3
>  CFLAGS += $(WERROR_FLAGS)
> +LDFLAGS += --no-as-needed
>
>  # this application needs libraries first
>  DEPDIRS-y += lib drivers
>

Re: [PATCH] mk: fix underlinking issues of most librte libraries

[Ferruh Yigit]

On 5/27/2016 6:59 AM, Panu Matilainen wrote:
> On 05/26/2016 04:13 PM, Ferruh Yigit wrote:
>> On 5/24/2016 11:11 AM, Christian Ehrhardt wrote:
>>> Hi Panu,
>>> I already agreed to Thomas on IRC but won't have time until next week.
>>> Thanks for making a patch that does that already - I'll give it a look and
>>> some test on my end next week then.
>>>
>>>
>>>
>>> Christian Ehrhardt
>>> Software Engineer, Ubuntu Server
>>> Canonical Ltd
>>>
>>> On Tue, May 24, 2016 at 11:56 AM, Panu Matilainen <pmatilai@redhat.com>
>>> wrote:
>>>
>>>> On 05/20/2016 08:08 PM, Thomas Monjalon wrote:
>>>>
>>>>> 2016-05-20 18:50, Christian Ehrhardt:
>>>>>
>>>>>> The individual libraries have various cross dependencies.
>>>>>> This is already refelcted in the DEPDIR dependency, but not yet in
>>>>>> proper DT_NEEDED flags in the .so's.
>>>>>> This adds the -l flags so that is properly stored in the .so's ELF
>>>>>> headers.
>>>>>>
>>>>>
>>>>> Why not filling LDLIBS by parsing DEPDIRS-y in rte.lib.mk?
>>>>>
>>>>>
>>>> A fair question :) The thought has passed my mind too, but I thought it'd
>>>> be too messy with differing library vs directory names and other
>>>> exceptions. But in reality manually maintained separate LDLIBS is only
>>>> going to get out of sync sooner or later, and turns out its not that bad
>>>> anyway:
>>>>
>>>> diff --git a/mk/rte.lib.mk b/mk/rte.lib.mk
>>>> index b420280..88b4e98 100644
>>>> --- a/mk/rte.lib.mk
>>>> +++ b/mk/rte.lib.mk
>>>> @@ -77,6 +77,12 @@ else
>>>>  _CPU_LDFLAGS := $(CPU_LDFLAGS)
>>>>  endif
>>>>
>>>> +# Translate DEPDIRS-y into LDLIBS
>>>> +IGNORE_DEPS = -lrte_eal/% -lrte_net -lrte_compat
>>>> +_LDDIRS = $(subst librte_ether,libethdev,$(DEPDIRS-y))
>>>> +_LDDEPS = $(subst lib/lib,-l,$(_LDDIRS))
>>>> +LDLIBS += $(filter-out $(IGNORE_DEPS), $(_LDDEPS))
>>>> +
>>>>  O_TO_A = $(AR) crDs $(LIB) $(OBJS-y)
>>>>  O_TO_A_STR = $(subst ','\'',$(O_TO_A)) #'# fix syntax highlight
>>>>  O_TO_A_DISP = $(if $(V),"$(O_TO_A_STR)","  AR $(@)")
>>>>
>>>> I'm also sure somebody more familiar with gmake could improve it, but it
>>>> seems to work quite fine here. I'll wait a bit to see if there are other
>>>> suggestions before sending it as a proper patch.
>>>>
>>>>         - Panu -
>>>>
>>
>> I did test the patch with minor diff in rte.vars.mk [1], otherwise all
>> libraries kept needed in final library.
>>
>> Patch itself looks good, overlinking problem in final binary seems fixed
>> [2] [3].
>>
>> But this cause a compilation error for cmdline_test [4]. This is because
>> of cyclic dependency between librte_eal <-> librte_mempool.
>>
>> Other applications don't have this compilation error because they use
>> librte_mempool, and since it is linked against application,  this error
>> not observed. cmdline_test itself doesn't have any direct call to
>> librte_mempool.
>>
>> It is possible to add workaround to cmdline_test [5], (introduce
>> overlinking back), but should we attack on cyclic dependency instead?
> 
> Cyclic dependencies are nasty, eliminating it would be good.
> 
> But in the meanwhile it can be worked around in a generic manner by 
> handling it in the libdpdk.so linker script, see 
> http://dpdk.org/ml/archives/dev/2016-May/039413.html
> 

Won't linker script has same problem, unless it is possible to select
some libraries to force link and some other to as-needed, than this can
be workaround, but I think using "--as-needed" as linker argument and
applying workaround to single test application is better.

Also when overlinking removed, getting compilation error for lpcap too.
Since -lpcap argument is before -lrte_pmd_pcap, symbols used by
rte_pmd_pcap not resolved, increasing --start-group scope fixes this.

Not related to overlinking, but while updating linker flags, as far as I
understand --whole-archive argument is only necessary for static pmd
libraries, but currently has wider scope, reducing its scope to pmd
slightly reduces final library sizes.

I will send a patch on top of your patch, with above updates.

Thanks,
ferruh

> 	- Panu -
> 
>>
>> Thanks,
>> ferruh
>>
>> ---
>>
>> [1]
>> diff --git a/mk/exec-env/linuxapp/rte.vars.mk
>> b/mk/exec-env/linuxapp/rte.vars.mk
>> index d51bd17..10d37d5 100644
>> --- a/mk/exec-env/linuxapp/rte.vars.mk
>> +++ b/mk/exec-env/linuxapp/rte.vars.mk
>> @@ -46,7 +46,7 @@ EXECENV_CFLAGS  = -pthread
>>  endif
>>
>>  # Workaround lack of DT_NEEDED entry
>> -EXECENV_LDFLAGS = --no-as-needed
>> +EXECENV_LDFLAGS = --as-needed
>>
>>  EXECENV_LDLIBS  =
>>  EXECENV_ASFLAGS =
>>
>>
>> [2]
>> # ldd build/app/testpmd
>>         linux-vdso.so.1 (0x00007ffcff92e000)
>>         librte_mbuf.so.2.1 => .../build/lib/librte_mbuf.so.2.1
>>         libethdev.so.4.1 => .../build/lib/libethdev.so.4.1
>>         librte_mempool.so.2.1 => .../build/lib/librte_mempool.so.2.1
>>         librte_ring.so.1.1 => .../build/lib/librte_ring.so.1.1
>>         librte_eal.so.2.1 => .../build/lib/librte_eal.so.2.1
>>         librte_cmdline.so.2.1 => .../build/lib/librte_cmdline.so.2.1
>>         librte_pmd_bond.so.1.1 => .../build/lib/librte_pmd_bond.so.1.1
>>         libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f6879ff1000)
>>         libc.so.6 => /lib64/libc.so.6 (0x00007f6879c30000)
>>         /lib64/ld-linux-x86-64.so.2 (0x0000564998dbd000)
>>         libdl.so.2 => /lib64/libdl.so.2 (0x00007f6879a2c000)
>>         libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f6879814000)
>>         librt.so.1 => /lib64/librt.so.1 (0x00007f687960c000)
>>         librte_kvargs.so.1.1 => .../build/lib/librte_kvargs.so.1.1
>>
>> [3]
>> # ldd -u -r build/app/testpmd
>> #
>>
>>
>> [4]
>> == Build app/cmdline_test
>>   CC cmdline_test.o
>>   CC commands.o
>>   LD cmdline_test
>> .../build/lib/librte_eal.so: undefined reference to `rte_mempool_lookup'
>> .../build/lib/librte_eal.so: undefined reference to `rte_mempool_create'
>> collect2: error: ld returned 1 exit status
>> .../mk/rte.app.mk:243: recipe for target 'cmdline_test' failed
>> make[3]: *** [cmdline_test] Error 1
>>
>>
>> [5]
>> diff --git a/app/cmdline_test/Makefile b/app/cmdline_test/Makefile
>> index c6169f5..69ebfb2 100644
>> --- a/app/cmdline_test/Makefile
>> +++ b/app/cmdline_test/Makefile
>> @@ -46,6 +46,7 @@ SRCS-y += commands.c
>>
>>  CFLAGS += -O3
>>  CFLAGS += $(WERROR_FLAGS)
>> +LDFLAGS += --no-as-needed
>>
>>  # this application needs libraries first
>>  DEPDIRS-y += lib drivers
>>
> 

[PATCH 1/2] mk: prevent overlinking in applications

[Ferruh Yigit]

Replace --no-as-needed linker flag with --as-needed flag, which will
only link libraries directly called by application. This requires inter
library dependencies resolved correctly.

Not linking all libraries cause a compile error for lpcap and possible
to have other similar compiler errors, so increasing the scope of
--start-group argument.

cmdline_test application causes compile error because of cyclic
dependency between librte_eal <-> librte_mempool. A workaround added to
cmdline_test for compile error.

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>

---

This patch is on top of patch:
http://dpdk.org/dev/patchwork/patch/12987/
---
 app/cmdline_test/Makefile        | 1 +
 mk/exec-env/linuxapp/rte.vars.mk | 2 +-
 mk/rte.app.mk                    | 3 +--
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/app/cmdline_test/Makefile b/app/cmdline_test/Makefile
index c6169f5..5b7a2a2 100644
--- a/app/cmdline_test/Makefile
+++ b/app/cmdline_test/Makefile
@@ -46,6 +46,7 @@ SRCS-y += commands.c
 
 CFLAGS += -O3
 CFLAGS += $(WERROR_FLAGS)
+LDFLAGS += -no-as-needed
 
 # this application needs libraries first
 DEPDIRS-y += lib drivers
diff --git a/mk/exec-env/linuxapp/rte.vars.mk b/mk/exec-env/linuxapp/rte.vars.mk
index d51bd17..10d37d5 100644
--- a/mk/exec-env/linuxapp/rte.vars.mk
+++ b/mk/exec-env/linuxapp/rte.vars.mk
@@ -46,7 +46,7 @@ EXECENV_CFLAGS  = -pthread
 endif
 
 # Workaround lack of DT_NEEDED entry
-EXECENV_LDFLAGS = --no-as-needed
+EXECENV_LDFLAGS = --as-needed
 
 EXECENV_LDLIBS  =
 EXECENV_ASFLAGS =
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index b84b56d..e12226c 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -58,6 +58,7 @@ _LDLIBS-y += -L$(RTE_SDK_BIN)/lib
 #
 
 _LDLIBS-y += --whole-archive
+_LDLIBS-y += --start-group
 
 _LDLIBS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR)    += -lrte_distributor
 _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER)        += -lrte_reorder
@@ -111,8 +112,6 @@ _LDLIBS-y                                   += -lcrypto
 endif
 endif # !CONFIG_RTE_BUILD_SHARED_LIBS
 
-_LDLIBS-y += --start-group
-
 _LDLIBS-$(CONFIG_RTE_LIBRTE_KVARGS)         += -lrte_kvargs
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MBUF)           += -lrte_mbuf
 _LDLIBS-$(CONFIG_RTE_LIBRTE_IP_FRAG)        += -lrte_ip_frag
-- 
2.5.5

[PATCH 2/2] mk: reduce scope of whole-archive to pmd libraries

[Ferruh Yigit]

--whole-archive argument only required for pmd libraries, and currently
it covers more libraries. Reducing scope of the argument to pmd
libraries slightly reduce final application size.

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
 mk/rte.app.mk | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index e12226c..8bfe240 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -57,7 +57,6 @@ _LDLIBS-y += -L$(RTE_SDK_BIN)/lib
 # Order is important: from higher level to lower level
 #
 
-_LDLIBS-y += --whole-archive
 _LDLIBS-y += --start-group
 
 _LDLIBS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR)    += -lrte_distributor
@@ -129,6 +128,8 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT)    += -lrte_pmd_xenvirt
 ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
 # plugins (link only if static libraries)
 
+_LDLIBS-y += --whole-archive
+
 _LDLIBS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD)    += -lrte_pmd_vmxnet3_uio
 _LDLIBS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD)     += -lrte_pmd_virtio
 _LDLIBS-$(CONFIG_RTE_LIBRTE_BNX2X_PMD)      += -lrte_pmd_bnx2x
@@ -174,11 +175,12 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_VHOST)      += -lrte_pmd_vhost
 
 endif # $(CONFIG_RTE_LIBRTE_VHOST)
 
+_LDLIBS-y += --no-whole-archive
+
 endif # ! $(CONFIG_RTE_BUILD_SHARED_LIB)
 
 _LDLIBS-y += $(EXECENV_LDLIBS)
 _LDLIBS-y += --end-group
-_LDLIBS-y += --no-whole-archive
 
 LDLIBS += $(_LDLIBS-y) $(CPU_LDLIBS) $(EXTRA_LDLIBS)
 
-- 
2.5.5

Re: [PATCH 2/2] mk: reduce scope of whole-archive to pmd libraries

[Thomas Monjalon]

2016-05-27 17:48, Ferruh Yigit:
> --whole-archive argument only required for pmd libraries, and currently
> it covers more libraries. Reducing scope of the argument to pmd
> libraries slightly reduce final application size.

In my understanding, --whole-archive is required for static libraries used
by plugins:
	http://dpdk.org/commit/20afd76a

If we want to restrict it, I would say it must be around EAL, ring,
mbuf, mempool, ethdev, cryptodev, etc.

Re: [PATCH 2/2] mk: reduce scope of whole-archive to pmd libraries

[Ferruh Yigit]

On 6/10/2016 10:03 AM, Thomas Monjalon wrote:
> 2016-05-27 17:48, Ferruh Yigit:
>> --whole-archive argument only required for pmd libraries, and currently
>> it covers more libraries. Reducing scope of the argument to pmd
>> libraries slightly reduce final application size.
> 
> In my understanding, --whole-archive is required for static libraries used
> by plugins:
> 	http://dpdk.org/commit/20afd76a
Right, required for static libraries. But more than used by plugins,
required for plugin to work.

> 
> If we want to restrict it, I would say it must be around EAL, ring,
> mbuf, mempool, ethdev, cryptodev, etc.
> 

We should restrict to plugins. What happens is, since there is no direct
call to PMDs from application, PMD code is not included into final
application. To force PMDs to be included into final binary,
--whole-archive is required.

But --whole-archive is not required for other libraries, and just cause
unnecessary increase in binary size.

Sample logs, with same setup, [1] compiled without --whole-archive, [2]
is current code:

[1]:
# ./testpmd -- -i
EAL: Detected 32 lcore(s)
EAL: Probing VFIO support...
EAL: No probed ethernet devices
Interactive-mode selected
USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=395456, size=2176,
socket=0
Done
testpmd> show port info all
testpmd> quit


[2]
# ./testpmd -- -i
EAL: Detected 32 lcore(s)
EAL: Probing VFIO support...
EAL: PCI device 0000:04:00.0 on NUMA socket 0
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL: PCI device 0000:08:00.0 on NUMA socket 0
EAL:   probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI device 0000:08:00.1 on NUMA socket 0
EAL:   probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI device 0000:0a:00.0 on NUMA socket 0
EAL:   probe driver: 8086:1584 rte_i40e_pmd
EAL: PCI device 0000:81:00.0 on NUMA socket 1
EAL:   probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI device 0000:81:00.1 on NUMA socket 1
EAL:   probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI device 0000:86:00.0 on NUMA socket 1
EAL:   probe driver: 8086:15a4 rte_pmd_fm10k
EAL: PCI device 0000:87:00.0 on NUMA socket 1
EAL:   probe driver: 8086:15a4 rte_pmd_fm10k
EAL: PCI device 0000:88:00.0 on NUMA socket 1
EAL:   probe driver: 8086:1572 rte_i40e_pmd
EAL: PCI device 0000:88:00.1 on NUMA socket 1
EAL:   probe driver: 8086:1572 rte_i40e_pmd
EAL: PCI device 0000:88:00.2 on NUMA socket 1
EAL:   probe driver: 8086:1572 rte_i40e_pmd
EAL: PCI device 0000:88:00.3 on NUMA socket 1
EAL:   probe driver: 8086:1572 rte_i40e_pmd
Interactive-mode selected
USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=395456, size=2176,
socket=0
Configuring Port 0 (socket 0)
Port 0: 90:E2:BA:0E:49:B9
Configuring Port 1 (socket 0)
Port 1: 00:1B:21:76:FA:21
Checking link statuses...
Port 0 Link Up - speed 10000 Mbps - full-duplex
Port 1 Link Up - speed 10000 Mbps - full-duplex
Done
testpmd>

Re: [PATCH 2/2] mk: reduce scope of whole-archive to pmd libraries

[Thomas Monjalon]

2016-06-10 10:57, Ferruh Yigit:
> On 6/10/2016 10:03 AM, Thomas Monjalon wrote:
> > 2016-05-27 17:48, Ferruh Yigit:
> >> --whole-archive argument only required for pmd libraries, and currently
> >> it covers more libraries. Reducing scope of the argument to pmd
> >> libraries slightly reduce final application size.
> > 
> > In my understanding, --whole-archive is required for static libraries used
> > by plugins:
> > 	http://dpdk.org/commit/20afd76a
> Right, required for static libraries. But more than used by plugins,
> required for plugin to work.
> 
> > If we want to restrict it, I would say it must be around EAL, ring,
> > mbuf, mempool, ethdev, cryptodev, etc.
> > 
> 
> We should restrict to plugins. What happens is, since there is no direct
> call to PMDs from application, PMD code is not included into final
> application. To force PMDs to be included into final binary,
> --whole-archive is required.
> 
> But --whole-archive is not required for other libraries, and just cause
> unnecessary increase in binary size.

Yes it is necessary for other libraries, otherwise some symbols needed by
plugins won't be available.
Example:
In function `rte_pmd_af_packet_devuninit':
rte_eth_af_packet.c:(.text+0x8bf): undefined reference to `rte_eth_dev_allocated'
rte_eth_af_packet.c:(.text+0x930): undefined reference to `rte_eth_dev_release_port'

Re: [PATCH 2/2] mk: reduce scope of whole-archive to pmd libraries

[Ferruh Yigit]

On 6/10/2016 11:18 AM, Thomas Monjalon wrote:
> 2016-06-10 10:57, Ferruh Yigit:
>> On 6/10/2016 10:03 AM, Thomas Monjalon wrote:
>>> 2016-05-27 17:48, Ferruh Yigit:
>>>> --whole-archive argument only required for pmd libraries, and currently
>>>> it covers more libraries. Reducing scope of the argument to pmd
>>>> libraries slightly reduce final application size.
>>>
>>> In my understanding, --whole-archive is required for static libraries used
>>> by plugins:
>>> 	http://dpdk.org/commit/20afd76a
>> Right, required for static libraries. But more than used by plugins,
>> required for plugin to work.
>>
>>> If we want to restrict it, I would say it must be around EAL, ring,
>>> mbuf, mempool, ethdev, cryptodev, etc.
>>>
>>
>> We should restrict to plugins. What happens is, since there is no direct
>> call to PMDs from application, PMD code is not included into final
>> application. To force PMDs to be included into final binary,
>> --whole-archive is required.
>>
>> But --whole-archive is not required for other libraries, and just cause
>> unnecessary increase in binary size.
> 
> Yes it is necessary for other libraries, otherwise some symbols needed by
> plugins won't be available.
> Example:
> In function `rte_pmd_af_packet_devuninit':
> rte_eth_af_packet.c:(.text+0x8bf): undefined reference to `rte_eth_dev_allocated'
> rte_eth_af_packet.c:(.text+0x930): undefined reference to `rte_eth_dev_release_port'
> 
If there is a direct call to that API, linker should include required
object file into final binary, --whole-archive shouldn't be required for
this.

What is special for PMD's is there are not directly called, but
initialized using constructors.

How are you getting above compilation error?

Thanks,
ferruh

Re: [PATCH 2/2] mk: reduce scope of whole-archive to pmd libraries

[Thomas Monjalon]

2016-06-10 12:06, Ferruh Yigit:
> On 6/10/2016 11:18 AM, Thomas Monjalon wrote:
> > 2016-06-10 10:57, Ferruh Yigit:
> >> On 6/10/2016 10:03 AM, Thomas Monjalon wrote:
> >>> 2016-05-27 17:48, Ferruh Yigit:
> >>>> --whole-archive argument only required for pmd libraries, and currently
> >>>> it covers more libraries. Reducing scope of the argument to pmd
> >>>> libraries slightly reduce final application size.
> >>>
> >>> In my understanding, --whole-archive is required for static libraries used
> >>> by plugins:
> >>> 	http://dpdk.org/commit/20afd76a
> >> Right, required for static libraries. But more than used by plugins,
> >> required for plugin to work.
> >>
> >>> If we want to restrict it, I would say it must be around EAL, ring,
> >>> mbuf, mempool, ethdev, cryptodev, etc.
> >>>
> >>
> >> We should restrict to plugins. What happens is, since there is no direct
> >> call to PMDs from application, PMD code is not included into final
> >> application. To force PMDs to be included into final binary,
> >> --whole-archive is required.
> >>
> >> But --whole-archive is not required for other libraries, and just cause
> >> unnecessary increase in binary size.
> > 
> > Yes it is necessary for other libraries, otherwise some symbols needed by
> > plugins won't be available.
> > Example:
> > In function `rte_pmd_af_packet_devuninit':
> > rte_eth_af_packet.c:(.text+0x8bf): undefined reference to `rte_eth_dev_allocated'
> > rte_eth_af_packet.c:(.text+0x930): undefined reference to `rte_eth_dev_release_port'
> > 
> If there is a direct call to that API, linker should include required
> object file into final binary, --whole-archive shouldn't be required for
> this.
> 
> What is special for PMD's is there are not directly called, but
> initialized using constructors.
> 
> How are you getting above compilation error?

The error was generated with your patch on top of others I'm going to send.
I suggest to continue the discussion/review based on the v2 coming soon.

Re: [PATCH 2/2] mk: reduce scope of whole-archive to pmd libraries

[Ferruh Yigit]

On 6/10/2016 1:21 PM, Thomas Monjalon wrote:
> 2016-06-10 12:06, Ferruh Yigit:
>> On 6/10/2016 11:18 AM, Thomas Monjalon wrote:
>>> 2016-06-10 10:57, Ferruh Yigit:
>>>> On 6/10/2016 10:03 AM, Thomas Monjalon wrote:
>>>>> 2016-05-27 17:48, Ferruh Yigit:
>>>>>> --whole-archive argument only required for pmd libraries, and currently
>>>>>> it covers more libraries. Reducing scope of the argument to pmd
>>>>>> libraries slightly reduce final application size.
>>>>>
>>>>> In my understanding, --whole-archive is required for static libraries used
>>>>> by plugins:
>>>>> 	http://dpdk.org/commit/20afd76a
>>>> Right, required for static libraries. But more than used by plugins,
>>>> required for plugin to work.
>>>>
>>>>> If we want to restrict it, I would say it must be around EAL, ring,
>>>>> mbuf, mempool, ethdev, cryptodev, etc.
>>>>>
>>>>
>>>> We should restrict to plugins. What happens is, since there is no direct
>>>> call to PMDs from application, PMD code is not included into final
>>>> application. To force PMDs to be included into final binary,
>>>> --whole-archive is required.
>>>>
>>>> But --whole-archive is not required for other libraries, and just cause
>>>> unnecessary increase in binary size.
>>>
>>> Yes it is necessary for other libraries, otherwise some symbols needed by
>>> plugins won't be available.
>>> Example:
>>> In function `rte_pmd_af_packet_devuninit':
>>> rte_eth_af_packet.c:(.text+0x8bf): undefined reference to `rte_eth_dev_allocated'
>>> rte_eth_af_packet.c:(.text+0x930): undefined reference to `rte_eth_dev_release_port'
>>>
>> If there is a direct call to that API, linker should include required
>> object file into final binary, --whole-archive shouldn't be required for
>> this.
>>
>> What is special for PMD's is there are not directly called, but
>> initialized using constructors.
>>
>> How are you getting above compilation error?
> 
> The error was generated with your patch on top of others I'm going to send.
> I suggest to continue the discussion/review based on the v2 coming soon.
> 

I have quickly checked v2, you are getting this compiler error because
--start-group/--end-group removed, so pmd not able to resolve the
references. And you are working around this by adding all object of
library unconditionally (--whole-archive). I will comment more of v2 patch.

Thanks,
ferruh

Re: [PATCH 1/2] mk: prevent overlinking in applications

[Thomas Monjalon]

Hi Ferruh,

2016-05-27 17:48, Ferruh Yigit:
> Replace --no-as-needed linker flag with --as-needed flag, which will
> only link libraries directly called by application. This requires inter
> library dependencies resolved correctly.
> 
> Not linking all libraries cause a compile error for lpcap and possible
> to have other similar compiler errors, so increasing the scope of
> --start-group argument.

What is the error?

> cmdline_test application causes compile error because of cyclic
> dependency between librte_eal <-> librte_mempool. A workaround added to
> cmdline_test for compile error.
> 
> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>

> --- a/app/cmdline_test/Makefile
> +++ b/app/cmdline_test/Makefile
> @@ -46,6 +46,7 @@ SRCS-y += commands.c
>  
>  CFLAGS += -O3
>  CFLAGS += $(WERROR_FLAGS)

A comment is required here to explain the workaround.

> +LDFLAGS += -no-as-needed

The option should be --no-as-needed.

> --- a/mk/exec-env/linuxapp/rte.vars.mk
> +++ b/mk/exec-env/linuxapp/rte.vars.mk
> @@ -46,7 +46,7 @@ EXECENV_CFLAGS  = -pthread
>  endif
>  
>  # Workaround lack of DT_NEEDED entry

This comment is obsolete now.

> -EXECENV_LDFLAGS = --no-as-needed
> +EXECENV_LDFLAGS = --as-needed

Why put this option for Linux only?
Shouldn't we restrict this option to app.mk only?

> --- a/mk/rte.app.mk
> +++ b/mk/rte.app.mk
> @@ -58,6 +58,7 @@ _LDLIBS-y += -L$(RTE_SDK_BIN)/lib
>  #
>  
>  _LDLIBS-y += --whole-archive
> +_LDLIBS-y += --start-group

--start-group must be used only to solve circular dependencies.
Ideally we should not use it. I think it's needed only because
of EAL logs using mempool (must be removed).
Why extending it?
I'm afraid we are masking some issues.

>  _LDLIBS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR)    += -lrte_distributor
>  _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER)        += -lrte_reorder
> @@ -111,8 +112,6 @@ _LDLIBS-y                                   += -lcrypto
>  endif
>  endif # !CONFIG_RTE_BUILD_SHARED_LIBS
>  
> -_LDLIBS-y += --start-group
> -
>  _LDLIBS-$(CONFIG_RTE_LIBRTE_KVARGS)         += -lrte_kvargs
>  _LDLIBS-$(CONFIG_RTE_LIBRTE_MBUF)           += -lrte_mbuf
>  _LDLIBS-$(CONFIG_RTE_LIBRTE_IP_FRAG)        += -lrte_ip_frag

Re: [PATCH 1/2] mk: prevent overlinking in applications

[Ferruh Yigit]

On 6/9/2016 11:10 AM, Thomas Monjalon wrote:
> Hi Ferruh,
> 
> 2016-05-27 17:48, Ferruh Yigit:
>> Replace --no-as-needed linker flag with --as-needed flag, which will
>> only link libraries directly called by application. This requires inter
>> library dependencies resolved correctly.
>>
>> Not linking all libraries cause a compile error for lpcap and possible
>> to have other similar compiler errors, so increasing the scope of
>> --start-group argument.
> 
> What is the error?
> 
This is with -as-needed flag, and static library compilation:

.../dpdk/build/lib/librte_pmd_pcap.a(rte_eth_pcap.o): In function
`eth_dev_stop':
rte_eth_pcap.c:(.text+0xcc1): undefined reference to `pcap_dump_close'
rte_eth_pcap.c:(.text+0xcd6): undefined reference to `pcap_close'
rte_eth_pcap.c:(.text+0xd19): undefined reference to `pcap_close'
rte_eth_pcap.c:(.text+0xd58): undefined reference to `pcap_close'
/root/development/dpdk/build/lib/librte_pmd_pcap.a(rte_eth_pcap.o): In
function `open_tx_pcap':
rte_eth_pcap.c:(.text+0x190b): undefined reference to `pcap_open_dead'
rte_eth_pcap.c:(.text+0x191b): undefined reference to `pcap_dump_open'
...

pcap pmd has libpcap calls, but while linking final application
(testpmd), linker is not able to find objects that has these.

The command line for compilation is:
gcc ... -o test ... -Wl,--whole-archive <dpdk libs> -Wl,-lpcap
-Wl,--start-group <more dpdk libs> -Wl,-lrte_pmd_pcap <more dpdk libs>
-Wl,--end-group -Wl,--no-whole-archive

-lpcap is provided, but since before -lrte_pmd_pcap, references not
resolved.

Previously, with "-no-as-needed" flag, all shared libraries linked
always, which was preventing compile error.


>> cmdline_test application causes compile error because of cyclic
>> dependency between librte_eal <-> librte_mempool. A workaround added to
>> cmdline_test for compile error.
>>
>> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> 
>> --- a/app/cmdline_test/Makefile
>> +++ b/app/cmdline_test/Makefile
>> @@ -46,6 +46,7 @@ SRCS-y += commands.c
>>  
>>  CFLAGS += -O3
>>  CFLAGS += $(WERROR_FLAGS)
> 
> A comment is required here to explain the workaround.
> 

Sure, I will add a comment and send a new version of the patch.

>> +LDFLAGS += -no-as-needed
> 
> The option should be --no-as-needed.
Right

> 
>> --- a/mk/exec-env/linuxapp/rte.vars.mk
>> +++ b/mk/exec-env/linuxapp/rte.vars.mk
>> @@ -46,7 +46,7 @@ EXECENV_CFLAGS  = -pthread
>>  endif
>>  
>>  # Workaround lack of DT_NEEDED entry
> 
> This comment is obsolete now.
Will remove.

> 
>> -EXECENV_LDFLAGS = --no-as-needed
>> +EXECENV_LDFLAGS = --as-needed
> 
> Why put this option for Linux only?
When this option not defined at all, different compilers behave
different, some has --as-needed as default and cause compilation issues,
I guess that is why for Linux --no-as-needed is forces. I assume BSD
compilers by default use --no-as-needed, so we don't have any
compilation problem.
And it looks like there is no specific reason to not force BSD to
--as-needed, I will test it.

> Shouldn't we restrict this option to app.mk only?
Having --no-needed flag only for app should be OK. Let me test first,
and as a result should we move this to app.mk?

> 
>> --- a/mk/rte.app.mk
>> +++ b/mk/rte.app.mk
>> @@ -58,6 +58,7 @@ _LDLIBS-y += -L$(RTE_SDK_BIN)/lib
>>  #
>>  
>>  _LDLIBS-y += --whole-archive
>> +_LDLIBS-y += --start-group
> 
> --start-group must be used only to solve circular dependencies.
> Ideally we should not use it. I think it's needed only because
> of EAL logs using mempool (must be removed).
No, this is not the reason, please check above lpcap compile error, that
is the reason of this update.

> Why extending it?
> I'm afraid we are masking some issues.
But if we have a convention to add external libraries after dpdk
libraries, that also should solve this issue, but that is more update
than just updating --start-group.

> 
>>  _LDLIBS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR)    += -lrte_distributor
>>  _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER)        += -lrte_reorder
>> @@ -111,8 +112,6 @@ _LDLIBS-y                                   += -lcrypto
>>  endif
>>  endif # !CONFIG_RTE_BUILD_SHARED_LIBS
>>  
>> -_LDLIBS-y += --start-group
>> -
>>  _LDLIBS-$(CONFIG_RTE_LIBRTE_KVARGS)         += -lrte_kvargs
>>  _LDLIBS-$(CONFIG_RTE_LIBRTE_MBUF)           += -lrte_mbuf
>>  _LDLIBS-$(CONFIG_RTE_LIBRTE_IP_FRAG)        += -lrte_ip_frag
> 

[PATCH v2 0/6] reduce overlinking in applications

[Thomas Monjalon]

This is a respin of the ideas of Christian and Ferruh to limit
the static application size or dynamic links to shared libraries.

It also brings some clean-up in rte.app.mk.

Ferruh Yigit (2):
  mk: prevent overlinking in applications
  mk: reduce scope of whole-archive static linking

Thomas Monjalon (4):
  mk: sort drivers in static application link list
  mk: fix driver dependencies order for static application
  mk: remove library grouping during application linking
  mk: sort libraries in level order when linking

 mk/exec-env/linuxapp/rte.vars.mk |   3 -
 mk/rte.app.mk                    | 134 ++++++++++++++++-----------------------
 2 files changed, 55 insertions(+), 82 deletions(-)

-- 
2.7.0

[PATCH v2 1/6] mk: sort drivers in static application link list

[Thomas Monjalon]

Just a clean up to prepare next patches.

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
---
 mk/rte.app.mk | 45 +++++++++++++++++----------------------------
 1 file changed, 17 insertions(+), 28 deletions(-)

diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index b84b56d..07be17c 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -124,57 +124,46 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL)            += -lrte_eal
 _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE)        += -lrte_cmdline
 _LDLIBS-$(CONFIG_RTE_LIBRTE_CFGFILE)        += -lrte_cfgfile
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_BOND)       += -lrte_pmd_bond
-
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT)    += -lrte_pmd_xenvirt
 
 ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
 # plugins (link only if static libraries)
 
-_LDLIBS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD)    += -lrte_pmd_vmxnet3_uio
-_LDLIBS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD)     += -lrte_pmd_virtio
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET)  += -lrte_pmd_af_packet
 _LDLIBS-$(CONFIG_RTE_LIBRTE_BNX2X_PMD)      += -lrte_pmd_bnx2x
 _LDLIBS-$(CONFIG_RTE_LIBRTE_CXGBE_PMD)      += -lrte_pmd_cxgbe
+_LDLIBS-$(CONFIG_RTE_LIBRTE_E1000_PMD)      += -lrte_pmd_e1000
+_LDLIBS-$(CONFIG_RTE_LIBRTE_ENA_PMD)        += -lrte_pmd_ena
 _LDLIBS-$(CONFIG_RTE_LIBRTE_ENIC_PMD)       += -lrte_pmd_enic
-_LDLIBS-$(CONFIG_RTE_LIBRTE_I40E_PMD)       += -lrte_pmd_i40e
 _LDLIBS-$(CONFIG_RTE_LIBRTE_FM10K_PMD)      += -lrte_pmd_fm10k
+_LDLIBS-$(CONFIG_RTE_LIBRTE_I40E_PMD)       += -lrte_pmd_i40e
 _LDLIBS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD)      += -lrte_pmd_ixgbe
-_LDLIBS-$(CONFIG_RTE_LIBRTE_E1000_PMD)      += -lrte_pmd_e1000
-_LDLIBS-$(CONFIG_RTE_LIBRTE_ENA_PMD)        += -lrte_pmd_ena
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD)       += -lrte_pmd_mlx4
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD)       += -lrte_pmd_mlx5
-_LDLIBS-$(CONFIG_RTE_LIBRTE_NFP_PMD)        += -lrte_pmd_nfp
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2)   += -lrte_pmd_szedata2
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD)      += -lrte_pmd_mpipe
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_RING)       += -lrte_pmd_ring
+_LDLIBS-$(CONFIG_RTE_LIBRTE_NFP_PMD)        += -lrte_pmd_nfp
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL)       += -lrte_pmd_null
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_PCAP)       += -lrte_pmd_pcap
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET)  += -lrte_pmd_af_packet
 _LDLIBS-$(CONFIG_RTE_LIBRTE_QEDE_PMD)       += -lrte_pmd_qede
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL)       += -lrte_pmd_null
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_RING)       += -lrte_pmd_ring
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2)   += -lrte_pmd_szedata2
+_LDLIBS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD)     += -lrte_pmd_virtio
+ifeq ($(CONFIG_RTE_LIBRTE_VHOST),y)
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_VHOST)      += -lrte_pmd_vhost
+endif # $(CONFIG_RTE_LIBRTE_VHOST)
+_LDLIBS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD)    += -lrte_pmd_vmxnet3_uio
 
 ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y)
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_QAT)        += -lrte_pmd_qat
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB)   += -lrte_pmd_aesni_mb
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM)   += -lrte_pmd_aesni_gcm
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB)   += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM)  += -lrte_pmd_aesni_gcm
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM)  += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO) += -lrte_pmd_null_crypto
-
-# AESNI MULTI BUFFER / GCM PMDs are dependent on the IPSec_MB library
-ifeq ($(CONFIG_RTE_LIBRTE_PMD_AESNI_MB),y)
-_LDLIBS-y += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
-else ifeq ($(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM),y)
-_LDLIBS-y += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
-endif
-
-# SNOW3G PMD is dependent on the LIBSSO library
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_QAT)        += -lrte_pmd_qat
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SNOW3G)     += -lrte_pmd_snow3g
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SNOW3G)     += -L$(LIBSSO_PATH)/build -lsso
 endif # CONFIG_RTE_LIBRTE_CRYPTODEV
 
-ifeq ($(CONFIG_RTE_LIBRTE_VHOST),y)
-
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_VHOST)      += -lrte_pmd_vhost
-
-endif # $(CONFIG_RTE_LIBRTE_VHOST)
-
 endif # ! $(CONFIG_RTE_BUILD_SHARED_LIB)
 
 _LDLIBS-y += $(EXECENV_LDLIBS)
-- 
2.7.0

[PATCH v2 2/6] mk: fix driver dependencies order for static application

[Thomas Monjalon]

On a linker command line, the dependencies must be declared after
the libraries using them.
It will avoid some issues when building an application with static
libraries and --as-needed option.

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
---
 mk/rte.app.mk | 38 +++++++++++---------------------------
 1 file changed, 11 insertions(+), 27 deletions(-)

diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 07be17c..8068e66 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -93,22 +93,6 @@ ifeq ($(CONFIG_RTE_LIBRTE_VHOST_USER),n)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST)          += -lfuse
 endif
 _LDLIBS-$(CONFIG_RTE_PORT_PCAP)             += -lpcap
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_PCAP)       += -lpcap
-_LDLIBS-$(CONFIG_RTE_LIBRTE_BNX2X_PMD)      += -lz
-_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD)       += -libverbs
-_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD)       += -libverbs
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2)   += -lsze2
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT)    += -lxenstore
-_LDLIBS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD)      += -lgxio
-_LDLIBS-$(CONFIG_RTE_LIBRTE_NFP_PMD)        += -lm
-_LDLIBS-$(CONFIG_RTE_LIBRTE_QEDE_PMD)       += -lz
-# QAT / AESNI GCM PMDs are dependent on libcrypto (from openssl)
-# for calculating HMAC precomputes
-ifeq ($(CONFIG_RTE_LIBRTE_PMD_QAT),y)
-_LDLIBS-y                                   += -lcrypto
-else ifeq ($(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM),y)
-_LDLIBS-y                                   += -lcrypto
-endif
 endif # !CONFIG_RTE_BUILD_SHARED_LIBS
 
 _LDLIBS-y += --start-group
@@ -124,13 +108,13 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL)            += -lrte_eal
 _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE)        += -lrte_cmdline
 _LDLIBS-$(CONFIG_RTE_LIBRTE_CFGFILE)        += -lrte_cfgfile
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_BOND)       += -lrte_pmd_bond
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT)    += -lrte_pmd_xenvirt
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT)    += -lrte_pmd_xenvirt -lxenstore
 
 ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
 # plugins (link only if static libraries)
 
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET)  += -lrte_pmd_af_packet
-_LDLIBS-$(CONFIG_RTE_LIBRTE_BNX2X_PMD)      += -lrte_pmd_bnx2x
+_LDLIBS-$(CONFIG_RTE_LIBRTE_BNX2X_PMD)      += -lrte_pmd_bnx2x -lz
 _LDLIBS-$(CONFIG_RTE_LIBRTE_CXGBE_PMD)      += -lrte_pmd_cxgbe
 _LDLIBS-$(CONFIG_RTE_LIBRTE_E1000_PMD)      += -lrte_pmd_e1000
 _LDLIBS-$(CONFIG_RTE_LIBRTE_ENA_PMD)        += -lrte_pmd_ena
@@ -138,15 +122,15 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_ENIC_PMD)       += -lrte_pmd_enic
 _LDLIBS-$(CONFIG_RTE_LIBRTE_FM10K_PMD)      += -lrte_pmd_fm10k
 _LDLIBS-$(CONFIG_RTE_LIBRTE_I40E_PMD)       += -lrte_pmd_i40e
 _LDLIBS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD)      += -lrte_pmd_ixgbe
-_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD)       += -lrte_pmd_mlx4
-_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD)       += -lrte_pmd_mlx5
-_LDLIBS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD)      += -lrte_pmd_mpipe
-_LDLIBS-$(CONFIG_RTE_LIBRTE_NFP_PMD)        += -lrte_pmd_nfp
+_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD)       += -lrte_pmd_mlx4 -libverbs
+_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD)       += -lrte_pmd_mlx5 -libverbs
+_LDLIBS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD)      += -lrte_pmd_mpipe -lgxio
+_LDLIBS-$(CONFIG_RTE_LIBRTE_NFP_PMD)        += -lrte_pmd_nfp -lm
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL)       += -lrte_pmd_null
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_PCAP)       += -lrte_pmd_pcap
-_LDLIBS-$(CONFIG_RTE_LIBRTE_QEDE_PMD)       += -lrte_pmd_qede
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_PCAP)       += -lrte_pmd_pcap -lpcap
+_LDLIBS-$(CONFIG_RTE_LIBRTE_QEDE_PMD)       += -lrte_pmd_qede -lz
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_RING)       += -lrte_pmd_ring
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2)   += -lrte_pmd_szedata2
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2)   += -lrte_pmd_szedata2 -lsze2
 _LDLIBS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD)     += -lrte_pmd_virtio
 ifeq ($(CONFIG_RTE_LIBRTE_VHOST),y)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_VHOST)      += -lrte_pmd_vhost
@@ -156,10 +140,10 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD)    += -lrte_pmd_vmxnet3_uio
 ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB)   += -lrte_pmd_aesni_mb
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB)   += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM)  += -lrte_pmd_aesni_gcm
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM)  += -lrte_pmd_aesni_gcm -lcrypto
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM)  += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO) += -lrte_pmd_null_crypto
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_QAT)        += -lrte_pmd_qat
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_QAT)        += -lrte_pmd_qat -lcrypto
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SNOW3G)     += -lrte_pmd_snow3g
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SNOW3G)     += -L$(LIBSSO_PATH)/build -lsso
 endif # CONFIG_RTE_LIBRTE_CRYPTODEV
-- 
2.7.0

Re: [PATCH v2 2/6] mk: fix driver dependencies order for static application

[Ferruh Yigit]

On 6/10/2016 2:19 PM, Thomas Monjalon wrote:
> On a linker command line, the dependencies must be declared after
> the libraries using them.
> It will avoid some issues when building an application with static
> libraries and --as-needed option.
> 
> Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
> ---

Do we need this which driver used which library detail?

I think it is enough to move all external library dependencies to the
last in command line, without paying attention who use it, Linker will
handle the rest, if that library is not required it won't be linked against.

Re: [PATCH v2 2/6] mk: fix driver dependencies order for static application

[Thomas Monjalon]

2016-06-10 15:49, Ferruh Yigit:
> On 6/10/2016 2:19 PM, Thomas Monjalon wrote:
> > On a linker command line, the dependencies must be declared after
> > the libraries using them.
> > It will avoid some issues when building an application with static
> > libraries and --as-needed option.
> > 
> > Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
> > ---
> 
> Do we need this which driver used which library detail?
> 
> I think it is enough to move all external library dependencies to the
> last in command line, without paying attention who use it, Linker will
> handle the rest, if that library is not required it won't be linked against.

I prefer avoiding garbage and have a justification for dependencies.

[PATCH v2 3/6] mk: remove library grouping during application linking

[Thomas Monjalon]

>From "man ld":
  Using this option has a significant performance cost.
  It is best to use it only when there are unavoidable
  circular references between two or more archives.

Remove the option since it does not seem necessary.

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
---
 mk/rte.app.mk | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 8068e66..9306b30 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -95,8 +95,6 @@ endif
 _LDLIBS-$(CONFIG_RTE_PORT_PCAP)             += -lpcap
 endif # !CONFIG_RTE_BUILD_SHARED_LIBS
 
-_LDLIBS-y += --start-group
-
 _LDLIBS-$(CONFIG_RTE_LIBRTE_KVARGS)         += -lrte_kvargs
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MBUF)           += -lrte_mbuf
 _LDLIBS-$(CONFIG_RTE_LIBRTE_IP_FRAG)        += -lrte_ip_frag
@@ -151,7 +149,7 @@ endif # CONFIG_RTE_LIBRTE_CRYPTODEV
 endif # ! $(CONFIG_RTE_BUILD_SHARED_LIB)
 
 _LDLIBS-y += $(EXECENV_LDLIBS)
-_LDLIBS-y += --end-group
+
 _LDLIBS-y += --no-whole-archive
 
 LDLIBS += $(_LDLIBS-y) $(CPU_LDLIBS) $(EXTRA_LDLIBS)
-- 
2.7.0

Re: [PATCH v2 3/6] mk: remove library grouping during application linking

[Ferruh Yigit]

On 6/10/2016 2:19 PM, Thomas Monjalon wrote:
> From "man ld":
>   Using this option has a significant performance cost.
>   It is best to use it only when there are unavoidable
>   circular references between two or more archives.
> 
> Remove the option since it does not seem necessary.
> 
> Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>

To remove this, I think we should order libraries in a way:

pmd -> librte -> external libraries.
Than we can remove --whole-archive too.

And perhaps keep it still to wrap around librte_eal and librte_mempool,
which still has cyclic dependencies.

Re: [PATCH v2 3/6] mk: remove library grouping during application linking

[Thomas Monjalon]

2016-06-10 15:54, Ferruh Yigit:
> On 6/10/2016 2:19 PM, Thomas Monjalon wrote:
> > From "man ld":
> >   Using this option has a significant performance cost.
> >   It is best to use it only when there are unavoidable
> >   circular references between two or more archives.
> > 
> > Remove the option since it does not seem necessary.
> > 
> > Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
> 
> To remove this, I think we should order libraries in a way:
> 
> pmd -> librte -> external libraries.
> Than we can remove --whole-archive too.
> 
> And perhaps keep it still to wrap around librte_eal and librte_mempool,
> which still has cyclic dependencies.

OK looks good
I let you do a v3

[PATCH v2 4/6] mk: prevent overlinking in applications

[Thomas Monjalon]

From: Ferruh Yigit <ferruh.yigit@intel.com>

Replace --no-as-needed linker flag with --as-needed flag, which will
only link libraries directly called by application.
It can be achieved now that the libraries dependencies are handled
properly.

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
---
v2: remove workarounds and workaround comment
---
 mk/exec-env/linuxapp/rte.vars.mk | 3 ---
 mk/rte.app.mk                    | 3 +++
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/mk/exec-env/linuxapp/rte.vars.mk b/mk/exec-env/linuxapp/rte.vars.mk
index d51bd17..a8a1ee4 100644
--- a/mk/exec-env/linuxapp/rte.vars.mk
+++ b/mk/exec-env/linuxapp/rte.vars.mk
@@ -45,9 +45,6 @@ else
 EXECENV_CFLAGS  = -pthread
 endif
 
-# Workaround lack of DT_NEEDED entry
-EXECENV_LDFLAGS = --no-as-needed
-
 EXECENV_LDLIBS  =
 EXECENV_ASFLAGS =
 
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 9306b30..d01a79f 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -50,6 +50,9 @@ ifeq ($(NO_LDSCRIPT),)
 LDSCRIPT = $(RTE_LDSCRIPT)
 endif
 
+# Link only the libraries used in the application
+LDFLAGS += --as-needed
+
 # default path for libs
 _LDLIBS-y += -L$(RTE_SDK_BIN)/lib
 
-- 
2.7.0

Re: [PATCH v2 4/6] mk: prevent overlinking in applications

[Ferruh Yigit]

On 6/10/2016 2:19 PM, Thomas Monjalon wrote:
> From: Ferruh Yigit <ferruh.yigit@intel.com>
> 
> Replace --no-as-needed linker flag with --as-needed flag, which will
> only link libraries directly called by application.
> It can be achieved now that the libraries dependencies are handled
> properly.
> 
> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
> ---
...
>  endif
>  
> +# Link only the libraries used in the application
> +LDFLAGS += --as-needed

For the case we need to have workaround for application, this doesn't
let because last argument wins.

Can we say as following :
LDFLAGS = --as-needed $(LDFLAGS)


> +
>  # default path for libs
>  _LDLIBS-y += -L$(RTE_SDK_BIN)/lib
>  
> 

Re: [PATCH v2 4/6] mk: prevent overlinking in applications

[Thomas Monjalon]

2016-06-10 15:56, Ferruh Yigit:
> On 6/10/2016 2:19 PM, Thomas Monjalon wrote:
> > From: Ferruh Yigit <ferruh.yigit@intel.com>
> > 
> > Replace --no-as-needed linker flag with --as-needed flag, which will
> > only link libraries directly called by application.
> > It can be achieved now that the libraries dependencies are handled
> > properly.
[...]
> > +# Link only the libraries used in the application
> > +LDFLAGS += --as-needed
> 
> For the case we need to have workaround for application, this doesn't
> let because last argument wins.
> 
> Can we say as following :
> LDFLAGS = --as-needed $(LDFLAGS)

It should be LDFLAGS := --as-needed $(LDFLAGS)
But I would prefer we avoid workarounds.

[PATCH v2 5/6] mk: sort libraries in level order when linking

[Thomas Monjalon]

As stated in the comment:
    Order is important: from higher level to lower level

This is an attempt to make the layering order better respected.
It will help to restrict the --whole-archive scope for plugins.

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
---
 mk/rte.app.mk | 54 ++++++++++++++++++++++++++++--------------------------
 1 file changed, 28 insertions(+), 26 deletions(-)

diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index d01a79f..99a7047 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -62,9 +62,6 @@ _LDLIBS-y += -L$(RTE_SDK_BIN)/lib
 
 _LDLIBS-y += --whole-archive
 
-_LDLIBS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR)    += -lrte_distributor
-_LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER)        += -lrte_reorder
-
 ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),y)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_KNI)            += -lrte_kni
 _LDLIBS-$(CONFIG_RTE_LIBRTE_IVSHMEM)        += -lrte_ivshmem
@@ -73,34 +70,23 @@ endif
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PIPELINE)       += -lrte_pipeline
 _LDLIBS-$(CONFIG_RTE_LIBRTE_TABLE)          += -lrte_table
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PORT)           += -lrte_port
-_LDLIBS-$(CONFIG_RTE_LIBRTE_TIMER)          += -lrte_timer
-_LDLIBS-$(CONFIG_RTE_LIBRTE_HASH)           += -lrte_hash
-_LDLIBS-$(CONFIG_RTE_LIBRTE_JOBSTATS)       += -lrte_jobstats
-_LDLIBS-$(CONFIG_RTE_LIBRTE_LPM)            += -lrte_lpm
-_LDLIBS-$(CONFIG_RTE_LIBRTE_POWER)          += -lrte_power
-_LDLIBS-$(CONFIG_RTE_LIBRTE_ACL)            += -lrte_acl
+
+_LDLIBS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR)    += -lrte_distributor
+_LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER)        += -lrte_reorder
+_LDLIBS-$(CONFIG_RTE_LIBRTE_IP_FRAG)        += -lrte_ip_frag
 _LDLIBS-$(CONFIG_RTE_LIBRTE_METER)          += -lrte_meter
 _LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED)          += -lrte_sched
-_LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST)          += -lrte_vhost
+_LDLIBS-$(CONFIG_RTE_LIBRTE_LPM)            += -lrte_lpm
+_LDLIBS-$(CONFIG_RTE_LIBRTE_ACL)            += -lrte_acl
+_LDLIBS-$(CONFIG_RTE_LIBRTE_JOBSTATS)       += -lrte_jobstats
+_LDLIBS-$(CONFIG_RTE_LIBRTE_POWER)          += -lrte_power
 
-# The static libraries do not know their dependencies.
-# So linking with static library requires explicit dependencies.
-ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
-_LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED)          += -lm
-_LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED)          += -lrt
-_LDLIBS-$(CONFIG_RTE_LIBRTE_METER)          += -lm
-ifeq ($(CONFIG_RTE_LIBRTE_VHOST_NUMA),y)
-_LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST)          += -lnuma
-endif
-ifeq ($(CONFIG_RTE_LIBRTE_VHOST_USER),n)
-_LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST)          += -lfuse
-endif
-_LDLIBS-$(CONFIG_RTE_PORT_PCAP)             += -lpcap
-endif # !CONFIG_RTE_BUILD_SHARED_LIBS
+_LDLIBS-$(CONFIG_RTE_LIBRTE_TIMER)          += -lrte_timer
+_LDLIBS-$(CONFIG_RTE_LIBRTE_HASH)           += -lrte_hash
+_LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST)          += -lrte_vhost
 
 _LDLIBS-$(CONFIG_RTE_LIBRTE_KVARGS)         += -lrte_kvargs
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MBUF)           += -lrte_mbuf
-_LDLIBS-$(CONFIG_RTE_LIBRTE_IP_FRAG)        += -lrte_ip_frag
 _LDLIBS-$(CONFIG_RTE_LIBRTE_ETHER)          += -lethdev
 _LDLIBS-$(CONFIG_RTE_LIBRTE_CRYPTODEV)      += -lrte_cryptodev
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MEMPOOL)        += -lrte_mempool
@@ -108,6 +94,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_RING)           += -lrte_ring
 _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL)            += -lrte_eal
 _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE)        += -lrte_cmdline
 _LDLIBS-$(CONFIG_RTE_LIBRTE_CFGFILE)        += -lrte_cfgfile
+
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_BOND)       += -lrte_pmd_bond
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT)    += -lrte_pmd_xenvirt -lxenstore
 
@@ -149,7 +136,22 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SNOW3G)     += -lrte_pmd_snow3g
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SNOW3G)     += -L$(LIBSSO_PATH)/build -lsso
 endif # CONFIG_RTE_LIBRTE_CRYPTODEV
 
-endif # ! $(CONFIG_RTE_BUILD_SHARED_LIB)
+endif # !CONFIG_RTE_BUILD_SHARED_LIBS
+
+ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
+# The static libraries do not know their dependencies.
+# So linking with static library requires explicit dependencies.
+_LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED)          += -lm
+_LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED)          += -lrt
+_LDLIBS-$(CONFIG_RTE_LIBRTE_METER)          += -lm
+ifeq ($(CONFIG_RTE_LIBRTE_VHOST_NUMA),y)
+_LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST)          += -lnuma
+endif
+ifeq ($(CONFIG_RTE_LIBRTE_VHOST_USER),n)
+_LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST)          += -lfuse
+endif
+_LDLIBS-$(CONFIG_RTE_PORT_PCAP)             += -lpcap
+endif # !CONFIG_RTE_BUILD_SHARED_LIBS
 
 _LDLIBS-y += $(EXECENV_LDLIBS)
 
-- 
2.7.0

[PATCH v2 6/6] mk: reduce scope of whole-archive static linking

[Thomas Monjalon]

From: Ferruh Yigit <ferruh.yigit@intel.com>

The --whole-archive argument is only required for plugins (drivers)
and libraries used by these plugins.
Currently it covers all libraries.
Reducing the scope of this argument slightly reduce final application size
when statically linked.

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
---
v2: keep some basic libs in the whole-archive scope
---
 mk/rte.app.mk | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 99a7047..e9969fc 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -60,8 +60,6 @@ _LDLIBS-y += -L$(RTE_SDK_BIN)/lib
 # Order is important: from higher level to lower level
 #
 
-_LDLIBS-y += --whole-archive
-
 ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),y)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_KNI)            += -lrte_kni
 _LDLIBS-$(CONFIG_RTE_LIBRTE_IVSHMEM)        += -lrte_ivshmem
@@ -81,6 +79,8 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_ACL)            += -lrte_acl
 _LDLIBS-$(CONFIG_RTE_LIBRTE_JOBSTATS)       += -lrte_jobstats
 _LDLIBS-$(CONFIG_RTE_LIBRTE_POWER)          += -lrte_power
 
+_LDLIBS-y += --whole-archive
+
 _LDLIBS-$(CONFIG_RTE_LIBRTE_TIMER)          += -lrte_timer
 _LDLIBS-$(CONFIG_RTE_LIBRTE_HASH)           += -lrte_hash
 _LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST)          += -lrte_vhost
@@ -138,6 +138,8 @@ endif # CONFIG_RTE_LIBRTE_CRYPTODEV
 
 endif # !CONFIG_RTE_BUILD_SHARED_LIBS
 
+_LDLIBS-y += --no-whole-archive
+
 ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
 # The static libraries do not know their dependencies.
 # So linking with static library requires explicit dependencies.
@@ -155,8 +157,6 @@ endif # !CONFIG_RTE_BUILD_SHARED_LIBS
 
 _LDLIBS-y += $(EXECENV_LDLIBS)
 
-_LDLIBS-y += --no-whole-archive
-
 LDLIBS += $(_LDLIBS-y) $(CPU_LDLIBS) $(EXTRA_LDLIBS)
 
 # Eliminate duplicates without sorting
-- 
2.7.0

Re: [PATCH v2 6/6] mk: reduce scope of whole-archive static linking

[Ferruh Yigit]

On 6/10/2016 2:19 PM, Thomas Monjalon wrote:
> From: Ferruh Yigit <ferruh.yigit@intel.com>
> 
> The --whole-archive argument is only required for plugins (drivers)
> and libraries used by these plugins.
> Currently it covers all libraries.
> Reducing the scope of this argument slightly reduce final application size
> when statically linked.
> 
> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
> ---

Technically the scope of --whole-archive should be PMDs, wider scope
required because of unresolved references, but I think this is not good
workaround.

If libraries ordered as suggested before: pmd->librte->external, won't
need to extend scope of parameter

Still some applications will have problem for now, because of cyclic
dependency of librte_eal and librte_mempool, which seems targeted to fix.

So, instead of adding --whole-archive as workaround that efffects whole
binaries, I am for adding workaround for effected application.
The be effected this problem, an application should have direct calls to
librte_eal, but shouldn't have any calls to librte_mempool, I think this
rare case, that is why only a unit test application effected. And when
dependency issue removed we can fix this.

To be able to apply workaround per application, LDFLAGS should be set in
a way to commented previous patch.

Thanks,
ferruh

Re: [PATCH v2 6/6] mk: reduce scope of whole-archive static linking

[Thomas Monjalon]

2016-06-10 16:03, Ferruh Yigit:
> On 6/10/2016 2:19 PM, Thomas Monjalon wrote:
> > From: Ferruh Yigit <ferruh.yigit@intel.com>
> > 
> > The --whole-archive argument is only required for plugins (drivers)
> > and libraries used by these plugins.
> > Currently it covers all libraries.
> > Reducing the scope of this argument slightly reduce final application size
> > when statically linked.
> > 
> > Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> > Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
> > ---
> 
> Technically the scope of --whole-archive should be PMDs, wider scope
> required because of unresolved references, but I think this is not good
> workaround.
> 
> If libraries ordered as suggested before: pmd->librte->external, won't
> need to extend scope of parameter
> 
> Still some applications will have problem for now, because of cyclic
> dependency of librte_eal and librte_mempool, which seems targeted to fix.
> 
> So, instead of adding --whole-archive as workaround that efffects whole
> binaries, I am for adding workaround for effected application.

I would prefer keeping a small workaround in rte.app.mk instead of in the
application makefile. We must be careful with applications which may break
because of our changes. This is the kind of issue that they won't fix
quickly.

> The be effected this problem, an application should have direct calls to
> librte_eal, but shouldn't have any calls to librte_mempool, I think this
> rare case, that is why only a unit test application effected. And when
> dependency issue removed we can fix this.
> 
> To be able to apply workaround per application, LDFLAGS should be set in
> a way to commented previous patch.

[PATCH v3 0/6] reduce overlinking in applications

[Ferruh Yigit]

This patch set updates application linking, main motivation is to reduce
overlinking in application, also removes library grouping by re-ordering
libraries and for static compilation reduces whole library linking to PMD
libs.

There is a workaround because of librte_eal <-> librte_mempool cyclic
dependency in rte.app.mk, which can be removed when issue solved.

It also brings some clean-up in rte.app.mk.

Ferruh Yigit (3):
  mk: sort libraries when linking, move pmd libs to higher level
  mk: sort libraries when linking, move external libs to lower level
  mk: prevent overlinking in applications

Thomas Monjalon (3):
  mk: sort drivers in static application link list
  mk: fix driver dependencies order for static application
  mk: remove library grouping during application linking

 mk/exec-env/linuxapp/rte.vars.mk |   3 -
 mk/rte.app.mk                    | 152 +++++++++++++++++----------------------
 2 files changed, 66 insertions(+), 89 deletions(-)

-- 
2.5.5

[PATCH v3 1/6] mk: sort drivers in static application link list

[Ferruh Yigit]

From: Thomas Monjalon <thomas.monjalon@6wind.com>

Just a clean up to prepare next patches.

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
---
 mk/rte.app.mk | 45 +++++++++++++++++----------------------------
 1 file changed, 17 insertions(+), 28 deletions(-)

diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index b84b56d..07be17c 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -124,57 +124,46 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL)            += -lrte_eal
 _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE)        += -lrte_cmdline
 _LDLIBS-$(CONFIG_RTE_LIBRTE_CFGFILE)        += -lrte_cfgfile
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_BOND)       += -lrte_pmd_bond
-
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT)    += -lrte_pmd_xenvirt
 
 ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
 # plugins (link only if static libraries)
 
-_LDLIBS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD)    += -lrte_pmd_vmxnet3_uio
-_LDLIBS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD)     += -lrte_pmd_virtio
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET)  += -lrte_pmd_af_packet
 _LDLIBS-$(CONFIG_RTE_LIBRTE_BNX2X_PMD)      += -lrte_pmd_bnx2x
 _LDLIBS-$(CONFIG_RTE_LIBRTE_CXGBE_PMD)      += -lrte_pmd_cxgbe
+_LDLIBS-$(CONFIG_RTE_LIBRTE_E1000_PMD)      += -lrte_pmd_e1000
+_LDLIBS-$(CONFIG_RTE_LIBRTE_ENA_PMD)        += -lrte_pmd_ena
 _LDLIBS-$(CONFIG_RTE_LIBRTE_ENIC_PMD)       += -lrte_pmd_enic
-_LDLIBS-$(CONFIG_RTE_LIBRTE_I40E_PMD)       += -lrte_pmd_i40e
 _LDLIBS-$(CONFIG_RTE_LIBRTE_FM10K_PMD)      += -lrte_pmd_fm10k
+_LDLIBS-$(CONFIG_RTE_LIBRTE_I40E_PMD)       += -lrte_pmd_i40e
 _LDLIBS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD)      += -lrte_pmd_ixgbe
-_LDLIBS-$(CONFIG_RTE_LIBRTE_E1000_PMD)      += -lrte_pmd_e1000
-_LDLIBS-$(CONFIG_RTE_LIBRTE_ENA_PMD)        += -lrte_pmd_ena
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD)       += -lrte_pmd_mlx4
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD)       += -lrte_pmd_mlx5
-_LDLIBS-$(CONFIG_RTE_LIBRTE_NFP_PMD)        += -lrte_pmd_nfp
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2)   += -lrte_pmd_szedata2
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD)      += -lrte_pmd_mpipe
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_RING)       += -lrte_pmd_ring
+_LDLIBS-$(CONFIG_RTE_LIBRTE_NFP_PMD)        += -lrte_pmd_nfp
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL)       += -lrte_pmd_null
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_PCAP)       += -lrte_pmd_pcap
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET)  += -lrte_pmd_af_packet
 _LDLIBS-$(CONFIG_RTE_LIBRTE_QEDE_PMD)       += -lrte_pmd_qede
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL)       += -lrte_pmd_null
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_RING)       += -lrte_pmd_ring
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2)   += -lrte_pmd_szedata2
+_LDLIBS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD)     += -lrte_pmd_virtio
+ifeq ($(CONFIG_RTE_LIBRTE_VHOST),y)
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_VHOST)      += -lrte_pmd_vhost
+endif # $(CONFIG_RTE_LIBRTE_VHOST)
+_LDLIBS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD)    += -lrte_pmd_vmxnet3_uio
 
 ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y)
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_QAT)        += -lrte_pmd_qat
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB)   += -lrte_pmd_aesni_mb
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM)   += -lrte_pmd_aesni_gcm
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB)   += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM)  += -lrte_pmd_aesni_gcm
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM)  += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO) += -lrte_pmd_null_crypto
-
-# AESNI MULTI BUFFER / GCM PMDs are dependent on the IPSec_MB library
-ifeq ($(CONFIG_RTE_LIBRTE_PMD_AESNI_MB),y)
-_LDLIBS-y += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
-else ifeq ($(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM),y)
-_LDLIBS-y += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
-endif
-
-# SNOW3G PMD is dependent on the LIBSSO library
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_QAT)        += -lrte_pmd_qat
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SNOW3G)     += -lrte_pmd_snow3g
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SNOW3G)     += -L$(LIBSSO_PATH)/build -lsso
 endif # CONFIG_RTE_LIBRTE_CRYPTODEV
 
-ifeq ($(CONFIG_RTE_LIBRTE_VHOST),y)
-
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_VHOST)      += -lrte_pmd_vhost
-
-endif # $(CONFIG_RTE_LIBRTE_VHOST)
-
 endif # ! $(CONFIG_RTE_BUILD_SHARED_LIB)
 
 _LDLIBS-y += $(EXECENV_LDLIBS)
-- 
2.5.5

Re: [PATCH v3 1/6] mk: sort drivers in static application link list

[Wiles, Keith]

On 6/10/16, 1:32 PM, "dev on behalf of Ferruh Yigit" <dev-bounces@dpdk.org on behalf of ferruh.yigit@intel.com> wrote:

>From: Thomas Monjalon <thomas.monjalon@6wind.com>
>
>Just a clean up to prepare next patches.

One thing you have or will do with these patches is create a set of groups PMD, Core, LIB, … and sort the items in each group. I was thinking it maybe more useful to use different names for the groups instead of _LDLIBS e.g. _PMD, _CORE, _LIB(this one may not work), _XYZ then create _LDLIBS from these groups. I know it is another change and sorry I did not catch it before. This only makes sense if we need say different PMD libs in different places within the _LDLIBS variable. Something to think about if you have to make changes again.

>
>Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
>---
> mk/rte.app.mk | 45 +++++++++++++++++----------------------------
> 1 file changed, 17 insertions(+), 28 deletions(-)
>
>diff --git a/mk/rte.app.mk b/mk/rte.app.mk
>index b84b56d..07be17c 100644
>--- a/mk/rte.app.mk
>+++ b/mk/rte.app.mk
>@@ -124,57 +124,46 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL)            += -lrte_eal
> _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE)        += -lrte_cmdline
> _LDLIBS-$(CONFIG_RTE_LIBRTE_CFGFILE)        += -lrte_cfgfile
> _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_BOND)       += -lrte_pmd_bond
>-
> _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT)    += -lrte_pmd_xenvirt
> 
> ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
> # plugins (link only if static libraries)
> 
>-_LDLIBS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD)    += -lrte_pmd_vmxnet3_uio
>-_LDLIBS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD)     += -lrte_pmd_virtio
>+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET)  += -lrte_pmd_af_packet
> _LDLIBS-$(CONFIG_RTE_LIBRTE_BNX2X_PMD)      += -lrte_pmd_bnx2x
> _LDLIBS-$(CONFIG_RTE_LIBRTE_CXGBE_PMD)      += -lrte_pmd_cxgbe
>+_LDLIBS-$(CONFIG_RTE_LIBRTE_E1000_PMD)      += -lrte_pmd_e1000
>+_LDLIBS-$(CONFIG_RTE_LIBRTE_ENA_PMD)        += -lrte_pmd_ena
> _LDLIBS-$(CONFIG_RTE_LIBRTE_ENIC_PMD)       += -lrte_pmd_enic
>-_LDLIBS-$(CONFIG_RTE_LIBRTE_I40E_PMD)       += -lrte_pmd_i40e
> _LDLIBS-$(CONFIG_RTE_LIBRTE_FM10K_PMD)      += -lrte_pmd_fm10k
>+_LDLIBS-$(CONFIG_RTE_LIBRTE_I40E_PMD)       += -lrte_pmd_i40e
> _LDLIBS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD)      += -lrte_pmd_ixgbe
>-_LDLIBS-$(CONFIG_RTE_LIBRTE_E1000_PMD)      += -lrte_pmd_e1000
>-_LDLIBS-$(CONFIG_RTE_LIBRTE_ENA_PMD)        += -lrte_pmd_ena
> _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD)       += -lrte_pmd_mlx4
> _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD)       += -lrte_pmd_mlx5
>-_LDLIBS-$(CONFIG_RTE_LIBRTE_NFP_PMD)        += -lrte_pmd_nfp
>-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2)   += -lrte_pmd_szedata2
> _LDLIBS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD)      += -lrte_pmd_mpipe
>-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_RING)       += -lrte_pmd_ring
>+_LDLIBS-$(CONFIG_RTE_LIBRTE_NFP_PMD)        += -lrte_pmd_nfp
>+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL)       += -lrte_pmd_null
> _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_PCAP)       += -lrte_pmd_pcap
>-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET)  += -lrte_pmd_af_packet
> _LDLIBS-$(CONFIG_RTE_LIBRTE_QEDE_PMD)       += -lrte_pmd_qede
>-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL)       += -lrte_pmd_null
>+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_RING)       += -lrte_pmd_ring
>+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2)   += -lrte_pmd_szedata2
>+_LDLIBS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD)     += -lrte_pmd_virtio
>+ifeq ($(CONFIG_RTE_LIBRTE_VHOST),y)
>+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_VHOST)      += -lrte_pmd_vhost
>+endif # $(CONFIG_RTE_LIBRTE_VHOST)
>+_LDLIBS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD)    += -lrte_pmd_vmxnet3_uio
> 
> ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y)
>-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_QAT)        += -lrte_pmd_qat
> _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB)   += -lrte_pmd_aesni_mb
>-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM)   += -lrte_pmd_aesni_gcm
>+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB)   += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
>+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM)  += -lrte_pmd_aesni_gcm
>+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM)  += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
> _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO) += -lrte_pmd_null_crypto
>-
>-# AESNI MULTI BUFFER / GCM PMDs are dependent on the IPSec_MB library
>-ifeq ($(CONFIG_RTE_LIBRTE_PMD_AESNI_MB),y)
>-_LDLIBS-y += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
>-else ifeq ($(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM),y)
>-_LDLIBS-y += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
>-endif
>-
>-# SNOW3G PMD is dependent on the LIBSSO library
>+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_QAT)        += -lrte_pmd_qat
> _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SNOW3G)     += -lrte_pmd_snow3g
> _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SNOW3G)     += -L$(LIBSSO_PATH)/build -lsso
> endif # CONFIG_RTE_LIBRTE_CRYPTODEV
> 
>-ifeq ($(CONFIG_RTE_LIBRTE_VHOST),y)
>-
>-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_VHOST)      += -lrte_pmd_vhost
>-
>-endif # $(CONFIG_RTE_LIBRTE_VHOST)
>-
> endif # ! $(CONFIG_RTE_BUILD_SHARED_LIB)
> 
> _LDLIBS-y += $(EXECENV_LDLIBS)
>-- 
>2.5.5
>
>

[PATCH v3 2/6] mk: fix driver dependencies order for static application

[Ferruh Yigit]

From: Thomas Monjalon <thomas.monjalon@6wind.com>

On a linker command line, the dependencies must be declared after
the libraries using them.
It will avoid some issues when building an application with static
libraries and --as-needed option.

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
---
 mk/rte.app.mk | 38 +++++++++++---------------------------
 1 file changed, 11 insertions(+), 27 deletions(-)

diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 07be17c..8068e66 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -93,22 +93,6 @@ ifeq ($(CONFIG_RTE_LIBRTE_VHOST_USER),n)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST)          += -lfuse
 endif
 _LDLIBS-$(CONFIG_RTE_PORT_PCAP)             += -lpcap
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_PCAP)       += -lpcap
-_LDLIBS-$(CONFIG_RTE_LIBRTE_BNX2X_PMD)      += -lz
-_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD)       += -libverbs
-_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD)       += -libverbs
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2)   += -lsze2
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT)    += -lxenstore
-_LDLIBS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD)      += -lgxio
-_LDLIBS-$(CONFIG_RTE_LIBRTE_NFP_PMD)        += -lm
-_LDLIBS-$(CONFIG_RTE_LIBRTE_QEDE_PMD)       += -lz
-# QAT / AESNI GCM PMDs are dependent on libcrypto (from openssl)
-# for calculating HMAC precomputes
-ifeq ($(CONFIG_RTE_LIBRTE_PMD_QAT),y)
-_LDLIBS-y                                   += -lcrypto
-else ifeq ($(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM),y)
-_LDLIBS-y                                   += -lcrypto
-endif
 endif # !CONFIG_RTE_BUILD_SHARED_LIBS
 
 _LDLIBS-y += --start-group
@@ -124,13 +108,13 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL)            += -lrte_eal
 _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE)        += -lrte_cmdline
 _LDLIBS-$(CONFIG_RTE_LIBRTE_CFGFILE)        += -lrte_cfgfile
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_BOND)       += -lrte_pmd_bond
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT)    += -lrte_pmd_xenvirt
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT)    += -lrte_pmd_xenvirt -lxenstore
 
 ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
 # plugins (link only if static libraries)
 
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET)  += -lrte_pmd_af_packet
-_LDLIBS-$(CONFIG_RTE_LIBRTE_BNX2X_PMD)      += -lrte_pmd_bnx2x
+_LDLIBS-$(CONFIG_RTE_LIBRTE_BNX2X_PMD)      += -lrte_pmd_bnx2x -lz
 _LDLIBS-$(CONFIG_RTE_LIBRTE_CXGBE_PMD)      += -lrte_pmd_cxgbe
 _LDLIBS-$(CONFIG_RTE_LIBRTE_E1000_PMD)      += -lrte_pmd_e1000
 _LDLIBS-$(CONFIG_RTE_LIBRTE_ENA_PMD)        += -lrte_pmd_ena
@@ -138,15 +122,15 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_ENIC_PMD)       += -lrte_pmd_enic
 _LDLIBS-$(CONFIG_RTE_LIBRTE_FM10K_PMD)      += -lrte_pmd_fm10k
 _LDLIBS-$(CONFIG_RTE_LIBRTE_I40E_PMD)       += -lrte_pmd_i40e
 _LDLIBS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD)      += -lrte_pmd_ixgbe
-_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD)       += -lrte_pmd_mlx4
-_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD)       += -lrte_pmd_mlx5
-_LDLIBS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD)      += -lrte_pmd_mpipe
-_LDLIBS-$(CONFIG_RTE_LIBRTE_NFP_PMD)        += -lrte_pmd_nfp
+_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD)       += -lrte_pmd_mlx4 -libverbs
+_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD)       += -lrte_pmd_mlx5 -libverbs
+_LDLIBS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD)      += -lrte_pmd_mpipe -lgxio
+_LDLIBS-$(CONFIG_RTE_LIBRTE_NFP_PMD)        += -lrte_pmd_nfp -lm
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL)       += -lrte_pmd_null
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_PCAP)       += -lrte_pmd_pcap
-_LDLIBS-$(CONFIG_RTE_LIBRTE_QEDE_PMD)       += -lrte_pmd_qede
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_PCAP)       += -lrte_pmd_pcap -lpcap
+_LDLIBS-$(CONFIG_RTE_LIBRTE_QEDE_PMD)       += -lrte_pmd_qede -lz
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_RING)       += -lrte_pmd_ring
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2)   += -lrte_pmd_szedata2
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2)   += -lrte_pmd_szedata2 -lsze2
 _LDLIBS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD)     += -lrte_pmd_virtio
 ifeq ($(CONFIG_RTE_LIBRTE_VHOST),y)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_VHOST)      += -lrte_pmd_vhost
@@ -156,10 +140,10 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD)    += -lrte_pmd_vmxnet3_uio
 ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB)   += -lrte_pmd_aesni_mb
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB)   += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM)  += -lrte_pmd_aesni_gcm
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM)  += -lrte_pmd_aesni_gcm -lcrypto
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM)  += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO) += -lrte_pmd_null_crypto
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_QAT)        += -lrte_pmd_qat
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_QAT)        += -lrte_pmd_qat -lcrypto
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SNOW3G)     += -lrte_pmd_snow3g
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SNOW3G)     += -L$(LIBSSO_PATH)/build -lsso
 endif # CONFIG_RTE_LIBRTE_CRYPTODEV
-- 
2.5.5

[PATCH v3 3/6] mk: sort libraries when linking, move pmd libs to higher level

[Ferruh Yigit]

As stated in the comment:
    Order is important: from higher level to lower level

This is an attempt to make the layering order better respected.

Limit scope of --whole-archive to pmd libraries

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
 mk/rte.app.mk | 88 ++++++++++++++++++++++++++++++-----------------------------
 1 file changed, 45 insertions(+), 43 deletions(-)

diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 8068e66..b8761ae 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -59,6 +59,51 @@ _LDLIBS-y += -L$(RTE_SDK_BIN)/lib
 
 _LDLIBS-y += --whole-archive
 
+ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
+# plugins (link only if static libraries)
+
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET)  += -lrte_pmd_af_packet
+_LDLIBS-$(CONFIG_RTE_LIBRTE_BNX2X_PMD)      += -lrte_pmd_bnx2x -lz
+_LDLIBS-$(CONFIG_RTE_LIBRTE_CXGBE_PMD)      += -lrte_pmd_cxgbe
+_LDLIBS-$(CONFIG_RTE_LIBRTE_E1000_PMD)      += -lrte_pmd_e1000
+_LDLIBS-$(CONFIG_RTE_LIBRTE_ENA_PMD)        += -lrte_pmd_ena
+_LDLIBS-$(CONFIG_RTE_LIBRTE_ENIC_PMD)       += -lrte_pmd_enic
+_LDLIBS-$(CONFIG_RTE_LIBRTE_FM10K_PMD)      += -lrte_pmd_fm10k
+_LDLIBS-$(CONFIG_RTE_LIBRTE_I40E_PMD)       += -lrte_pmd_i40e
+_LDLIBS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD)      += -lrte_pmd_ixgbe
+_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD)       += -lrte_pmd_mlx4 -libverbs
+_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD)       += -lrte_pmd_mlx5 -libverbs
+_LDLIBS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD)      += -lrte_pmd_mpipe -lgxio
+_LDLIBS-$(CONFIG_RTE_LIBRTE_NFP_PMD)        += -lrte_pmd_nfp -lm
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL)       += -lrte_pmd_null
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_PCAP)       += -lrte_pmd_pcap -lpcap
+_LDLIBS-$(CONFIG_RTE_LIBRTE_QEDE_PMD)       += -lrte_pmd_qede -lz
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_RING)       += -lrte_pmd_ring
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2)   += -lrte_pmd_szedata2 -lsze2
+_LDLIBS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD)     += -lrte_pmd_virtio
+ifeq ($(CONFIG_RTE_LIBRTE_VHOST),y)
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_VHOST)      += -lrte_pmd_vhost
+endif # $(CONFIG_RTE_LIBRTE_VHOST)
+_LDLIBS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD)    += -lrte_pmd_vmxnet3_uio
+
+ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y)
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB)   += -lrte_pmd_aesni_mb
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB)   += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM)  += -lrte_pmd_aesni_gcm -lcrypto
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM)  += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO) += -lrte_pmd_null_crypto
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_QAT)        += -lrte_pmd_qat -lcrypto
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SNOW3G)     += -lrte_pmd_snow3g
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SNOW3G)     += -L$(LIBSSO_PATH)/build -lsso
+endif # CONFIG_RTE_LIBRTE_CRYPTODEV
+
+endif # ! $(CONFIG_RTE_BUILD_SHARED_LIB)
+
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_BOND)       += -lrte_pmd_bond
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT)    += -lrte_pmd_xenvirt -lxenstore
+
+_LDLIBS-y += --no-whole-archive
+
 _LDLIBS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR)    += -lrte_distributor
 _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER)        += -lrte_reorder
 
@@ -107,52 +152,9 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_RING)           += -lrte_ring
 _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL)            += -lrte_eal
 _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE)        += -lrte_cmdline
 _LDLIBS-$(CONFIG_RTE_LIBRTE_CFGFILE)        += -lrte_cfgfile
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_BOND)       += -lrte_pmd_bond
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT)    += -lrte_pmd_xenvirt -lxenstore
-
-ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
-# plugins (link only if static libraries)
-
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET)  += -lrte_pmd_af_packet
-_LDLIBS-$(CONFIG_RTE_LIBRTE_BNX2X_PMD)      += -lrte_pmd_bnx2x -lz
-_LDLIBS-$(CONFIG_RTE_LIBRTE_CXGBE_PMD)      += -lrte_pmd_cxgbe
-_LDLIBS-$(CONFIG_RTE_LIBRTE_E1000_PMD)      += -lrte_pmd_e1000
-_LDLIBS-$(CONFIG_RTE_LIBRTE_ENA_PMD)        += -lrte_pmd_ena
-_LDLIBS-$(CONFIG_RTE_LIBRTE_ENIC_PMD)       += -lrte_pmd_enic
-_LDLIBS-$(CONFIG_RTE_LIBRTE_FM10K_PMD)      += -lrte_pmd_fm10k
-_LDLIBS-$(CONFIG_RTE_LIBRTE_I40E_PMD)       += -lrte_pmd_i40e
-_LDLIBS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD)      += -lrte_pmd_ixgbe
-_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD)       += -lrte_pmd_mlx4 -libverbs
-_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD)       += -lrte_pmd_mlx5 -libverbs
-_LDLIBS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD)      += -lrte_pmd_mpipe -lgxio
-_LDLIBS-$(CONFIG_RTE_LIBRTE_NFP_PMD)        += -lrte_pmd_nfp -lm
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL)       += -lrte_pmd_null
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_PCAP)       += -lrte_pmd_pcap -lpcap
-_LDLIBS-$(CONFIG_RTE_LIBRTE_QEDE_PMD)       += -lrte_pmd_qede -lz
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_RING)       += -lrte_pmd_ring
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2)   += -lrte_pmd_szedata2 -lsze2
-_LDLIBS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD)     += -lrte_pmd_virtio
-ifeq ($(CONFIG_RTE_LIBRTE_VHOST),y)
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_VHOST)      += -lrte_pmd_vhost
-endif # $(CONFIG_RTE_LIBRTE_VHOST)
-_LDLIBS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD)    += -lrte_pmd_vmxnet3_uio
-
-ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y)
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB)   += -lrte_pmd_aesni_mb
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB)   += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM)  += -lrte_pmd_aesni_gcm -lcrypto
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM)  += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO) += -lrte_pmd_null_crypto
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_QAT)        += -lrte_pmd_qat -lcrypto
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SNOW3G)     += -lrte_pmd_snow3g
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SNOW3G)     += -L$(LIBSSO_PATH)/build -lsso
-endif # CONFIG_RTE_LIBRTE_CRYPTODEV
-
-endif # ! $(CONFIG_RTE_BUILD_SHARED_LIB)
 
 _LDLIBS-y += $(EXECENV_LDLIBS)
 _LDLIBS-y += --end-group
-_LDLIBS-y += --no-whole-archive
 
 LDLIBS += $(_LDLIBS-y) $(CPU_LDLIBS) $(EXTRA_LDLIBS)
 
-- 
2.5.5

Re: [PATCH v3 3/6] mk: sort libraries when linking, move pmd libs to higher level

[Thomas Monjalon]

2016-06-10 19:32, Ferruh Yigit:
> As stated in the comment:
>     Order is important: from higher level to lower level
> 
> This is an attempt to make the layering order better respected.
> 
> Limit scope of --whole-archive to pmd libraries

Compared to the link order in the v2, you did two things:
1/ move PMDs to the first position
2/ restrict --whole-archive to PMDs only

Having the PMDs first, helps the linker to get all the PMD dependencies
in the static binary. Thus it seems we do not need --whole-archive
for the PMD dependencies (ethdev, mbuf, etc).
But, if an external PMD is loaded via dlopen, it is possible that it
needs a symbol which was not used by the internal PMDs. So it will not
be found in the statically linked binary.
Let's take another example: if we disable the internal PMDs with their
config options from the static build, and we decide to build them
separately as DSOs. We won't be able to load them as plugins because
they depend on symbols which won't be found in the static binary.

To make it short, the PMDs must be considered as plugins. Therefore, we
must not rely on their availability to link the required symbols in
a static binary.
To make sure the plugin loading will be always well achieved, we must
link the static PMDs at the last position.

If you agree, I vote for the v2 of this patchset.

Re: [PATCH v3 3/6] mk: sort libraries when linking, move pmd libs to higher level

[Ferruh Yigit]

On 6/13/2016 10:29 AM, Thomas Monjalon wrote:
> 2016-06-10 19:32, Ferruh Yigit:
>> As stated in the comment:
>>     Order is important: from higher level to lower level
>>
>> This is an attempt to make the layering order better respected.
>>
>> Limit scope of --whole-archive to pmd libraries
> 
> Compared to the link order in the v2, you did two things:
> 1/ move PMDs to the first position
> 2/ restrict --whole-archive to PMDs only
> 
> Having the PMDs first, helps the linker to get all the PMD dependencies
> in the static binary. Thus it seems we do not need --whole-archive
> for the PMD dependencies (ethdev, mbuf, etc).
> But, if an external PMD is loaded via dlopen, it is possible that it
> needs a symbol which was not used by the internal PMDs. So it will not
> be found in the statically linked binary.
> Let's take another example: if we disable the internal PMDs with their
> config options from the static build, and we decide to build them
> separately as DSOs. We won't be able to load them as plugins because
> they depend on symbols which won't be found in the static binary.
So you want to keep all objects in final binary (used or unused) because
of the possibility that any plugin may use them.

But what is the list to include here, whole dpdk?, -since we may not
know what API will plugin call.

What I am confused is --whole-archive only used when dpdk compiled as
static, if dpdk compiled as shared, each PMD should have proper
dependencies [1], and if external PMD compiled properly there shouldn't
be a problem. So do we have a case that dpdk compiled statically into
final binary but we still want to load some plugins dynamically?

> 
> To make it short, the PMDs must be considered as plugins. Therefore, we
> must not rely on their availability to link the required symbols in
> a static binary.
> To make sure the plugin loading will be always well achieved, we must
> link the static PMDs at the last position.
I think this is not the issue of linking PMD's first or last, but
expanding --whole-archive to cover other libraries other than PMDs.

> 
> If you agree, I vote for the v2 of this patchset.
If this is breaking something and best way to fix is not in external PMD
but in here, please feel free to go with v2.


[1] Like all has mbuf.so dependency:
# ldd build/lib/librte_pmd_ring.so
        linux-vdso.so.1 (0x00007ffe3d78a000)
        librte_eal.so.2.1 => not found
        librte_ring.so.1.1 => not found
        librte_mbuf.so.2.1 => not found
        libethdev.so.4.1 => not found
        librte_kvargs.so.1.1 => not found
        libc.so.6 => /lib64/libc.so.6 (0x00007f8ab5ae1000)
        /lib64/ld-linux-x86-64.so.2 (0x0000556616488000)

# ldd build/lib/librte_pmd_af_packet.so
        linux-vdso.so.1 (0x00007ffcb0572000)
        librte_mbuf.so.2.1 => not found
        libethdev.so.4.1 => not found
        librte_kvargs.so.1.1 => not found
        libc.so.6 => /lib64/libc.so.6 (0x00007f7fbd4aa000)
        /lib64/ld-linux-x86-64.so.2 (0x00005604556d2000)

# ldd build/lib/librte_pmd_null.so
        linux-vdso.so.1 (0x00007ffdf08cf000)
        librte_mbuf.so.2.1 => not found
        libethdev.so.4.1 => not found
        librte_kvargs.so.1.1 => not found
        libc.so.6 => /lib64/libc.so.6 (0x00007fca6fbfc000)
        /lib64/ld-linux-x86-64.so.2 (0x000055fe7dd4a000)


Thanks,
ferruh

Re: [PATCH v3 3/6] mk: sort libraries when linking, move pmd libs to higher level

[Thomas Monjalon]

2016-06-13 11:04, Ferruh Yigit:
> On 6/13/2016 10:29 AM, Thomas Monjalon wrote:
> > 2016-06-10 19:32, Ferruh Yigit:
> >> As stated in the comment:
> >>     Order is important: from higher level to lower level
> >>
> >> This is an attempt to make the layering order better respected.
> >>
> >> Limit scope of --whole-archive to pmd libraries
> > 
> > Compared to the link order in the v2, you did two things:
> > 1/ move PMDs to the first position
> > 2/ restrict --whole-archive to PMDs only
> > 
> > Having the PMDs first, helps the linker to get all the PMD dependencies
> > in the static binary. Thus it seems we do not need --whole-archive
> > for the PMD dependencies (ethdev, mbuf, etc).
> > But, if an external PMD is loaded via dlopen, it is possible that it
> > needs a symbol which was not used by the internal PMDs. So it will not
> > be found in the statically linked binary.
> > Let's take another example: if we disable the internal PMDs with their
> > config options from the static build, and we decide to build them
> > separately as DSOs. We won't be able to load them as plugins because
> > they depend on symbols which won't be found in the static binary.
> 
> So you want to keep all objects in final binary (used or unused) because
> of the possibility that any plugin may use them.

Yes

> But what is the list to include here, whole dpdk?, -since we may not
> know what API will plugin call.

The list of the libraries candidates to be called from a driver can be
discussed. The top layers like lpm or distributor should not be in this
list I think.

> What I am confused is --whole-archive only used when dpdk compiled as
> static, if dpdk compiled as shared, each PMD should have proper
> dependencies [1], and if external PMD compiled properly there shouldn't
> be a problem. So do we have a case that dpdk compiled statically into
> final binary but we still want to load some plugins dynamically?

Yes a plugin can be loaded from a static binary.
Breaking this feature would need a separate discussion and notices.

> > To make it short, the PMDs must be considered as plugins. Therefore, we
> > must not rely on their availability to link the required symbols in
> > a static binary.
> > To make sure the plugin loading will be always well achieved, we must
> > link the static PMDs at the last position.
> 
> I think this is not the issue of linking PMD's first or last, but
> expanding --whole-archive to cover other libraries other than PMDs.

Yes, linking PMD at the end is a way to force us to keep some libraries
in --whole-archive.

> > If you agree, I vote for the v2 of this patchset.
> 
> If this is breaking something and best way to fix is not in external PMD
> but in here, please feel free to go with v2.

I don't see any other solution.
But I'm sure we could discuss it more and/or improve it in the future.

Re: [PATCH v3 3/6] mk: sort libraries when linking, move pmd libs to higher level

[Ferruh Yigit]

On 6/13/2016 11:21 AM, Thomas Monjalon wrote:
> 2016-06-13 11:04, Ferruh Yigit:
>> On 6/13/2016 10:29 AM, Thomas Monjalon wrote:
>>> 2016-06-10 19:32, Ferruh Yigit:
>>>> As stated in the comment:
>>>>     Order is important: from higher level to lower level
>>>>
>>>> This is an attempt to make the layering order better respected.
>>>>
>>>> Limit scope of --whole-archive to pmd libraries
>>>
>>> Compared to the link order in the v2, you did two things:
>>> 1/ move PMDs to the first position
>>> 2/ restrict --whole-archive to PMDs only
>>>
>>> Having the PMDs first, helps the linker to get all the PMD dependencies
>>> in the static binary. Thus it seems we do not need --whole-archive
>>> for the PMD dependencies (ethdev, mbuf, etc).
>>> But, if an external PMD is loaded via dlopen, it is possible that it
>>> needs a symbol which was not used by the internal PMDs. So it will not
>>> be found in the statically linked binary.
>>> Let's take another example: if we disable the internal PMDs with their
>>> config options from the static build, and we decide to build them
>>> separately as DSOs. We won't be able to load them as plugins because
>>> they depend on symbols which won't be found in the static binary.
>>
>> So you want to keep all objects in final binary (used or unused) because
>> of the possibility that any plugin may use them.
> 
> Yes
> 
>> But what is the list to include here, whole dpdk?, -since we may not
>> know what API will plugin call.
> 
> The list of the libraries candidates to be called from a driver can be
> discussed. The top layers like lpm or distributor should not be in this
> list I think.
> 
>> What I am confused is --whole-archive only used when dpdk compiled as
>> static, if dpdk compiled as shared, each PMD should have proper
>> dependencies [1], and if external PMD compiled properly there shouldn't
>> be a problem. So do we have a case that dpdk compiled statically into
>> final binary but we still want to load some plugins dynamically?
> 
> Yes a plugin can be loaded from a static binary.
> Breaking this feature would need a separate discussion and notices.
Thanks for clarifying, I wasn't aware of this.
So I am leaving this patch, please just ping if anything required from me.

> 
>>> To make it short, the PMDs must be considered as plugins. Therefore, we
>>> must not rely on their availability to link the required symbols in
>>> a static binary.
>>> To make sure the plugin loading will be always well achieved, we must
>>> link the static PMDs at the last position.
>>
>> I think this is not the issue of linking PMD's first or last, but
>> expanding --whole-archive to cover other libraries other than PMDs.
> 
> Yes, linking PMD at the end is a way to force us to keep some libraries
> in --whole-archive.
> 
>>> If you agree, I vote for the v2 of this patchset.
>>
>> If this is breaking something and best way to fix is not in external PMD
>> but in here, please feel free to go with v2.
> 
> I don't see any other solution.
> But I'm sure we could discuss it more and/or improve it in the future.
> 

[PATCH v3 4/6] mk: sort libraries when linking, move external libs to lower level

[Ferruh Yigit]

As stated in the comment:
    Order is important: from higher level to lower level

This is an attempt to make the layering order better respected.

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
 mk/rte.app.mk | 30 +++++++++++++++---------------
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index b8761ae..a518527 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -125,21 +125,6 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_METER)          += -lrte_meter
 _LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED)          += -lrte_sched
 _LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST)          += -lrte_vhost
 
-# The static libraries do not know their dependencies.
-# So linking with static library requires explicit dependencies.
-ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
-_LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED)          += -lm
-_LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED)          += -lrt
-_LDLIBS-$(CONFIG_RTE_LIBRTE_METER)          += -lm
-ifeq ($(CONFIG_RTE_LIBRTE_VHOST_NUMA),y)
-_LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST)          += -lnuma
-endif
-ifeq ($(CONFIG_RTE_LIBRTE_VHOST_USER),n)
-_LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST)          += -lfuse
-endif
-_LDLIBS-$(CONFIG_RTE_PORT_PCAP)             += -lpcap
-endif # !CONFIG_RTE_BUILD_SHARED_LIBS
-
 _LDLIBS-y += --start-group
 
 _LDLIBS-$(CONFIG_RTE_LIBRTE_KVARGS)         += -lrte_kvargs
@@ -156,6 +141,21 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_CFGFILE)        += -lrte_cfgfile
 _LDLIBS-y += $(EXECENV_LDLIBS)
 _LDLIBS-y += --end-group
 
+# The static libraries do not know their dependencies.
+# So linking with static library requires explicit dependencies.
+ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
+_LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED)          += -lm
+_LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED)          += -lrt
+_LDLIBS-$(CONFIG_RTE_LIBRTE_METER)          += -lm
+ifeq ($(CONFIG_RTE_LIBRTE_VHOST_NUMA),y)
+_LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST)          += -lnuma
+endif
+ifeq ($(CONFIG_RTE_LIBRTE_VHOST_USER),n)
+_LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST)          += -lfuse
+endif
+_LDLIBS-$(CONFIG_RTE_PORT_PCAP)             += -lpcap
+endif # !CONFIG_RTE_BUILD_SHARED_LIBS
+
 LDLIBS += $(_LDLIBS-y) $(CPU_LDLIBS) $(EXTRA_LDLIBS)
 
 # Eliminate duplicates without sorting
-- 
2.5.5

[PATCH v3 5/6] mk: remove library grouping during application linking

[Ferruh Yigit]

From: Thomas Monjalon <thomas.monjalon@6wind.com>

>From "man ld":
  Using this option has a significant performance cost.
  It is best to use it only when there are unavoidable
  circular references between two or more archives.

Remove the option since it does not seem necessary.

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
 mk/rte.app.mk | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index a518527..f60e5bf 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -125,8 +125,6 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_METER)          += -lrte_meter
 _LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED)          += -lrte_sched
 _LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST)          += -lrte_vhost
 
-_LDLIBS-y += --start-group
-
 _LDLIBS-$(CONFIG_RTE_LIBRTE_KVARGS)         += -lrte_kvargs
 _LDLIBS-$(CONFIG_RTE_LIBRTE_MBUF)           += -lrte_mbuf
 _LDLIBS-$(CONFIG_RTE_LIBRTE_IP_FRAG)        += -lrte_ip_frag
@@ -139,7 +137,6 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE)        += -lrte_cmdline
 _LDLIBS-$(CONFIG_RTE_LIBRTE_CFGFILE)        += -lrte_cfgfile
 
 _LDLIBS-y += $(EXECENV_LDLIBS)
-_LDLIBS-y += --end-group
 
 # The static libraries do not know their dependencies.
 # So linking with static library requires explicit dependencies.
-- 
2.5.5

[PATCH v3 6/6] mk: prevent overlinking in applications

[Ferruh Yigit]

Replace --no-as-needed linker flag with --as-needed flag, which will
only link libraries directly called by application.
It can be achieved now that the libraries dependencies are handled
properly.

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
---
 mk/exec-env/linuxapp/rte.vars.mk | 3 ---
 mk/rte.app.mk                    | 8 ++++++++
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/mk/exec-env/linuxapp/rte.vars.mk b/mk/exec-env/linuxapp/rte.vars.mk
index d51bd17..a8a1ee4 100644
--- a/mk/exec-env/linuxapp/rte.vars.mk
+++ b/mk/exec-env/linuxapp/rte.vars.mk
@@ -45,9 +45,6 @@ else
 EXECENV_CFLAGS  = -pthread
 endif
 
-# Workaround lack of DT_NEEDED entry
-EXECENV_LDFLAGS = --no-as-needed
-
 EXECENV_LDLIBS  =
 EXECENV_ASFLAGS =
 
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index f60e5bf..aa794bc 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -50,6 +50,14 @@ ifeq ($(NO_LDSCRIPT),)
 LDSCRIPT = $(RTE_LDSCRIPT)
 endif
 
+ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y)
+# Workaround to eal <-> mempool cyclic dependency
+_LDLIBS-$(CONFIG_RTE_LIBRTE_MEMPOOL)        += -lrte_mempool
+endif
+
+# Link only the libraries used in the application
+_LDLIBS-y += --as-needed
+
 # default path for libs
 _LDLIBS-y += -L$(RTE_SDK_BIN)/lib
 
-- 
2.5.5

Re: [PATCH v3 6/6] mk: prevent overlinking in applications

[Thomas Monjalon]

Hi Ferruh,

2016-06-10 19:32, Ferruh Yigit:
> --- a/mk/rte.app.mk
> +++ b/mk/rte.app.mk
> @@ -50,6 +50,14 @@ ifeq ($(NO_LDSCRIPT),)
>  LDSCRIPT = $(RTE_LDSCRIPT)
>  endif
>  
> +ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y)
> +# Workaround to eal <-> mempool cyclic dependency
> +_LDLIBS-$(CONFIG_RTE_LIBRTE_MEMPOOL)        += -lrte_mempool
> +endif
> +
> +# Link only the libraries used in the application
> +_LDLIBS-y += --as-needed

I think we do not need this workaround.
The dependency of EAL logs on mempool is now removed.
We only have the dependency of EAL ivshmem on librte_ivshmem header
(which needs mempool and rings headers).

Re: [PATCH v3 6/6] mk: prevent overlinking in applications

[Ferruh Yigit]

On 6/11/2016 7:34 AM, Thomas Monjalon wrote:
> Hi Ferruh,
> 
> 2016-06-10 19:32, Ferruh Yigit:
>> --- a/mk/rte.app.mk
>> +++ b/mk/rte.app.mk
>> @@ -50,6 +50,14 @@ ifeq ($(NO_LDSCRIPT),)
>>  LDSCRIPT = $(RTE_LDSCRIPT)
>>  endif
>>  
>> +ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y)
>> +# Workaround to eal <-> mempool cyclic dependency
>> +_LDLIBS-$(CONFIG_RTE_LIBRTE_MEMPOOL)        += -lrte_mempool
>> +endif
>> +
>> +# Link only the libraries used in the application
>> +_LDLIBS-y += --as-needed
> 
> I think we do not need this workaround.
> The dependency of EAL logs on mempool is now removed.
> We only have the dependency of EAL ivshmem on librte_ivshmem header
> (which needs mempool and rings headers).
> 

I missed that dependency removed, this is good, I will send a new
version of the patch.

Thanks,
ferruh

Re: [PATCH v3 6/6] mk: prevent overlinking in applications

[Christian Ehrhardt]

Hi,
great work on cleaning that up already.
I applied v3 of the series and see already much better results.

But I found some more that should be part of this.
I don't know how generally available a similar solution is, but usually in
our build dpkg-shlibdeps checks for under/overlinking issues.
I let that run on a dir built from git plus v3 of the series in discussion
here.

There are two way of open dependency issues left:
1. some libraries use symbols of other libraries they are not linked with
correctly yet
2. the appliactions are still overlinking

We need to finalize #1 first before being able to tackle #2 without
breaking something.

I'll send a patch for the following so it can be part of your v4

$ for binfile in $(find debian/build/shared-root/lib -executable -type f;
find debian/build/shared-root/app -executable -type f;); do printf
"\n\n\nTesting %s\n" $binfile; dpkg-shlibdeps -e $binfile
-ldebian/build/shared-root/lib/ | grep -v "installed in their package's
directory"; done

Testing debian/build/shared-root/lib/librte_port.so.3
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory
dpkg-shlibdeps: warning: symbol rte_sched_port_dequeue used by
debian/build/shared-root/lib/librte_port.so.3 found in none of the libraries
dpkg-shlibdeps: warning: symbol rte_sched_port_enqueue used by
debian/build/shared-root/lib/librte_port.so.3 found in none of the libraries
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory

Testing debian/build/shared-root/lib/librte_pmd_bond.so.1
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory
dpkg-shlibdeps: warning: symbol rte_mempool_create used by
debian/build/shared-root/lib/librte_pmd_bond.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_ring_create used by
debian/build/shared-root/lib/librte_pmd_bond.so.1 found in none of the
libraries

Testing debian/build/shared-root/lib/librte_pipeline.so.3
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory
dpkg-shlibdeps: warning: symbol per_lcore__lcore_id used by
debian/build/shared-root/lib/librte_pipeline.so.3 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_eal_has_hugepages used by
debian/build/shared-root/lib/librte_pipeline.so.3 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_mem_virt2phy used by
debian/build/shared-root/lib/librte_pipeline.so.3 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_free used by
debian/build/shared-root/lib/librte_pipeline.so.3 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_log used by
debian/build/shared-root/lib/librte_pipeline.so.3 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_zmalloc_socket used by
debian/build/shared-root/lib/librte_pipeline.so.3 found in none of the
libraries

Testing debian/build/shared-root/lib/librte_pmd_null_crypto.so.1
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory
dpkg-shlibdeps: warning: symbol rte_ring_create used by
debian/build/shared-root/lib/librte_pmd_null_crypto.so.1 found in none of
the libraries
dpkg-shlibdeps: warning: symbol rte_kvargs_parse used by
debian/build/shared-root/lib/librte_pmd_null_crypto.so.1 found in none of
the libraries
dpkg-shlibdeps: warning: symbol rte_kvargs_free used by
debian/build/shared-root/lib/librte_pmd_null_crypto.so.1 found in none of
the libraries
dpkg-shlibdeps: warning: symbol rte_kvargs_process used by
debian/build/shared-root/lib/librte_pmd_null_crypto.so.1 found in none of
the libraries
dpkg-shlibdeps: warning: symbol rte_ring_lookup used by
debian/build/shared-root/lib/librte_pmd_null_crypto.so.1 found in none of
the libraries
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory

Testing debian/build/shared-root/lib/librte_pmd_vhost.so.1
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory
dpkg-shlibdeps: warning: symbol rte_eal_driver_register used by
debian/build/shared-root/lib/librte_pmd_vhost.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_log used by
debian/build/shared-root/lib/librte_pmd_vhost.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol pthread_join used by
debian/build/shared-root/lib/librte_pmd_vhost.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol pthread_cancel used by
debian/build/shared-root/lib/librte_pmd_vhost.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_eal_has_hugepages used by
debian/build/shared-root/lib/librte_pmd_vhost.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_mem_virt2phy used by
debian/build/shared-root/lib/librte_pmd_vhost.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_zmalloc_socket used by
debian/build/shared-root/lib/librte_pmd_vhost.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol pthread_create used by
debian/build/shared-root/lib/librte_pmd_vhost.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol per_lcore__lcore_id used by
debian/build/shared-root/lib/librte_pmd_vhost.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_socket_id used by
debian/build/shared-root/lib/librte_pmd_vhost.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: 2 other similar warnings have been skipped (use -v
to see them all)
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory

Testing debian/build/shared-root/lib/librte_pmd_fm10k.so.1
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory
dpkg-shlibdeps: warning: symbol rte_kvargs_count used by
debian/build/shared-root/lib/librte_pmd_fm10k.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_kvargs_free used by
debian/build/shared-root/lib/librte_pmd_fm10k.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_kvargs_parse used by
debian/build/shared-root/lib/librte_pmd_fm10k.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_kvargs_process used by
debian/build/shared-root/lib/librte_pmd_fm10k.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory

Testing debian/build/shared-root/lib/librte_pmd_pcap.so.1
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory
dpkg-shlibdeps: warning: symbol rte_eal_driver_register used by
debian/build/shared-root/lib/librte_pmd_pcap.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_cycles_vmware_tsc_map used by
debian/build/shared-root/lib/librte_pmd_pcap.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_zmalloc_socket used by
debian/build/shared-root/lib/librte_pmd_pcap.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_get_tsc_hz used by
debian/build/shared-root/lib/librte_pmd_pcap.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_free used by
debian/build/shared-root/lib/librte_pmd_pcap.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_eal_has_hugepages used by
debian/build/shared-root/lib/librte_pmd_pcap.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_socket_id used by
debian/build/shared-root/lib/librte_pmd_pcap.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol eal_timer_source used by
debian/build/shared-root/lib/librte_pmd_pcap.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_mem_virt2phy used by
debian/build/shared-root/lib/librte_pmd_pcap.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_log used by
debian/build/shared-root/lib/librte_pmd_pcap.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: 2 other similar warnings have been skipped (use -v
to see them all)
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory

Testing debian/build/shared-root/lib/librte_sched.so.1
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory
dpkg-shlibdeps: warning: symbol rte_zmalloc used by
debian/build/shared-root/lib/librte_sched.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol __rte_panic used by
debian/build/shared-root/lib/librte_sched.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol per_lcore__lcore_id used by
debian/build/shared-root/lib/librte_sched.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_free used by
debian/build/shared-root/lib/librte_sched.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_eal_has_hugepages used by
debian/build/shared-root/lib/librte_sched.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_get_tsc_hz used by
debian/build/shared-root/lib/librte_sched.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_mem_virt2phy used by
debian/build/shared-root/lib/librte_sched.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_log used by
debian/build/shared-root/lib/librte_sched.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_cycles_vmware_tsc_map used by
debian/build/shared-root/lib/librte_sched.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory

Testing debian/build/shared-root/lib/librte_pmd_af_packet.so.1
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory
dpkg-shlibdeps: warning: symbol rte_eal_has_hugepages used by
debian/build/shared-root/lib/librte_pmd_af_packet.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_zmalloc_socket used by
debian/build/shared-root/lib/librte_pmd_af_packet.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_free used by
debian/build/shared-root/lib/librte_pmd_af_packet.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_socket_id used by
debian/build/shared-root/lib/librte_pmd_af_packet.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_mem_virt2phy used by
debian/build/shared-root/lib/librte_pmd_af_packet.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_eal_driver_register used by
debian/build/shared-root/lib/librte_pmd_af_packet.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol per_lcore__lcore_id used by
debian/build/shared-root/lib/librte_pmd_af_packet.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_log used by
debian/build/shared-root/lib/librte_pmd_af_packet.so.1 found in none of the
libraries

Testing debian/build/shared-root/lib/librte_port.so.2
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory
dpkg-shlibdeps: warning: symbol rte_sched_port_enqueue used by
debian/build/shared-root/lib/librte_port.so.2 found in none of the libraries
dpkg-shlibdeps: warning: symbol rte_sched_port_dequeue used by
debian/build/shared-root/lib/librte_port.so.2 found in none of the libraries
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory

Testing debian/build/shared-root/lib/librte_pmd_null.so.1
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory
dpkg-shlibdeps: warning: symbol rte_zmalloc_socket used by
debian/build/shared-root/lib/librte_pmd_null.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_mem_virt2phy used by
debian/build/shared-root/lib/librte_pmd_null.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_socket_id used by
debian/build/shared-root/lib/librte_pmd_null.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_cpu_get_flag_enabled used by
debian/build/shared-root/lib/librte_pmd_null.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_log used by
debian/build/shared-root/lib/librte_pmd_null.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_eal_has_hugepages used by
debian/build/shared-root/lib/librte_pmd_null.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_free used by
debian/build/shared-root/lib/librte_pmd_null.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol per_lcore__lcore_id used by
debian/build/shared-root/lib/librte_pmd_null.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_eal_driver_register used by
debian/build/shared-root/lib/librte_pmd_null.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory

Testing debian/build/shared-root/lib/librte_ip_frag.so.1
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory
dpkg-shlibdeps: warning: symbol rte_eal_has_hugepages used by
debian/build/shared-root/lib/librte_ip_frag.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_log used by
debian/build/shared-root/lib/librte_ip_frag.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol per_lcore__lcore_id used by
debian/build/shared-root/lib/librte_ip_frag.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_zmalloc_socket used by
debian/build/shared-root/lib/librte_ip_frag.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_cpu_get_flag_enabled used by
debian/build/shared-root/lib/librte_ip_frag.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: symbol rte_mem_virt2phy used by
debian/build/shared-root/lib/librte_ip_frag.so.1 found in none of the
libraries
dpkg-shlibdeps: warning: binaries to analyze should already be installed in
their package's directory


Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd

On Mon, Jun 13, 2016 at 11:05 AM, Ferruh Yigit <ferruh.yigit@intel.com>
wrote:

> On 6/11/2016 7:34 AM, Thomas Monjalon wrote:
> > Hi Ferruh,
> >
> > 2016-06-10 19:32, Ferruh Yigit:
> >> --- a/mk/rte.app.mk
> >> +++ b/mk/rte.app.mk
> >> @@ -50,6 +50,14 @@ ifeq ($(NO_LDSCRIPT),)
> >>  LDSCRIPT = $(RTE_LDSCRIPT)
> >>  endif
> >>
> >> +ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y)
> >> +# Workaround to eal <-> mempool cyclic dependency
> >> +_LDLIBS-$(CONFIG_RTE_LIBRTE_MEMPOOL)        += -lrte_mempool
> >> +endif
> >> +
> >> +# Link only the libraries used in the application
> >> +_LDLIBS-y += --as-needed
> >
> > I think we do not need this workaround.
> > The dependency of EAL logs on mempool is now removed.
> > We only have the dependency of EAL ivshmem on librte_ivshmem header
> > (which needs mempool and rings headers).
> >
>
> I missed that dependency removed, this is good, I will send a new
> version of the patch.
>
> Thanks,
> ferruh
>

[PATCH] mk: fix missing DEPDIRS to avoid libarary underlinking

[Christian Ehrhardt]

After several cleanups libraries are now linked against the libs the
refer as DEPDIR which is great. But that now made it visible that some DEPDIRS
where still missing.

This patch is adding the missing depdirs to avoid any library referring to
symbols they are not linked against (found with dpkg-shlibs).

Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
---
 drivers/crypto/null/Makefile   | 2 ++
 drivers/net/af_packet/Makefile | 1 +
 drivers/net/bonding/Makefile   | 2 ++
 drivers/net/fm10k/Makefile     | 1 +
 drivers/net/null/Makefile      | 1 +
 drivers/net/pcap/Makefile      | 1 +
 drivers/net/vhost/Makefile     | 3 +++
 lib/librte_ip_frag/Makefile    | 1 +
 lib/librte_pipeline/Makefile   | 1 +
 lib/librte_port/Makefile       | 1 +
 lib/librte_sched/Makefile      | 1 +
 11 files changed, 15 insertions(+)

diff --git a/drivers/crypto/null/Makefile b/drivers/crypto/null/Makefile
index 2173277..573894f 100644
--- a/drivers/crypto/null/Makefile
+++ b/drivers/crypto/null/Makefile
@@ -55,5 +55,7 @@ SYMLINK-y-include +=
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO) += lib/librte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO) += lib/librte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO) += lib/librte_cryptodev
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO) += lib/librte_ring
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO) += lib/librte_kvargs
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/net/af_packet/Makefile b/drivers/net/af_packet/Makefile
index cb1a7ae..132f1d9 100644
--- a/drivers/net/af_packet/Makefile
+++ b/drivers/net/af_packet/Makefile
@@ -54,5 +54,6 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += rte_eth_af_packet.c
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += lib/librte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += lib/librte_ether
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += lib/librte_kvargs
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += lib/librte_eal
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/net/bonding/Makefile b/drivers/net/bonding/Makefile
index 10c794c..504f2e8 100644
--- a/drivers/net/bonding/Makefile
+++ b/drivers/net/bonding/Makefile
@@ -64,5 +64,7 @@ DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += lib/librte_ether
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += lib/librte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += lib/librte_kvargs
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += lib/librte_cmdline
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += lib/librte_mempool
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += lib/librte_ring
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index cf141ae..afcbd1d 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -100,5 +100,6 @@ SRCS-$(CONFIG_RTE_LIBRTE_FM10K_INC_VECTOR) += fm10k_rxtx_vec.c
 DEPDIRS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += lib/librte_eal lib/librte_ether
 DEPDIRS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += lib/librte_mempool lib/librte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += lib/librte_net
+DEPDIRS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += lib/librte_kvargs
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/net/null/Makefile b/drivers/net/null/Makefile
index 2202389..3502095 100644
--- a/drivers/net/null/Makefile
+++ b/drivers/net/null/Makefile
@@ -57,5 +57,6 @@ SYMLINK-y-include += rte_eth_null.h
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += lib/librte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += lib/librte_ether
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += lib/librte_kvargs
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += lib/librte_eal
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/net/pcap/Makefile b/drivers/net/pcap/Makefile
index b41d8a2..85179ad 100644
--- a/drivers/net/pcap/Makefile
+++ b/drivers/net/pcap/Makefile
@@ -59,5 +59,6 @@ SYMLINK-y-include +=
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_PCAP) += lib/librte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_PCAP) += lib/librte_ether
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_PCAP) += lib/librte_kvargs
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_PCAP) += lib/librte_eal
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/net/vhost/Makefile b/drivers/net/vhost/Makefile
index f49a69b..dd318a8 100644
--- a/drivers/net/vhost/Makefile
+++ b/drivers/net/vhost/Makefile
@@ -36,6 +36,8 @@ include $(RTE_SDK)/mk/rte.vars.mk
 #
 LIB = librte_pmd_vhost.a
 
+LDLIBS += -lpthread
+
 CFLAGS += -O3
 CFLAGS += $(WERROR_FLAGS)
 
@@ -58,5 +60,6 @@ DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += lib/librte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += lib/librte_ether
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += lib/librte_kvargs
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += lib/librte_vhost
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += lib/librte_eal
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_ip_frag/Makefile b/lib/librte_ip_frag/Makefile
index 9d06780..2b91f73 100644
--- a/lib/librte_ip_frag/Makefile
+++ b/lib/librte_ip_frag/Makefile
@@ -55,5 +55,6 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_IP_FRAG)-include += rte_ip_frag.h
 
 # this library depends on rte_ether
 DEPDIRS-$(CONFIG_RTE_LIBRTE_IP_FRAG) += lib/librte_mempool lib/librte_ether
+DEPDIRS-$(CONFIG_RTE_LIBRTE_IP_FRAG) += lib/librte_eal
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_pipeline/Makefile b/lib/librte_pipeline/Makefile
index 822fd41..a3b00f4 100644
--- a/lib/librte_pipeline/Makefile
+++ b/lib/librte_pipeline/Makefile
@@ -54,5 +54,6 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_pipeline.h
 # this lib depends upon:
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) := lib/librte_table
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += lib/librte_port
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += lib/librte_eal
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_port/Makefile b/lib/librte_port/Makefile
index d4de5af..dc6a601 100644
--- a/lib/librte_port/Makefile
+++ b/lib/librte_port/Makefile
@@ -75,5 +75,6 @@ DEPDIRS-$(CONFIG_RTE_LIBRTE_PORT) += lib/librte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PORT) += lib/librte_mempool
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PORT) += lib/librte_ether
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PORT) += lib/librte_ip_frag
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PORT) += lib/librte_sched
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_sched/Makefile b/lib/librte_sched/Makefile
index a782cd1..0766fb9 100644
--- a/lib/librte_sched/Makefile
+++ b/lib/librte_sched/Makefile
@@ -61,5 +61,6 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_SCHED)-include += rte_reciprocal.h
 # this lib depends upon:
 DEPDIRS-$(CONFIG_RTE_LIBRTE_SCHED) += lib/librte_mempool lib/librte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_SCHED) += lib/librte_net lib/librte_timer
+DEPDIRS-$(CONFIG_RTE_LIBRTE_SCHED) += lib/librte_eal
 
 include $(RTE_SDK)/mk/rte.lib.mk
-- 
2.7.4

Re: [PATCH] mk: fix missing DEPDIRS to avoid libarary underlinking

[Thomas Monjalon]

2016-06-13 11:48, Christian Ehrhardt:
> --- a/drivers/net/af_packet/Makefile
> +++ b/drivers/net/af_packet/Makefile
> @@ -54,5 +54,6 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += rte_eth_af_packet.c
>  DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += lib/librte_mbuf
>  DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += lib/librte_ether
>  DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += lib/librte_kvargs
> +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += lib/librte_eal

It looks more logical to have eal as first DEPDIR in these lists.

> --- a/drivers/net/vhost/Makefile
> +++ b/drivers/net/vhost/Makefile
> @@ -36,6 +36,8 @@ include $(RTE_SDK)/mk/rte.vars.mk
>  #
>  LIB = librte_pmd_vhost.a
>  
> +LDLIBS += -lpthread
> +

It is not a DEPDIR but an external dependency.
It deserves a separate patch.
Do we need it in rte.app.mk? It is an EAL dependency as well.

Re: [PATCH] mk: fix missing DEPDIRS to avoid libarary underlinking

[Christian Ehrhardt]

On Mon, Jun 13, 2016 at 12:08 PM, Thomas Monjalon <thomas.monjalon@6wind.com
> wrote:

> 2016-06-13 11:48, Christian Ehrhardt:
> > --- a/drivers/net/af_packet/Makefile
> > +++ b/drivers/net/af_packet/Makefile
> > @@ -54,5 +54,6 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) +=
> rte_eth_af_packet.c
> >  DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += lib/librte_mbuf
> >  DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += lib/librte_ether
> >  DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += lib/librte_kvargs
> > +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += lib/librte_eal
>
> It looks more logical to have eal as first DEPDIR in these lists.


Yeah, thought about alphanumeric order, but eal first should be most
appropriate.


> > --- a/drivers/net/vhost/Makefile
> > +++ b/drivers/net/vhost/Makefile
> > @@ -36,6 +36,8 @@ include $(RTE_SDK)/mk/rte.vars.mk
> >  #
> >  LIB = librte_pmd_vhost.a
> >
> > +LDLIBS += -lpthread
> > +
>
> It is not a DEPDIR but an external dependency.

It deserves a separate patch.
>

Yes, sorry for just wrapping it in.
Will send a v2 with both changes later.


> Do we need it in rte.app.mk? It is an EAL dependency as well.
>

Not IMHO:
EAL depends on it so librte_eal.so depends on it as it should (just
reverified by ldd).
I don't see a reason that any app created should depend on pthread other
than if it actually uses libpthread.

[PATCH v2 0/2] mk: fix more library underlinking

[Christian Ehrhardt]

After several cleanups libraries are now linked against the libs they
refer as DEPDIR which is great.
But that now also made it visible that some references where still missing.

The following two patches try to fix that:
[PATCH v2 1/2] mk: fix missing vhost dependency to pthread to avoid
[PATCH v2 2/2] mk: fix missing DEPDIRS to avoid libarary underlinking

[PATCH v2 1/2] mk: fix missing vhost dependency to pthread to avoid libarary underlinking

[Christian Ehrhardt]

Add the missing external dependency to pthread to avoid referring to
symbols the library is not linked against.

Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
---
 drivers/net/vhost/Makefile | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/net/vhost/Makefile b/drivers/net/vhost/Makefile
index f49a69b..545d998 100644
--- a/drivers/net/vhost/Makefile
+++ b/drivers/net/vhost/Makefile
@@ -36,6 +36,8 @@ include $(RTE_SDK)/mk/rte.vars.mk
 #
 LIB = librte_pmd_vhost.a
 
+LDLIBS += -lpthread
+
 CFLAGS += -O3
 CFLAGS += $(WERROR_FLAGS)
 
-- 
2.7.4

[PATCH v2 2/2] mk: fix missing DEPDIRS to avoid libarary underlinking

[Christian Ehrhardt]

This patch adds missing DEPDIRS to avoid any library referring to
symbols they are not linked against.

Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
---
 drivers/crypto/null/Makefile   | 2 ++
 drivers/net/af_packet/Makefile | 1 +
 drivers/net/bonding/Makefile   | 2 ++
 drivers/net/fm10k/Makefile     | 1 +
 drivers/net/null/Makefile      | 1 +
 drivers/net/pcap/Makefile      | 1 +
 drivers/net/vhost/Makefile     | 1 +
 lib/librte_ip_frag/Makefile    | 1 +
 lib/librte_pipeline/Makefile   | 1 +
 lib/librte_port/Makefile       | 1 +
 lib/librte_sched/Makefile      | 1 +
 11 files changed, 13 insertions(+)

diff --git a/drivers/crypto/null/Makefile b/drivers/crypto/null/Makefile
index 2173277..573894f 100644
--- a/drivers/crypto/null/Makefile
+++ b/drivers/crypto/null/Makefile
@@ -55,5 +55,7 @@ SYMLINK-y-include +=
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO) += lib/librte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO) += lib/librte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO) += lib/librte_cryptodev
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO) += lib/librte_ring
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO) += lib/librte_kvargs
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/net/af_packet/Makefile b/drivers/net/af_packet/Makefile
index cb1a7ae..c6de1c6 100644
--- a/drivers/net/af_packet/Makefile
+++ b/drivers/net/af_packet/Makefile
@@ -51,6 +51,7 @@ CFLAGS += $(WERROR_FLAGS)
 SRCS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += rte_eth_af_packet.c
 
 # this lib depends upon:
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += lib/librte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += lib/librte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += lib/librte_ether
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += lib/librte_kvargs
diff --git a/drivers/net/bonding/Makefile b/drivers/net/bonding/Makefile
index 10c794c..504f2e8 100644
--- a/drivers/net/bonding/Makefile
+++ b/drivers/net/bonding/Makefile
@@ -64,5 +64,7 @@ DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += lib/librte_ether
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += lib/librte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += lib/librte_kvargs
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += lib/librte_cmdline
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += lib/librte_mempool
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += lib/librte_ring
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index cf141ae..afcbd1d 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -100,5 +100,6 @@ SRCS-$(CONFIG_RTE_LIBRTE_FM10K_INC_VECTOR) += fm10k_rxtx_vec.c
 DEPDIRS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += lib/librte_eal lib/librte_ether
 DEPDIRS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += lib/librte_mempool lib/librte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += lib/librte_net
+DEPDIRS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += lib/librte_kvargs
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/net/null/Makefile b/drivers/net/null/Makefile
index 2202389..2d115de 100644
--- a/drivers/net/null/Makefile
+++ b/drivers/net/null/Makefile
@@ -54,6 +54,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += rte_eth_null.c
 SYMLINK-y-include += rte_eth_null.h
 
 # this lib depends upon:
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += lib/librte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += lib/librte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += lib/librte_ether
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += lib/librte_kvargs
diff --git a/drivers/net/pcap/Makefile b/drivers/net/pcap/Makefile
index b41d8a2..b2071e8 100644
--- a/drivers/net/pcap/Makefile
+++ b/drivers/net/pcap/Makefile
@@ -56,6 +56,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_PCAP) += rte_eth_pcap.c
 SYMLINK-y-include +=
 
 # this lib depends upon:
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_PCAP) += lib/librte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_PCAP) += lib/librte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_PCAP) += lib/librte_ether
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_PCAP) += lib/librte_kvargs
diff --git a/drivers/net/vhost/Makefile b/drivers/net/vhost/Makefile
index 545d998..d3d3d05 100644
--- a/drivers/net/vhost/Makefile
+++ b/drivers/net/vhost/Makefile
@@ -56,6 +56,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += rte_eth_vhost.c
 SYMLINK-y-include += rte_eth_vhost.h
 
 # this lib depends upon:
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += lib/librte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += lib/librte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += lib/librte_ether
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += lib/librte_kvargs
diff --git a/lib/librte_ip_frag/Makefile b/lib/librte_ip_frag/Makefile
index 9d06780..a375222 100644
--- a/lib/librte_ip_frag/Makefile
+++ b/lib/librte_ip_frag/Makefile
@@ -54,6 +54,7 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_IP_FRAG)-include += rte_ip_frag.h
 
 
 # this library depends on rte_ether
+DEPDIRS-$(CONFIG_RTE_LIBRTE_IP_FRAG) += lib/librte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_IP_FRAG) += lib/librte_mempool lib/librte_ether
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_pipeline/Makefile b/lib/librte_pipeline/Makefile
index 822fd41..95387aa 100644
--- a/lib/librte_pipeline/Makefile
+++ b/lib/librte_pipeline/Makefile
@@ -52,6 +52,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) := rte_pipeline.c
 SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_pipeline.h
 
 # this lib depends upon:
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += lib/librte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) := lib/librte_table
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += lib/librte_port
 
diff --git a/lib/librte_port/Makefile b/lib/librte_port/Makefile
index d4de5af..dc6a601 100644
--- a/lib/librte_port/Makefile
+++ b/lib/librte_port/Makefile
@@ -75,5 +75,6 @@ DEPDIRS-$(CONFIG_RTE_LIBRTE_PORT) += lib/librte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PORT) += lib/librte_mempool
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PORT) += lib/librte_ether
 DEPDIRS-$(CONFIG_RTE_LIBRTE_PORT) += lib/librte_ip_frag
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PORT) += lib/librte_sched
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_sched/Makefile b/lib/librte_sched/Makefile
index a782cd1..44cb780 100644
--- a/lib/librte_sched/Makefile
+++ b/lib/librte_sched/Makefile
@@ -59,6 +59,7 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_SCHED)-include := rte_sched.h rte_bitmap.h rte_sched
 SYMLINK-$(CONFIG_RTE_LIBRTE_SCHED)-include += rte_reciprocal.h
 
 # this lib depends upon:
+DEPDIRS-$(CONFIG_RTE_LIBRTE_SCHED) += lib/librte_eal
 DEPDIRS-$(CONFIG_RTE_LIBRTE_SCHED) += lib/librte_mempool lib/librte_mbuf
 DEPDIRS-$(CONFIG_RTE_LIBRTE_SCHED) += lib/librte_net lib/librte_timer
 
-- 
2.7.4

Re: [PATCH v2 0/2] mk: fix more library underlinking

[Thomas Monjalon]

2016-06-13 13:06, Christian Ehrhardt:
> After several cleanups libraries are now linked against the libs they
> refer as DEPDIR which is great.
> But that now also made it visible that some references where still missing.
> 
> The following two patches try to fix that:
> [PATCH v2 1/2] mk: fix missing vhost dependency to pthread to avoid
> [PATCH v2 2/2] mk: fix missing DEPDIRS to avoid libarary underlinking

Applied, thanks

Re: [PATCH v2 0/6] reduce overlinking in applications

[Thomas Monjalon]

2016-06-10 15:19, Thomas Monjalon:
> This is a respin of the ideas of Christian and Ferruh to limit
> the static application size or dynamic links to shared libraries.
> 
> It also brings some clean-up in rte.app.mk.
> 
> Ferruh Yigit (2):
>   mk: prevent overlinking in applications
>   mk: reduce scope of whole-archive static linking
> 
> Thomas Monjalon (4):
>   mk: sort drivers in static application link list
>   mk: fix driver dependencies order for static application
>   mk: remove library grouping during application linking
>   mk: sort libraries in level order when linking

Applied

Re: [PATCH] mk: fix underlinking issues of most librte libraries

[Thomas Monjalon]

2016-05-24 12:56, Panu Matilainen:
> On 05/20/2016 08:08 PM, Thomas Monjalon wrote:
> > 2016-05-20 18:50, Christian Ehrhardt:
> >> The individual libraries have various cross dependencies.
> >> This is already refelcted in the DEPDIR dependency, but not yet in
> >> proper DT_NEEDED flags in the .so's.
> >> This adds the -l flags so that is properly stored in the .so's ELF
> >> headers.
> >
> > Why not filling LDLIBS by parsing DEPDIRS-y in rte.lib.mk?
> >
> 
> A fair question :) The thought has passed my mind too, but I thought 
> it'd be too messy with differing library vs directory names and other 
> exceptions. But in reality manually maintained separate LDLIBS is only 
> going to get out of sync sooner or later, and turns out its not that bad 
> anyway:
> 
> diff --git a/mk/rte.lib.mk b/mk/rte.lib.mk
> index b420280..88b4e98 100644
> --- a/mk/rte.lib.mk
> +++ b/mk/rte.lib.mk
> @@ -77,6 +77,12 @@ else
>   _CPU_LDFLAGS := $(CPU_LDFLAGS)
>   endif
> 
> +# Translate DEPDIRS-y into LDLIBS
> +IGNORE_DEPS = -lrte_eal/% -lrte_net -lrte_compat

We need some comments to explain why they are ignored.

> +_LDDIRS = $(subst librte_ether,libethdev,$(DEPDIRS-y))
> +_LDDEPS = $(subst lib/lib,-l,$(_LDDIRS))
> +LDLIBS += $(filter-out $(IGNORE_DEPS), $(_LDDEPS))

> I'm also sure somebody more familiar with gmake could improve it, but it 
> seems to work quite fine here. I'll wait a bit to see if there are other 
> suggestions before sending it as a proper patch.

I'm familiar with gmake and I do not think it can be improved more ;)
Please send a patch. Thanks

Re: [PATCH] mk: fix underlinking issues of most librte libraries

[Panu Matilainen]

On 06/07/2016 11:23 AM, Thomas Monjalon wrote:
> 2016-05-24 12:56, Panu Matilainen:
>> On 05/20/2016 08:08 PM, Thomas Monjalon wrote:
>>> 2016-05-20 18:50, Christian Ehrhardt:
>>>> The individual libraries have various cross dependencies.
>>>> This is already refelcted in the DEPDIR dependency, but not yet in
>>>> proper DT_NEEDED flags in the .so's.
>>>> This adds the -l flags so that is properly stored in the .so's ELF
>>>> headers.
>>>
>>> Why not filling LDLIBS by parsing DEPDIRS-y in rte.lib.mk?
>>>
>>
>> A fair question :) The thought has passed my mind too, but I thought
>> it'd be too messy with differing library vs directory names and other
>> exceptions. But in reality manually maintained separate LDLIBS is only
>> going to get out of sync sooner or later, and turns out its not that bad
>> anyway:
>>
>> diff --git a/mk/rte.lib.mk b/mk/rte.lib.mk
>> index b420280..88b4e98 100644
>> --- a/mk/rte.lib.mk
>> +++ b/mk/rte.lib.mk
>> @@ -77,6 +77,12 @@ else
>>   _CPU_LDFLAGS := $(CPU_LDFLAGS)
>>   endif
>>
>> +# Translate DEPDIRS-y into LDLIBS
>> +IGNORE_DEPS = -lrte_eal/% -lrte_net -lrte_compat
>
> We need some comments to explain why they are ignored.

Yup. It'd be more clear if this filtering was done before translating 
directories to -lrte_foo but this was the first variant I came across 
that worked :) The issue is that librte_eal/common, librte_net and 
librte_compat are common directory dependencies but no actual library is 
generated from them so they just need to be weeded out of the equation, 
the revised version makes this more clear.

>
>> +_LDDIRS = $(subst librte_ether,libethdev,$(DEPDIRS-y))
>> +_LDDEPS = $(subst lib/lib,-l,$(_LDDIRS))
>> +LDLIBS += $(filter-out $(IGNORE_DEPS), $(_LDDEPS))
>
>> I'm also sure somebody more familiar with gmake could improve it, but it
>> seems to work quite fine here. I'll wait a bit to see if there are other
>> suggestions before sending it as a proper patch.
>
> I'm familiar with gmake and I do not think it can be improved more ;)
> Please send a patch. Thanks
>

Ok, will do.

	- Panu -