Re: [PATCH] Fix linking errors when CONFIG_RTE_BUILD_SHARED_LIB is enabled

[Sergio Gonzalez Monroy <sergio.gonzalez.monroy-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>]

On Thu, Oct 02, 2014 at 06:05:09PM +0900, Tetsuya Mukawa wrote:
> (2014/10/02 17:53), Sergio Gonzalez Monroy wrote:
> > On Thu, Oct 02, 2014 at 05:28:04PM +0900, Tetsuya Mukawa wrote:
> >> (2014/10/02 17:12), Sergio Gonzalez Monroy wrote:
> >>> On Thu, Oct 02, 2014 at 11:48:37AM +0900, Tetsuya Mukawa wrote:
> >>>
> >>>> $ gcc -m64 -pthread -fPIC -march=native -DRTE_MACHINE_CPUFLAG_SSE
> >>>> -DRTE_MACHINE_CPUFLAG_SSE2 -DRTE_MACHINE_CPUFLAG_SSE3
> >>>> -DRTE_MACHINE_CPUFLAG_SSSE3 -DRTE_MACHINE_CPUFLAG_SSE4_1
> >>>> -DRTE_MACHINE_CPUFLAG_SSE4_2 -DRTE_MACHINE_CPUFLAG_AES
> >>>> -DRTE_MACHINE_CPUFLAG_PCLMULQDQ -DRTE_MACHINE_CPUFLAG_AVX
> >>>> -DRTE_MACHINE_CPUFLAG_F16C
> >>>> -DRTE_COMPILE_TIME_CPUFLAGS=RTE_CPUFLAG_SSE,RTE_CPUFLAG_SSE2,RTE_CPUFLAG_SSE3,RTE_CPUFLAG_SSSE3,RTE_CPUFLAG_SSE4_1,RTE_CPUFLAG_SSE4_2,RTE_CPUFLAG_AES,RTE_CPUFLAG_PCLMULQDQ,RTE_CPUFLAG_AVX,RTE_CPUFLAG_F16C
> >>>> -I/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/include -include
> >>>> /home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/include/rte_config.h
> >>>> -W -Wall -Werror -Wstrict-prototypes -Wmissing-prototypes
> >>>> -Wmissing-declarations -Wold-style-definition -Wpointer-arith
> >>>> -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral
> >>>> -Wformat-security -Wundef -Wwrite-strings -Wl,-Map=testacl.map,--cref
> >>>> -o testacl main.o -Wl,-export-dynamic
> >>>> -L/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib
> >>>> -L/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib
> >>>> -Wl,--whole-archive -Wl,-lrte_distributor -Wl,-lrte_kni
> >>>> -Wl,-lrte_pipeline -Wl,-lrte_table -Wl,-lrte_port -Wl,-lrte_timer
> >>>> -Wl,-lrte_hash -Wl,-lrte_lpm -Wl,-lrte_power -Wl,-lrte_acl
> >>>> -Wl,-lrte_meter -Wl,-lrte_sched -Wl,-lm -Wl,-lrt -Wl,--start-group
> >>>> -Wl,-lrte_kvargs -Wl,-lrte_mbuf -Wl,-lrte_ip_frag -Wl,-lethdev
> >>>> -Wl,-lrte_malloc -Wl,-lrte_mempool -Wl,-lrte_ring -Wl,-lrte_eal
> >>>> -Wl,-lrte_cmdline -Wl,-lrte_cfgfile -Wl,-lrte_pmd_bond -Wl,-lrt
> >>>> -Wl,-lm -Wl,-lgcc_s -Wl,-ldl -Wl,--end-group -Wl,--no-whole-archive
> >>>> /home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib/librte_eal.so:
> >>>> undefined reference to `rte_mempool_lookup'
> >>>> /home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib/librte_eal.so:
> >>>> undefined reference to `rte_mempool_create' collect2: error: ld
> >>>> returned 1 exit status 
> >>> Hi Tetsuya,
> >>>
> >>> Would you mind posting the output of the last command adding the option '-v'?
> >> Sure, here is.
> >>
> >> $ gcc -m64 -pthread -fPIC  -march=native -DRTE_MACHINE_CPUFLAG_SSE
> >> -DRTE_MACHINE_CPUFLAG_SSE2 -DRTE_MACHINE_CPUFLAG_SSE3
> >> -DRTE_MACHINE_CPUFLAG_SSSE3 -DRTE_MACHINE_CPUFLAG_SSE4_1
> >> -DRTE_MACHINE_CPUFLAG_SSE4_2 -DRTE_MACHINE_CPUFLAG_AES
> >> -DRTE_MACHINE_CPUFLAG_PCLMULQDQ -DRTE_MACHINE_CPUFLAG_AVX
> >> -DRTE_MACHINE_CPUFLAG_F16C
> >> -DRTE_COMPILE_TIME_CPUFLAGS=RTE_CPUFLAG_SSE,RTE_CPUFLAG_SSE2,RTE_CPUFLAG_SSE3,RTE_CPUFLAG_SSSE3,RTE_CPUFLAG_SSE4_1,RTE_CPUFLAG_SSE4_2,RTE_CPUFLAG_AES,RTE_CPUFLAG_PCLMULQDQ,RTE_CPUFLAG_AVX,RTE_CPUFLAG_F16C 
> >> -I/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/include -include
> >> /home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/include/rte_config.h -W
> >> -Wall -Werror -Wstrict-prototypes -Wmissing-prototypes
> >> -Wmissing-declarations -Wold-style-definition -Wpointer-arith
> >> -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral
> >> -Wformat-security -Wundef -Wwrite-strings  -Wl,-Map=testacl.map,--cref
> >> -o testacl main.o -Wl,-export-dynamic
> >> -L/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib 
> >> -L/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib
> >> -Wl,--whole-archive -Wl,-lrte_distributor -Wl,-lrte_kni
> >> -Wl,-lrte_pipeline -Wl,-lrte_table -Wl,-lrte_port -Wl,-lrte_timer
> >> -Wl,-lrte_hash -Wl,-lrte_lpm -Wl,-lrte_power -Wl,-lrte_acl
> >> -Wl,-lrte_meter -Wl,-lrte_sched -Wl,-lm -Wl,-lrt -Wl,--start-group
> >> -Wl,-lrte_kvargs -Wl,-lrte_mbuf -Wl,-lrte_ip_frag -Wl,-lethdev
> >> -Wl,-lrte_malloc -Wl,-lrte_mempool -Wl,-lrte_ring -Wl,-lrte_eal
> >> -Wl,-lrte_cmdline -Wl,-lrte_cfgfile -Wl,-lrte_pmd_bond -Wl,-lrt -Wl,-lm
> >> -Wl,-lgcc_s -Wl,-ldl -Wl,--end-group -Wl,--no-whole-archive -v
> >> Using built-in specs.
> >> COLLECT_GCC=gcc
> >> COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.8/lto-wrapper
> >> Target: x86_64-linux-gnu
> >> Configured with: ../src/configure -v --with-pkgversion='Ubuntu
> >> 4.8.2-19ubuntu1' --with-bugurl=file:///usr/share/doc/gcc-4.8/README.Bugs
> >> --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr
> >> --program-suffix=-4.8 --enable-shared --enable-linker-build-id
> >> --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix
> >> --with-gxx-include-dir=/usr/include/c++/4.8 --libdir=/usr/lib
> >> --enable-nls --with-sysroot=/ --enable-clocale=gnu
> >> --enable-libstdcxx-debug --enable-libstdcxx-time=yes
> >> --enable-gnu-unique-object --disable-libmudflap --enable-plugin
> >> --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk
> >> --enable-gtk-cairo
> >> --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64/jre
> >> --enable-java-home
> >> --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64
> >> --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.8-amd64
> >> --with-arch-directory=amd64
> >> --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc
> >> --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64
> >> --with-multilib-list=m32,m64,mx32 --with-tune=generic
> >> --enable-checking=release --build=x86_64-linux-gnu
> >> --host=x86_64-linux-gnu --target=x86_64-linux-gnu
> >> Thread model: posix
> >> gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1)
> >> COMPILER_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/
> >> LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib/:/lib/x86_64-linux-gnu/:/lib/../lib/:/usr/lib/x86_64-linux-gnu/:/usr/lib/../lib/:/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../:/lib/:/usr/lib/
> >> COLLECT_GCC_OPTIONS='-m64' '-pthread' '-fPIC' '-march=native' '-D'
> >> 'RTE_MACHINE_CPUFLAG_SSE' '-D' 'RTE_MACHINE_CPUFLAG_SSE2' '-D'
> >> 'RTE_MACHINE_CPUFLAG_SSE3' '-D' 'RTE_MACHINE_CPUFLAG_SSSE3' '-D'
> >> 'RTE_MACHINE_CPUFLAG_SSE4_1' '-D' 'RTE_MACHINE_CPUFLAG_SSE4_2' '-D'
> >> 'RTE_MACHINE_CPUFLAG_AES' '-D' 'RTE_MACHINE_CPUFLAG_PCLMULQDQ' '-D'
> >> 'RTE_MACHINE_CPUFLAG_AVX' '-D' 'RTE_MACHINE_CPUFLAG_F16C' '-D'
> >> 'RTE_COMPILE_TIME_CPUFLAGS=RTE_CPUFLAG_SSE,RTE_CPUFLAG_SSE2,RTE_CPUFLAG_SSE3,RTE_CPUFLAG_SSSE3,RTE_CPUFLAG_SSE4_1,RTE_CPUFLAG_SSE4_2,RTE_CPUFLAG_AES,RTE_CPUFLAG_PCLMULQDQ,RTE_CPUFLAG_AVX,RTE_CPUFLAG_F16C'
> >> '-I' '/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/include'
> >> '-include'
> >> '/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/include/rte_config.h'
> >> '-Wextra' '-Wall' '-Werror' '-Wstrict-prototypes' '-Wmissing-prototypes'
> >> '-Wmissing-declarations' '-Wold-style-definition' '-Wpointer-arith'
> >> '-Wcast-align' '-Wnested-externs' '-Wcast-qual' '-Wformat-nonliteral'
> >> '-Wformat-security' '-Wundef' '-Wwrite-strings' '-o' 'testacl'
> >> '-L/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib'
> >> '-L/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib' '-v'
> > We have been looking at this issue over the last couple of days.
> > It seems to be an Ubuntu GCC related bug as it works on other distros.
> > GCC is passing incorrectly the option '--as-needed' to LD resulting in wrong linking.
> >

Just a quick follow up on this issue and to rectify myself. 
This is not a bug but expected behavior:
https://wiki.debian.org/ToolChain/DSOLinking#Onlylinkwithneededlibraries

Sergio

> >>  /usr/lib/gcc/x86_64-linux-gnu/4.8/collect2 --sysroot=/ --build-id
> >> --eh-frame-hdr -m elf_x86_64 --hash-style=gnu --as-needed
> > The '--as-needed' just at the end of the above line is incorrect and not matching
> > a closing '--no-as-needed'.
> > You may get away with it by adding 'EXTRA_LDFLAGS=--no-as-needed'. 
> > ie. $ make install T=x86_64-native-linuxapp-gcc EXTRA_LDFLAGS=--no-as-needed
> 
> I can compile DPDK with above options.
> Thank you everyone for helping to solve this issue.
> 
> Regards,
> Tetsuya
> 
> >
> > Sergio
> >
> >> -dynamic-linker /lib64/ld-linux-x86-64.so.2 -z relro -o testacl
> >> /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crt1.o
> >> /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crti.o
> >> /usr/lib/gcc/x86_64-linux-gnu/4.8/crtbegin.o
> >> -L/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib
> >> -L/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib
> >> -L/usr/lib/gcc/x86_64-linux-gnu/4.8
> >> -L/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu
> >> -L/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib
> >> -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu
> >> -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/4.8/../../..
> >> -Map=testacl.map --cref main.o -export-dynamic --whole-archive
> >> -lrte_distributor -lrte_kni -lrte_pipeline -lrte_table -lrte_port
> >> -lrte_timer -lrte_hash -lrte_lpm -lrte_power -lrte_acl -lrte_meter
> >> -lrte_sched -lm -lrt --start-group -lrte_kvargs -lrte_mbuf -lrte_ip_frag
> >> -lethdev -lrte_malloc -lrte_mempool -lrte_ring -lrte_eal -lrte_cmdline
> >> -lrte_cfgfile -lrte_pmd_bond -lrt -lm -lgcc_s -ldl --end-group
> >> --no-whole-archive -lgcc --as-needed -lgcc_s --no-as-needed -lpthread
> >> -lc -lgcc --as-needed -lgcc_s --no-as-needed
> >> /usr/lib/gcc/x86_64-linux-gnu/4.8/crtend.o
> >> /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crtn.o
> >> /home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib/librte_eal.so:
> >> undefined reference to `rte_mempool_lookup'
> >> /home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib/librte_eal.so:
> >> undefined reference to `rte_mempool_create'
> >> collect2: error: ld returned 1 exit status
> >>
> >> Regards,
> >> Tetsuya
> >>
> >>> Sergio
> >>>
> >>>
> >>>> ----------------------------------------------------------------------
> >>>>
> >>>> I just enabled the following option, then compile DPDK.
> >>>> CONFIG_RTE_BUILD_SHARED_LIB
> >>>> Is this collect to compile PMDs as dynamic link libraries?
> >>>>
> >>>> With the option, all libraries include are compiled as dynamic link library.
> >>>> Does "--start-group/--end-group" options work with dynamic link libraries?
> >>>>
> >>>> Regards,
> >>>> Tetsuya
> >>>>
> 
>