Simon,

Thanks for this - so after I updated to gcc 9.2, the shared libraries command ran correctly and the build finished. All the tests pass and I was able to read in a CTF trace from a python script.  

The issue was that the system gcc was gcc 4.X.X, so too old.  So, I had to manually add the newer gcc to the path alongside the intel compiler.  I can try to option of setting LDSHARED, but at this point that seems even more confusing than just using the system gcc, which apparently has no trouble linking intel object files into a shared library.  Maybe there are good reasons why this is the case and this is why distutils does not use the right compiler?

Ideally the build would just "work" with ./configure, make, make install since these kinds of compilation issues can quickly get over people's heads and cause them to abandon the library altogether.  I wonder if there is a way for the configure/make to set the LDSHARED automatically so that is is transparent to the user.  Certainly we are not the first to run into this issue with distutils.

Rocky

On Fri, Mar 20, 2020 at 4:20 PM Simon Marchi <simark@simark.ca> wrote:
On 2020-03-20 5:55 p.m., Rocky Dunlap wrote:
> Simon,
>
> I was able to get past the issue by passing "--enable-compile-warnings=yes" to configure.
>
> It get a lot further, then fails here:
>
> gcc -pthread -shared -L/apps/intel/intelpython3/lib -Wl,-rpath=/apps/intel/intelpython3/lib,--no-as-needed -Wl,-z,noexecstack,-z,relro,-z,now -L../../../../src/lib/.libs -pthread -lgmodule-2.0 -lglib-2.0 -pthread -lgmodule-2.0 -lglib-2.0 -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -pthread -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -fno-strict-aliasing -Wnested-externs -Wmissing-prototypes -Wstrict-prototypes -Wdeclaration-after-statement -Wimplicit-function-declaration -Wold-style-definition -Wjump-misses-init -Wall -Wextra -Wundef -Wwrite-strings -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wno-unused-parameter -Wno-missing-field-initializers -Wformat=2 -Wcast-align -Wformat-nonliteral -Wformat-security -Wsign-compare -Wstrict-aliasing -Wshadow -Winline -Wpacked -Wmissing-format-attribute -Wmissing-noreturn -Winit-self -Wmissing-include-dirs -Wunused-but-set-variable -Warray-bounds -Wreturn-type -Wswitch-enum
> -Wswitch-default -Wduplicated-cond -Wduplicated-branches -Wlogical-op -Wrestrict -Wnull-dereference -Wdouble-promotion -Wno-sign-compare -Wno-inline -Wno-declaration-after-statement -Wno-switch-enum -Wno-switch-default -Wno-packed -Wno-pointer-arith -Wno-format-nonliteral -Wno-double-promotion -Wno-cast-align -Wno-cast-function-type -Wno-error=unused-parameter -Wno-error=missing-field-initializers -Wno-error=sign-compare -Wno-error=inline -Wno-error=declaration-after-statement -Wno-error=switch-enum -Wno-error=switch-default -Wno-error=packed -Wno-error=pointer-arith -Wno-error=format-nonliteral -Wno-error=double-promotion -Wno-error=cast-align -Wno-error=cast-function-type -Wold-style-definition -Werror=implicit-function-declaration -g -O2 -Wno-shadow -Wno-null-dereference -I../../../../include -I../../../../src -I../../../../src -include common/config.h -I./bt2 build/temp.linux-x86_64-3.6/bt2/native_bt.o build/temp.linux-x86_64-3.6/./bt2/logging.o
> ../../../../src/autodisc/.libs/libbabeltrace2-autodisc.a ../../../../src/logging/.libs/libbabeltrace2-logging.a ../../../../src/common/.libs/libbabeltrace2-common.a ../../../../src/py-common/.libs/libbabeltrace2-py-common.a ../../../../src/string-format/.libs/libbabeltrace2-string-format.a -L/apps/intel/intelpython3/lib -lbabeltrace2 -lglib-2.0 -lpython3.6m -o build/build_lib/bt2/_native_bt.cpython-36m-x86_64-linux-gnu.so <http://native_bt.cpython-36m-x86_64-linux-gnu.so>
> gcc: error: unrecognized command line option ‘-Wduplicated-cond’
> gcc: error: unrecognized command line option ‘-Wduplicated-branches’
> gcc: error: unrecognized command line option ‘-Wrestrict’
> gcc: error: unrecognized command line option ‘-Wnull-dereference’
> error: command 'gcc' failed with exit status 1
> make[4]: *** [build-python-bindings.stamp] Error 1
>
> I am surprised that this is using "gcc" here, shouldn't it be using "icc" since I have CC=icc?
>
> Or maybe it is supposed to be using gcc?  I think if I update the GCC compiler version then it will get past this issue as well, but I just wanted confirmation that this indeed should be using a combination of gcc and icc?
>
> If it should be icc, can you point me to where in the configure/make system I can make this change to force it to use what's sets as CC instead of defaulting to GCC?
>
> Rocky

I've hit this exact issue by trying to compile with CC=clang.  This is due to an
oddity of distutils when it builds the native module, I don't really know what to
do about it.

When it compiles native_bt.o, distutils uses the compiler specified in CC, which is
what we want.  But when it links the shared object, it:

  - uses the default system compiler (often gcc), and
  - passes the flags specified in CFLAGS

However, CFLAGS contains the warning flags that our configure script determined
$CC accepts (note, that detection might not work well with icc, we haven't tested).

So we end up passing warning flags recognized by $CC (e.g. clang) to gcc, which makes
gcc error out.

distutils allows to override the command used to link shared libraries by setting the
LDSHARED variable.  So you could try to set LDSHARED to "icc" plus the right flags
required to link a shared lib.  For reference, the default LDSHARED value for your Python
installation can be checked with:

    $ python3 -c 'import sysconfig; print(sysconfig.get_config_var("LDSHARED"))'
    x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro

But even worse, when building with CC=clang on Arch Linux, I noticed that distutils
passes some gcc-specific flags (which don't come from us) when compiling.  I'm not
sure what we're going to do about that.

Simon