[LTP] [PATCH 1/2 v2] Fix build with libnuma-dev on some distributions

[LTP] [PATCH 1/2 v2] Fix build with libnuma-dev on some distributions

[Cyril Hrubis]

On some distributions when libnuma-dev is installed the tst_numa.o,
which was added to libltp.a pulls in symbols from numa library to random
unrealted tests which breaks the build.

This commit hence moves the tst_numa to a separate libltpnuma.a library
that is used for linking only for tests that actually use it and link
with -lnuma.

It also adds a libs/ directory to the top level directory which is
intended to collect all possible library code that is used from
different tests but couldn't be included in the main library.

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
CC: Jan Stancek <jstancek@redhat.com>
CC: Petr Vorel <pvorel@suse.cz>
---
 Makefile                                         |  6 +++---
 libs/Makefile                                    |  8 ++++++++
 libs/libltpnuma/Makefile                         | 12 ++++++++++++
 {lib => libs/libltpnuma}/tst_numa.c              |  0
 testcases/kernel/syscalls/set_mempolicy/Makefile |  3 ++-
 5 files changed, 25 insertions(+), 4 deletions(-)
 create mode 100644 libs/Makefile
 create mode 100644 libs/libltpnuma/Makefile
 rename {lib => libs/libltpnuma}/tst_numa.c (100%)

diff --git a/Makefile b/Makefile
index bcadd21b2..bc6e9a3e0 100644
--- a/Makefile
+++ b/Makefile
@@ -77,7 +77,7 @@ INSTALL_TARGETS		+= runtest scenario_groups testscripts
 CLEAN_TARGETS		+= include runtest scenario_groups testscripts
 endif
 INSTALL_TARGETS		+= $(COMMON_TARGETS)
-CLEAN_TARGETS		+= $(COMMON_TARGETS) lib
+CLEAN_TARGETS		+= $(COMMON_TARGETS) lib libs
 BOOTSTRAP_TARGETS	:= $(sort $(COMMON_TARGETS) $(CLEAN_TARGETS) $(INSTALL_TARGETS))
 
 CLEAN_TARGETS		:= $(addsuffix -clean,$(CLEAN_TARGETS))
@@ -89,7 +89,7 @@ MAKE_TARGETS		:= $(addsuffix -all,$(filter-out lib,$(COMMON_TARGETS)))
 # overtaxed one, or one where -j => 1 was specified.
 all: $(addsuffix -all,$(COMMON_TARGETS)) Version
 
-$(MAKE_TARGETS): lib-all
+$(MAKE_TARGETS): lib-all libs-all
 
 .PHONY: include-all include-install
 include-install: $(top_builddir)/include/config.h include/mk/config.mk include-all
@@ -110,7 +110,7 @@ $(sort $(addprefix $(abs_top_builddir)/,$(BOOTSTRAP_TARGETS)) $(INSTALL_DIR) $(D
 ## Pattern based subtarget rules.
 lib-install: lib-all
 
-$(MAKE_TARGETS) include-all lib-all:
+$(MAKE_TARGETS) include-all lib-all libs-all:
 	$(MAKE) -C "$(subst -all,,$@)" \
 		-f "$(abs_top_srcdir)/$(subst -all,,$@)/Makefile" all
 
diff --git a/libs/Makefile b/libs/Makefile
new file mode 100644
index 000000000..16ed3aa6d
--- /dev/null
+++ b/libs/Makefile
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Copyright (C) Cyril Hrubis <chrubis@suse.cz>
+
+top_srcdir		?= ..
+
+include $(top_srcdir)/include/mk/env_pre.mk
+include $(top_srcdir)/include/mk/generic_trunk_target.mk
diff --git a/libs/libltpnuma/Makefile b/libs/libltpnuma/Makefile
new file mode 100644
index 000000000..56d6b7762
--- /dev/null
+++ b/libs/libltpnuma/Makefile
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Copyright (C) Cyril Hrubis <chrubis@suse.cz>
+
+top_srcdir		?= ../..
+
+include $(top_srcdir)/include/mk/env_pre.mk
+
+LIB			:= libltpnuma.a
+
+include $(top_srcdir)/include/mk/lib.mk
+include $(top_srcdir)/include/mk/generic_leaf_target.mk
diff --git a/lib/tst_numa.c b/libs/libltpnuma/tst_numa.c
similarity index 100%
rename from lib/tst_numa.c
rename to libs/libltpnuma/tst_numa.c
diff --git a/testcases/kernel/syscalls/set_mempolicy/Makefile b/testcases/kernel/syscalls/set_mempolicy/Makefile
index d273b432b..b79a53faf 100644
--- a/testcases/kernel/syscalls/set_mempolicy/Makefile
+++ b/testcases/kernel/syscalls/set_mempolicy/Makefile
@@ -2,6 +2,7 @@ top_srcdir		?= ../../../..
 
 include $(top_srcdir)/include/mk/testcases.mk
 
-LDLIBS  += $(NUMA_LIBS)
+LDFLAGS += -L$(top_builddir)/libs/libltpnuma
+LDLIBS  += $(NUMA_LIBS) -lltpnuma
 
 include $(top_srcdir)/include/mk/generic_leaf_target.mk
-- 
2.19.2

[LTP] [PATCH 2/2 v2] libs: Hook up libs into the build system

[Cyril Hrubis]

This hooks up libs/ subdirectories into the build system so that tests
needs only specify which library from libs/ directory we should link
againts. Which sets up the path to the library in LDFLAGS and also
causes the library to be rebuild even when make is executed from the
directory with testcases.

Now the test only needs to set up LTPLIBS variable with a list of
optional libraries to link against and LDLIBS, since as far as I know we
have to maintain the order of the libraries manually.

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
CC: Jan Stancek <jstancek@redhat.com>
CC: Petr Vorel <pvorel@suse.cz>
---
 include/mk/testcases.mk                          | 16 +++++++++++++++-
 testcases/kernel/syscalls/set_mempolicy/Makefile |  3 ++-
 2 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/include/mk/testcases.mk b/include/mk/testcases.mk
index bf97384a7..131854ec7 100644
--- a/include/mk/testcases.mk
+++ b/include/mk/testcases.mk
@@ -41,5 +41,19 @@ INSTALL_DIR	:= testcases/bin
 
 LDLIBS		+= -lltp
 
-$(APICMDS_DIR) $(LIBLTP_DIR): %:
+ifdef LTPLIBS
+
+LTPLIBS_DIRS = $(addprefix $(abs_top_builddir)/libs/lib, $(LTPLIBS))
+LTPLIBS_FILES = $(addsuffix .a, $(addprefix $(abs_top_builddir)/libs/, $(foreach LIB,$(LTPLIBS),lib$(LIB)/lib$(LIB))))
+
+MAKE_DEPS += $(LTPLIBS_FILES)
+
+$(LTPLIBS_FILES): $(LTPLIBS_DIRS)
+	$(MAKE) -C "$^" -f "$^/Makefile" all
+
+LDFLAGS += $(addprefix -L$(top_builddir)/libs/lib, $(LTPLIBS))
+
+endif
+
+$(LTPLIBS_DIRS) $(APICMDS_DIR) $(LIBLTP_DIR): %:
 	mkdir -p "$@"
diff --git a/testcases/kernel/syscalls/set_mempolicy/Makefile b/testcases/kernel/syscalls/set_mempolicy/Makefile
index b79a53faf..a0b79d6e1 100644
--- a/testcases/kernel/syscalls/set_mempolicy/Makefile
+++ b/testcases/kernel/syscalls/set_mempolicy/Makefile
@@ -1,8 +1,9 @@
 top_srcdir		?= ../../../..
 
+LTPLIBS = ltpnuma
+
 include $(top_srcdir)/include/mk/testcases.mk
 
-LDFLAGS += -L$(top_builddir)/libs/libltpnuma
 LDLIBS  += $(NUMA_LIBS) -lltpnuma
 
 include $(top_srcdir)/include/mk/generic_leaf_target.mk
-- 
2.19.2

[LTP] [PATCH 2/2 v2] libs: Hook up libs into the build system

[Jan Stancek]

----- Original Message -----
> This hooks up libs/ subdirectories into the build system so that tests
> needs only specify which library from libs/ directory we should link
> againts. Which sets up the path to the library in LDFLAGS and also
> causes the library to be rebuild even when make is executed from the
> directory with testcases.
> 
> Now the test only needs to set up LTPLIBS variable with a list of
> optional libraries to link against and LDLIBS, since as far as I know we
> have to maintain the order of the libraries manually.

What if we added "libs" as dependency for all tests and always expanded
LDFLAGS for all libraries even if tests don't link against them?
Then we wouldn't need to specify "LTPLIBS". Just a thought.

I tested v2 as far back as RHEL6.2, ACK.

Regards,
Jan

> 
> Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
> CC: Jan Stancek <jstancek@redhat.com>
> CC: Petr Vorel <pvorel@suse.cz>
> ---
>  include/mk/testcases.mk                          | 16 +++++++++++++++-
>  testcases/kernel/syscalls/set_mempolicy/Makefile |  3 ++-
>  2 files changed, 17 insertions(+), 2 deletions(-)
> 
> diff --git a/include/mk/testcases.mk b/include/mk/testcases.mk
> index bf97384a7..131854ec7 100644
> --- a/include/mk/testcases.mk
> +++ b/include/mk/testcases.mk
> @@ -41,5 +41,19 @@ INSTALL_DIR	:= testcases/bin
>  
>  LDLIBS		+= -lltp
>  
> -$(APICMDS_DIR) $(LIBLTP_DIR): %:
> +ifdef LTPLIBS
> +
> +LTPLIBS_DIRS = $(addprefix $(abs_top_builddir)/libs/lib, $(LTPLIBS))
> +LTPLIBS_FILES = $(addsuffix .a, $(addprefix $(abs_top_builddir)/libs/,
> $(foreach LIB,$(LTPLIBS),lib$(LIB)/lib$(LIB))))
> +
> +MAKE_DEPS += $(LTPLIBS_FILES)
> +
> +$(LTPLIBS_FILES): $(LTPLIBS_DIRS)
> +	$(MAKE) -C "$^" -f "$^/Makefile" all
> +
> +LDFLAGS += $(addprefix -L$(top_builddir)/libs/lib, $(LTPLIBS))
> +
> +endif
> +
> +$(LTPLIBS_DIRS) $(APICMDS_DIR) $(LIBLTP_DIR): %:
>  	mkdir -p "$@"
> diff --git a/testcases/kernel/syscalls/set_mempolicy/Makefile
> b/testcases/kernel/syscalls/set_mempolicy/Makefile
> index b79a53faf..a0b79d6e1 100644
> --- a/testcases/kernel/syscalls/set_mempolicy/Makefile
> +++ b/testcases/kernel/syscalls/set_mempolicy/Makefile
> @@ -1,8 +1,9 @@
>  top_srcdir		?= ../../../..
>  
> +LTPLIBS = ltpnuma
> +
>  include $(top_srcdir)/include/mk/testcases.mk
>  
> -LDFLAGS += -L$(top_builddir)/libs/libltpnuma
>  LDLIBS  += $(NUMA_LIBS) -lltpnuma
>  
>  include $(top_srcdir)/include/mk/generic_leaf_target.mk
> --
> 2.19.2
> 
> 

[LTP] [PATCH 2/2 v2] libs: Hook up libs into the build system

[Petr Vorel]

Hi Jan, Cyril,

> I tested v2 as far back as RHEL6.2, ACK.
Jan, have you tested out of tree build?

Still failing [1], mostly for out-of-tree builds [2]:
make: Entering directory '/usr/src/ltp-build'
make -C "lib" \
	-f "/usr/src/ltp/lib/Makefile" all
make -C "libs" \
	-f "/usr/src/ltp/libs/Makefile" all
make[1]: Entering directory '/usr/src/ltp-build/lib'
mkdir -m 00755 -p "newlib_tests"
make[1]: Entering directory '/usr/src/ltp-build'
make[1]: *** libs: No such file or directory.  Stop.
make[1]: Leaving directory '/usr/src/ltp-build'
/usr/src/ltp/Makefile:114: recipe for target 'libs-all' failed
make: *** [libs-all] Error 2
make: *** Waiting for unfinished jobs....

And i386 for Debian has still the same error [3]:
/usr/src/ltp/libs/libltpnuma/tst_numa.c:200: undefined /tmp/ccO9srN7.o: In function `verify_set_mempolicy':
/usr/src/ltp/testcases/kernel/syscalls/set_mempolicy/set_mempolicy04.c:105: undefined reference to `numa_allocate_nodemask'
/usr/src/ltp/testcases/kernel/syscalls/set_mempolicy/set_mempolicy04.c:110: undefined reference to `numa_bitmask_setbit'
/usr/src/ltp/testcases/kernel/syscalls/set_mempolicy/set_mempolicy04.c:112: undefined reference to `set_mempolicy'

[1] https://travis-ci.org/pevik/ltp/builds/503593388
[2] https://api.travis-ci.org/v3/job/503593397/log.txt
[3] https://api.travis-ci.org/v3/job/503593389/log.txt

Kind regards,
Petr

[LTP] [PATCH 2/2 v2] libs: Hook up libs into the build system

[Cyril Hrubis]

Hi!
> What if we added "libs" as dependency for all tests and always expanded
> LDFLAGS for all libraries even if tests don't link against them?
> Then we wouldn't need to specify "LTPLIBS". Just a thought.

That way if we ever wanted to fix the "rebuild tests after library
change" it will will rebuild a whole LTP even after change in a library
that is used in just a few packages...

-- 
Cyril Hrubis
chrubis@suse.cz

[LTP] [PATCH 2/2 v2] libs: Hook up libs into the build system

[Jan Stancek]

----- Original Message -----
> Hi Jan, Cyril,
> 
> > I tested v2 as far back as RHEL6.2, ACK.
> Jan, have you tested out of tree build?

I didn't. Only in-tree builds.

> 
> Still failing [1], mostly for out-of-tree builds [2]:
> make: Entering directory '/usr/src/ltp-build'
> make -C "lib" \
> 	-f "/usr/src/ltp/lib/Makefile" all
> make -C "libs" \
> 	-f "/usr/src/ltp/libs/Makefile" all
> make[1]: Entering directory '/usr/src/ltp-build/lib'
> mkdir -m 00755 -p "newlib_tests"
> make[1]: Entering directory '/usr/src/ltp-build'
> make[1]: *** libs: No such file or directory.  Stop.
> make[1]: Leaving directory '/usr/src/ltp-build'
> /usr/src/ltp/Makefile:114: recipe for target 'libs-all' failed
> make: *** [libs-all] Error 2
> make: *** Waiting for unfinished jobs....
> 
> And i386 for Debian has still the same error [3]:
> /usr/src/ltp/libs/libltpnuma/tst_numa.c:200: undefined /tmp/ccO9srN7.o: In
> function `verify_set_mempolicy':
> /usr/src/ltp/testcases/kernel/syscalls/set_mempolicy/set_mempolicy04.c:105:
> undefined reference to `numa_allocate_nodemask'
> /usr/src/ltp/testcases/kernel/syscalls/set_mempolicy/set_mempolicy04.c:110:
> undefined reference to `numa_bitmask_setbit'
> /usr/src/ltp/testcases/kernel/syscalls/set_mempolicy/set_mempolicy04.c:112:
> undefined reference to `set_mempolicy'

can you try to reverse linking order of libnuma and libltpnuma?

> 
> [1] https://travis-ci.org/pevik/ltp/builds/503593388
> [2] https://api.travis-ci.org/v3/job/503593397/log.txt
> [3] https://api.travis-ci.org/v3/job/503593389/log.txt
> 
> Kind regards,
> Petr
> 

[LTP] [PATCH 2/2 v2] libs: Hook up libs into the build system

[Cyril Hrubis]

Hi!
> > I tested v2 as far back as RHEL6.2, ACK.
> Jan, have you tested out of tree build?
> 
> Still failing [1], mostly for out-of-tree builds [2]:
> make: Entering directory '/usr/src/ltp-build'
> make -C "lib" \
> 	-f "/usr/src/ltp/lib/Makefile" all
> make -C "libs" \
> 	-f "/usr/src/ltp/libs/Makefile" all
> make[1]: Entering directory '/usr/src/ltp-build/lib'
> mkdir -m 00755 -p "newlib_tests"
> make[1]: Entering directory '/usr/src/ltp-build'
> make[1]: *** libs: No such file or directory.  Stop.
> make[1]: Leaving directory '/usr/src/ltp-build'
> /usr/src/ltp/Makefile:114: recipe for target 'libs-all' failed
> make: *** [libs-all] Error 2
> make: *** Waiting for unfinished jobs....

Sigh, the out-of-tree build is always making things unnecessary
difficuilt. I have no idea why the directory is not created in the build
tree.

> And i386 for Debian has still the same error [3]:
> /usr/src/ltp/libs/libltpnuma/tst_numa.c:200: undefined /tmp/ccO9srN7.o: In function `verify_set_mempolicy':
> /usr/src/ltp/testcases/kernel/syscalls/set_mempolicy/set_mempolicy04.c:105: undefined reference to `numa_allocate_nodemask'
> /usr/src/ltp/testcases/kernel/syscalls/set_mempolicy/set_mempolicy04.c:110: undefined reference to `numa_bitmask_setbit'
> /usr/src/ltp/testcases/kernel/syscalls/set_mempolicy/set_mempolicy04.c:112: undefined reference to `set_mempolicy'

Not at all, this is very different problem :-). Now it fails to build in
these tests that actually use the library. So original problem was fixed
and this is quite possibly wrong config guards used in the tst_numa.c.

I do use ifdef HAVE_NUMA_H in the library in the library and tests it
should be probably replaced by HAVE_NUMA_V2.

Still I would say that the original patches should be pushed and that we
should deal with the rest of the problems in separate ones.

-- 
Cyril Hrubis
chrubis@suse.cz

[LTP] [PATCH 2/2 v2] libs: Hook up libs into the build system

[Cyril Hrubis]

Hi!
> > > I tested v2 as far back as RHEL6.2, ACK.
> > Jan, have you tested out of tree build?
> > 
> > Still failing [1], mostly for out-of-tree builds [2]:
> > make: Entering directory '/usr/src/ltp-build'
> > make -C "lib" \
> > 	-f "/usr/src/ltp/lib/Makefile" all
> > make -C "libs" \
> > 	-f "/usr/src/ltp/libs/Makefile" all
> > make[1]: Entering directory '/usr/src/ltp-build/lib'
> > mkdir -m 00755 -p "newlib_tests"
> > make[1]: Entering directory '/usr/src/ltp-build'
> > make[1]: *** libs: No such file or directory.  Stop.
> > make[1]: Leaving directory '/usr/src/ltp-build'
> > /usr/src/ltp/Makefile:114: recipe for target 'libs-all' failed
> > make: *** [libs-all] Error 2
> > make: *** Waiting for unfinished jobs....
> 
> Sigh, the out-of-tree build is always making things unnecessary
> difficuilt. I have no idea why the directory is not created in the build
> tree.

And on my machine out-of-tree build does not managed to get past
configure step, it fails somewhere in the ffsb configure step.

-- 
Cyril Hrubis
chrubis@suse.cz

[LTP] [PATCH 2/2 v2] libs: Hook up libs into the build system

[Petr Vorel]

Hi,

> And on my machine out-of-tree build does not managed to get past
> configure step, it fails somewhere in the ffsb configure step.
I often get
configure:2889: error: source directory already configured; run "make distclean" there first

So I build it with a bit drastic way:
rm -rfv *; git reset --hard; ./build.sh -o out


Kind regards,
Petr

[LTP] [PATCH 2/2 v2] libs: Hook up libs into the build system

[Cyril Hrubis]

Hi!
> > And on my machine out-of-tree build does not managed to get past
> > configure step, it fails somewhere in the ffsb configure step.
> I often get
> configure:2889: error: source directory already configured; run "make distclean" there first
> 
> So I build it with a bit drastic way:
> rm -rfv *; git reset --hard; ./build.sh -o out

Mine is different, I got exit 1 without any reasonable explanation.

-- 
Cyril Hrubis
chrubis@suse.cz