* [LTP] [RFC PATCH 0/3] build: make check target
@ 2021-06-03 18:38 Petr Vorel
2021-06-03 18:38 ` [LTP] [RFC PATCH 1/3] make: Add make check{,-c,-shell} targets Petr Vorel
` (3 more replies)
0 siblings, 4 replies; 9+ messages in thread
From: Petr Vorel @ 2021-06-03 18:38 UTC (permalink / raw)
To: ltp
Hi,
actually, there are 3 targets: check, check-c, check-shell.
Previous attempt: make: Add make check target + run.sh [1].
This is built on the top of patchset tst_net.sh: Allow to skip
initialization [2]. Hope this time library check will made it :).
I plan to wrap tests with a script in the future, which will parse
expected output, i.e. approach from #312 [3].
It's not ready due several problems, but for curious here is a test run:
https://github.com/pevik/ltp/runs/2739826773
1) I made something wrong, I have no idea how to fix this error:
make[1]: Entering directory 'ltp/lib'
../include/mk/generic_trunk_target.inc:105: warning: overriding recipe for target 'check-c'
../include/mk/generic_leaf_target.inc:110: warning: ignoring old recipe for target 'check-c'
../include/mk/generic_trunk_target.inc:105: warning: overriding recipe for target 'check-shell'
../include/mk/generic_leaf_target.inc:118: warning: ignoring old recipe for target 'check-shell'
2) I obviously have problem with out-of-tree build
https://github.com/pevik/ltp/runs/2739826612?check_suite_focus=true
where path of shell tests (which are in src directory only) is obviously
broken:
make[2]: *** No rule to make target 'shell/tst_check_driver.sh', needed by 'check-shell'. Stop.
20
make[1]: *** [/__w/ltp/ltp/include/mk/generic_trunk_target.inc:105: check-shell] Error 2
21
make[1]: Leaving directory '/__w/ltp/ltp-build/lib'
3) tst_bool_expr fails on some systems when run via make check:
https://github.com/pevik/ltp/runs/2739826773?check_suite_focus=true
But it runs ok when running normally. It might be dash issue
(it's on Debian).
Kind regards,
Petr
[1] https://patchwork.ozlabs.org/project/ltp/patch/20190924182841.4528-1-pvorel@suse.cz/
[2] https://patchwork.ozlabs.org/project/ltp/list/?series=247078
[3] https://github.com/linux-test-project/ltp/issues/312
Petr Vorel (3):
make: Add make check{,-c,-shell} targets
build.sh: Add support for make check
CI: Run also make check
.github/workflows/ci.yml | 3 +++
Makefile | 21 ++++++++++++++++++++-
build.sh | 18 +++++++++++++++++-
include/mk/generic_leaf_target.inc | 25 ++++++++++++++++++++++---
include/mk/generic_trunk_target.inc | 4 ++--
lib/newlib_tests/Makefile | 8 ++++++++
6 files changed, 72 insertions(+), 7 deletions(-)
--
2.31.1
^ permalink raw reply [flat|nested] 9+ messages in thread* [LTP] [RFC PATCH 1/3] make: Add make check{,-c,-shell} targets 2021-06-03 18:38 [LTP] [RFC PATCH 0/3] build: make check target Petr Vorel @ 2021-06-03 18:38 ` Petr Vorel 2021-06-04 14:49 ` Cyril Hrubis 2021-06-04 18:01 ` Enji Cooper 2021-06-03 18:38 ` [LTP] [RFC PATCH 2/3] build.sh: Add support for make check Petr Vorel ` (2 subsequent siblings) 3 siblings, 2 replies; 9+ messages in thread From: Petr Vorel @ 2021-06-03 18:38 UTC (permalink / raw) To: ltp For testing C and shell API. Signed-off-by: Petr Vorel <pvorel@suse.cz> --- Makefile | 21 ++++++++++++++++++++- include/mk/generic_leaf_target.inc | 25 ++++++++++++++++++++++--- include/mk/generic_trunk_target.inc | 4 ++-- lib/newlib_tests/Makefile | 8 ++++++++ 4 files changed, 52 insertions(+), 6 deletions(-) diff --git a/Makefile b/Makefile index 56812d77b..acea4551e 100644 --- a/Makefile +++ b/Makefile @@ -75,10 +75,14 @@ endif INSTALL_TARGETS += $(COMMON_TARGETS) CLEAN_TARGETS += $(COMMON_TARGETS) lib libs BOOTSTRAP_TARGETS := $(sort $(COMMON_TARGETS) $(CLEAN_TARGETS) $(INSTALL_TARGETS)) +TEST_TARGETS := lib +TEST_SHELL_TARGETS := lib CLEAN_TARGETS := $(addsuffix -clean,$(CLEAN_TARGETS)) INSTALL_TARGETS := $(addsuffix -install,$(INSTALL_TARGETS)) MAKE_TARGETS := $(addsuffix -all,$(filter-out lib,$(COMMON_TARGETS))) +TEST_TARGETS := $(addsuffix -check-c,$(TEST_TARGETS)) +TEST_SHELL_TARGETS := $(addsuffix -check-shell,$(TEST_SHELL_TARGETS)) # There's no reason why we should run `all' twice. Otherwise we're just wasting # 3+ mins of useful CPU cycles on a modern machine, and even more time on an @@ -113,6 +117,14 @@ $(filter-out include-clean,$(CLEAN_TARGETS)):: -$(MAKE) -C "$(subst -clean,,$@)" \ -f "$(abs_top_srcdir)/$(subst -clean,,$@)/Makefile" clean +$(TEST_TARGETS): lib-all + $(MAKE) -C "$(subst -check-c,,$@)" \ + -f "$(abs_top_srcdir)/$(subst -check-c,,$@)/Makefile" check-c + +$(TEST_SHELL_TARGETS): + $(MAKE) -C "$(subst -check-shell,,$@)" \ + -f "$(abs_top_srcdir)/$(subst -check-shell,,$@)/Makefile" check-shell + # Just like everything depends on include-all / -install, we need to get rid # of include last to ensure that things won't be monkey screwed up. Only do # this if we're invoking clean or a subclean directly though. @@ -189,8 +201,15 @@ INSTALL_TARGETS += $(addprefix $(DESTDIR)/$(bindir)/,$(BINDIR_INSTALL_SCRIPTS)) $(INSTALL_TARGETS): $(INSTALL_DIR) $(DESTDIR)/$(bindir) +.PHONY: check check-c check-shell + +## Check +check-c: $(TEST_TARGETS) +check-shell: $(TEST_SHELL_TARGETS) +check: check-c check-shell + ## Install -install: $(INSTALL_TARGETS) +install: lib-all $(INSTALL_TARGETS) ## Help .PHONY: help diff --git a/include/mk/generic_leaf_target.inc b/include/mk/generic_leaf_target.inc index 64953f89a..d53623456 100644 --- a/include/mk/generic_leaf_target.inc +++ b/include/mk/generic_leaf_target.inc @@ -63,8 +63,12 @@ # real files or directories). This will automatically append # adds the .o suffix to all files referenced by # $(MAKE_TARGETS)) to CLEAN_TARGETS, if MAKE_TARGETS wasn't -# defined (see -# $(MAKE_TARGETS)). +# defined (see $(MAKE_TARGETS)). +# +# $(TEST_TARGETS) : LTP C API tests. +# +# $(TEST_SHELL_TARGETS) : LTP shell API tests. +# # $(INSTALL_MODE) : What mode should we using when calling # install(1)? # @@ -92,7 +96,7 @@ # INSTALL_DIR := $(libdir) # -.PHONY: all clean install +.PHONY: all check-c check-shell clean install ifneq ($(strip $(MAKE_TARGETS)),) $(MAKE_TARGETS) += $(HOST_MAKE_TARGETS) @@ -102,6 +106,21 @@ $(MAKE_TARGETS): | $(MAKE_DEPS) all: $(MAKE_TARGETS) +check-c: $(TEST_TARGETS) + @set -e; echo; echo "===== Test C API ====="; \ + for i in $(TEST_TARGETS); do \ + echo; echo "* $$i"; \ + echo "PATH $(top_srcdir)/testcases/lib:$$PATH"; \ + PATH="$(top_srcdir)/testcases/lib:$$PATH" ./$$i || [ $$? -eq 32 ]; \ + done + +check-shell: $(TEST_SHELL_TARGETS) + @set -e; echo; echo "===== Test shell API ====="; \ + for i in $(TEST_SHELL_TARGETS); do \ + echo; echo "* $$i"; \ + PATH="$(abs_top_srcdir)/testcases/lib:$(abs_top_builddir)/testcases/lib:$$PATH" $(abs_srcdir)/$$i || [ $$? -eq 32 ]; \ + done + clean:: $(CLEAN_DEPS) -$(RM) -f -r $(CLEAN_TARGETS) diff --git a/include/mk/generic_trunk_target.inc b/include/mk/generic_trunk_target.inc index fc59f944f..d195fe3bd 100644 --- a/include/mk/generic_trunk_target.inc +++ b/include/mk/generic_trunk_target.inc @@ -48,7 +48,7 @@ include $(top_srcdir)/include/mk/functions.mk -RECURSIVE_TARGETS ?= all install +RECURSIVE_TARGETS ?= all check check-c check-shell install $(eval $(get_make_dirs)) @@ -71,7 +71,7 @@ trunk-install: $(INSTALL_FILES) # Avoid creating duplicate .PHONY references to all, clean, and install. IIRC, # I've seen some indeterministic behavior when one does this in the past with # GNU Make... -.PHONY: $(filter-out $(RECURSIVE_TARGETS),all clean install) +.PHONY: $(filter-out $(RECURSIVE_TARGETS),all check clean install) all: trunk-all clean:: trunk-clean diff --git a/lib/newlib_tests/Makefile b/lib/newlib_tests/Makefile index 30ca6810c..182ef5689 100644 --- a/lib/newlib_tests/Makefile +++ b/lib/newlib_tests/Makefile @@ -19,5 +19,13 @@ ifeq ($(ANDROID),1) FILTER_OUT_MAKE_TARGETS += test08 endif +# C API tests (which exit 0) +TEST_TARGETS += test0[579] test1[2568] test_exec test_timer +TEST_TARGETS += tst_bool_expr tst_res_hexd tst_strstatus +TEST_TARGETS += tst_fuzzy_sync0[1-2] + +# shell API tests (which exit 0) +TEST_SHELL_TARGETS += shell/tst_check_driver.sh +TEST_SHELL_TARGETS += shell/net/*.sh include $(top_srcdir)/include/mk/generic_leaf_target.mk -- 2.31.1 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [LTP] [RFC PATCH 1/3] make: Add make check{,-c,-shell} targets 2021-06-03 18:38 ` [LTP] [RFC PATCH 1/3] make: Add make check{,-c,-shell} targets Petr Vorel @ 2021-06-04 14:49 ` Cyril Hrubis 2021-06-04 16:56 ` Petr Vorel 2021-06-04 18:01 ` Enji Cooper 1 sibling, 1 reply; 9+ messages in thread From: Cyril Hrubis @ 2021-06-04 14:49 UTC (permalink / raw) To: ltp Hi! > For testing C and shell API. > > Signed-off-by: Petr Vorel <pvorel@suse.cz> > --- > Makefile | 21 ++++++++++++++++++++- > include/mk/generic_leaf_target.inc | 25 ++++++++++++++++++++++--- > include/mk/generic_trunk_target.inc | 4 ++-- > lib/newlib_tests/Makefile | 8 ++++++++ > 4 files changed, 52 insertions(+), 6 deletions(-) > > diff --git a/Makefile b/Makefile > index 56812d77b..acea4551e 100644 > --- a/Makefile > +++ b/Makefile > @@ -75,10 +75,14 @@ endif > INSTALL_TARGETS += $(COMMON_TARGETS) > CLEAN_TARGETS += $(COMMON_TARGETS) lib libs > BOOTSTRAP_TARGETS := $(sort $(COMMON_TARGETS) $(CLEAN_TARGETS) $(INSTALL_TARGETS)) > +TEST_TARGETS := lib > +TEST_SHELL_TARGETS := lib > > CLEAN_TARGETS := $(addsuffix -clean,$(CLEAN_TARGETS)) > INSTALL_TARGETS := $(addsuffix -install,$(INSTALL_TARGETS)) > MAKE_TARGETS := $(addsuffix -all,$(filter-out lib,$(COMMON_TARGETS))) > +TEST_TARGETS := $(addsuffix -check-c,$(TEST_TARGETS)) > +TEST_SHELL_TARGETS := $(addsuffix -check-shell,$(TEST_SHELL_TARGETS)) > > # There's no reason why we should run `all' twice. Otherwise we're just wasting > # 3+ mins of useful CPU cycles on a modern machine, and even more time on an > @@ -113,6 +117,14 @@ $(filter-out include-clean,$(CLEAN_TARGETS)):: > -$(MAKE) -C "$(subst -clean,,$@)" \ > -f "$(abs_top_srcdir)/$(subst -clean,,$@)/Makefile" clean > > +$(TEST_TARGETS): lib-all > + $(MAKE) -C "$(subst -check-c,,$@)" \ > + -f "$(abs_top_srcdir)/$(subst -check-c,,$@)/Makefile" check-c > + > +$(TEST_SHELL_TARGETS): > + $(MAKE) -C "$(subst -check-shell,,$@)" \ > + -f "$(abs_top_srcdir)/$(subst -check-shell,,$@)/Makefile" check-shell > + > # Just like everything depends on include-all / -install, we need to get rid > # of include last to ensure that things won't be monkey screwed up. Only do > # this if we're invoking clean or a subclean directly though. > @@ -189,8 +201,15 @@ INSTALL_TARGETS += $(addprefix $(DESTDIR)/$(bindir)/,$(BINDIR_INSTALL_SCRIPTS)) > > $(INSTALL_TARGETS): $(INSTALL_DIR) $(DESTDIR)/$(bindir) > > +.PHONY: check check-c check-shell > + > +## Check > +check-c: $(TEST_TARGETS) > +check-shell: $(TEST_SHELL_TARGETS) > +check: check-c check-shell > + > ## Install > -install: $(INSTALL_TARGETS) > +install: lib-all $(INSTALL_TARGETS) > > ## Help > .PHONY: help > diff --git a/include/mk/generic_leaf_target.inc b/include/mk/generic_leaf_target.inc > index 64953f89a..d53623456 100644 > --- a/include/mk/generic_leaf_target.inc > +++ b/include/mk/generic_leaf_target.inc > @@ -63,8 +63,12 @@ > # real files or directories). This will automatically append > # adds the .o suffix to all files referenced by > # $(MAKE_TARGETS)) to CLEAN_TARGETS, if MAKE_TARGETS wasn't > -# defined (see > -# $(MAKE_TARGETS)). > +# defined (see $(MAKE_TARGETS)). > +# > +# $(TEST_TARGETS) : LTP C API tests. > +# > +# $(TEST_SHELL_TARGETS) : LTP shell API tests. > +# > # $(INSTALL_MODE) : What mode should we using when calling > # install(1)? > # > @@ -92,7 +96,7 @@ > # INSTALL_DIR := $(libdir) > # > > -.PHONY: all clean install > +.PHONY: all check-c check-shell clean install > > ifneq ($(strip $(MAKE_TARGETS)),) > $(MAKE_TARGETS) += $(HOST_MAKE_TARGETS) > @@ -102,6 +106,21 @@ $(MAKE_TARGETS): | $(MAKE_DEPS) > > all: $(MAKE_TARGETS) > > +check-c: $(TEST_TARGETS) > + @set -e; echo; echo "===== Test C API ====="; \ > + for i in $(TEST_TARGETS); do \ > + echo; echo "* $$i"; \ > + echo "PATH $(top_srcdir)/testcases/lib:$$PATH"; \ > + PATH="$(top_srcdir)/testcases/lib:$$PATH" ./$$i || [ $$? -eq 32 ]; \ > + done > + > +check-shell: $(TEST_SHELL_TARGETS) > + @set -e; echo; echo "===== Test shell API ====="; \ > + for i in $(TEST_SHELL_TARGETS); do \ > + echo; echo "* $$i"; \ > + PATH="$(abs_top_srcdir)/testcases/lib:$(abs_top_builddir)/testcases/lib:$$PATH" $(abs_srcdir)/$$i || [ $$? -eq 32 ]; \ > + done > + > clean:: $(CLEAN_DEPS) > -$(RM) -f -r $(CLEAN_TARGETS) > > diff --git a/include/mk/generic_trunk_target.inc b/include/mk/generic_trunk_target.inc > index fc59f944f..d195fe3bd 100644 > --- a/include/mk/generic_trunk_target.inc > +++ b/include/mk/generic_trunk_target.inc > @@ -48,7 +48,7 @@ > > include $(top_srcdir)/include/mk/functions.mk > > -RECURSIVE_TARGETS ?= all install > +RECURSIVE_TARGETS ?= all check check-c check-shell install If you add a target to RECURSIVE_TARGETS such target automatically traverses directories, that what the target is for. See the rule at the end of the generic_trunk_target.inc. Which means that you redefine targets if you define the same targets to loop and run the tests above. I guess that you want add another target that depends on check targets to actually run the tests? Maybe: check-c: check-c-run check-c-run: $(TEST_TARGETS) for i in $(TEST_TARGETS); do ... done in order to run the tests. In this case the check* targets would be used for directory traversal and the check-c-run would be used to run the tests. > $(eval $(get_make_dirs)) > > @@ -71,7 +71,7 @@ trunk-install: $(INSTALL_FILES) > # Avoid creating duplicate .PHONY references to all, clean, and install. IIRC, > # I've seen some indeterministic behavior when one does this in the past with > # GNU Make... > -.PHONY: $(filter-out $(RECURSIVE_TARGETS),all clean install) > +.PHONY: $(filter-out $(RECURSIVE_TARGETS),all check clean install) The check is already in RECURSIVE_TARGETS > all: trunk-all > > clean:: trunk-clean > diff --git a/lib/newlib_tests/Makefile b/lib/newlib_tests/Makefile > index 30ca6810c..182ef5689 100644 > --- a/lib/newlib_tests/Makefile > +++ b/lib/newlib_tests/Makefile > @@ -19,5 +19,13 @@ ifeq ($(ANDROID),1) > FILTER_OUT_MAKE_TARGETS += test08 > endif > > +# C API tests (which exit 0) > +TEST_TARGETS += test0[579] test1[2568] test_exec test_timer > +TEST_TARGETS += tst_bool_expr tst_res_hexd tst_strstatus > +TEST_TARGETS += tst_fuzzy_sync0[1-2] > + > +# shell API tests (which exit 0) > +TEST_SHELL_TARGETS += shell/tst_check_driver.sh > +TEST_SHELL_TARGETS += shell/net/*.sh > > include $(top_srcdir)/include/mk/generic_leaf_target.mk > -- > 2.31.1 > -- Cyril Hrubis chrubis@suse.cz ^ permalink raw reply [flat|nested] 9+ messages in thread
* [LTP] [RFC PATCH 1/3] make: Add make check{,-c,-shell} targets 2021-06-04 14:49 ` Cyril Hrubis @ 2021-06-04 16:56 ` Petr Vorel 0 siblings, 0 replies; 9+ messages in thread From: Petr Vorel @ 2021-06-04 16:56 UTC (permalink / raw) To: ltp Hi Cyril, ... > > +++ b/include/mk/generic_trunk_target.inc > > @@ -48,7 +48,7 @@ > > include $(top_srcdir)/include/mk/functions.mk > > -RECURSIVE_TARGETS ?= all install > > +RECURSIVE_TARGETS ?= all check check-c check-shell install > If you add a target to RECURSIVE_TARGETS such target automatically > traverses directories, that what the target is for. See the rule at the > end of the generic_trunk_target.inc. Which means that you redefine > targets if you define the same targets to loop and run the tests above. Ah, thanks! > I guess that you want add another target that depends on check targets > to actually run the tests? > Maybe: > check-c: check-c-run > check-c-run: $(TEST_TARGETS) > for i in $(TEST_TARGETS); do > ... > done > in order to run the tests. > In this case the check* targets would be used for directory traversal > and the check-c-run would be used to run the tests. Sounds reasonable. > > $(eval $(get_make_dirs)) > > @@ -71,7 +71,7 @@ trunk-install: $(INSTALL_FILES) > > # Avoid creating duplicate .PHONY references to all, clean, and install. IIRC, > > # I've seen some indeterministic behavior when one does this in the past with > > # GNU Make... > > -.PHONY: $(filter-out $(RECURSIVE_TARGETS),all clean install) > > +.PHONY: $(filter-out $(RECURSIVE_TARGETS),all check clean install) > The check is already in RECURSIVE_TARGETS Thanks! I'll send v2 on Monday, which also use test* as target name instead of check* as we agreed on Richie's Libclang based analyzer patchset [1]. Kind regards, Petr [1] https://lists.linux.it/pipermail/ltp/2021-June/023008.html ^ permalink raw reply [flat|nested] 9+ messages in thread
* [LTP] [RFC PATCH 1/3] make: Add make check{,-c,-shell} targets 2021-06-03 18:38 ` [LTP] [RFC PATCH 1/3] make: Add make check{,-c,-shell} targets Petr Vorel 2021-06-04 14:49 ` Cyril Hrubis @ 2021-06-04 18:01 ` Enji Cooper 2021-06-04 18:49 ` Petr Vorel 1 sibling, 1 reply; 9+ messages in thread From: Enji Cooper @ 2021-06-04 18:01 UTC (permalink / raw) To: ltp > On Jun 3, 2021, at 11:38 AM, Petr Vorel <pvorel@suse.cz> wrote: > > For testing C and shell API. Why not just add a single ?make check? target that calls the C and shell targets to match what most open source projects do in terms of testing? Also, continuing on in light of errors seems like a good default behavior to have as it would prevent a single error/failure from gathering all other results. My 2 cents, -Enji -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: <http://lists.linux.it/pipermail/ltp/attachments/20210604/1b27c9a2/attachment-0001.sig> ^ permalink raw reply [flat|nested] 9+ messages in thread
* [LTP] [RFC PATCH 1/3] make: Add make check{,-c,-shell} targets 2021-06-04 18:01 ` Enji Cooper @ 2021-06-04 18:49 ` Petr Vorel 0 siblings, 0 replies; 9+ messages in thread From: Petr Vorel @ 2021-06-04 18:49 UTC (permalink / raw) To: ltp Hi Enji, > > On Jun 3, 2021, at 11:38 AM, Petr Vorel <pvorel@suse.cz> wrote: > > For testing C and shell API. > Why not just add a single ?make check? target that calls the C and shell targets to match what most open source projects do in terms of testing? Also, continuing on in light of errors seems like a good default behavior to have as it would prevent a single error/failure from gathering all other results. Yes, the original approach in this patchset is make check, which consists of two make-c and make-shell. One of tst_fuzzy_sync checks is quite long and in longer term the number of tests for both C and shell API might grow to prolong the testing. Thus one may appreciate to test for local development just one of these two. Or, if you mean how it has been implemented (adding tests into variables), I plan to add docparse tests (into docparse/tests/) - i.e. into different directory. Also, result of Richie's Libclang based analyzer patchset [1], where he also added make check, we decided to use make test here [2]. > My 2 cents, Thanks, comments are always welcome. > -Enji Kind regards, Petr [1] https://lists.linux.it/pipermail/ltp/2021-June/023008.html [2] https://lists.linux.it/pipermail/ltp/2021-June/023021.html ^ permalink raw reply [flat|nested] 9+ messages in thread
* [LTP] [RFC PATCH 2/3] build.sh: Add support for make check 2021-06-03 18:38 [LTP] [RFC PATCH 0/3] build: make check target Petr Vorel 2021-06-03 18:38 ` [LTP] [RFC PATCH 1/3] make: Add make check{,-c,-shell} targets Petr Vorel @ 2021-06-03 18:38 ` Petr Vorel 2021-06-03 18:38 ` [LTP] [RFC PATCH 3/3] CI: Run also " Petr Vorel 2021-06-04 14:43 ` [LTP] [RFC PATCH 0/3] build: make check target Richard Palethorpe 3 siblings, 0 replies; 9+ messages in thread From: Petr Vorel @ 2021-06-03 18:38 UTC (permalink / raw) To: ltp Signed-off-by: Petr Vorel <pvorel@suse.cz> --- build.sh | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/build.sh b/build.sh index 240ce8e68..c892fa87b 100755 --- a/build.sh +++ b/build.sh @@ -119,6 +119,17 @@ build_out_tree() make $MAKE_OPTS_OUT_TREE } +check_in_tree() +{ + make check +} + +check_out_tree() +{ + cd $BUILD_DIR + make $MAKE_OPTS_OUT_TREE check +} + install_in_tree() { make $MAKE_OPTS install @@ -165,6 +176,7 @@ RUN: autotools run only 'make autotools' configure run only 'configure' build run only 'make' +check run only 'make check' install run only 'make install' Default configure options: @@ -192,7 +204,7 @@ while getopts "c:hio:p:r:t:" opt; do esac;; p) prefix="$OPTARG";; r) case "$OPTARG" in - autotools|configure|build|install) run="$OPTARG";; + autotools|configure|build|check|install) run="$OPTARG";; *) echo "Wrong run type '$OPTARG'" >&2; usage; exit 1;; esac;; t) case "$OPTARG" in @@ -218,6 +230,10 @@ if [ -z "$run" -o "$run" = "build" ]; then eval build_${tree}_tree fi +if [ -z "$run" -o "$run" = "check" ]; then + eval check_${tree}_tree +fi + if [ -z "$run" -o "$run" = "install" ]; then if [ "$install" = 1 ]; then eval install_${tree}_tree -- 2.31.1 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [LTP] [RFC PATCH 3/3] CI: Run also make check 2021-06-03 18:38 [LTP] [RFC PATCH 0/3] build: make check target Petr Vorel 2021-06-03 18:38 ` [LTP] [RFC PATCH 1/3] make: Add make check{,-c,-shell} targets Petr Vorel 2021-06-03 18:38 ` [LTP] [RFC PATCH 2/3] build.sh: Add support for make check Petr Vorel @ 2021-06-03 18:38 ` Petr Vorel 2021-06-04 14:43 ` [LTP] [RFC PATCH 0/3] build: make check target Richard Palethorpe 3 siblings, 0 replies; 9+ messages in thread From: Petr Vorel @ 2021-06-03 18:38 UTC (permalink / raw) To: ltp ona all targets Signed-off-by: Petr Vorel <pvorel@suse.cz> --- .github/workflows/ci.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 185f49e41..01e2578fa 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -151,6 +151,9 @@ jobs: - name: Compile run: ./build.sh -r build -o ${TREE:-in} + - name: Test + run: ./build.sh -r check -o ${TREE:-in} + - name: Install run: | if [ "$MAKE_INSTALL" = 1 ]; then INSTALL_OPT="-i"; fi -- 2.31.1 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [LTP] [RFC PATCH 0/3] build: make check target 2021-06-03 18:38 [LTP] [RFC PATCH 0/3] build: make check target Petr Vorel ` (2 preceding siblings ...) 2021-06-03 18:38 ` [LTP] [RFC PATCH 3/3] CI: Run also " Petr Vorel @ 2021-06-04 14:43 ` Richard Palethorpe 3 siblings, 0 replies; 9+ messages in thread From: Richard Palethorpe @ 2021-06-04 14:43 UTC (permalink / raw) To: ltp Hello Petr, Petr Vorel <pvorel@suse.cz> writes: > Hi, > > actually, there are 3 targets: check, check-c, check-shell. Usually, if you only want to make a sub group of tests. Then you cd into the sub directory and do 'make'. So why do we need three targets? Just an idea, but maybe in a future patch this could run some actual tests as well? A lot of the old syscall tests would be fine to run in CI. More likely if they fail it is due to LTP library changes than a kernel bug on the host. > > Previous attempt: make: Add make check target + run.sh [1]. > This is built on the top of patchset tst_net.sh: Allow to skip > initialization [2]. Hope this time library check will made it :). > > I plan to wrap tests with a script in the future, which will parse > expected output, i.e. approach from #312 [3]. > > It's not ready due several problems, but for curious here is a test run: > https://github.com/pevik/ltp/runs/2739826773 > > 1) I made something wrong, I have no idea how to fix this error: > make[1]: Entering directory 'ltp/lib' > ../include/mk/generic_trunk_target.inc:105: warning: overriding recipe for target 'check-c' > ../include/mk/generic_leaf_target.inc:110: warning: ignoring old recipe for target 'check-c' > ../include/mk/generic_trunk_target.inc:105: warning: overriding recipe for target 'check-shell' > ../include/mk/generic_leaf_target.inc:118: warning: ignoring old recipe for target 'check-shell' > > 2) I obviously have problem with out-of-tree build > https://github.com/pevik/ltp/runs/2739826612?check_suite_focus=true > where path of shell tests (which are in src directory only) is obviously > broken: > make[2]: *** No rule to make target 'shell/tst_check_driver.sh', needed by 'check-shell'. Stop. > 20 > make[1]: *** [/__w/ltp/ltp/include/mk/generic_trunk_target.inc:105: check-shell] Error 2 > 21 > make[1]: Leaving directory '/__w/ltp/ltp-build/lib' > > 3) tst_bool_expr fails on some systems when run via make check: > https://github.com/pevik/ltp/runs/2739826773?check_suite_focus=true > But it runs ok when running normally. It might be dash issue > (it's on Debian). > > Kind regards, > Petr > > [1] https://patchwork.ozlabs.org/project/ltp/patch/20190924182841.4528-1-pvorel@suse.cz/ > [2] https://patchwork.ozlabs.org/project/ltp/list/?series=247078 > [3] https://github.com/linux-test-project/ltp/issues/312 > > Petr Vorel (3): > make: Add make check{,-c,-shell} targets > build.sh: Add support for make check > CI: Run also make check > > .github/workflows/ci.yml | 3 +++ > Makefile | 21 ++++++++++++++++++++- > build.sh | 18 +++++++++++++++++- > include/mk/generic_leaf_target.inc | 25 ++++++++++++++++++++++--- > include/mk/generic_trunk_target.inc | 4 ++-- > lib/newlib_tests/Makefile | 8 ++++++++ > 6 files changed, 72 insertions(+), 7 deletions(-) > > -- > 2.31.1 -- Thank you, Richard. ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2021-06-04 18:49 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-03 18:38 [LTP] [RFC PATCH 0/3] build: make check target Petr Vorel
2021-06-03 18:38 ` [LTP] [RFC PATCH 1/3] make: Add make check{,-c,-shell} targets Petr Vorel
2021-06-04 14:49 ` Cyril Hrubis
2021-06-04 16:56 ` Petr Vorel
2021-06-04 18:01 ` Enji Cooper
2021-06-04 18:49 ` Petr Vorel
2021-06-03 18:38 ` [LTP] [RFC PATCH 2/3] build.sh: Add support for make check Petr Vorel
2021-06-03 18:38 ` [LTP] [RFC PATCH 3/3] CI: Run also " Petr Vorel
2021-06-04 14:43 ` [LTP] [RFC PATCH 0/3] build: make check target Richard Palethorpe
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.