[LTP] Building LTP despite failures

[LTP] Building LTP despite failures

[Vineet Gupta]

Hi,

We use LTP regularly to check the health of ARC Linux. One the pain points have
been out of tree patches to LTP to avoid building lot of stuff which are either
not provided by C library (say uClibc) or not configured in uClibc etc. It seems
we can't use make -k to keep ignoring errors and continue with whatever can be
built. I'm not too bright with makefiles so perhaps there is something I'm missing.
Do people have ideas or have solved this already with LTP.

Thx,
-Vineet
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20170117/5cf93895/attachment.html>

[LTP] Building LTP despite failures

[Vineet Gupta]

Hi,

We use LTP regularly to check the health of ARC Linux. One the pain points have
been out of tree patches to LTP to avoid building lot of stuff which are either
not provided by C library (say uClibc) or not configured in uClibc etc. It seems
we can't use make -k to keep ignoring errors and continue with whatever can be
built. I'm not too bright with makefiles so perhaps there is something I'm missing.
Do people have ideas or have solved this already with LTP.

Thx,
-Vineet

[LTP] Building LTP despite failures

[Cyril Hrubis]

Hi!
> We use LTP regularly to check the health of ARC Linux. One the pain points have
> been out of tree patches to LTP to avoid building lot of stuff which are either
> not provided by C library (say uClibc) or not configured in uClibc etc.

The obvious answer is here are:

* If there is a functionality not provided by you C library you should
  write a configure check like we do for many testcases already which
  is something that could most likely be then accepted upstream

* There is also some added support for skipping test compilation under
  UCLIBC, see testcases/kernel/syscalls/Makefile that skips profil
  directory if UCLIBC has been defined for build. We may add more if
  there are features not provided by the uClibc library.

-- 
Cyril Hrubis
chrubis@suse.cz

[LTP] Building LTP despite failures

[Vineet Gupta]

On 01/18/2017 02:39 AM, Cyril Hrubis wrote:
> Hi!
>> We use LTP regularly to check the health of ARC Linux. One the pain points have
>> been out of tree patches to LTP to avoid building lot of stuff which are either
>> not provided by C library (say uClibc) or not configured in uClibc etc.
> The obvious answer is here are:
>
> * If there is a functionality not provided by you C library you should
>   write a configure check like we do for many testcases already which
>   is something that could most likely be then accepted upstream
>
> * There is also some added support for skipping test compilation under
>   UCLIBC, see testcases/kernel/syscalls/Makefile that skips profil
>   directory if UCLIBC has been defined for build. We may add more if
>   there are features not provided by the uClibc library.

Indeed, my colleague Yuriy contributed the profil fix.

However above still doesn't help answer my question.
Some things are indeed provided in uClibc but just not configured by default and I
would like to test whatever is currently present and build (make -k style) but
that doesn't seem to be supported with current build system.

There's another issue - we need to test various hardware configs, not all of which
are available in silicon and only in slower FPGA targets (say 50 MHz). I would
like to reduce the number of iterations for some of the tests (e.g. float_exp_log,
hackbench01, fork13 ...). The easy way is to make a copy of corresponding runtest
file but thn the issue is keeping it in sync etc. Is there a approach where we can
optionally pass the reduced iters, based on some runtime env variable (akin to
make flag UCLIBC for affecting build)

Thx,
-Vineet

[LTP] Building LTP despite failures

[Cyril Hrubis]

Hi!
> However above still doesn't help answer my question.
> Some things are indeed provided in uClibc but just not configured by default and I
> would like to test whatever is currently present and build (make -k style) but
> that doesn't seem to be supported with current build system.

Hmm, maybe you can workaround that with ignoring the return value from
the compilation command line. You would have to write implicit pattern
rules to generic_leaf_target.inc and prepend them with hyphen.

Something like (beware untested):

diff --git a/include/mk/generic_leaf_target.inc b/include/mk/generic_leaf_target.inc
index a88586f..1e0d2a3 100644
--- a/include/mk/generic_leaf_target.inc
+++ b/include/mk/generic_leaf_target.inc
@@ -103,4 +103,10 @@ $(INSTALL_FILES): | $(INSTALL_DEPS)
 
 install: $(INSTALL_FILES)
 
+%.o : %.c
+       -$(CC) -c $(CFLAGS) $(CPPFLAGS) $< -o $@
+
+% : %.c
+       -$(CC) $(CFLAGS) $(CPPFLAGS) $(LDFLAGS) $< $(LOADLIBES) $(LDLIBS) -o $@
+
 # vim: syntax=make

> There's another issue - we need to test various hardware configs, not all of which
> are available in silicon and only in slower FPGA targets (say 50 MHz). I would
> like to reduce the number of iterations for some of the tests (e.g. float_exp_log,
> hackbench01, fork13 ...). The easy way is to make a copy of corresponding runtest
> file but thn the issue is keeping it in sync etc. Is there a approach where we can
> optionally pass the reduced iters, based on some runtime env variable (akin to
> make flag UCLIBC for affecting build)

Most of the LTP testcases are as fast as possible, the fork13 for instance does
minimal amount of iterations that was guaranteed to trigger the bug it tests
for. So reducing it does not make much sense since it's close to disabling it
completely.

The best you can do here is to process the runtest files locally to
remove/adjust parameters for this kind of tests.

-- 
Cyril Hrubis
chrubis@suse.cz