Re: [PATCH 1/1] coreutils: add ptest

[Trevor Gamblin <trevor.gamblin@windriver.com>]

On 11/23/19 7:50 AM, Richard Purdie wrote:
> On Thu, 2019-11-21 at 12:43 -0500, Trevor Gamblin wrote:
>> From: Trevor Gamblin <trevor.gamblin@windriver.com>
>>
>> coreutils has a large number of tests, (and potential RDEPENDS
>> to support them), including tests added with the flags
>> RUN_EXPENSIVE_TESTS and RUN_VERY_EXPENSIVE_TESTS that add
>> significant run time for their coverage. The RUN_VERY_EXPENSIVE_TESTS
>> option has been omitted from the run-ptest script to reduce
>> some of this run time, even though this increases the SKIP count
>> in the results. Also, the ptest directory for coreutils is given
>> blanket permissions at runtime with chmod -R 777 to ensure that
>> the user created for the tests will be able to run the test
>> scripts and create the necessary files in the process.
>>
>> There is still room to improve the results of this ptest without
>> the aforementioned additions. Of the tests marked SKIP, there
>> are 30 tests that are currently counted as SKIP because they
>> require root permissions, and another 21 that require membership
>> in multiple user groups. It is important to know that coreutils
>> has tests for both root and non-root users. Testing showed that
>> 42 tests are skipped when running as root versus 30 when running
>> as a non-root user, so the decision was made to run the suite as
>> the latter.
>>
>> Finally, note that the ptest suite for coreutils has a short
>> runtime on x86-64/kvm of approximately 4.5 minutes, in contrast
>> to the arm64 runtime of ~70 minutes.
> Its good to see this coming together thanks. Unfortunately it failed in
> testing.
>
>> Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
>> ---
>>   .../coreutils/coreutils/run-ptest             | 17 +++++++++
>>   meta/recipes-core/coreutils/coreutils_8.31.bb | 38 +++++++++++++++++++
>>   2 files changed, 55 insertions(+)
>>   create mode 100755 meta/recipes-core/coreutils/coreutils/run-ptest
>>
>> diff --git a/meta/recipes-core/coreutils/coreutils/run-ptest b/meta/recipes-core/coreutils/coreutils/run-ptest
>> new file mode 100755
>> index 0000000000..683fedee7a
>> --- /dev/null
>> +++ b/meta/recipes-core/coreutils/coreutils/run-ptest
>> @@ -0,0 +1,17 @@
>> +#!/bin/bash
>> +
>> +COREUTILSLIB=/usr/lib/coreutils
>> +LOG="${COREUTILSLIB}/ptest/coreutils_ptest_$(date +%Y%m%d-%H%M%S).log"
>> +
>> +addgroup usergroup1
>> +adduser --ingroup usergroup1 tester
>> +
>> +su tester -c "make check-TESTS RUN_EXPENSIVE_TESTS=yes top_srcdir=. srcdir=." | tee -a ${LOG}
>> +deluser tester
>> +delgroup usergroup1
>> +
>> +passed=`grep PASS ${LOG}|wc -l`
>> +failed=`grep FAIL ${LOG}|wc -l`
>> +skipped=`grep -E SKIP ${LOG}|wc -l`
>> +all=$((passed + failed + skipped))
>> +
>> diff --git a/meta/recipes-core/coreutils/coreutils_8.31.bb b/meta/recipes-core/coreutils/coreutils_8.31.bb
>> index 57b2c1bdba..5ce2730048 100644
>> --- a/meta/recipes-core/coreutils/coreutils_8.31.bb
>> +++ b/meta/recipes-core/coreutils/coreutils_8.31.bb
>> @@ -143,3 +143,41 @@ python __anonymous() {
>>   }
>>   
>>   BBCLASSEXTEND = "native nativesdk"
>> +
>> +inherit ptest
>> +
>> +FILESEXTRAPATHS_prepend := "${THISDIR}/files:"
>> +
>> +SRC_URI += 	"file://run-ptest"
>
> Parsing recipes...WARNING: /home/pokybuild/yocto-worker/musl-qemux86/build/meta/recipes-core/coreutils/coreutils_8.31.bb: Unable to get checksum for coreutils SRC_URI entry run-ptestfile: file could not be found
>
> I think this is a whitespace problem in SRC_URI where file from another
> url is being added to the string above.
>
>> +RDEPENDS_${PN}-ptest += "bash findutils gawk gdb glibc libconvert-asn1-perl liberror-perl libmodule-build-perl libtimedate-perl liburi-perl make perl perl-module-file-stat python strace"
> Using glibc here breaks musl builds.
>
>
> In the next version could you please also update:
>
> meta/conf/distro/include/ptest-packagelists.inc
>
> (this would be a slow ptest)
Will do - v2 incoming. Thanks for the catch.
> Cheers,
>
> Richard
>