[PATCH RFC 7/7] selftests/nolibc: proof of concept for TAP output

From: Zhangjin Wu <falcon@tinylab.org>
To: linux@weissschuh.net
Cc: falcon@tinylab.org, linux-kernel@vger.kernel.org,
	linux-kselftest@vger.kernel.org, w@1wt.eu
Subject: [PATCH RFC 7/7] selftests/nolibc: proof of concept for TAP output
Date: Thu, 20 Jul 2023 23:29:40 +0800	[thread overview]
Message-ID: <20230720152940.7623-1-falcon@tinylab.org> (raw)
In-Reply-To: <20230719-nolibc-ktap-tmp-v1-7-930bd0c52ff1@weissschuh.net>

Hi, Thomas

The result looks very similar and the change seems not that big, thanks.

I have a generic question: do we need to compile nolibc-test.c
independently or at least let users easily compile nolibc-test.c in the
other places no just in kernel source code, for example, the other libcs
may want to download and compile it directly.

The functions used in this change seems not many, is it able to provide
our clones for them or only provide the clones when we compile them
out-of-kernel.

for example:

    #ifdef NOLIBC_TEST_IN_KERNEL
    /* -DNOLIBC_TEST_IN_KERNEL from Makefile, for future compatibility */
    #include "../kselftest.h"
    #else
    // our clones of the used functions, for standalone usage
    #endif

Best regards,
Zhangjin

> Dirty proof of concept to show how (K)TAP output can look and how it can
> be used.
> 
> Currently test selection is not supported and for simplicity only the
> startup tests are enabled.
> 
> Example output:
> 
> $ ./nolibc-test
>  KTAP version 1
>  1..15
>  ok 1 argc = 1
>  ok 2 argv_addr = <0x7ffdc66173a8>
>  ok 3 argv_environ = <0x7ffdc66173a8>
>  ok 4 argv_total = 1
>  ok 5 argv0_addr = <0x7ffdc6618bca>
>  ok 6 argv0_str = <0x7ffdc6618bca>
>  ok 7 argv0_len = 13
>  ok 8 environ_addr = <0x7ffdc66173b8>
>  ok 9 environ_envp = <0x7ffdc66173b8>
>  ok 10 environ_auxv = <0x7ffdc66173b8>
>  ok 11 environ_total = 271
>  ok 12 environ_HOME = <0x7ffdc6618cc7>
>  ok 13 auxv_addr = <0x7ffdc66174c8>
>  ok 14 auxv_AT_UID = 1000
>  ok 15 auxv_AT_PAGESZ = 4096
>  # Exiting with status 0
>  # Totals: pass:15 fail:0 xfail:0 xpass:0 skip:0 error:0
> 
> $ ./libc-test
>  KTAP version 1
>  1..15
>  ok 1 argc = 1
>  ok 2 argv_addr = <0x7ffd5f3d43e8>
>  ok 3 argv_environ = <0x7ffd5f3d43e8>
>  ok 4 argv_total = 1
>  ok 5 argv0_addr = <0x7ffd5f3d5bd0>
>  ok 6 argv0_str = <0x7ffd5f3d5bd0>
>  ok 7 argv0_len = 11
>  ok 8 environ_addr = <0x7ffd5f3d43f8>
>  ok 9 environ_envp = <0x7ffd5f3d43f8>
>  ok 10 environ_auxv # SKIP test_auxv != (void *)-1
>  ok 11 environ_total # SKIP test_auxv != (void *)-1
>  ok 12 environ_HOME = <0x7ffd5f3d5ccb>
>  ok 13 auxv_addr # SKIP test_auxv != (void *)-1
>  ok 14 auxv_AT_UID = 1000
>  ok 15 auxv_AT_PAGESZ = 4096
>  # Exiting with status 0
>  # Totals: pass:12 fail:0 xfail:0 xpass:0 skip:3 error:0
> 
> ./run-all-tests.sh | $SRC/tools/testing/kunit/kunit.py parse
>  [23:47:26] ============================================================
>  [23:47:26] ====================== (15 subtests) =======================
>  [23:47:26] [PASSED] argc = 1
>  [23:47:26] [PASSED] argv_addr = <0x7ffcac1b8bc8>
>  [23:47:26] [PASSED] argv_environ = <0x7ffcac1b8bc8>
>  [23:47:26] [PASSED] argv_total = 1
>  [23:47:26] [PASSED] argv0_addr = <0x7ffcac1b9bd0>
>  [23:47:26] [PASSED] argv0_str = <0x7ffcac1b9bd0>
>  [23:47:26] [PASSED] argv0_len = 11
>  [23:47:26] [PASSED] environ_addr = <0x7ffcac1b8bd8>
>  [23:47:26] [PASSED] environ_envp = <0x7ffcac1b8bd8>
>  [23:47:26] [SKIPPED] environ_auxv
>  [23:47:26] [SKIPPED] environ_total
>  [23:47:26] [PASSED] environ_HOME = <0x7ffcac1b9ccb>
>  [23:47:26] [SKIPPED] auxv_addr
>  [23:47:26] [PASSED] auxv_AT_UID = 1000
>  [23:47:26] [PASSED] auxv_AT_PAGESZ = 4096
>  [23:47:26] ====================== [PASSED] arm64 ======================
>  [23:47:26] ====================== (15 subtests) =======================
>  [23:47:26] [PASSED] argc = 1
>  [23:47:26] [PASSED] argv_addr = <0x7ffdee178188>
>  [23:47:26] [PASSED] argv_environ = <0x7ffdee178188>
>  [23:47:26] [PASSED] argv_total = 1
>  [23:47:26] [PASSED] argv0_addr = <0x7ffdee178bd0>
>  [23:47:26] [PASSED] argv0_str = <0x7ffdee178bd0>
>  [23:47:26] [PASSED] argv0_len = 11
>  [23:47:26] [PASSED] environ_addr = <0x7ffdee178198>
>  [23:47:26] [PASSED] environ_envp = <0x7ffdee178198>
>  [23:47:26] [SKIPPED] environ_auxv
>  [23:47:26] [SKIPPED] environ_total
>  [23:47:26] [PASSED] environ_HOME = <0x7ffdee178ccb>
>  [23:47:26] [SKIPPED] auxv_addr
>  [23:47:26] [PASSED] auxv_AT_UID = 1000
>  [23:47:26] [PASSED] auxv_AT_PAGESZ = 4096
>  [23:47:26] ===================== [PASSED] x86_64 ======================
>  [23:47:26] ====================== (15 subtests) =======================
>  [23:47:26] [PASSED] argc = 1
>  [23:47:26] [PASSED] argv_addr = <0x7ffc16bf3628>
>  [23:47:26] [PASSED] argv_environ = <0x7ffc16bf3628>
>  [23:47:26] [PASSED] argv_total = 1
>  [23:47:26] [PASSED] argv0_addr = <0x7ffc16bf4bd0>
>  [23:47:26] [PASSED] argv0_str = <0x7ffc16bf4bd0>
>  [23:47:26] [PASSED] argv0_len = 11
>  [23:47:26] [PASSED] environ_addr = <0x7ffc16bf3638>
>  [23:47:26] [PASSED] environ_envp = <0x7ffc16bf3638>
>  [23:47:26] [SKIPPED] environ_auxv
>  [23:47:26] [SKIPPED] environ_total
>  [23:47:26] [PASSED] environ_HOME = <0x7ffc16bf4ccb>
>  [23:47:26] [SKIPPED] auxv_addr
>  [23:47:26] [PASSED] auxv_AT_UID = 1000
>  [23:47:26] [PASSED] auxv_AT_PAGESZ = 4096
>  [23:47:26] ===================== [PASSED] riscv64 =====================
>  [23:47:26] ============================================================
>  [23:47:26] Testing complete. Ran 45 tests: passed: 36, skipped: 9
> 
> The output of kunit.py is colored after the test results.
> 
> Not-signed-off
> ---
>  tools/testing/selftests/nolibc/nolibc-test.c    | 121 ++++++++----------------
>  tools/testing/selftests/nolibc/run-all-tests.sh |  22 +++++
>  2 files changed, 63 insertions(+), 80 deletions(-)
> 