From: Shuah Khan <firstname.lastname@example.org> To: "Bird, Tim" <Tim.Bird@sony.com>, Paolo Bonzini <email@example.com>, "firstname.lastname@example.org" <email@example.com>, "firstname.lastname@example.org" <email@example.com> Cc: "firstname.lastname@example.org" <email@example.com>, "firstname.lastname@example.org" <email@example.com>, Shuah Khan <firstname.lastname@example.org> Subject: Re: [PATCH] kselftest: runner: fix TAP output for skipped tests Date: Wed, 10 Jun 2020 14:02:37 -0600 [thread overview] Message-ID: <email@example.com> (raw) In-Reply-To: <CY4PR13MB1175A17F29B281642DF05A6DFD830@CY4PR13MB1175.namprd13.prod.outlook.com> On 6/10/20 11:43 AM, Bird, Tim wrote: > > >> -----Original Message----- >> From: firstname.lastname@example.org <email@example.com> On Behalf Of Shuah Khan >> >> On 6/10/20 9:44 AM, Paolo Bonzini wrote: >>> According to the TAP specification, a skipped test must be marked as "ok" >>> and annotated with the SKIP directive, for example >>> >>> ok 23 # skip Insufficient flogiston pressure. >>> (https://testanything.org/tap-specification.html) >>> >>> Fix the runner script to match this. >>> >>> Signed-off-by: Paolo Bonzini <firstname.lastname@example.org> >>> --- >>> tools/testing/selftests/kselftest/runner.sh | 2 +- >>> 1 file changed, 1 insertion(+), 1 deletion(-) >>> >>> diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh >>> index 676b3a8b114d..f4815cbcd60f 100644 >>> --- a/tools/testing/selftests/kselftest/runner.sh >>> +++ b/tools/testing/selftests/kselftest/runner.sh >>> @@ -77,7 +77,7 @@ run_one() >>> echo "ok $test_num $TEST_HDR_MSG") || >>> (rc=$?; \ >>> if [ $rc -eq $skip_rc ]; then \ >>> - echo "not ok $test_num $TEST_HDR_MSG # SKIP" >>> + echo "ok $test_num $TEST_HDR_MSG # SKIP" > > This is a pretty big change, and might break upstream CIs that have come to > rely on kselftest's existing behavior. I know it's going to break Fuego's parsing > of results. > Thanks for chiming in. We don't want to break CI workflow. > kselftest has a few conventions that are different from the TAP spec, > and a few items it does that are extensions to the TAP spec. > IMHO, the TAP spec got this one wrong, but I could be convinced > otherwise. But I think we should discuss this among CI users of > kselftest before making the change. > > I started work quite a while ago on an effort to document the > conventions used by kselftest (particularly where it deviates > from the TAP spec), but never submitted it. > > I'm going to submit what I've got as an RFC now, for discussion, > even though it's not finished. I'll do that in a separate thread. > > >>> elif [ $rc -eq $timeout_rc ]; then \ >>> echo "#" >>> echo "not ok $test_num $TEST_HDR_MSG # TIMEOUT" >>> >> >> Thanks. I will pull this in for Linux 5.8-rc2 > Shuah - can you hold off on this until we discuss it? > Of course. Thanks for getting my attention before I pulled it in. thanks, -- Shuah
prev parent reply other threads:[~2020-06-10 20:02 UTC|newest] Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-06-10 15:44 Paolo Bonzini 2020-06-10 16:19 ` Shuah Khan 2020-06-10 17:43 ` Bird, Tim 2020-06-10 18:22 ` Paolo Bonzini 2020-06-15 19:28 ` Bird, Tim 2020-06-10 20:02 ` Shuah Khan [this message]
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --email@example.com \ --firstname.lastname@example.org \ --cc=Tim.Bird@sony.com \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --subject='Re: [PATCH] kselftest: runner: fix TAP output for skipped tests' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).