[LTP] [PATCH 1/1] request_key04: Use TFAIL instead of TBROK

[LTP] [PATCH 1/1] request_key04: Use TFAIL instead of TBROK

[Petr Vorel]

To fix unreported results error:
tst_test.c:1036: BROK: Test haven't reported results!

which also prevents to display kernel commit fix and CVE.

Fixes: 8ac26843e ("syscalls/request_key04: new test for request_key()
permission check bug")

Signed-off-by: Petr Vorel <pvorel@suse.cz>
---
Hi,

obvious fix which I'd normally merge without sending to ML.
But there are 19 tests which use tst_res(TBROK, and at least some of
them are affected by this (i.e. don't unconditionally report any result
before tst_res(TBROK), so we might want to fix it different way.

Kind regards,
Petr

 testcases/kernel/syscalls/request_key/request_key04.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/testcases/kernel/syscalls/request_key/request_key04.c b/testcases/kernel/syscalls/request_key/request_key04.c
index da036bf3a..c125f4261 100644
--- a/testcases/kernel/syscalls/request_key/request_key04.c
+++ b/testcases/kernel/syscalls/request_key/request_key04.c
@@ -65,7 +65,7 @@ static void do_test(void)
 	if (TST_ERR == EACCES) {
 		tst_res(TPASS, "request_key() failed with EACCES as expected");
 	} else {
-		tst_res(TBROK | TTERRNO,
+		tst_res(TFAIL | TTERRNO,
 			"request_key() failed with unexpected error code");
 	}
 }
-- 
2.25.1

[LTP] [PATCH 1/1] request_key04: Use TFAIL instead of TBROK

[Li Wang]

On Wed, Feb 26, 2020 at 4:43 PM Petr Vorel <pvorel@suse.cz> wrote:

> To fix unreported results error:
> tst_test.c:1036: BROK: Test haven't reported results!
>
> which also prevents to display kernel commit fix and CVE.
>
> Fixes: 8ac26843e ("syscalls/request_key04: new test for request_key()
> permission check bug")
>
> Signed-off-by: Petr Vorel <pvorel@suse.cz>
> ---
> Hi,
>
> obvious fix which I'd normally merge without sending to ML.
> But there are 19 tests which use tst_res(TBROK, and at least some of
> them are affected by this (i.e. don't unconditionally report any result
> before tst_res(TBROK), so we might want to fix it different way.
>
+1 for this fix.

By the way, do you think it makes sense to add build check parameters for
tst_res() too? Just like what we do for tst_brk() in commit 0738e3753c51.

Any thoughts?



>
> Kind regards,
> Petr
>
>  testcases/kernel/syscalls/request_key/request_key04.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/testcases/kernel/syscalls/request_key/request_key04.c
> b/testcases/kernel/syscalls/request_key/request_key04.c
> index da036bf3a..c125f4261 100644
> --- a/testcases/kernel/syscalls/request_key/request_key04.c
> +++ b/testcases/kernel/syscalls/request_key/request_key04.c
> @@ -65,7 +65,7 @@ static void do_test(void)
>         if (TST_ERR == EACCES) {
>                 tst_res(TPASS, "request_key() failed with EACCES as
> expected");
>         } else {
> -               tst_res(TBROK | TTERRNO,
> +               tst_res(TFAIL | TTERRNO,
>                         "request_key() failed with unexpected error code");
>         }
>  }
> --
> 2.25.1
>
>

-- 
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20200226/2c786714/attachment.htm>

[LTP] [PATCH 1/1] request_key04: Use TFAIL instead of TBROK

[Petr Vorel]

Hi Li,

...
> > obvious fix which I'd normally merge without sending to ML.
> > But there are 19 tests which use tst_res(TBROK, and at least some of
> > them are affected by this (i.e. don't unconditionally report any result
> > before tst_res(TBROK), so we might want to fix it different way.

> +1 for this fix.
Thanks for your review.

> By the way, do you think it makes sense to add build check parameters for
> tst_res() too? Just like what we do for tst_brk() in commit 0738e3753c51.

IMHO if we agree that tst_res(TBROK) is harmful and should be avoided entirely,
I'd be for adding this check.

> Any thoughts?

Kind regards,
Petr

[LTP] [PATCH 1/1] request_key04: Use TFAIL instead of TBROK

[Jan Stancek]

----- Original Message -----
> To fix unreported results error:
> tst_test.c:1036: BROK: Test haven't reported results!
> 
> which also prevents to display kernel commit fix and CVE.
> 
> Fixes: 8ac26843e ("syscalls/request_key04: new test for request_key()
> permission check bug")
> 
> Signed-off-by: Petr Vorel <pvorel@suse.cz>
> ---
> Hi,
> 
> obvious fix which I'd normally merge without sending to ML.
> But there are 19 tests which use tst_res(TBROK, and at least some of
> them are affected by this (i.e. don't unconditionally report any result
> before tst_res(TBROK), so we might want to fix it different way.
> 
> Kind regards,
> Petr
> 
>  testcases/kernel/syscalls/request_key/request_key04.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/testcases/kernel/syscalls/request_key/request_key04.c
> b/testcases/kernel/syscalls/request_key/request_key04.c
> index da036bf3a..c125f4261 100644
> --- a/testcases/kernel/syscalls/request_key/request_key04.c
> +++ b/testcases/kernel/syscalls/request_key/request_key04.c
> @@ -65,7 +65,7 @@ static void do_test(void)
>  	if (TST_ERR == EACCES) {
>  		tst_res(TPASS, "request_key() failed with EACCES as expected");
>  	} else {
> -		tst_res(TBROK | TTERRNO,
> +		tst_res(TFAIL | TTERRNO,
>  			"request_key() failed with unexpected error code");
>  	}
>  }
> --
> 2.25.1

Ack

[LTP] [PATCH 1/1] request_key04: Use TFAIL instead of TBROK

[Li Wang]

On Wed, Feb 26, 2020 at 5:18 PM Petr Vorel <pvorel@suse.cz> wrote:

> Hi Li,
>
> ...
> > > obvious fix which I'd normally merge without sending to ML.
> > > But there are 19 tests which use tst_res(TBROK, and at least some of
> > > them are affected by this (i.e. don't unconditionally report any result
> > > before tst_res(TBROK), so we might want to fix it different way.
>
> > +1 for this fix.
> Thanks for your review.
>
> > By the way, do you think it makes sense to add build check parameters for
> > tst_res() too? Just like what we do for tst_brk() in commit 0738e3753c51.
>
> IMHO if we agree that tst_res(TBROK) is harmful and should be avoided
> entirely,
>
I agree.


> I'd be for adding this check.
>

In personally, I'd let tst_res() only accept 'TPASS, TFAIL, TINFO, TCONF,
TWARN'.

-- 
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20200226/36e6d613/attachment.htm>

[LTP] [PATCH 1/1] request_key04: Use TFAIL instead of TBROK

[Petr Vorel]

Hi Li,

> > ...
> > > > obvious fix which I'd normally merge without sending to ML.
> > > > But there are 19 tests which use tst_res(TBROK, and at least some of
> > > > them are affected by this (i.e. don't unconditionally report any result
> > > > before tst_res(TBROK), so we might want to fix it different way.

> > > +1 for this fix.
> > Thanks for your review.

> > > By the way, do you think it makes sense to add build check parameters for
> > > tst_res() too? Just like what we do for tst_brk() in commit 0738e3753c51.

> > IMHO if we agree that tst_res(TBROK) is harmful and should be avoided
> > entirely,

> I agree.

> > I'd be for adding this check.


> In personally, I'd let tst_res() only accept 'TPASS, TFAIL, TINFO, TCONF,
> TWARN'.
+1. Cyril, Jan?

Kind regards,
Petr

[LTP] [PATCH 1/1] request_key04: Use TFAIL instead of TBROK

[Xiao Yang]

On 2020/2/26 19:37, Petr Vorel wrote:
> Hi Li,
>
>>> ...
>>>>> obvious fix which I'd normally merge without sending to ML.
>>>>> But there are 19 tests which use tst_res(TBROK, and at least some of
>>>>> them are affected by this (i.e. don't unconditionally report any result
>>>>> before tst_res(TBROK), so we might want to fix it different way.
>>>> +1 for this fix.
>>> Thanks for your review.
>>>> By the way, do you think it makes sense to add build check parameters for
>>>> tst_res() too? Just like what we do for tst_brk() in commit 0738e3753c51.
>>> IMHO if we agree that tst_res(TBROK) is harmful and should be avoided
>>> entirely,
>> I agree.
>>> I'd be for adding this check.
>
>> In personally, I'd let tst_res() only accept 'TPASS, TFAIL, TINFO, TCONF,
>> TWARN'.
> +1. Cyril, Jan?
Hi Petr,

It is OK for me to add this check. :-)

Thanks,
Xiao Yang
> Kind regards,
> Petr
>

[LTP] [PATCH 1/1] request_key04: Use TFAIL instead of TBROK

[Cyril Hrubis]

Hi!
> > In personally, I'd let tst_res() only accept 'TPASS, TFAIL, TINFO, TCONF,
> > TWARN'.
> +1. Cyril, Jan?

I guess that it does not make much sense to report TBROK and then
continue the test. But that all depends on how exactly are these states
defined. Have we ever wrote them down?

I guess that TPASS and TFAIL are obvious.

TINFO is just additional information.

TCONF is test skipped.

TWARN something went wrong but we decided to continue?

TBROK something went wrong and we decided to exit?

-- 
Cyril Hrubis
chrubis@suse.cz

[LTP] [PATCH 1/1] request_key04: Use TFAIL instead of TBROK

[Jan Stancek]

----- Original Message -----
> Hi!
> > > In personally, I'd let tst_res() only accept 'TPASS, TFAIL, TINFO, TCONF,
> > > TWARN'.
> > +1. Cyril, Jan?
> 
> I guess that it does not make much sense to report TBROK and then
> continue the test. But that all depends on how exactly are these states
> defined. Have we ever wrote them down?
> 
> I guess that TPASS and TFAIL are obvious.
> 
> TINFO is just additional information.
> 
> TCONF is test skipped.
> 
> TWARN something went wrong but we decided to continue?

This appears to be mostly used in cleanup() when something goes wrong.
So a failure that's not critical, not related to goal of test,
but still something we want to draw attention to.

> 
> TBROK something went wrong and we decided to exit?

We do have following in style-guide.txt:

Use +TBROK+ when an unexpected failure unrelated to the goal of the testcase
occurred, and use +TFAIL+ when an unexpected failure related to the goal of
the testcase occurred.

I agree that tst_res with TBROK doesn't make much sense. I see TBROK as
something we can't recover from and need to end the test. Otherwise
if we want to skip part of test, then TCONF looks more fitting.

[LTP] [PATCH 1/1] request_key04: Use TFAIL instead of TBROK

[Petr Vorel]

Hi,

> > > > In personally, I'd let tst_res() only accept 'TPASS, TFAIL, TINFO, TCONF,
> > > > TWARN'.
> > > +1. Cyril, Jan?

> > I guess that it does not make much sense to report TBROK and then
> > continue the test. But that all depends on how exactly are these states
> > defined. Have we ever wrote them down?

> > I guess that TPASS and TFAIL are obvious.

> > TINFO is just additional information.

> > TCONF is test skipped.

> > TWARN something went wrong but we decided to continue?
+1

> This appears to be mostly used in cleanup() when something goes wrong.
> So a failure that's not critical, not related to goal of test,
> but still something we want to draw attention to.


> > TBROK something went wrong and we decided to exit?

> We do have following in style-guide.txt:

> Use +TBROK+ when an unexpected failure unrelated to the goal of the testcase
> occurred, and use +TFAIL+ when an unexpected failure related to the goal of
> the testcase occurred.

> I agree that tst_res with TBROK doesn't make much sense. I see TBROK as
> something we can't recover from and need to end the test. Otherwise
> if we want to skip part of test, then TCONF looks more fitting.

Thanks all for their input.
I'll merge this test with your ack and create macro + fix remaining tests
(sent patchset to ML).

Kind regards,
Petr