Re: [Fuego] Trả lời: [fuego-core 6/7] LTP: execute all tests when spec specifies "all"

[<Tim.Bird@sony.com>]

The patch looks good. Thanks for the changes.
But I can't figure out how to apply it.

The patch content is encoded in base64, and when I tried to manually 
decode it at the command line, I got an error "invalid input".

Can you please re-send this patch as an attachment instead of inline
in the message body?  Something is going wrong with the mail
handling of the text.
 -- Tim

> -----Original Message-----
> From: tho1.nguyendat@toshiba.co.jp <tho1.nguyendat@toshiba.co.jp>
> Dear Tim,
> 
> I would like to send my update.
> 
> From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
> 
> 
> Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> 
> 
> ---
>  tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++-------------
>  tests/Functional.LTP/spec.json     |  6 +++++
>  2 files changed, 30 insertions(+), 18 deletions(-)
> 
> diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh
> index 85adabf..689abfe 100755
> --- a/tests/Functional.LTP/fuego_test.sh
> +++ b/tests/Functional.LTP/fuego_test.sh
> @@ -414,25 +414,31 @@ function test_run {
>          # on the type of the test (regular, posize, or realtime)
>          # Separate the test list by type, and pass the tests in different
>          # variables
> -        for a in $FUNCTIONAL_LTP_TESTS; do
> -            for b in $ALLTESTS; do
> -                if [ "$a" == "$b" ]; then
> -                    TESTS+="$a "
> -                fi
> -            done
> -
> -            for b in $ALLPTSTESTS; do
> -                if [ "$a" == "$b" ]; then
> -                    PTSTESTS+="$a "
> -                fi
> -            done
> -
> -            for b in $ALLRTTESTS; do
> -                if [ "$a" == "$b" ]; then
> -                    RTTESTS+="$a "
> -                fi
> +        if [ "$FUNCTIONAL_LTP_TESTS" == "all" ]; then
> +            TESTS=$ALLTESTS
> +            PTSTESTS=$ALLPTSTESTS
> +            RTTESTS=$ALLRTTESTS
> +        else
> +            for a in $FUNCTIONAL_LTP_TESTS; do
> +                for b in $ALLTESTS; do
> +                    if [ "$a" == "$b" ]; then
> +                        TESTS+="$a "
> +                    fi
> +                done
> +
> +                for b in $ALLPTSTESTS; do
> +                    if [ "$a" == "$b" ]; then
> +                        PTSTESTS+="$a "
> +                    fi
> +                done
> +
> +                for b in $ALLRTTESTS; do
> +                    if [ "$a" == "$b" ]; then
> +                        RTTESTS+="$a "
> +                    fi
> +                done
>              done
> -        done
> +        fi
> 
>          # Let some of the tests fail, the information will be in the result xlsx file
>          report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\";
> ./ltp_target_run.sh"
> diff --git a/tests/Functional.LTP/spec.json b/tests/Functional.LTP/spec.json
> index d30ba4c..53ee115 100644
> --- a/tests/Functional.LTP/spec.json
> +++ b/tests/Functional.LTP/spec.json
> @@ -1,6 +1,12 @@
>  {
>      "testName": "Functional.LTP",
>      "specs": {
> +        "all": {
> +            "tests": "all",
> +            "noautoskip": "true",
> +            "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"},
> +            "extra_fail_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"}
> +        },
>          "default": {
>              "tests": "syscalls SEM",
>              "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"},
> --
> 2.20.1
> 
> ________________________________
> 
> Từ: Tim.Bird@sony.com <Tim.Bird@sony.com>
> Đã gửi: 23 Tháng Bảy 2021 2:03 SA
> Đến: pyla venkata(ＴＳＩＰ) <Venkata.Pyla@toshiba-tsip.com>
> Cc: nguyen dat tho(ＴＳＤＶ Ｅｎｇ　１) <tho1.nguyendat@toshiba.co.jp>; sangorrin daniel(サンゴリン ダニエル □ＳＷＣ◯ＡＣＴ)
> <daniel.sangorrin@toshiba.co.jp>; fuego@lists.linuxfoundation.org <fuego@lists.linuxfoundation.org>
> Chủ đề: RE: [fuego-core 6/7] LTP: execute all tests when spec specifies "all"
> 
> This one has problems.
> 
> > -----Original Message-----
> > From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com>
> > Subject: [fuego-core 6/7] LTP: execute all tests when spec specifies "all"
> >
> I'm not sure that the summary line of this change matches what it does.
> 
> From reading the code, it appears that "tests" in the spec needs to be
> missing or empty, in order to activate this.  If "tests" is "all",
> then it appears that none of the tests will match, and you'd just
> end up with empty lists.
> 
> Also, I had to intuit the reasoning here, since there is no description
> for this.  I don't mind the concept of being able to easily run all
> LTP tests, but it would be nice to use the string "all" to signal
> this instead of just leaving FUNCTIONAL_LTP_TESTS empty.
> 
> It should probably go into a comment in the code as well, and maybe
> in the test.yaml file, so that users and future developers will see
> it and understand the behaviour.
> 
> Also, you might want to add a spec to the spec.json file for "all", to
> show it's usage.
> 
> > From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
> >
> > Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
> > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> > ---
> >  tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++-------------
> >  1 file changed, 24 insertions(+), 18 deletions(-)
> >
> > diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh
> > index e620b17..16d556f 100755
> > --- a/tests/Functional.LTP/fuego_test.sh
> > +++ b/tests/Functional.LTP/fuego_test.sh
> > @@ -188,7 +188,6 @@ function test_pre_check {
> >      assert_define AR
> >      assert_define RANLIB
> >      #assert_define LDFLAGS
> > -    assert_define FUNCTIONAL_LTP_TESTS
> >
> >      # FIXTHIS: use regex for selecting tests to skip once merged on LTP upstream
> >      echo "Tests skipped by default in Fuego for now"
> > @@ -453,25 +452,32 @@ function test_run {
> >          # on the type of the test (regular, posize, or realtime)
> >          # Separate the test list by type, and pass the tests in different
> >          # variables
> > -        for a in $FUNCTIONAL_LTP_TESTS; do
> > -            for b in $ALLTESTS; do
> > -                if [ "$a" == "$b" ]; then
> > -                    TESTS+="$a "
> > -                fi
> > -            done
> > -
> > -            for b in $ALLPTSTESTS; do
> > -                if [ "$a" == "$b" ]; then
> > -                    PTSTESTS+="$a "
> > -                fi
> > -            done
> >
> > -            for b in $ALLRTTESTS; do
> > -                if [ "$a" == "$b" ]; then
> > -                    RTTESTS+="$a "
> > -                fi
> > +        if [ -n "$FUNCTIONAL_LTP_TESTS" ]; then
> > +            for a in $FUNCTIONAL_LTP_TESTS; do
> > +                for b in $ALLTESTS; do
> > +                    if [ "$a" == "$b" ]; then
> > +                        TESTS+="$a "
> > +                    fi
> > +                done
> > +
> > +                for b in $ALLPTSTESTS; do
> > +                    if [ "$a" == "$b" ]; then
> > +                        PTSTESTS+="$a "
> > +                    fi
> > +                done
> > +
> > +                for b in $ALLRTTESTS; do
> > +                    if [ "$a" == "$b" ]; then
> > +                        RTTESTS+="$a "
> > +                    fi
> > +                done
> >              done
> > -        done
> > +        else
> > +            TESTS=$ALLTESTS
> > +            PTSTESTS=$ALLPTSTESTS
> > +            RTTESTS=$ALLRTTESTS
> > +        fi
> >
> >          # Let some of the tests fail, the information will be in the result xlsx file
> >          report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\";
> > ./ltp_target_run.sh"
> > --
> > 2.20.1
> >
> 
> I have NOT applied this one.  Please change this to check for the string "all"
> in FUNCTIONAL_LTP_TESTS (and please submit a sample spec to go with this).
> 
> Thanks,
>  -- Tim
>