All of lore.kernel.org
 help / color / mirror / Atom feed
* [LTP] [PATCH v3] runltp: add -R option for randomize test order
@ 2016-09-24 12:35 Zorro Lang
  2016-09-25  9:38 ` Li Wang
  0 siblings, 1 reply; 4+ messages in thread
From: Zorro Lang @ 2016-09-24 12:35 UTC (permalink / raw)
  To: ltp

Generally we write test entries into runtests/${testfile}, then
runltp reads and runs lines sequentially everytime, nearly no one
will change the test order.

Recently we find bugs by change the test order，different test
combination maybe find more bugs. So I add this new option -R to
randomize test order.

Signed-off-by: Zorro Lang <zlang@redhat.com>
---

V2 use 'sort -R' to instead of 'randlines.awk'.
V3 remove change log and someone useless comment

Thanks,
Zorro

 runltp | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/runltp b/runltp
index bb2e5d8..33e23ab 100755
--- a/runltp
+++ b/runltp
@@ -153,6 +153,7 @@ usage()
     -p              Human readable format logfiles.
     -q              Print less verbose output to screen.
     -r LTPROOT      Fully qualified path where testsuite is installed.
+    -R              Randomize test order.
     -s PATTERN      Only run test cases which match PATTERN.
     -S SKIPFILE     Skip tests specified in SKIPFILE
     -t DURATION     Execute the testsuite for given duration. Examples:
@@ -213,12 +214,13 @@ main()
     local EMAIL_TO=""
     local TAG_RESTRICT_STRING=""
     local PAN_COMMAND=""
+    local RANDOMRUN=0
     local DEFAULT_FILE_NAME_GENERATION_TIME=`date +"%Y_%m_%d-%Hh_%Mm_%Ss"`
     local scenfile=
 
     version_date=$(cat "$LTPROOT/Version")
 
-    while getopts a:c:C:T:d:D:f:F:ehi:I:K:g:l:m:M:Nno:pqr:s:S:t:T:w:x:b:B:z:Z: arg
+    while getopts a:c:C:T:d:D:f:F:ehi:I:K:g:l:m:M:Nno:pqr:Rs:S:t:T:w:x:b:B:z:Z: arg
     do  case $arg in
         a)  EMAIL_TO=$OPTARG
             ALT_EMAIL_OUT=1;;
@@ -410,6 +412,8 @@ main()
 
         r)  LTPROOT=$OPTARG;;
 
+        R)  RANDOMRUN=1;;
+
         s)  TAG_RESTRICT_STRING=$OPTARG;;
 
 	S)  case $OPTARG in
@@ -740,6 +744,11 @@ main()
          done
     fi
 
+    if [ "$RANDOMRUN" != "0" ]; then
+        sort -R ${TMP}/alltests > ${TMP}/alltests.temp
+        cat ${TMP}/alltests.temp > ${TMP}/alltests
+    fi
+
     [ ! -z "$QUIET_MODE" ] && { echo "INFO: Test start time: $(date)" ; }
     PAN_COMMAND="${LTPROOT}/bin/ltp-pan $QUIET_MODE -e -S $INSTANCES $DURATION -a $$ \
     -n $$ $PRETTY_PRT -f ${TMP}/alltests $LOGFILE $OUTPUTFILE $FAILCMDFILE $TCONFCMDFILE"
-- 
2.7.4


^ permalink raw reply related	[flat|nested] 4+ messages in thread
* [LTP] [PATCH v3] runltp: add -R option for randomize test order
  2016-09-24 12:35 [LTP] [PATCH v3] runltp: add -R option for randomize test order Zorro Lang
@ 2016-09-25  9:38 ` Li Wang
  2016-09-25 13:47   ` Zorro Lang
  0 siblings, 1 reply; 4+ messages in thread
From: Li Wang @ 2016-09-25  9:38 UTC (permalink / raw)
  To: ltp

On Sat, Sep 24, 2016 at 8:35 PM, Zorro Lang <zlang@redhat.com> wrote:
> Generally we write test entries into runtests/${testfile}, then
> runltp reads and runs lines sequentially everytime, nearly no one
> will change the test order.
>
> Recently we find bugs by change the test order，different test
> combination maybe find more bugs. So I add this new option -R to
> randomize test order.
>
> Signed-off-by: Zorro Lang <zlang@redhat.com>
> ---
>
> V2 use 'sort -R' to instead of 'randlines.awk'.
> V3 remove change log and someone useless comment
>
> Thanks,
> Zorro
>
>  runltp | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/runltp b/runltp
> index bb2e5d8..33e23ab 100755
> --- a/runltp
> +++ b/runltp
> @@ -153,6 +153,7 @@ usage()
>      -p              Human readable format logfiles.
>      -q              Print less verbose output to screen.
>      -r LTPROOT      Fully qualified path where testsuite is installed.
> +    -R              Randomize test order.
>      -s PATTERN      Only run test cases which match PATTERN.
>      -S SKIPFILE     Skip tests specified in SKIPFILE
>      -t DURATION     Execute the testsuite for given duration. Examples:
> @@ -213,12 +214,13 @@ main()
>      local EMAIL_TO=""
>      local TAG_RESTRICT_STRING=""
>      local PAN_COMMAND=""
> +    local RANDOMRUN=0
>      local DEFAULT_FILE_NAME_GENERATION_TIME=`date +"%Y_%m_%d-%Hh_%Mm_%Ss"`
>      local scenfile=
>
>      version_date=$(cat "$LTPROOT/Version")
>
> -    while getopts a:c:C:T:d:D:f:F:ehi:I:K:g:l:m:M:Nno:pqr:s:S:t:T:w:x:b:B:z:Z: arg
> +    while getopts a:c:C:T:d:D:f:F:ehi:I:K:g:l:m:M:Nno:pqr:Rs:S:t:T:w:x:b:B:z:Z: arg
>      do  case $arg in
>          a)  EMAIL_TO=$OPTARG
>              ALT_EMAIL_OUT=1;;
> @@ -410,6 +412,8 @@ main()
>
>          r)  LTPROOT=$OPTARG;;
>
> +        R)  RANDOMRUN=1;;
> +
>          s)  TAG_RESTRICT_STRING=$OPTARG;;
>
>         S)  case $OPTARG in
> @@ -740,6 +744,11 @@ main()
>           done
>      fi
>
> +    if [ "$RANDOMRUN" != "0" ]; then
> +        sort -R ${TMP}/alltests > ${TMP}/alltests.temp
> +        cat ${TMP}/alltests.temp > ${TMP}/alltests

How about combine them in one line?

  shuf ${TMP}/alltests -o ${TMP}/alltests
or,
  sort -R ${TMP}/alltests -o ${TMP}/alltests

> +    fi
> +
>      [ ! -z "$QUIET_MODE" ] && { echo "INFO: Test start time: $(date)" ; }
>      PAN_COMMAND="${LTPROOT}/bin/ltp-pan $QUIET_MODE -e -S $INSTANCES $DURATION -a $$ \
>      -n $$ $PRETTY_PRT -f ${TMP}/alltests $LOGFILE $OUTPUTFILE $FAILCMDFILE $TCONFCMDFILE"
> --
> 2.7.4
>
>
> --
> Mailing list info: https://lists.linux.it/listinfo/ltp



-- 
Regards,
Li Wang
Email: liwang@redhat.com

^ permalink raw reply	[flat|nested] 4+ messages in thread
* [LTP] [PATCH v3] runltp: add -R option for randomize test order
  2016-09-25  9:38 ` Li Wang
@ 2016-09-25 13:47   ` Zorro Lang
  2016-09-26  8:25     ` Cyril Hrubis
  0 siblings, 1 reply; 4+ messages in thread
From: Zorro Lang @ 2016-09-25 13:47 UTC (permalink / raw)
  To: ltp

On Sun, Sep 25, 2016 at 05:38:07PM +0800, Li Wang wrote:
> On Sat, Sep 24, 2016 at 8:35 PM, Zorro Lang <zlang@redhat.com> wrote:
> > Generally we write test entries into runtests/${testfile}, then
> > runltp reads and runs lines sequentially everytime, nearly no one
> > will change the test order.
> >
> > Recently we find bugs by change the test order，different test
> > combination maybe find more bugs. So I add this new option -R to
> > randomize test order.
> >
> > Signed-off-by: Zorro Lang <zlang@redhat.com>
> > ---
> >
> > V2 use 'sort -R' to instead of 'randlines.awk'.
> > V3 remove change log and someone useless comment
> >
> > Thanks,
> > Zorro
> >
> >  runltp | 11 ++++++++++-
> >  1 file changed, 10 insertions(+), 1 deletion(-)
> >
> > diff --git a/runltp b/runltp
> > index bb2e5d8..33e23ab 100755
> > --- a/runltp
> > +++ b/runltp
> > @@ -153,6 +153,7 @@ usage()
> >      -p              Human readable format logfiles.
> >      -q              Print less verbose output to screen.
> >      -r LTPROOT      Fully qualified path where testsuite is installed.
> > +    -R              Randomize test order.
> >      -s PATTERN      Only run test cases which match PATTERN.
> >      -S SKIPFILE     Skip tests specified in SKIPFILE
> >      -t DURATION     Execute the testsuite for given duration. Examples:
> > @@ -213,12 +214,13 @@ main()
> >      local EMAIL_TO=""
> >      local TAG_RESTRICT_STRING=""
> >      local PAN_COMMAND=""
> > +    local RANDOMRUN=0
> >      local DEFAULT_FILE_NAME_GENERATION_TIME=`date +"%Y_%m_%d-%Hh_%Mm_%Ss"`
> >      local scenfile=
> >
> >      version_date=$(cat "$LTPROOT/Version")
> >
> > -    while getopts a:c:C:T:d:D:f:F:ehi:I:K:g:l:m:M:Nno:pqr:s:S:t:T:w:x:b:B:z:Z: arg
> > +    while getopts a:c:C:T:d:D:f:F:ehi:I:K:g:l:m:M:Nno:pqr:Rs:S:t:T:w:x:b:B:z:Z: arg
> >      do  case $arg in
> >          a)  EMAIL_TO=$OPTARG
> >              ALT_EMAIL_OUT=1;;
> > @@ -410,6 +412,8 @@ main()
> >
> >          r)  LTPROOT=$OPTARG;;
> >
> > +        R)  RANDOMRUN=1;;
> > +
> >          s)  TAG_RESTRICT_STRING=$OPTARG;;
> >
> >         S)  case $OPTARG in
> > @@ -740,6 +744,11 @@ main()
> >           done
> >      fi
> >
> > +    if [ "$RANDOMRUN" != "0" ]; then
> > +        sort -R ${TMP}/alltests > ${TMP}/alltests.temp
> > +        cat ${TMP}/alltests.temp > ${TMP}/alltests
> 
> How about combine them in one line?
> 
>   shuf ${TMP}/alltests -o ${TMP}/alltests
> or,
>   sort -R ${TMP}/alltests -o ${TMP}/alltests

Thanks Li:)

Hm, it works, even works for large file (>pagesize). I thought it maybe make
mistake if read and rewrite same file at same time when I sent this patch.
Now I just read the code of sort.c, it'll read the file into buffer and temp
file at first, then close the fopen(r) file and go to fopen(w) the file:

          if (buf.eof && !nfiles && !ntemps && !buf.left)
            {
              xfclose (fp, file);
              tfp = xfopen (output_file, "w");
	      ...
	      ...

So it looks safe to read and write the same file together ... But I don't know
if it worth changing this patch (the old way looks safer:). Can maintainers
choose a favorite way?

Thanks,
Zorro

> 
> > +    fi
> > +
> >      [ ! -z "$QUIET_MODE" ] && { echo "INFO: Test start time: $(date)" ; }
> >      PAN_COMMAND="${LTPROOT}/bin/ltp-pan $QUIET_MODE -e -S $INSTANCES $DURATION -a $$ \
> >      -n $$ $PRETTY_PRT -f ${TMP}/alltests $LOGFILE $OUTPUTFILE $FAILCMDFILE $TCONFCMDFILE"
> > --
> > 2.7.4
> >
> >
> > --
> > Mailing list info: https://lists.linux.it/listinfo/ltp
> 
> 
> 
> -- 
> Regards,
> Li Wang
> Email: liwang@redhat.com

^ permalink raw reply	[flat|nested] 4+ messages in thread
* [LTP] [PATCH v3] runltp: add -R option for randomize test order
  2016-09-25 13:47   ` Zorro Lang
@ 2016-09-26  8:25     ` Cyril Hrubis
  0 siblings, 0 replies; 4+ messages in thread
From: Cyril Hrubis @ 2016-09-26  8:25 UTC (permalink / raw)
  To: ltp

Hi!
> Hm, it works, even works for large file (>pagesize). I thought it maybe make
> mistake if read and rewrite same file at same time when I sent this patch.
> Now I just read the code of sort.c, it'll read the file into buffer and temp
> file at first, then close the fopen(r) file and go to fopen(w) the file:
> 
>           if (buf.eof && !nfiles && !ntemps && !buf.left)
>             {
>               xfclose (fp, file);
>               tfp = xfopen (output_file, "w");
> 	      ...
> 	      ...
> 
> So it looks safe to read and write the same file together ... But I don't know
> if it worth changing this patch (the old way looks safer:). Can maintainers
> choose a favorite way?

The POSIX specification for sort explicitly states:

-o  output
    Specify the name of an output file to be used instead of the
    standard output. This file can be the same as one of the input
    files.

So I would say that using the same filename for input and output is
pretty much safe.

-- 
Cyril Hrubis
chrubis@suse.cz

^ permalink raw reply	[flat|nested] 4+ messages in thread
end of thread, other threads:[~2016-09-26  8:25 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-09-24 12:35 [LTP] [PATCH v3] runltp: add -R option for randomize test order Zorro Lang
2016-09-25  9:38 ` Li Wang
2016-09-25 13:47   ` Zorro Lang
2016-09-26  8:25     ` Cyril Hrubis

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.