[Qemu-devel] [PATCH v2] qemu-iotests: Make debugging python tests easier

[Qemu-devel] [PATCH v2] qemu-iotests: Make debugging python tests easier

[Fam Zheng]

Adding "-d" option. The output goes to "tee" so it appears in your
console. Also, raise the verbosity of unnitest runner.

When testing a topic branch, it's possible that a bug introduced by a
code change makes the python test case hang, with debug output, it is
much easier to locate the problem.

This can also be helpful if you want to watch the progress of a python
test, it offers you a way to sense the speed of each test case method
you're writing.

Note: because there is no easy way to get *both* the verbose output and
the output expected by ./check comparison, the case would always fail
with an "output mismatch". The sole purpose of using this option is
giving developers a quick way to debug when things go wrong.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/qemu-iotests/check      | 12 +++++++++---
 tests/qemu-iotests/common     |  6 ++++++
 tests/qemu-iotests/iotests.py | 14 +++++++++++---
 3 files changed, 26 insertions(+), 6 deletions(-)

diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
index baeae80..1fa6319 100755
--- a/tests/qemu-iotests/check
+++ b/tests/qemu-iotests/check
@@ -296,9 +296,15 @@ do
             run_command="./$seq"
         fi
         export OUTPUT_DIR=$PWD
-        (cd "$source_iotests";
-        MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(($RANDOM % 255 + 1))} \
-                $run_command >$tmp.out 2>&1)
+        if $debug; then
+            (cd "$source_iotests";
+            MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(($RANDOM % 255 + 1))} \
+                    $run_command -d 2>&1 | tee $tmp.out)
+        else
+            (cd "$source_iotests";
+            MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(($RANDOM % 255 + 1))} \
+                    $run_command >$tmp.out 2>&1)
+        fi
         sts=$?
         $timestamp && _timestamp
         stop=`_wallclock`
diff --git a/tests/qemu-iotests/common b/tests/qemu-iotests/common
index 1e556bb..1030aaf 100644
--- a/tests/qemu-iotests/common
+++ b/tests/qemu-iotests/common
@@ -32,6 +32,7 @@ check=${check-true}
 
 diff="diff -u"
 verbose=false
+debug=false
 group=false
 xgroup=false
 imgopts=false
@@ -132,6 +133,7 @@ s/ .*//p
 
 common options
     -v                  verbose
+    -d                  debug
 
 check options
     -raw                test raw (default)
@@ -322,6 +324,10 @@ testlist options
             verbose=true
             xpand=false
             ;;
+        -d)
+            debug=true
+            xpand=false
+            ;;
         -x)        # -x group ... exclude from group file
             xgroup=true
             xpand=false
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index e93e623..04a294d 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -338,6 +338,8 @@ def notrun(reason):
 def main(supported_fmts=[], supported_oses=['linux']):
     '''Run tests'''
 
+    debug = '-d' in sys.argv
+    verbosity = 1
     if supported_fmts and (imgfmt not in supported_fmts):
         notrun('not suitable for this image format: %s' % imgfmt)
 
@@ -347,14 +349,20 @@ def main(supported_fmts=[], supported_oses=['linux']):
     # We need to filter out the time taken from the output so that qemu-iotest
     # can reliably diff the results against master output.
     import StringIO
-    output = StringIO.StringIO()
+    if debug:
+        output = sys.stdout
+        verbosity = 2
+        sys.argv.remove('-d')
+    else:
+        output = StringIO.StringIO()
 
     class MyTestRunner(unittest.TextTestRunner):
-        def __init__(self, stream=output, descriptions=True, verbosity=1):
+        def __init__(self, stream=output, descriptions=True, verbosity=verbosity):
             unittest.TextTestRunner.__init__(self, stream, descriptions, verbosity)
 
     # unittest.main() will use sys.exit() so expect a SystemExit exception
     try:
         unittest.main(testRunner=MyTestRunner)
     finally:
-        sys.stderr.write(re.sub(r'Ran (\d+) tests? in [\d.]+s', r'Ran \1 tests', output.getvalue()))
+        if not debug:
+            sys.stderr.write(re.sub(r'Ran (\d+) tests? in [\d.]+s', r'Ran \1 tests', output.getvalue()))
-- 
2.4.0

Re: [Qemu-devel] [Qemu-block] [PATCH v2] qemu-iotests: Make debugging python tests easier

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1150 bytes --]

On Mon, May 18, 2015 at 09:39:12AM +0800, Fam Zheng wrote:
> Adding "-d" option. The output goes to "tee" so it appears in your
> console. Also, raise the verbosity of unnitest runner.
> 
> When testing a topic branch, it's possible that a bug introduced by a
> code change makes the python test case hang, with debug output, it is
> much easier to locate the problem.
> 
> This can also be helpful if you want to watch the progress of a python
> test, it offers you a way to sense the speed of each test case method
> you're writing.
> 
> Note: because there is no easy way to get *both* the verbose output and
> the output expected by ./check comparison, the case would always fail
> with an "output mismatch". The sole purpose of using this option is
> giving developers a quick way to debug when things go wrong.
> 
> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
>  tests/qemu-iotests/check      | 12 +++++++++---
>  tests/qemu-iotests/common     |  6 ++++++
>  tests/qemu-iotests/iotests.py | 14 +++++++++++---
>  3 files changed, 26 insertions(+), 6 deletions(-)

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

[-- Attachment #2: Type: application/pgp-signature, Size: 473 bytes --]

Re: [Qemu-devel] [PATCH v2] qemu-iotests: Make debugging python tests easier

[Kevin Wolf]

Am 18.05.2015 um 03:39 hat Fam Zheng geschrieben:
> Adding "-d" option. The output goes to "tee" so it appears in your
> console. Also, raise the verbosity of unnitest runner.
> 
> When testing a topic branch, it's possible that a bug introduced by a
> code change makes the python test case hang, with debug output, it is
> much easier to locate the problem.
> 
> This can also be helpful if you want to watch the progress of a python
> test, it offers you a way to sense the speed of each test case method
> you're writing.
> 
> Note: because there is no easy way to get *both* the verbose output and
> the output expected by ./check comparison, the case would always fail
> with an "output mismatch". The sole purpose of using this option is
> giving developers a quick way to debug when things go wrong.
> 
> Signed-off-by: Fam Zheng <famz@redhat.com>

Thanks, applied to the block branch.

Next step: Log all qemu-img/qemu-io output and QMP traffic when -d is
specified. :-)

Kevin