[PATCH v5 00/16] qemu_iotests: improve debugging options

[PATCH v5 00/16] qemu_iotests: improve debugging options

[Emanuele Giuseppe Esposito]

This series adds the option to attach gdbserver and valgrind
to the QEMU binary running in qemu_iotests.
It also allows to redirect QEMU binaries output of the python tests
to the stdout, instead of a log file.

Patches 1-9 introduce the -gdb option to both python and bash tests, 
10-14 extend the already existing -valgrind flag to work also on 
python tests, and patch 15-16 introduces -p to enable logging to stdout.

In particular, patches 1,6,11 focus on extending the QMP socket timers
when using gdb/valgrind, otherwise the python tests will fail due to
delays in the QMP responses.

This series is tested on the previous serie
"qemu-iotests: quality of life improvements"
but independent from it, so it can be applied separately.

It is also based on "iotests/check: move general long options to double dash"
by Vladimir Sementsov-Ogievskiy.

Based-on: <20210526181659.365531-1-vsementsov@virtuozzo.com>

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
---
v5:
* base this serie on the double dash options, so define --gdb instead of -gdb
* return error if -gdb and -valgrind are used together [Vladimir]
* style changes (do not use subprocess_check_valgrind() but do the operation
  inside the caller) [Vladimir]
* minor fixes in commit messages [Vladimir]
* create a common public function in patch 15 instead of using
  _private fields [Vladimir]
* use @contextmanager in patch 6 to skip gdb timeouts involving the
  Timeout class [Vladimir]
* apply John Snow patch to silence pylint warning in the CI tests

Emanuele Giuseppe Esposito (15):
  python: qemu: add timer parameter for qmp.accept socket
  python: qemu: pass the wrapper field from QEMUQtestmachine to
    QEMUMachine
  docs/devel/testing: add debug section to the QEMU iotests chapter
  qemu-iotests: add option to attach gdbserver
  qemu-iotests: delay QMP socket timers
  qemu_iotests: insert gdbserver command line as wrapper for qemu binary
  qemu-iotests: add gdbserver option to script tests too
  docs/devel/testing: add --gdb option to the debugging section of QEMU
    iotests
  qemu-iotests: extend the check script to prepare supporting valgrind
    for python tests
  qemu-iotests: extend QMP socket timeout when using valgrind
  qemu-iotests: allow valgrind to read/delete the generated log file
  qemu-iotests: insert valgrind command line as wrapper for qemu binary
  docs/devel/testing: add --valgrind option to the debug section of QEMU
    iotests
  qemu-iotests: add option to show qemu binary logs on stdout
  docs/devel/testing: add -p option to the debug section of QEMU iotests

John Snow (1):
  python: Reduce strictness of pylint's duplicate-code check

 docs/devel/testing.rst         | 30 ++++++++++++++++++++
 python/qemu/machine/machine.py | 16 +++++++----
 python/qemu/machine/qtest.py   |  9 ++++--
 python/setup.cfg               |  5 ++++
 tests/qemu-iotests/check       | 15 +++++++---
 tests/qemu-iotests/common.rc   |  8 +++++-
 tests/qemu-iotests/iotests.py  | 52 ++++++++++++++++++++++++++++++++--
 tests/qemu-iotests/testenv.py  | 23 +++++++++++++--
 8 files changed, 141 insertions(+), 17 deletions(-)

-- 
2.30.2

[PATCH v5 01/16] python: qemu: add timer parameter for qmp.accept socket

[Emanuele Giuseppe Esposito]

Also add a new _qmp_timer field to the QEMUMachine class.

Let's change the default socket timeout to None, so that if
a subclass needs to add a timer, it can be done by modifying
this private field.

At the same time, restore the timer to be 15 seconds in iotests.py, to
give an upper bound to the QMP monitor test command execution.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 python/qemu/machine/machine.py | 7 +++++--
 python/qemu/machine/qtest.py   | 5 +++--
 tests/qemu-iotests/iotests.py  | 3 ++-
 3 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py
index b62435528e..fdf2fc0e9c 100644
--- a/python/qemu/machine/machine.py
+++ b/python/qemu/machine/machine.py
@@ -95,7 +95,8 @@ def __init__(self,
                  socket_scm_helper: Optional[str] = None,
                  sock_dir: Optional[str] = None,
                  drain_console: bool = False,
-                 console_log: Optional[str] = None):
+                 console_log: Optional[str] = None,
+                 qmp_timer: Optional[float] = None):
         '''
         Initialize a QEMUMachine
 
@@ -109,6 +110,7 @@ def __init__(self,
         @param sock_dir: where to create socket (defaults to base_temp_dir)
         @param drain_console: (optional) True to drain console socket to buffer
         @param console_log: (optional) path to console log file
+        @param qmp_timer: (optional) default QMP socket timeout
         @note: Qemu process is not started until launch() is used.
         '''
         # Direct user configuration
@@ -116,6 +118,7 @@ def __init__(self,
         self._binary = binary
         self._args = list(args)
         self._wrapper = wrapper
+        self._qmp_timer = qmp_timer
 
         self._name = name or "qemu-%d" % os.getpid()
         self._base_temp_dir = base_temp_dir
@@ -333,7 +336,7 @@ def _pre_launch(self) -> None:
 
     def _post_launch(self) -> None:
         if self._qmp_connection:
-            self._qmp.accept()
+            self._qmp.accept(self._qmp_timer)
 
     def _post_shutdown(self) -> None:
         """
diff --git a/python/qemu/machine/qtest.py b/python/qemu/machine/qtest.py
index 93700684d1..33a86a9d69 100644
--- a/python/qemu/machine/qtest.py
+++ b/python/qemu/machine/qtest.py
@@ -115,14 +115,15 @@ def __init__(self,
                  name: Optional[str] = None,
                  base_temp_dir: str = "/var/tmp",
                  socket_scm_helper: Optional[str] = None,
-                 sock_dir: Optional[str] = None):
+                 sock_dir: Optional[str] = None,
+                 qmp_timer: Optional[float] = None):
         if name is None:
             name = "qemu-%d" % os.getpid()
         if sock_dir is None:
             sock_dir = base_temp_dir
         super().__init__(binary, args, name=name, base_temp_dir=base_temp_dir,
                          socket_scm_helper=socket_scm_helper,
-                         sock_dir=sock_dir)
+                         sock_dir=sock_dir, qmp_timer=qmp_timer)
         self._qtest: Optional[QEMUQtestProtocol] = None
         self._qtest_path = os.path.join(sock_dir, name + "-qtest.sock")
 
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 89663dac06..6b0db4ce54 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -570,10 +570,11 @@ class VM(qtest.QEMUQtestMachine):
 
     def __init__(self, path_suffix=''):
         name = "qemu%s-%d" % (path_suffix, os.getpid())
+        timer = 15.0
         super().__init__(qemu_prog, qemu_opts, name=name,
                          base_temp_dir=test_dir,
                          socket_scm_helper=socket_scm_helper,
-                         sock_dir=sock_dir)
+                         sock_dir=sock_dir, qmp_timer=timer)
         self._num_drives = 0
 
     def add_object(self, opts):
-- 
2.30.2

[PATCH v5 02/16] python: Reduce strictness of pylint's duplicate-code check

[Emanuele Giuseppe Esposito]

From: John Snow <jsnow@redhat.com>

Pylint prior to 2.8.3 (We pin at >= 2.8.0) includes function and method
signatures as part of its duplicate checking algorithm. This check does
not listen to pragmas, so the only way to disable it is to turn it off
completely or increase the minimum duplicate lines so that it doesn't
trigger for functions with long, multi-line signatures.

When we decide to upgrade to pylint 2.8.3 or greater, we will be able to
use 'ignore-signatures = true' to the config instead.

I'd prefer not to keep us on the very bleeding edge of pylint if I can
help it -- 2.8.3 came out only three days ago at time of writing.

See: https://github.com/PyCQA/pylint/pull/4474
Signed-off-by: John Snow <jsnow@redhat.com>
---
 python/setup.cfg | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/python/setup.cfg b/python/setup.cfg
index 0fcdec6f32..d82c39aa46 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -78,6 +78,11 @@ good-names=i,
 # Ignore imports when computing similarities.
 ignore-imports=yes
 
+# Minimum lines number of a similarity.
+# TODO: Remove after we opt in to Pylint 2.8.3. See commit msg.
+min-similarity-lines=6
+
+
 [isort]
 force_grid_wrap=4
 force_sort_within_sections=True
-- 
2.30.2

Re: [PATCH v5 02/16] python: Reduce strictness of pylint's duplicate-code check

[Vladimir Sementsov-Ogievskiy]

04.06.2021 12:17, Emanuele Giuseppe Esposito wrote:
> From: John Snow<jsnow@redhat.com>
> 
> Pylint prior to 2.8.3 (We pin at >= 2.8.0) includes function and method
> signatures as part of its duplicate checking algorithm. This check does
> not listen to pragmas, so the only way to disable it is to turn it off
> completely or increase the minimum duplicate lines so that it doesn't
> trigger for functions with long, multi-line signatures.
> 
> When we decide to upgrade to pylint 2.8.3 or greater, we will be able to
> use 'ignore-signatures = true' to the config instead.
> 
> I'd prefer not to keep us on the very bleeding edge of pylint if I can
> help it -- 2.8.3 came out only three days ago at time of writing.
> 
> See:https://github.com/PyCQA/pylint/pull/4474
> Signed-off-by: John Snow<jsnow@redhat.com>

Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

-- 
Best regards,
Vladimir

[PATCH v5 03/16] python: qemu: pass the wrapper field from QEMUQtestmachine to QEMUMachine

[Emanuele Giuseppe Esposito]

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
---
 python/qemu/machine/qtest.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/python/qemu/machine/qtest.py b/python/qemu/machine/qtest.py
index 33a86a9d69..dc2b5ccfb1 100644
--- a/python/qemu/machine/qtest.py
+++ b/python/qemu/machine/qtest.py
@@ -112,6 +112,7 @@ class QEMUQtestMachine(QEMUMachine):
     def __init__(self,
                  binary: str,
                  args: Sequence[str] = (),
+                 wrapper: Sequence[str] = (),
                  name: Optional[str] = None,
                  base_temp_dir: str = "/var/tmp",
                  socket_scm_helper: Optional[str] = None,
@@ -121,7 +122,8 @@ def __init__(self,
             name = "qemu-%d" % os.getpid()
         if sock_dir is None:
             sock_dir = base_temp_dir
-        super().__init__(binary, args, name=name, base_temp_dir=base_temp_dir,
+        super().__init__(binary, args, wrapper=wrapper, name=name,
+                         base_temp_dir=base_temp_dir,
                          socket_scm_helper=socket_scm_helper,
                          sock_dir=sock_dir, qmp_timer=qmp_timer)
         self._qtest: Optional[QEMUQtestProtocol] = None
-- 
2.30.2

[PATCH v5 04/16] docs/devel/testing: add debug section to the QEMU iotests chapter

[Emanuele Giuseppe Esposito]

Introduce the "Debugging a test case" section, in preparation
to the additional flags that will be added in the next patches.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 docs/devel/testing.rst | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 4e42392810..9d6a8f8636 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -224,6 +224,14 @@ another application on the host may have locked the file, possibly leading to a
 test failure.  If using such devices are explicitly desired, consider adding
 ``locking=off`` option to disable image locking.
 
+Debugging a test case
+-----------------------
+The following options to the ``check`` script can be useful when debugging
+a failing test:
+
+* ``-d`` (debug) just increases the logging verbosity, showing
+  for example the QMP commands and answers.
+
 Test case groups
 ----------------
 
-- 
2.30.2

[PATCH v5 05/16] qemu-iotests: add option to attach gdbserver

[Emanuele Giuseppe Esposito]

Define --gdb flag and GDB_OPTIONS environment variable
to python tests to attach a gdbserver to each qemu instance.
This patch only adds and parses this flag, it does not yet add
the implementation for it.

if --gdb is not provided but $GDB_OPTIONS is set, ignore the
environment variable.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 tests/qemu-iotests/check      |  6 +++++-
 tests/qemu-iotests/iotests.py |  5 +++++
 tests/qemu-iotests/testenv.py | 17 +++++++++++++++--
 3 files changed, 25 insertions(+), 3 deletions(-)

diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
index 3f3962dd75..1dba4218c0 100755
--- a/tests/qemu-iotests/check
+++ b/tests/qemu-iotests/check
@@ -36,6 +36,9 @@ def make_argparser() -> argparse.ArgumentParser:
                    help='pretty print output for make check')
 
     p.add_argument('-d', dest='debug', action='store_true', help='debug')
+    p.add_argument('--gdb', action='store_true',
+                   help="start gdbserver with $GDB_OPTIONS options \
+                        ('localhost:12345' if $GDB_OPTIONS is empty)")
     p.add_argument('--misalign', action='store_true',
                    help='misalign memory allocations')
     p.add_argument('--color', choices=['on', 'off', 'auto'],
@@ -114,7 +117,8 @@ if __name__ == '__main__':
     env = TestEnv(imgfmt=args.imgfmt, imgproto=args.imgproto,
                   aiomode=args.aiomode, cachemode=args.cachemode,
                   imgopts=args.imgopts, misalign=args.misalign,
-                  debug=args.debug, valgrind=args.valgrind)
+                  debug=args.debug, valgrind=args.valgrind,
+                  gdb=args.gdb)
 
     if len(sys.argv) > 1 and sys.argv[-len(args.tests)-1] == '--':
         if not args.tests:
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 6b0db4ce54..c86f239d81 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -74,6 +74,11 @@
 qemu_prog = os.environ.get('QEMU_PROG', 'qemu')
 qemu_opts = os.environ.get('QEMU_OPTIONS', '').strip().split(' ')
 
+gdb_qemu_env = os.environ.get('GDB_OPTIONS')
+qemu_gdb = []
+if gdb_qemu_env:
+    qemu_gdb = ['gdbserver'] + gdb_qemu_env.strip().split(' ')
+
 imgfmt = os.environ.get('IMGFMT', 'raw')
 imgproto = os.environ.get('IMGPROTO', 'file')
 output_dir = os.environ.get('OUTPUT_DIR', '.')
diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py
index 0c3fe75636..8501c6caf5 100644
--- a/tests/qemu-iotests/testenv.py
+++ b/tests/qemu-iotests/testenv.py
@@ -27,6 +27,7 @@
 import glob
 from typing import List, Dict, Any, Optional, ContextManager
 
+DEF_GDB_OPTIONS = 'localhost:12345'
 
 def isxfile(path: str) -> bool:
     return os.path.isfile(path) and os.access(path, os.X_OK)
@@ -72,7 +73,8 @@ class TestEnv(ContextManager['TestEnv']):
                      'QEMU_NBD_OPTIONS', 'IMGOPTS', 'IMGFMT', 'IMGPROTO',
                      'AIOMODE', 'CACHEMODE', 'VALGRIND_QEMU',
                      'CACHEMODE_IS_DEFAULT', 'IMGFMT_GENERIC', 'IMGOPTSSYNTAX',
-                     'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE', 'MALLOC_PERTURB_']
+                     'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE', 'MALLOC_PERTURB_',
+                     'GDB_OPTIONS']
 
     def prepare_subprocess(self, args: List[str]) -> Dict[str, str]:
         if self.debug:
@@ -178,7 +180,8 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str,
                  imgopts: Optional[str] = None,
                  misalign: bool = False,
                  debug: bool = False,
-                 valgrind: bool = False) -> None:
+                 valgrind: bool = False,
+                 gdb: bool = False) -> None:
         self.imgfmt = imgfmt
         self.imgproto = imgproto
         self.aiomode = aiomode
@@ -186,6 +189,15 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str,
         self.misalign = misalign
         self.debug = debug
 
+        if gdb:
+            self.gdb_options = os.getenv('GDB_OPTIONS', DEF_GDB_OPTIONS)
+            if not self.gdb_options:
+                # cover the case 'export GDB_OPTIONS='
+                self.gdb_options = DEF_GDB_OPTIONS
+        elif 'GDB_OPTIONS' in os.environ:
+            # to not propagate it in prepare_subprocess()
+            del os.environ['GDB_OPTIONS']
+
         if valgrind:
             self.valgrind_qemu = 'y'
 
@@ -285,6 +297,7 @@ def print_env(self) -> None:
 TEST_DIR      -- {TEST_DIR}
 SOCK_DIR      -- {SOCK_DIR}
 SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER}
+GDB_OPTIONS   -- {GDB_OPTIONS}
 """
 
         args = collections.defaultdict(str, self.get_env())
-- 
2.30.2

[PATCH v5 06/16] qemu-iotests: delay QMP socket timers

[Emanuele Giuseppe Esposito]

Attaching gdbserver implies that the qmp socket
should wait indefinitely for an answer from QEMU.

For Timeout class, create a @contextmanager that
switches Timeout with NoTimeout (empty context manager)
so that if --gdb is set, no timeout will be triggered.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 tests/qemu-iotests/iotests.py | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index c86f239d81..d4bfd8f1d6 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -486,6 +486,13 @@ def __exit__(self, exc_type, value, traceback):
     def timeout(self, signum, frame):
         raise Exception(self.errmsg)
 
+@contextmanager
+def NoTimeout():
+    yield
+
+if qemu_gdb:
+    Timeout = NoTimeout
+
 def file_pattern(name):
     return "{0}-{1}".format(os.getpid(), name)
 
@@ -575,7 +582,7 @@ class VM(qtest.QEMUQtestMachine):
 
     def __init__(self, path_suffix=''):
         name = "qemu%s-%d" % (path_suffix, os.getpid())
-        timer = 15.0
+        timer = 15.0 if not qemu_gdb else None
         super().__init__(qemu_prog, qemu_opts, name=name,
                          base_temp_dir=test_dir,
                          socket_scm_helper=socket_scm_helper,
-- 
2.30.2

Re: [PATCH v5 06/16] qemu-iotests: delay QMP socket timers

[Emanuele Giuseppe Esposito]

On 04/06/2021 11:17, Emanuele Giuseppe Esposito wrote:
> Attaching gdbserver implies that the qmp socket
> should wait indefinitely for an answer from QEMU.
> 
> For Timeout class, create a @contextmanager that
> switches Timeout with NoTimeout (empty context manager)
> so that if --gdb is set, no timeout will be triggered.
> 
> Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>   tests/qemu-iotests/iotests.py | 9 ++++++++-
>   1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
> index c86f239d81..d4bfd8f1d6 100644
> --- a/tests/qemu-iotests/iotests.py
> +++ b/tests/qemu-iotests/iotests.py
> @@ -486,6 +486,13 @@ def __exit__(self, exc_type, value, traceback):
>       def timeout(self, signum, frame):
>           raise Exception(self.errmsg)
>   
> +@contextmanager
> +def NoTimeout():
> +    yield
> +
> +if qemu_gdb:
> +    Timeout = NoTimeout
> +

@Vladimir or anyone expert enough in python:
This fix above works, but I just noticed that makes pylint (test 297) 
fail. My bad, I should have tried it before.

The reason for that is
>> +    Timeout = NoTimeout

They obviously have different types.

> +iotests.py:507: error: Cannot assign to a type
> +iotests.py:507: error: Incompatible types in assignment (expression has type "Callable[[], _GeneratorContextManager[Any]]", variable has type "Type[Timeout]")
> +Found 2 errors in 1 file (checked 1 source file)

Any ideas on how to fix this? Otherwise I will get rid of it.

Thank you,
Emanuele

>   def file_pattern(name):
>       return "{0}-{1}".format(os.getpid(), name)
>   
> @@ -575,7 +582,7 @@ class VM(qtest.QEMUQtestMachine):
>   
>       def __init__(self, path_suffix=''):
>           name = "qemu%s-%d" % (path_suffix, os.getpid())
> -        timer = 15.0
> +        timer = 15.0 if not qemu_gdb else None
>           super().__init__(qemu_prog, qemu_opts, name=name,
>                            base_temp_dir=test_dir,
>                            socket_scm_helper=socket_scm_helper,
> 

Re: [PATCH v5 06/16] qemu-iotests: delay QMP socket timers

[Vladimir Sementsov-Ogievskiy]

14.06.2021 13:36, Emanuele Giuseppe Esposito wrote:
> 
> 
> On 04/06/2021 11:17, Emanuele Giuseppe Esposito wrote:
>> Attaching gdbserver implies that the qmp socket
>> should wait indefinitely for an answer from QEMU.
>>
>> For Timeout class, create a @contextmanager that
>> switches Timeout with NoTimeout (empty context manager)
>> so that if --gdb is set, no timeout will be triggered.
>>
>> Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
>> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>   tests/qemu-iotests/iotests.py | 9 ++++++++-
>>   1 file changed, 8 insertions(+), 1 deletion(-)
>>
>> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
>> index c86f239d81..d4bfd8f1d6 100644
>> --- a/tests/qemu-iotests/iotests.py
>> +++ b/tests/qemu-iotests/iotests.py
>> @@ -486,6 +486,13 @@ def __exit__(self, exc_type, value, traceback):
>>       def timeout(self, signum, frame):
>>           raise Exception(self.errmsg)
>> +@contextmanager
>> +def NoTimeout():
>> +    yield
>> +
>> +if qemu_gdb:
>> +    Timeout = NoTimeout
>> +
> 
> @Vladimir or anyone expert enough in python:
> This fix above works, but I just noticed that makes pylint (test 297) fail. My bad, I should have tried it before.

I think, just make mypy ignore it, like:

    Timeout = NoTimeout # type: ignore


> 
> The reason for that is
>>> +    Timeout = NoTimeout
> 
> They obviously have different types.
> 
>> +iotests.py:507: error: Cannot assign to a type
>> +iotests.py:507: error: Incompatible types in assignment (expression has type "Callable[[], _GeneratorContextManager[Any]]", variable has type "Type[Timeout]")
>> +Found 2 errors in 1 file (checked 1 source file)
> 
> Any ideas on how to fix this? Otherwise I will get rid of it.
> 
> Thank you,
> Emanuele
> 
>>   def file_pattern(name):
>>       return "{0}-{1}".format(os.getpid(), name)
>> @@ -575,7 +582,7 @@ class VM(qtest.QEMUQtestMachine):
>>       def __init__(self, path_suffix=''):
>>           name = "qemu%s-%d" % (path_suffix, os.getpid())
>> -        timer = 15.0
>> +        timer = 15.0 if not qemu_gdb else None
>>           super().__init__(qemu_prog, qemu_opts, name=name,
>>                            base_temp_dir=test_dir,
>>                            socket_scm_helper=socket_scm_helper,
>>
> 


-- 
Best regards,
Vladimir

Re: [PATCH v5 06/16] qemu-iotests: delay QMP socket timers

[Emanuele Giuseppe Esposito]

On 15/06/2021 09:57, Vladimir Sementsov-Ogievskiy wrote:
> 14.06.2021 13:36, Emanuele Giuseppe Esposito wrote:
>>
>>
>> On 04/06/2021 11:17, Emanuele Giuseppe Esposito wrote:
>>> Attaching gdbserver implies that the qmp socket
>>> should wait indefinitely for an answer from QEMU.
>>>
>>> For Timeout class, create a @contextmanager that
>>> switches Timeout with NoTimeout (empty context manager)
>>> so that if --gdb is set, no timeout will be triggered.
>>>
>>> Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
>>> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>> ---
>>>   tests/qemu-iotests/iotests.py | 9 ++++++++-
>>>   1 file changed, 8 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/tests/qemu-iotests/iotests.py 
>>> b/tests/qemu-iotests/iotests.py
>>> index c86f239d81..d4bfd8f1d6 100644
>>> --- a/tests/qemu-iotests/iotests.py
>>> +++ b/tests/qemu-iotests/iotests.py
>>> @@ -486,6 +486,13 @@ def __exit__(self, exc_type, value, traceback):
>>>       def timeout(self, signum, frame):
>>>           raise Exception(self.errmsg)
>>> +@contextmanager
>>> +def NoTimeout():
>>> +    yield
>>> +
>>> +if qemu_gdb:
>>> +    Timeout = NoTimeout
>>> +
>>
>> @Vladimir or anyone expert enough in python:
>> This fix above works, but I just noticed that makes pylint (test 297) 
>> fail. My bad, I should have tried it before.
> 
> I think, just make mypy ignore it, like:
> 
>     Timeout = NoTimeout # type: ignore
> 
> 

I think I am going to drop this change.
This series already has patch 2 to ignore another pylint warning, plus 
another separate patch was sent to silence a warning that pops out with 
newer versions of pylint.

So once this gets in, feel free to add a patch with this change.

Emanuele
>>
>> The reason for that is
>>>> +    Timeout = NoTimeout
>>
>> They obviously have different types.
>>
>>> +iotests.py:507: error: Cannot assign to a type
>>> +iotests.py:507: error: Incompatible types in assignment (expression 
>>> has type "Callable[[], _GeneratorContextManager[Any]]", variable has 
>>> type "Type[Timeout]")
>>> +Found 2 errors in 1 file (checked 1 source file)
>>
>> Any ideas on how to fix this? Otherwise I will get rid of it.
>>
>> Thank you,
>> Emanuele
>>
>>>   def file_pattern(name):
>>>       return "{0}-{1}".format(os.getpid(), name)
>>> @@ -575,7 +582,7 @@ class VM(qtest.QEMUQtestMachine):
>>>       def __init__(self, path_suffix=''):
>>>           name = "qemu%s-%d" % (path_suffix, os.getpid())
>>> -        timer = 15.0
>>> +        timer = 15.0 if not qemu_gdb else None
>>>           super().__init__(qemu_prog, qemu_opts, name=name,
>>>                            base_temp_dir=test_dir,
>>>                            socket_scm_helper=socket_scm_helper,
>>>
>>
> 
> 

Re: [PATCH v5 06/16] qemu-iotests: delay QMP socket timers

[Vladimir Sementsov-Ogievskiy]

16.06.2021 10:09, Emanuele Giuseppe Esposito wrote:
> 
> 
> On 15/06/2021 09:57, Vladimir Sementsov-Ogievskiy wrote:
>> 14.06.2021 13:36, Emanuele Giuseppe Esposito wrote:
>>>
>>>
>>> On 04/06/2021 11:17, Emanuele Giuseppe Esposito wrote:
>>>> Attaching gdbserver implies that the qmp socket
>>>> should wait indefinitely for an answer from QEMU.
>>>>
>>>> For Timeout class, create a @contextmanager that
>>>> switches Timeout with NoTimeout (empty context manager)
>>>> so that if --gdb is set, no timeout will be triggered.
>>>>
>>>> Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
>>>> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>> ---
>>>>   tests/qemu-iotests/iotests.py | 9 ++++++++-
>>>>   1 file changed, 8 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
>>>> index c86f239d81..d4bfd8f1d6 100644
>>>> --- a/tests/qemu-iotests/iotests.py
>>>> +++ b/tests/qemu-iotests/iotests.py
>>>> @@ -486,6 +486,13 @@ def __exit__(self, exc_type, value, traceback):
>>>>       def timeout(self, signum, frame):
>>>>           raise Exception(self.errmsg)
>>>> +@contextmanager
>>>> +def NoTimeout():
>>>> +    yield
>>>> +
>>>> +if qemu_gdb:
>>>> +    Timeout = NoTimeout
>>>> +
>>>
>>> @Vladimir or anyone expert enough in python:
>>> This fix above works, but I just noticed that makes pylint (test 297) fail. My bad, I should have tried it before.
>>
>> I think, just make mypy ignore it, like:
>>
>>     Timeout = NoTimeout # type: ignore
>>
>>
> 
> I think I am going to drop this change.
> This series already has patch 2 to ignore another pylint warning, plus another separate patch was sent to silence a warning that pops out with newer versions of pylint.

pylint complains should not be a reason for bad pattern. Shadowing the whole class is not good too, but it's at least a separate small hack, instead of silencing the whole logic of existing well-defined class internally with help of global variable.

-- 
Best regards,
Vladimir

[PATCH v5 07/16] qemu_iotests: insert gdbserver command line as wrapper for qemu binary

[Emanuele Giuseppe Esposito]

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 tests/qemu-iotests/iotests.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index d4bfd8f1d6..c547e8c07b 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -583,7 +583,8 @@ class VM(qtest.QEMUQtestMachine):
     def __init__(self, path_suffix=''):
         name = "qemu%s-%d" % (path_suffix, os.getpid())
         timer = 15.0 if not qemu_gdb else None
-        super().__init__(qemu_prog, qemu_opts, name=name,
+        super().__init__(qemu_prog, qemu_opts, wrapper=qemu_gdb,
+                         name=name,
                          base_temp_dir=test_dir,
                          socket_scm_helper=socket_scm_helper,
                          sock_dir=sock_dir, qmp_timer=timer)
-- 
2.30.2

[PATCH v5 08/16] qemu-iotests: add gdbserver option to script tests too

[Emanuele Giuseppe Esposito]

The only limitation here is that running a script with gdbserver
will make the test output mismatch with the expected
results, making the test fail.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 tests/qemu-iotests/common.rc | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index e2f81cd41b..ad0c449863 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -166,8 +166,14 @@ _qemu_wrapper()
         if [ -n "${QEMU_NEED_PID}" ]; then
             echo $BASHPID > "${QEMU_TEST_DIR}/qemu-${_QEMU_HANDLE}.pid"
         fi
+
+        GDB=""
+        if [ ! -z ${GDB_OPTIONS} ]; then
+            GDB="gdbserver ${GDB_OPTIONS}"
+        fi
+
         VALGRIND_QEMU="${VALGRIND_QEMU_VM}" _qemu_proc_exec "${VALGRIND_LOGFILE}" \
-            "$QEMU_PROG" $QEMU_OPTIONS "$@"
+            $GDB "$QEMU_PROG" $QEMU_OPTIONS "$@"
     )
     RETVAL=$?
     _qemu_proc_valgrind_log "${VALGRIND_LOGFILE}" $RETVAL
-- 
2.30.2

[PATCH v5 09/16] docs/devel/testing: add --gdb option to the debugging section of QEMU iotests

[Emanuele Giuseppe Esposito]

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 docs/devel/testing.rst | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 9d6a8f8636..11a0359218 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -229,6 +229,17 @@ Debugging a test case
 The following options to the ``check`` script can be useful when debugging
 a failing test:
 
+* ``--gdb`` wraps every QEMU invocation in a ``gdbserver``, which waits for a
+  connection from a gdb client.  The options given to ``gdbserver`` (e.g. the
+  address on which to listen for connections) are taken from the ``$GDB_OPTIONS``
+  environment variable.  By default (if ``$GDB_OPTIONS`` is empty), it listens on
+  ``localhost:12345``.
+  It is possible to connect to it for example with
+  ``gdb -iex "target remote $addr"``, where ``$addr`` is the address
+  ``gdbserver`` listens on.
+  If the ``-gdb`` option is not used, ``$GDB_OPTIONS`` is ignored,
+  regardless on whether it is set or not.
+
 * ``-d`` (debug) just increases the logging verbosity, showing
   for example the QMP commands and answers.
 
-- 
2.30.2

Re: [PATCH v5 09/16] docs/devel/testing: add --gdb option to the debugging section of QEMU iotests

[Vladimir Sementsov-Ogievskiy]

04.06.2021 12:17, Emanuele Giuseppe Esposito wrote:
> Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>   docs/devel/testing.rst | 11 +++++++++++
>   1 file changed, 11 insertions(+)
> 
> diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
> index 9d6a8f8636..11a0359218 100644
> --- a/docs/devel/testing.rst
> +++ b/docs/devel/testing.rst
> @@ -229,6 +229,17 @@ Debugging a test case
>   The following options to the ``check`` script can be useful when debugging
>   a failing test:
>   
> +* ``--gdb`` wraps every QEMU invocation in a ``gdbserver``, which waits for a
> +  connection from a gdb client.  The options given to ``gdbserver`` (e.g. the
> +  address on which to listen for connections) are taken from the ``$GDB_OPTIONS``
> +  environment variable.  By default (if ``$GDB_OPTIONS`` is empty), it listens on
> +  ``localhost:12345``.
> +  It is possible to connect to it for example with
> +  ``gdb -iex "target remote $addr"``, where ``$addr`` is the address
> +  ``gdbserver`` listens on.
> +  If the ``-gdb`` option is not used, ``$GDB_OPTIONS`` is ignored,

Then, --gdb here too

> +  regardless on whether it is set or not.
> +
>   * ``-d`` (debug) just increases the logging verbosity, showing
>     for example the QMP commands and answers.
>   
> 


-- 
Best regards,
Vladimir

[PATCH v5 10/16] qemu-iotests: extend the check script to prepare supporting valgrind for python tests

[Emanuele Giuseppe Esposito]

Currently, the check script only parses the option and sets the
VALGRIND_QEMU environmental variable to "y".
Add another local python variable that prepares the command line,
identical to the one provided in the test scripts.

Because the python script does not know in advance the valgrind
PID to assign to the log file name, use the "%p" flag in valgrind
log file name that automatically puts the process PID at runtime.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/check      |  7 ++++---
 tests/qemu-iotests/iotests.py | 11 +++++++++++
 tests/qemu-iotests/testenv.py |  1 +
 3 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
index 1dba4218c0..e6aa110715 100755
--- a/tests/qemu-iotests/check
+++ b/tests/qemu-iotests/check
@@ -39,6 +39,10 @@ def make_argparser() -> argparse.ArgumentParser:
     p.add_argument('--gdb', action='store_true',
                    help="start gdbserver with $GDB_OPTIONS options \
                         ('localhost:12345' if $GDB_OPTIONS is empty)")
+    p.add_argument('--valgrind', action='store_true',
+                    help='use valgrind, sets VALGRIND_QEMU environment '
+                    'variable')
+
     p.add_argument('--misalign', action='store_true',
                    help='misalign memory allocations')
     p.add_argument('--color', choices=['on', 'off', 'auto'],
@@ -88,9 +92,6 @@ def make_argparser() -> argparse.ArgumentParser:
     g_bash.add_argument('-o', dest='imgopts',
                         help='options to pass to qemu-img create/convert, '
                         'sets IMGOPTS environment variable')
-    g_bash.add_argument('--valgrind', action='store_true',
-                        help='use valgrind, sets VALGRIND_QEMU environment '
-                        'variable')
 
     g_sel = p.add_argument_group('test selecting options',
                                  'The following options specify test set '
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index c547e8c07b..3fa1bd0ab5 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -96,6 +96,17 @@
     sys.stderr.write('Please run this test via the "check" script\n')
     sys.exit(os.EX_USAGE)
 
+qemu_valgrind = []
+if os.environ.get('VALGRIND_QEMU') == "y" and \
+    os.environ.get('NO_VALGRIND') != "y":
+    valgrind_logfile = "--log-file=" + test_dir
+    # %p allows to put the valgrind process PID, since
+    # we don't know it a priori (subprocess.Popen is
+    # not yet invoked)
+    valgrind_logfile += "/%p.valgrind"
+
+    qemu_valgrind = ['valgrind', valgrind_logfile, '--error-exitcode=99']
+
 socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper')
 
 luks_default_secret_object = 'secret,id=keysec0,data=' + \
diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py
index 8501c6caf5..8bf154376f 100644
--- a/tests/qemu-iotests/testenv.py
+++ b/tests/qemu-iotests/testenv.py
@@ -298,6 +298,7 @@ def print_env(self) -> None:
 SOCK_DIR      -- {SOCK_DIR}
 SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER}
 GDB_OPTIONS   -- {GDB_OPTIONS}
+VALGRIND_QEMU -- {VALGRIND_QEMU}
 """
 
         args = collections.defaultdict(str, self.get_env())
-- 
2.30.2

Re: [PATCH v5 10/16] qemu-iotests: extend the check script to prepare supporting valgrind for python tests

[Vladimir Sementsov-Ogievskiy]

04.06.2021 12:17, Emanuele Giuseppe Esposito wrote:
> Currently, the check script only parses the option and sets the
> VALGRIND_QEMU environmental variable to "y".
> Add another local python variable that prepares the command line,
> identical to the one provided in the test scripts.
> 
> Because the python script does not know in advance the valgrind
> PID to assign to the log file name, use the "%p" flag in valgrind
> log file name that automatically puts the process PID at runtime.
> 
> Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
> Reviewed-by: Max Reitz <mreitz@redhat.com>
> ---
>   tests/qemu-iotests/check      |  7 ++++---
>   tests/qemu-iotests/iotests.py | 11 +++++++++++
>   tests/qemu-iotests/testenv.py |  1 +
>   3 files changed, 16 insertions(+), 3 deletions(-)
> 
> diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
> index 1dba4218c0..e6aa110715 100755
> --- a/tests/qemu-iotests/check
> +++ b/tests/qemu-iotests/check
> @@ -39,6 +39,10 @@ def make_argparser() -> argparse.ArgumentParser:
>       p.add_argument('--gdb', action='store_true',
>                      help="start gdbserver with $GDB_OPTIONS options \
>                           ('localhost:12345' if $GDB_OPTIONS is empty)")
> +    p.add_argument('--valgrind', action='store_true',
> +                    help='use valgrind, sets VALGRIND_QEMU environment '
> +                    'variable')
> +
>       p.add_argument('--misalign', action='store_true',
>                      help='misalign memory allocations')
>       p.add_argument('--color', choices=['on', 'off', 'auto'],
> @@ -88,9 +92,6 @@ def make_argparser() -> argparse.ArgumentParser:
>       g_bash.add_argument('-o', dest='imgopts',
>                           help='options to pass to qemu-img create/convert, '
>                           'sets IMGOPTS environment variable')
> -    g_bash.add_argument('--valgrind', action='store_true',
> -                        help='use valgrind, sets VALGRIND_QEMU environment '
> -                        'variable')
>   
>       g_sel = p.add_argument_group('test selecting options',
>                                    'The following options specify test set '
> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
> index c547e8c07b..3fa1bd0ab5 100644
> --- a/tests/qemu-iotests/iotests.py
> +++ b/tests/qemu-iotests/iotests.py
> @@ -96,6 +96,17 @@
>       sys.stderr.write('Please run this test via the "check" script\n')
>       sys.exit(os.EX_USAGE)
>   
> +qemu_valgrind = []
> +if os.environ.get('VALGRIND_QEMU') == "y" and \
> +    os.environ.get('NO_VALGRIND') != "y":

Hmm, interesting, why do you need additional NO_VALGRIND variable

> +    valgrind_logfile = "--log-file=" + test_dir
> +    # %p allows to put the valgrind process PID, since
> +    # we don't know it a priori (subprocess.Popen is
> +    # not yet invoked)
> +    valgrind_logfile += "/%p.valgrind"
> +
> +    qemu_valgrind = ['valgrind', valgrind_logfile, '--error-exitcode=99']
> +
>   socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper')
>   
>   luks_default_secret_object = 'secret,id=keysec0,data=' + \
> diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py
> index 8501c6caf5..8bf154376f 100644
> --- a/tests/qemu-iotests/testenv.py
> +++ b/tests/qemu-iotests/testenv.py
> @@ -298,6 +298,7 @@ def print_env(self) -> None:
>   SOCK_DIR      -- {SOCK_DIR}
>   SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER}
>   GDB_OPTIONS   -- {GDB_OPTIONS}
> +VALGRIND_QEMU -- {VALGRIND_QEMU}
>   """
>   
>           args = collections.defaultdict(str, self.get_env())
> 



Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

-- 
Best regards,
Vladimir

Re: [PATCH v5 10/16] qemu-iotests: extend the check script to prepare supporting valgrind for python tests

[Emanuele Giuseppe Esposito]

On 05/06/2021 15:28, Vladimir Sementsov-Ogievskiy wrote:
> 04.06.2021 12:17, Emanuele Giuseppe Esposito wrote:
>> Currently, the check script only parses the option and sets the
>> VALGRIND_QEMU environmental variable to "y".
>> Add another local python variable that prepares the command line,
>> identical to the one provided in the test scripts.
>>
>> Because the python script does not know in advance the valgrind
>> PID to assign to the log file name, use the "%p" flag in valgrind
>> log file name that automatically puts the process PID at runtime.
>>
>> Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
>> Reviewed-by: Max Reitz <mreitz@redhat.com>
>> ---
>>   tests/qemu-iotests/check      |  7 ++++---
>>   tests/qemu-iotests/iotests.py | 11 +++++++++++
>>   tests/qemu-iotests/testenv.py |  1 +
>>   3 files changed, 16 insertions(+), 3 deletions(-)
>>
>> diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
>> index 1dba4218c0..e6aa110715 100755
>> --- a/tests/qemu-iotests/check
>> +++ b/tests/qemu-iotests/check
>> @@ -39,6 +39,10 @@ def make_argparser() -> argparse.ArgumentParser:
>>       p.add_argument('--gdb', action='store_true',
>>                      help="start gdbserver with $GDB_OPTIONS options \
>>                           ('localhost:12345' if $GDB_OPTIONS is empty)")
>> +    p.add_argument('--valgrind', action='store_true',
>> +                    help='use valgrind, sets VALGRIND_QEMU environment '
>> +                    'variable')
>> +
>>       p.add_argument('--misalign', action='store_true',
>>                      help='misalign memory allocations')
>>       p.add_argument('--color', choices=['on', 'off', 'auto'],
>> @@ -88,9 +92,6 @@ def make_argparser() -> argparse.ArgumentParser:
>>       g_bash.add_argument('-o', dest='imgopts',
>>                           help='options to pass to qemu-img 
>> create/convert, '
>>                           'sets IMGOPTS environment variable')
>> -    g_bash.add_argument('--valgrind', action='store_true',
>> -                        help='use valgrind, sets VALGRIND_QEMU 
>> environment '
>> -                        'variable')
>>       g_sel = p.add_argument_group('test selecting options',
>>                                    'The following options specify test 
>> set '
>> diff --git a/tests/qemu-iotests/iotests.py 
>> b/tests/qemu-iotests/iotests.py
>> index c547e8c07b..3fa1bd0ab5 100644
>> --- a/tests/qemu-iotests/iotests.py
>> +++ b/tests/qemu-iotests/iotests.py
>> @@ -96,6 +96,17 @@
>>       sys.stderr.write('Please run this test via the "check" script\n')
>>       sys.exit(os.EX_USAGE)
>> +qemu_valgrind = []
>> +if os.environ.get('VALGRIND_QEMU') == "y" and \
>> +    os.environ.get('NO_VALGRIND') != "y":
> 
> Hmm, interesting, why do you need additional NO_VALGRIND variable

To maintain consistency with the bash tests, where we have:

# Valgrind bug #409141 https://bugs.kde.org/show_bug.cgi?id=409141
# Until valgrind 3.16+ is ubiquitous, we must work around a hang in
# valgrind when issuing sigkill. Disable valgrind for this invocation.
_NO_VALGRIND()
{
     NO_VALGRIND="y" "$@"
}


> 
>> +    valgrind_logfile = "--log-file=" + test_dir
>> +    # %p allows to put the valgrind process PID, since
>> +    # we don't know it a priori (subprocess.Popen is
>> +    # not yet invoked)
>> +    valgrind_logfile += "/%p.valgrind"
>> +
>> +    qemu_valgrind = ['valgrind', valgrind_logfile, 
>> '--error-exitcode=99']
>> +
>>   socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 
>> 'socket_scm_helper')
>>   luks_default_secret_object = 'secret,id=keysec0,data=' + \
>> diff --git a/tests/qemu-iotests/testenv.py 
>> b/tests/qemu-iotests/testenv.py
>> index 8501c6caf5..8bf154376f 100644
>> --- a/tests/qemu-iotests/testenv.py
>> +++ b/tests/qemu-iotests/testenv.py
>> @@ -298,6 +298,7 @@ def print_env(self) -> None:
>>   SOCK_DIR      -- {SOCK_DIR}
>>   SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER}
>>   GDB_OPTIONS   -- {GDB_OPTIONS}
>> +VALGRIND_QEMU -- {VALGRIND_QEMU}
>>   """
>>           args = collections.defaultdict(str, self.get_env())
>>
> 
> 
> 
> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> 

Re: [PATCH v5 10/16] qemu-iotests: extend the check script to prepare supporting valgrind for python tests

[Vladimir Sementsov-Ogievskiy]

05.06.2021 20:38, Emanuele Giuseppe Esposito wrote:
> 
> 
> On 05/06/2021 15:28, Vladimir Sementsov-Ogievskiy wrote:
>> 04.06.2021 12:17, Emanuele Giuseppe Esposito wrote:
>>> Currently, the check script only parses the option and sets the
>>> VALGRIND_QEMU environmental variable to "y".
>>> Add another local python variable that prepares the command line,
>>> identical to the one provided in the test scripts.
>>>
>>> Because the python script does not know in advance the valgrind
>>> PID to assign to the log file name, use the "%p" flag in valgrind
>>> log file name that automatically puts the process PID at runtime.
>>>
>>> Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
>>> Reviewed-by: Max Reitz <mreitz@redhat.com>
>>> ---
>>>   tests/qemu-iotests/check      |  7 ++++---
>>>   tests/qemu-iotests/iotests.py | 11 +++++++++++
>>>   tests/qemu-iotests/testenv.py |  1 +
>>>   3 files changed, 16 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
>>> index 1dba4218c0..e6aa110715 100755
>>> --- a/tests/qemu-iotests/check
>>> +++ b/tests/qemu-iotests/check
>>> @@ -39,6 +39,10 @@ def make_argparser() -> argparse.ArgumentParser:
>>>       p.add_argument('--gdb', action='store_true',
>>>                      help="start gdbserver with $GDB_OPTIONS options \
>>>                           ('localhost:12345' if $GDB_OPTIONS is empty)")
>>> +    p.add_argument('--valgrind', action='store_true',
>>> +                    help='use valgrind, sets VALGRIND_QEMU environment '
>>> +                    'variable')
>>> +
>>>       p.add_argument('--misalign', action='store_true',
>>>                      help='misalign memory allocations')
>>>       p.add_argument('--color', choices=['on', 'off', 'auto'],
>>> @@ -88,9 +92,6 @@ def make_argparser() -> argparse.ArgumentParser:
>>>       g_bash.add_argument('-o', dest='imgopts',
>>>                           help='options to pass to qemu-img create/convert, '
>>>                           'sets IMGOPTS environment variable')
>>> -    g_bash.add_argument('--valgrind', action='store_true',
>>> -                        help='use valgrind, sets VALGRIND_QEMU environment '
>>> -                        'variable')
>>>       g_sel = p.add_argument_group('test selecting options',
>>>                                    'The following options specify test set '
>>> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
>>> index c547e8c07b..3fa1bd0ab5 100644
>>> --- a/tests/qemu-iotests/iotests.py
>>> +++ b/tests/qemu-iotests/iotests.py
>>> @@ -96,6 +96,17 @@
>>>       sys.stderr.write('Please run this test via the "check" script\n')
>>>       sys.exit(os.EX_USAGE)
>>> +qemu_valgrind = []
>>> +if os.environ.get('VALGRIND_QEMU') == "y" and \
>>> +    os.environ.get('NO_VALGRIND') != "y":
>>
>> Hmm, interesting, why do you need additional NO_VALGRIND variable
> 
> To maintain consistency with the bash tests, where we have:
> 
> # Valgrind bug #409141 https://bugs.kde.org/show_bug.cgi?id=409141
> # Until valgrind 3.16+ is ubiquitous, we must work around a hang in
> # valgrind when issuing sigkill. Disable valgrind for this invocation.
> _NO_VALGRIND()
> {
>      NO_VALGRIND="y" "$@"
> }
> 

A, hm, I see it in bash tests. So, it's intended to not set by user but by test.. But I doubt that python test will want to set environment variable to disable valgrind. Most probably they will want to set some valgrind_supported=False near supported_fmts=['qcow2']. So, we'll need valgrind_supported argument for execute_setup_common. But no reason to implement it until we don't need.

> 
>>
>>> +    valgrind_logfile = "--log-file=" + test_dir
>>> +    # %p allows to put the valgrind process PID, since
>>> +    # we don't know it a priori (subprocess.Popen is
>>> +    # not yet invoked)
>>> +    valgrind_logfile += "/%p.valgrind"
>>> +
>>> +    qemu_valgrind = ['valgrind', valgrind_logfile, '--error-exitcode=99']
>>> +
>>>   socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper')
>>>   luks_default_secret_object = 'secret,id=keysec0,data=' + \
>>> diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py
>>> index 8501c6caf5..8bf154376f 100644
>>> --- a/tests/qemu-iotests/testenv.py
>>> +++ b/tests/qemu-iotests/testenv.py
>>> @@ -298,6 +298,7 @@ def print_env(self) -> None:
>>>   SOCK_DIR      -- {SOCK_DIR}
>>>   SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER}
>>>   GDB_OPTIONS   -- {GDB_OPTIONS}
>>> +VALGRIND_QEMU -- {VALGRIND_QEMU}
>>>   """
>>>           args = collections.defaultdict(str, self.get_env())
>>>
>>
>>
>>
>> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>
> 


-- 
Best regards,
Vladimir

[PATCH v5 11/16] qemu-iotests: extend QMP socket timeout when using valgrind

[Emanuele Giuseppe Esposito]

As with gdbserver, valgrind delays the test execution, so
the default QMP socket timeout and the generic class
Timeout in iotests.py timeouts too soon.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 tests/qemu-iotests/iotests.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 3fa1bd0ab5..e4a299b9f3 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -501,7 +501,7 @@ def timeout(self, signum, frame):
 def NoTimeout():
     yield
 
-if qemu_gdb:
+if qemu_gdb or qemu_valgrind:
     Timeout = NoTimeout
 
 def file_pattern(name):
@@ -593,7 +593,7 @@ class VM(qtest.QEMUQtestMachine):
 
     def __init__(self, path_suffix=''):
         name = "qemu%s-%d" % (path_suffix, os.getpid())
-        timer = 15.0 if not qemu_gdb else None
+        timer = 15.0 if not (qemu_gdb or qemu_valgrind) else None
         super().__init__(qemu_prog, qemu_opts, wrapper=qemu_gdb,
                          name=name,
                          base_temp_dir=test_dir,
-- 
2.30.2

[PATCH v5 12/16] qemu-iotests: allow valgrind to read/delete the generated log file

[Emanuele Giuseppe Esposito]

When using --valgrind on the script tests, it generates a log file
in $TEST_DIR that is either read (if valgrind finds problems) or
otherwise deleted. Provide the same exact behavior when using
--valgrind on the python tests.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 tests/qemu-iotests/iotests.py | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index e4a299b9f3..89099c13e6 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -601,6 +601,17 @@ def __init__(self, path_suffix=''):
                          sock_dir=sock_dir, qmp_timer=timer)
         self._num_drives = 0
 
+    def _post_shutdown(self) -> None:
+        super()._post_shutdown()
+        if not qemu_valgrind or not self._popen:
+            return
+        valgrind_filename =  f"{test_dir}/{self._popen.pid}.valgrind"
+        if self.exitcode() == 99:
+            with open(valgrind_filename) as f:
+                print(f.read())
+        else:
+            os.remove(valgrind_filename)
+
     def add_object(self, opts):
         self._args.append('-object')
         self._args.append(opts)
-- 
2.30.2

[PATCH v5 13/16] qemu-iotests: insert valgrind command line as wrapper for qemu binary

[Emanuele Giuseppe Esposito]

If --gdb and --valgrind are both defined, return an error.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 tests/qemu-iotests/iotests.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 89099c13e6..b9f0ac0ddd 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -594,7 +594,11 @@ class VM(qtest.QEMUQtestMachine):
     def __init__(self, path_suffix=''):
         name = "qemu%s-%d" % (path_suffix, os.getpid())
         timer = 15.0 if not (qemu_gdb or qemu_valgrind) else None
-        super().__init__(qemu_prog, qemu_opts, wrapper=qemu_gdb,
+        if qemu_gdb and qemu_valgrind:
+            sys.stderr.write('Either use gdb or valgrind, not together\n')
+            sys.exit(1)
+        wrapper = qemu_gdb if qemu_gdb else qemu_valgrind
+        super().__init__(qemu_prog, qemu_opts, wrapper=wrapper,
                          name=name,
                          base_temp_dir=test_dir,
                          socket_scm_helper=socket_scm_helper,
-- 
2.30.2

[PATCH v5 14/16] docs/devel/testing: add --valgrind option to the debug section of QEMU iotests

[Emanuele Giuseppe Esposito]

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 docs/devel/testing.rst | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 11a0359218..9cac707ee5 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -240,6 +240,13 @@ a failing test:
   If the ``-gdb`` option is not used, ``$GDB_OPTIONS`` is ignored,
   regardless on whether it is set or not.
 
+* ``--valgrind`` attaches a valgrind instance to QEMU. If it detects
+  warnings, it will print and save the log in
+  ``$TEST_DIR/<valgrind_pid>.valgrind``.
+  The final command line will be ``valgrind --log-file=$TEST_DIR/
+  <valgrind_pid>.valgrind --error-exitcode=99 $QEMU ...``
+  Note: if used together with ``-gdb``, this command will be ignored.
+
 * ``-d`` (debug) just increases the logging verbosity, showing
   for example the QMP commands and answers.
 
-- 
2.30.2

[PATCH v5 15/16] qemu-iotests: add option to show qemu binary logs on stdout

[Emanuele Giuseppe Esposito]

Using the flag -p, allow the qemu binary to print to stdout.

Also create the common function _close_qemu_log_file() to
avoid accessing machine.py private fields directly and have
duplicate code.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 python/qemu/machine/machine.py | 9 ++++++---
 tests/qemu-iotests/check       | 4 +++-
 tests/qemu-iotests/iotests.py  | 8 ++++++++
 tests/qemu-iotests/testenv.py  | 9 +++++++--
 4 files changed, 24 insertions(+), 6 deletions(-)

diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py
index fdf2fc0e9c..c9d344d955 100644
--- a/python/qemu/machine/machine.py
+++ b/python/qemu/machine/machine.py
@@ -338,6 +338,11 @@ def _post_launch(self) -> None:
         if self._qmp_connection:
             self._qmp.accept(self._qmp_timer)
 
+    def _close_qemu_log_file(self) -> None:
+        if self._qemu_log_file is not None:
+            self._qemu_log_file.close()
+            self._qemu_log_file = None
+
     def _post_shutdown(self) -> None:
         """
         Called to cleanup the VM instance after the process has exited.
@@ -350,9 +355,7 @@ def _post_shutdown(self) -> None:
             self._qmp.close()
             self._qmp_connection = None
 
-        if self._qemu_log_file is not None:
-            self._qemu_log_file.close()
-            self._qemu_log_file = None
+        self._close_qemu_log_file()
 
         self._load_io_log()
 
diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
index e6aa110715..2371d13c8e 100755
--- a/tests/qemu-iotests/check
+++ b/tests/qemu-iotests/check
@@ -36,6 +36,8 @@ def make_argparser() -> argparse.ArgumentParser:
                    help='pretty print output for make check')
 
     p.add_argument('-d', dest='debug', action='store_true', help='debug')
+    p.add_argument('-p', dest='print', action='store_true',
+                help='redirects qemu\'s stdout and stderr to the test output')
     p.add_argument('--gdb', action='store_true',
                    help="start gdbserver with $GDB_OPTIONS options \
                         ('localhost:12345' if $GDB_OPTIONS is empty)")
@@ -119,7 +121,7 @@ if __name__ == '__main__':
                   aiomode=args.aiomode, cachemode=args.cachemode,
                   imgopts=args.imgopts, misalign=args.misalign,
                   debug=args.debug, valgrind=args.valgrind,
-                  gdb=args.gdb)
+                  gdb=args.gdb, qprint=args.print)
 
     if len(sys.argv) > 1 and sys.argv[-len(args.tests)-1] == '--':
         if not args.tests:
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index b9f0ac0ddd..ac2391f380 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -79,6 +79,8 @@
 if gdb_qemu_env:
     qemu_gdb = ['gdbserver'] + gdb_qemu_env.strip().split(' ')
 
+qemu_print = os.environ.get('PRINT_QEMU', False)
+
 imgfmt = os.environ.get('IMGFMT', 'raw')
 imgproto = os.environ.get('IMGPROTO', 'file')
 output_dir = os.environ.get('OUTPUT_DIR', '.')
@@ -616,6 +618,12 @@ def _post_shutdown(self) -> None:
         else:
             os.remove(valgrind_filename)
 
+    def _pre_launch(self) -> None:
+        super()._pre_launch()
+        if qemu_print:
+            # set QEMU binary output to stdout
+            self._close_qemu_log_file()
+
     def add_object(self, opts):
         self._args.append('-object')
         self._args.append(opts)
diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py
index 8bf154376f..70da0d60c8 100644
--- a/tests/qemu-iotests/testenv.py
+++ b/tests/qemu-iotests/testenv.py
@@ -74,7 +74,7 @@ class TestEnv(ContextManager['TestEnv']):
                      'AIOMODE', 'CACHEMODE', 'VALGRIND_QEMU',
                      'CACHEMODE_IS_DEFAULT', 'IMGFMT_GENERIC', 'IMGOPTSSYNTAX',
                      'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE', 'MALLOC_PERTURB_',
-                     'GDB_OPTIONS']
+                     'GDB_OPTIONS', 'PRINT_QEMU']
 
     def prepare_subprocess(self, args: List[str]) -> Dict[str, str]:
         if self.debug:
@@ -181,7 +181,8 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str,
                  misalign: bool = False,
                  debug: bool = False,
                  valgrind: bool = False,
-                 gdb: bool = False) -> None:
+                 gdb: bool = False,
+                 qprint: bool = False) -> None:
         self.imgfmt = imgfmt
         self.imgproto = imgproto
         self.aiomode = aiomode
@@ -189,6 +190,9 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str,
         self.misalign = misalign
         self.debug = debug
 
+        if qprint:
+            self.print_qemu = 'y'
+
         if gdb:
             self.gdb_options = os.getenv('GDB_OPTIONS', DEF_GDB_OPTIONS)
             if not self.gdb_options:
@@ -299,6 +303,7 @@ def print_env(self) -> None:
 SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER}
 GDB_OPTIONS   -- {GDB_OPTIONS}
 VALGRIND_QEMU -- {VALGRIND_QEMU}
+PRINT_QEMU_OUTPUT -- {PRINT_QEMU}
 """
 
         args = collections.defaultdict(str, self.get_env())
-- 
2.30.2

[PATCH v5 16/16] docs/devel/testing: add -p option to the debug section of QEMU iotests

[Emanuele Giuseppe Esposito]

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 docs/devel/testing.rst | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 9cac707ee5..c038ca93d4 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -250,6 +250,10 @@ a failing test:
 * ``-d`` (debug) just increases the logging verbosity, showing
   for example the QMP commands and answers.
 
+* ``-p`` (print) redirect QEMU’s stdout and stderr to the test output,
+  instead of saving it into a log file in
+  ``$TEST_DIR/qemu-machine-<random_string>``.
+
 Test case groups
 ----------------
 
-- 
2.30.2

Re: [PATCH v5 00/16] qemu_iotests: improve debugging options

[Emanuele Giuseppe Esposito]

On 04/06/2021 11:17, Emanuele Giuseppe Esposito wrote:
> This series adds the option to attach gdbserver and valgrind
> to the QEMU binary running in qemu_iotests.
> It also allows to redirect QEMU binaries output of the python tests
> to the stdout, instead of a log file.
> 
> Patches 1-9 introduce the -gdb option to both python and bash tests,
> 10-14 extend the already existing -valgrind flag to work also on
> python tests, and patch 15-16 introduces -p to enable logging to stdout.
> 
> In particular, patches 1,6,11 focus on extending the QMP socket timers
> when using gdb/valgrind, otherwise the python tests will fail due to
> delays in the QMP responses.
> 
> This series is tested on the previous serie
> "qemu-iotests: quality of life improvements"
> but independent from it, so it can be applied separately.
> 
> It is also based on "iotests/check: move general long options to double dash"
> by Vladimir Sementsov-Ogievskiy.

Apologies to all the people following this series,

I realized that there is a discussion[*] on whether to use the 
"--option" notation vs "-option".

Therefore I am going to re-send this series without Vladimir's based-on 
(also because I got all necessary reviews, so it should be ready to be 
merged).

Thank you,
Emanuele

[*] = 
https://patchew.org/QEMU/20210526181659.365531-1-vsementsov@virtuozzo.com/#955ce8e3-eafe-79ab-ad4e-f81ecf08d20f@virtuozzo.com
> 
> Based-on: <20210526181659.365531-1-vsementsov@virtuozzo.com>
> 
> Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
> ---
> v5:
> * base this serie on the double dash options, so define --gdb instead of -gdb
> * return error if -gdb and -valgrind are used together [Vladimir]
> * style changes (do not use subprocess_check_valgrind() but do the operation
>    inside the caller) [Vladimir]
> * minor fixes in commit messages [Vladimir]
> * create a common public function in patch 15 instead of using
>    _private fields [Vladimir]
> * use @contextmanager in patch 6 to skip gdb timeouts involving the
>    Timeout class [Vladimir]
> * apply John Snow patch to silence pylint warning in the CI tests
> 
> Emanuele Giuseppe Esposito (15):
>    python: qemu: add timer parameter for qmp.accept socket
>    python: qemu: pass the wrapper field from QEMUQtestmachine to
>      QEMUMachine
>    docs/devel/testing: add debug section to the QEMU iotests chapter
>    qemu-iotests: add option to attach gdbserver
>    qemu-iotests: delay QMP socket timers
>    qemu_iotests: insert gdbserver command line as wrapper for qemu binary
>    qemu-iotests: add gdbserver option to script tests too
>    docs/devel/testing: add --gdb option to the debugging section of QEMU
>      iotests
>    qemu-iotests: extend the check script to prepare supporting valgrind
>      for python tests
>    qemu-iotests: extend QMP socket timeout when using valgrind
>    qemu-iotests: allow valgrind to read/delete the generated log file
>    qemu-iotests: insert valgrind command line as wrapper for qemu binary
>    docs/devel/testing: add --valgrind option to the debug section of QEMU
>      iotests
>    qemu-iotests: add option to show qemu binary logs on stdout
>    docs/devel/testing: add -p option to the debug section of QEMU iotests
> 
> John Snow (1):
>    python: Reduce strictness of pylint's duplicate-code check
> 
>   docs/devel/testing.rst         | 30 ++++++++++++++++++++
>   python/qemu/machine/machine.py | 16 +++++++----
>   python/qemu/machine/qtest.py   |  9 ++++--
>   python/setup.cfg               |  5 ++++
>   tests/qemu-iotests/check       | 15 +++++++---
>   tests/qemu-iotests/common.rc   |  8 +++++-
>   tests/qemu-iotests/iotests.py  | 52 ++++++++++++++++++++++++++++++++--
>   tests/qemu-iotests/testenv.py  | 23 +++++++++++++--
>   8 files changed, 141 insertions(+), 17 deletions(-)
>