* [PATCH v6 01/11] iotests/277: use dot slash for nbd-fault-injector.py running
2021-01-09 12:26 [PATCH v6 00/11] Rework iotests/check Vladimir Sementsov-Ogievskiy
@ 2021-01-09 12:26 ` Vladimir Sementsov-Ogievskiy
2021-01-13 22:37 ` Eric Blake
2021-01-09 12:26 ` [PATCH v6 02/11] iotests/303: use dot slash for qcow2.py running Vladimir Sementsov-Ogievskiy
` (9 subsequent siblings)
10 siblings, 1 reply; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-09 12:26 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, vsementsov, jsnow, qemu-devel, mreitz, den
If you run './check 277', check includes common.config which adjusts
$PATH to include '.' first, and therefore finds nbd-fault-injector.py
on PATH. But if you run './277' directly, there is nothing to adjust
PATH, and if '.' is not already on your PATH by other means, the test
fails because the executable is not found. Adjust how we invoke the
helper executable to avoid needing a PATH search in the first place.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
tests/qemu-iotests/277 | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/qemu-iotests/277 b/tests/qemu-iotests/277
index d34f87021f..a39ce2d873 100755
--- a/tests/qemu-iotests/277
+++ b/tests/qemu-iotests/277
@@ -42,7 +42,7 @@ def make_conf_file(event):
def start_server_NBD(event):
make_conf_file(event)
- srv = subprocess.Popen(['nbd-fault-injector.py', '--classic-negotiation',
+ srv = subprocess.Popen(['./nbd-fault-injector.py', '--classic-negotiation',
nbd_sock, conf_file], stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, universal_newlines=True)
line = srv.stdout.readline()
--
2.29.2
^ permalink raw reply related [flat|nested] 43+ messages in thread* Re: [PATCH v6 01/11] iotests/277: use dot slash for nbd-fault-injector.py running
2021-01-09 12:26 ` [PATCH v6 01/11] iotests/277: use dot slash for nbd-fault-injector.py running Vladimir Sementsov-Ogievskiy
@ 2021-01-13 22:37 ` Eric Blake
0 siblings, 0 replies; 43+ messages in thread
From: Eric Blake @ 2021-01-13 22:37 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, den, jsnow, qemu-devel, mreitz
On 1/9/21 6:26 AM, Vladimir Sementsov-Ogievskiy wrote:
> If you run './check 277', check includes common.config which adjusts
> $PATH to include '.' first, and therefore finds nbd-fault-injector.py
> on PATH. But if you run './277' directly, there is nothing to adjust
> PATH, and if '.' is not already on your PATH by other means, the test
> fails because the executable is not found. Adjust how we invoke the
> helper executable to avoid needing a PATH search in the first place.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
> tests/qemu-iotests/277 | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
As this one touches NBD, I'm happy to queue it in my next NBD pull
request if the entire series takes longer to collect enough reviews.
>
> diff --git a/tests/qemu-iotests/277 b/tests/qemu-iotests/277
> index d34f87021f..a39ce2d873 100755
> --- a/tests/qemu-iotests/277
> +++ b/tests/qemu-iotests/277
> @@ -42,7 +42,7 @@ def make_conf_file(event):
> def start_server_NBD(event):
> make_conf_file(event)
>
> - srv = subprocess.Popen(['nbd-fault-injector.py', '--classic-negotiation',
> + srv = subprocess.Popen(['./nbd-fault-injector.py', '--classic-negotiation',
> nbd_sock, conf_file], stdout=subprocess.PIPE,
> stderr=subprocess.STDOUT, universal_newlines=True)
> line = srv.stdout.readline()
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 43+ messages in thread
* [PATCH v6 02/11] iotests/303: use dot slash for qcow2.py running
2021-01-09 12:26 [PATCH v6 00/11] Rework iotests/check Vladimir Sementsov-Ogievskiy
2021-01-09 12:26 ` [PATCH v6 01/11] iotests/277: use dot slash for nbd-fault-injector.py running Vladimir Sementsov-Ogievskiy
@ 2021-01-09 12:26 ` Vladimir Sementsov-Ogievskiy
2021-01-13 22:38 ` Eric Blake
2021-01-09 12:26 ` [PATCH v6 03/11] iotests: fix some whitespaces in test output files Vladimir Sementsov-Ogievskiy
` (8 subsequent siblings)
10 siblings, 1 reply; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-09 12:26 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, vsementsov, jsnow, qemu-devel, mreitz, den
If you run './check 303', check includes common.config which adjusts
$PATH to include '.' first, and therefore finds qcow2.py on PATH. But
if you run './303' directly, there is nothing to adjust PATH, and if
'.' is not already on your PATH by other means, the test fails because
the executable is not found. Adjust how we invoke the helper
executable to avoid needing a PATH search in the first place.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
tests/qemu-iotests/303 | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/qemu-iotests/303 b/tests/qemu-iotests/303
index 6c21774483..11cd9eeb26 100755
--- a/tests/qemu-iotests/303
+++ b/tests/qemu-iotests/303
@@ -56,7 +56,7 @@ qemu_img_create('-f', iotests.imgfmt, disk, '10M')
add_bitmap(1, 0, 6, False)
add_bitmap(2, 6, 8, True)
-dump = ['qcow2.py', disk, 'dump-header']
+dump = ['./qcow2.py', disk, 'dump-header']
subprocess.run(dump)
# Dump the metadata in JSON format
dump.append('-j')
--
2.29.2
^ permalink raw reply related [flat|nested] 43+ messages in thread* Re: [PATCH v6 02/11] iotests/303: use dot slash for qcow2.py running
2021-01-09 12:26 ` [PATCH v6 02/11] iotests/303: use dot slash for qcow2.py running Vladimir Sementsov-Ogievskiy
@ 2021-01-13 22:38 ` Eric Blake
0 siblings, 0 replies; 43+ messages in thread
From: Eric Blake @ 2021-01-13 22:38 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, den, jsnow, qemu-devel, mreitz
On 1/9/21 6:26 AM, Vladimir Sementsov-Ogievskiy wrote:
> If you run './check 303', check includes common.config which adjusts
> $PATH to include '.' first, and therefore finds qcow2.py on PATH. But
> if you run './303' directly, there is nothing to adjust PATH, and if
> '.' is not already on your PATH by other means, the test fails because
> the executable is not found. Adjust how we invoke the helper
> executable to avoid needing a PATH search in the first place.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> tests/qemu-iotests/303 | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
Reviewed-by: Eric Blake <eblake@redhat.com>
>
> diff --git a/tests/qemu-iotests/303 b/tests/qemu-iotests/303
> index 6c21774483..11cd9eeb26 100755
> --- a/tests/qemu-iotests/303
> +++ b/tests/qemu-iotests/303
> @@ -56,7 +56,7 @@ qemu_img_create('-f', iotests.imgfmt, disk, '10M')
>
> add_bitmap(1, 0, 6, False)
> add_bitmap(2, 6, 8, True)
> -dump = ['qcow2.py', disk, 'dump-header']
> +dump = ['./qcow2.py', disk, 'dump-header']
> subprocess.run(dump)
> # Dump the metadata in JSON format
> dump.append('-j')
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 43+ messages in thread
* [PATCH v6 03/11] iotests: fix some whitespaces in test output files
2021-01-09 12:26 [PATCH v6 00/11] Rework iotests/check Vladimir Sementsov-Ogievskiy
2021-01-09 12:26 ` [PATCH v6 01/11] iotests/277: use dot slash for nbd-fault-injector.py running Vladimir Sementsov-Ogievskiy
2021-01-09 12:26 ` [PATCH v6 02/11] iotests/303: use dot slash for qcow2.py running Vladimir Sementsov-Ogievskiy
@ 2021-01-09 12:26 ` Vladimir Sementsov-Ogievskiy
2021-01-13 22:51 ` Eric Blake
2021-01-09 12:26 ` [PATCH v6 04/11] iotests: make tests executable Vladimir Sementsov-Ogievskiy
` (7 subsequent siblings)
10 siblings, 1 reply; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-09 12:26 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, vsementsov, jsnow, qemu-devel, mreitz, den
We are going to be stricter about comparing test result with .out
files. So, fix some whitespaces now.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
tests/qemu-iotests/175.out | 2 +-
tests/qemu-iotests/271.out | 12 ++++++------
tests/qemu-iotests/287.out | 10 +++++-----
3 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/tests/qemu-iotests/175.out b/tests/qemu-iotests/175.out
index 39c2ee0f62..40a5bd1ce6 100644
--- a/tests/qemu-iotests/175.out
+++ b/tests/qemu-iotests/175.out
@@ -23,4 +23,4 @@ size=4096, min allocation
== resize empty image with block_resize ==
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=0
size=1048576, min allocation
- *** done
+*** done
diff --git a/tests/qemu-iotests/271.out b/tests/qemu-iotests/271.out
index 92deb7ebb0..81043ba4d7 100644
--- a/tests/qemu-iotests/271.out
+++ b/tests/qemu-iotests/271.out
@@ -500,7 +500,7 @@ L2 entry #0: 0x8000000000050001 0000000000000001
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
L2 entry #1: 0x8000000000060000 00000001ffffffff
-qcow2: Marking image as corrupt: Invalid cluster entry found (L2 offset: 0x40000, L2 index: 0x1); further corruption events will be suppressed
+qcow2: Marking image as corrupt: Invalid cluster entry found (L2 offset: 0x40000, L2 index: 0x1); further corruption events will be suppressed
write failed: Input/output error
### Corrupted L2 entries - write test (unallocated) ###
@@ -515,14 +515,14 @@ L2 entry #0: 0x8000000000060000 0000000000000001
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
L2 entry #0: 0x0000000000000000 0000000000000001
-qcow2: Marking image as corrupt: Invalid cluster entry found (L2 offset: 0x40000, L2 index: 0); further corruption events will be suppressed
+qcow2: Marking image as corrupt: Invalid cluster entry found (L2 offset: 0x40000, L2 index: 0); further corruption events will be suppressed
write failed: Input/output error
# Both 'subcluster is zero' and 'subcluster is allocated' bits set
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
L2 entry #1: 0x0000000000000000 0000000100000001
-qcow2: Marking image as corrupt: Invalid cluster entry found (L2 offset: 0x40000, L2 index: 0x1); further corruption events will be suppressed
+qcow2: Marking image as corrupt: Invalid cluster entry found (L2 offset: 0x40000, L2 index: 0x1); further corruption events will be suppressed
write failed: Input/output error
### Compressed cluster with subcluster bitmap != 0 - write test ###
@@ -583,7 +583,7 @@ read 524288/524288 bytes at offset 0
read 524288/524288 bytes at offset 524288
512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
Offset Length Mapped to File
-0 0x80000 0 TEST_DIR/t.qcow2.base
+0 0x80000 0 TEST_DIR/t.qcow2.base
# backing file and preallocation=falloc
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=524288 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=raw preallocation=falloc
Image resized.
@@ -592,7 +592,7 @@ read 524288/524288 bytes at offset 0
read 524288/524288 bytes at offset 524288
512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
Offset Length Mapped to File
-0 0x80000 0 TEST_DIR/t.qcow2.base
+0 0x80000 0 TEST_DIR/t.qcow2.base
# backing file and preallocation=full
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=524288 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=raw preallocation=full
Image resized.
@@ -601,7 +601,7 @@ read 524288/524288 bytes at offset 0
read 524288/524288 bytes at offset 524288
512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
Offset Length Mapped to File
-0 0x80000 0 TEST_DIR/t.qcow2.base
+0 0x80000 0 TEST_DIR/t.qcow2.base
### Image resizing with preallocation and backing files ###
diff --git a/tests/qemu-iotests/287.out b/tests/qemu-iotests/287.out
index 6b9dfb4af0..49ab6a27d5 100644
--- a/tests/qemu-iotests/287.out
+++ b/tests/qemu-iotests/287.out
@@ -10,22 +10,22 @@ incompatible_features []
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
incompatible_features [3]
-=== Testing zlib with incompatible bit set ===
+=== Testing zlib with incompatible bit set ===
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
incompatible_features [3]
-=== Testing zstd with incompatible bit unset ===
+=== Testing zstd with incompatible bit unset ===
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
incompatible_features []
-=== Testing compression type values ===
+=== Testing compression type values ===
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
- 0
+0
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
- 1
+1
=== Testing simple reading and writing with zstd ===
--
2.29.2
^ permalink raw reply related [flat|nested] 43+ messages in thread* Re: [PATCH v6 03/11] iotests: fix some whitespaces in test output files
2021-01-09 12:26 ` [PATCH v6 03/11] iotests: fix some whitespaces in test output files Vladimir Sementsov-Ogievskiy
@ 2021-01-13 22:51 ` Eric Blake
0 siblings, 0 replies; 43+ messages in thread
From: Eric Blake @ 2021-01-13 22:51 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, den, jsnow, qemu-devel, mreitz
On 1/9/21 6:26 AM, Vladimir Sementsov-Ogievskiy wrote:
> We are going to be stricter about comparing test result with .out
> files. So, fix some whitespaces now.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> tests/qemu-iotests/175.out | 2 +-
> tests/qemu-iotests/271.out | 12 ++++++------
> tests/qemu-iotests/287.out | 10 +++++-----
> 3 files changed, 12 insertions(+), 12 deletions(-)
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 43+ messages in thread
* [PATCH v6 04/11] iotests: make tests executable
2021-01-09 12:26 [PATCH v6 00/11] Rework iotests/check Vladimir Sementsov-Ogievskiy
` (2 preceding siblings ...)
2021-01-09 12:26 ` [PATCH v6 03/11] iotests: fix some whitespaces in test output files Vladimir Sementsov-Ogievskiy
@ 2021-01-09 12:26 ` Vladimir Sementsov-Ogievskiy
2021-01-13 22:53 ` Eric Blake
2021-01-09 12:26 ` [PATCH v6 05/11] iotests/294: add shebang line Vladimir Sementsov-Ogievskiy
` (6 subsequent siblings)
10 siblings, 1 reply; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-09 12:26 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, vsementsov, jsnow, qemu-devel, mreitz, den
All other test files are executable. Fix these.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
tests/qemu-iotests/283 | 0
tests/qemu-iotests/298 | 0
tests/qemu-iotests/299 | 0
3 files changed, 0 insertions(+), 0 deletions(-)
mode change 100644 => 100755 tests/qemu-iotests/283
mode change 100644 => 100755 tests/qemu-iotests/298
mode change 100644 => 100755 tests/qemu-iotests/299
diff --git a/tests/qemu-iotests/283 b/tests/qemu-iotests/283
old mode 100644
new mode 100755
diff --git a/tests/qemu-iotests/298 b/tests/qemu-iotests/298
old mode 100644
new mode 100755
diff --git a/tests/qemu-iotests/299 b/tests/qemu-iotests/299
old mode 100644
new mode 100755
--
2.29.2
^ permalink raw reply [flat|nested] 43+ messages in thread* Re: [PATCH v6 04/11] iotests: make tests executable
2021-01-09 12:26 ` [PATCH v6 04/11] iotests: make tests executable Vladimir Sementsov-Ogievskiy
@ 2021-01-13 22:53 ` Eric Blake
0 siblings, 0 replies; 43+ messages in thread
From: Eric Blake @ 2021-01-13 22:53 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, den, jsnow, qemu-devel, mreitz
On 1/9/21 6:26 AM, Vladimir Sementsov-Ogievskiy wrote:
> All other test files are executable. Fix these.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> tests/qemu-iotests/283 | 0
> tests/qemu-iotests/298 | 0
> tests/qemu-iotests/299 | 0
> 3 files changed, 0 insertions(+), 0 deletions(-)
> mode change 100644 => 100755 tests/qemu-iotests/283
> mode change 100644 => 100755 tests/qemu-iotests/298
> mode change 100644 => 100755 tests/qemu-iotests/299
Reviewed-by: Eric Blake <eblake@redhat.com>
>
> diff --git a/tests/qemu-iotests/283 b/tests/qemu-iotests/283
> old mode 100644
> new mode 100755
> diff --git a/tests/qemu-iotests/298 b/tests/qemu-iotests/298
> old mode 100644
> new mode 100755
> diff --git a/tests/qemu-iotests/299 b/tests/qemu-iotests/299
> old mode 100644
> new mode 100755
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 43+ messages in thread
* [PATCH v6 05/11] iotests/294: add shebang line
2021-01-09 12:26 [PATCH v6 00/11] Rework iotests/check Vladimir Sementsov-Ogievskiy
` (3 preceding siblings ...)
2021-01-09 12:26 ` [PATCH v6 04/11] iotests: make tests executable Vladimir Sementsov-Ogievskiy
@ 2021-01-09 12:26 ` Vladimir Sementsov-Ogievskiy
2021-01-13 22:54 ` Eric Blake
2021-01-09 12:26 ` [PATCH v6 06/11] iotests: define group in each iotest Vladimir Sementsov-Ogievskiy
` (5 subsequent siblings)
10 siblings, 1 reply; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-09 12:26 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, vsementsov, jsnow, qemu-devel, mreitz, den
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
tests/qemu-iotests/294 | 1 +
1 file changed, 1 insertion(+)
diff --git a/tests/qemu-iotests/294 b/tests/qemu-iotests/294
index 87da35db49..4c375ed609 100755
--- a/tests/qemu-iotests/294
+++ b/tests/qemu-iotests/294
@@ -1,3 +1,4 @@
+#!/usr/bin/env bash
#
# Copyright (C) 2019 Red Hat, Inc.
#
--
2.29.2
^ permalink raw reply related [flat|nested] 43+ messages in thread* Re: [PATCH v6 05/11] iotests/294: add shebang line
2021-01-09 12:26 ` [PATCH v6 05/11] iotests/294: add shebang line Vladimir Sementsov-Ogievskiy
@ 2021-01-13 22:54 ` Eric Blake
0 siblings, 0 replies; 43+ messages in thread
From: Eric Blake @ 2021-01-13 22:54 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, den, jsnow, qemu-devel, mreitz
On 1/9/21 6:26 AM, Vladimir Sementsov-Ogievskiy wrote:
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> tests/qemu-iotests/294 | 1 +
> 1 file changed, 1 insertion(+)
Reviewed-by: Eric Blake <eblake@redhat.com>
>
> diff --git a/tests/qemu-iotests/294 b/tests/qemu-iotests/294
> index 87da35db49..4c375ed609 100755
> --- a/tests/qemu-iotests/294
> +++ b/tests/qemu-iotests/294
> @@ -1,3 +1,4 @@
> +#!/usr/bin/env bash
> #
> # Copyright (C) 2019 Red Hat, Inc.
> #
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 43+ messages in thread
* [PATCH v6 06/11] iotests: define group in each iotest
2021-01-09 12:26 [PATCH v6 00/11] Rework iotests/check Vladimir Sementsov-Ogievskiy
` (4 preceding siblings ...)
2021-01-09 12:26 ` [PATCH v6 05/11] iotests/294: add shebang line Vladimir Sementsov-Ogievskiy
@ 2021-01-09 12:26 ` Vladimir Sementsov-Ogievskiy
2021-01-13 23:01 ` Eric Blake
2021-01-09 12:26 ` [PATCH v6 07/11] iotests: add findtests.py Vladimir Sementsov-Ogievskiy
` (4 subsequent siblings)
10 siblings, 1 reply; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-09 12:26 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, vsementsov, jsnow, qemu-devel, mreitz, den
We are going to drop group file. Define group in tests as a preparatory
step.
The patch is generated by
cd tests/qemu-iotests
grep '^[0-9]\{3\} ' group | while read line; do
file=$(awk '{print $1}' <<< "$line");
groups=$(sed -e 's/^... //' <<< "$line");
awk "NR==2{print \"# group: $groups\"}1" $file > tmp;
cat tmp > $file;
done
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
tests/qemu-iotests/001 | 1 +
tests/qemu-iotests/002 | 1 +
tests/qemu-iotests/003 | 1 +
tests/qemu-iotests/004 | 1 +
tests/qemu-iotests/005 | 1 +
tests/qemu-iotests/007 | 1 +
tests/qemu-iotests/008 | 1 +
tests/qemu-iotests/009 | 1 +
tests/qemu-iotests/010 | 1 +
tests/qemu-iotests/011 | 1 +
tests/qemu-iotests/012 | 1 +
tests/qemu-iotests/013 | 1 +
tests/qemu-iotests/014 | 1 +
tests/qemu-iotests/015 | 1 +
tests/qemu-iotests/017 | 1 +
tests/qemu-iotests/018 | 1 +
tests/qemu-iotests/019 | 1 +
tests/qemu-iotests/020 | 1 +
tests/qemu-iotests/021 | 1 +
tests/qemu-iotests/022 | 1 +
tests/qemu-iotests/023 | 1 +
tests/qemu-iotests/024 | 1 +
tests/qemu-iotests/025 | 1 +
tests/qemu-iotests/026 | 1 +
tests/qemu-iotests/027 | 1 +
tests/qemu-iotests/028 | 1 +
tests/qemu-iotests/029 | 1 +
tests/qemu-iotests/030 | 1 +
tests/qemu-iotests/031 | 1 +
tests/qemu-iotests/032 | 1 +
tests/qemu-iotests/033 | 1 +
tests/qemu-iotests/034 | 1 +
tests/qemu-iotests/035 | 1 +
tests/qemu-iotests/036 | 1 +
tests/qemu-iotests/037 | 1 +
tests/qemu-iotests/038 | 1 +
tests/qemu-iotests/039 | 1 +
tests/qemu-iotests/040 | 1 +
tests/qemu-iotests/041 | 1 +
tests/qemu-iotests/042 | 1 +
tests/qemu-iotests/043 | 1 +
tests/qemu-iotests/044 | 1 +
tests/qemu-iotests/045 | 1 +
tests/qemu-iotests/046 | 1 +
tests/qemu-iotests/047 | 1 +
tests/qemu-iotests/048 | 1 +
tests/qemu-iotests/049 | 1 +
tests/qemu-iotests/050 | 1 +
tests/qemu-iotests/051 | 1 +
tests/qemu-iotests/052 | 1 +
tests/qemu-iotests/053 | 1 +
tests/qemu-iotests/054 | 1 +
tests/qemu-iotests/055 | 1 +
tests/qemu-iotests/056 | 1 +
tests/qemu-iotests/057 | 1 +
tests/qemu-iotests/058 | 1 +
tests/qemu-iotests/059 | 1 +
tests/qemu-iotests/060 | 1 +
tests/qemu-iotests/061 | 1 +
tests/qemu-iotests/062 | 1 +
tests/qemu-iotests/063 | 1 +
tests/qemu-iotests/064 | 1 +
tests/qemu-iotests/065 | 1 +
tests/qemu-iotests/066 | 1 +
tests/qemu-iotests/068 | 1 +
tests/qemu-iotests/069 | 1 +
tests/qemu-iotests/070 | 1 +
tests/qemu-iotests/071 | 1 +
tests/qemu-iotests/072 | 1 +
tests/qemu-iotests/073 | 1 +
tests/qemu-iotests/074 | 1 +
tests/qemu-iotests/075 | 1 +
tests/qemu-iotests/076 | 1 +
tests/qemu-iotests/077 | 1 +
tests/qemu-iotests/078 | 1 +
tests/qemu-iotests/079 | 1 +
tests/qemu-iotests/080 | 1 +
tests/qemu-iotests/081 | 1 +
tests/qemu-iotests/082 | 1 +
tests/qemu-iotests/083 | 1 +
tests/qemu-iotests/084 | 1 +
tests/qemu-iotests/085 | 1 +
tests/qemu-iotests/086 | 1 +
tests/qemu-iotests/087 | 1 +
tests/qemu-iotests/088 | 1 +
tests/qemu-iotests/089 | 1 +
tests/qemu-iotests/090 | 1 +
tests/qemu-iotests/091 | 1 +
tests/qemu-iotests/092 | 1 +
tests/qemu-iotests/093 | 1 +
tests/qemu-iotests/094 | 1 +
tests/qemu-iotests/095 | 1 +
tests/qemu-iotests/096 | 1 +
tests/qemu-iotests/097 | 1 +
tests/qemu-iotests/098 | 1 +
tests/qemu-iotests/099 | 1 +
tests/qemu-iotests/101 | 1 +
tests/qemu-iotests/102 | 1 +
tests/qemu-iotests/103 | 1 +
tests/qemu-iotests/104 | 1 +
tests/qemu-iotests/105 | 1 +
tests/qemu-iotests/106 | 1 +
tests/qemu-iotests/107 | 1 +
tests/qemu-iotests/108 | 1 +
tests/qemu-iotests/109 | 1 +
tests/qemu-iotests/110 | 1 +
tests/qemu-iotests/111 | 1 +
tests/qemu-iotests/112 | 1 +
tests/qemu-iotests/113 | 1 +
tests/qemu-iotests/114 | 1 +
tests/qemu-iotests/115 | 1 +
tests/qemu-iotests/116 | 1 +
tests/qemu-iotests/117 | 1 +
tests/qemu-iotests/118 | 1 +
tests/qemu-iotests/119 | 1 +
tests/qemu-iotests/120 | 1 +
tests/qemu-iotests/121 | 1 +
tests/qemu-iotests/122 | 1 +
tests/qemu-iotests/123 | 1 +
tests/qemu-iotests/124 | 1 +
tests/qemu-iotests/125 | 1 +
tests/qemu-iotests/126 | 1 +
tests/qemu-iotests/127 | 1 +
tests/qemu-iotests/128 | 1 +
tests/qemu-iotests/129 | 1 +
tests/qemu-iotests/130 | 1 +
tests/qemu-iotests/131 | 1 +
tests/qemu-iotests/132 | 1 +
tests/qemu-iotests/133 | 1 +
tests/qemu-iotests/134 | 1 +
tests/qemu-iotests/135 | 1 +
tests/qemu-iotests/136 | 1 +
tests/qemu-iotests/137 | 1 +
tests/qemu-iotests/138 | 1 +
tests/qemu-iotests/139 | 1 +
tests/qemu-iotests/140 | 1 +
tests/qemu-iotests/141 | 1 +
tests/qemu-iotests/143 | 1 +
tests/qemu-iotests/144 | 1 +
tests/qemu-iotests/145 | 1 +
tests/qemu-iotests/146 | 1 +
tests/qemu-iotests/147 | 1 +
tests/qemu-iotests/148 | 1 +
tests/qemu-iotests/149 | 1 +
tests/qemu-iotests/150 | 1 +
tests/qemu-iotests/151 | 1 +
tests/qemu-iotests/152 | 1 +
tests/qemu-iotests/153 | 1 +
tests/qemu-iotests/154 | 1 +
tests/qemu-iotests/155 | 1 +
tests/qemu-iotests/156 | 1 +
tests/qemu-iotests/157 | 1 +
tests/qemu-iotests/158 | 1 +
tests/qemu-iotests/159 | 1 +
tests/qemu-iotests/160 | 1 +
tests/qemu-iotests/161 | 1 +
tests/qemu-iotests/162 | 1 +
tests/qemu-iotests/163 | 1 +
tests/qemu-iotests/165 | 1 +
tests/qemu-iotests/169 | 1 +
tests/qemu-iotests/170 | 1 +
tests/qemu-iotests/171 | 1 +
tests/qemu-iotests/172 | 1 +
tests/qemu-iotests/173 | 1 +
tests/qemu-iotests/174 | 1 +
tests/qemu-iotests/175 | 1 +
tests/qemu-iotests/176 | 1 +
tests/qemu-iotests/177 | 1 +
tests/qemu-iotests/178 | 1 +
tests/qemu-iotests/179 | 1 +
tests/qemu-iotests/181 | 1 +
tests/qemu-iotests/182 | 1 +
tests/qemu-iotests/183 | 1 +
tests/qemu-iotests/184 | 1 +
tests/qemu-iotests/185 | 1 +
tests/qemu-iotests/186 | 1 +
tests/qemu-iotests/187 | 1 +
tests/qemu-iotests/188 | 1 +
tests/qemu-iotests/189 | 1 +
tests/qemu-iotests/190 | 1 +
tests/qemu-iotests/191 | 1 +
tests/qemu-iotests/192 | 1 +
tests/qemu-iotests/194 | 1 +
tests/qemu-iotests/195 | 1 +
tests/qemu-iotests/196 | 1 +
tests/qemu-iotests/197 | 1 +
tests/qemu-iotests/198 | 1 +
tests/qemu-iotests/199 | 1 +
tests/qemu-iotests/200 | 1 +
tests/qemu-iotests/201 | 1 +
tests/qemu-iotests/202 | 1 +
tests/qemu-iotests/203 | 1 +
tests/qemu-iotests/204 | 1 +
tests/qemu-iotests/205 | 1 +
tests/qemu-iotests/206 | 1 +
tests/qemu-iotests/207 | 1 +
tests/qemu-iotests/208 | 1 +
tests/qemu-iotests/209 | 1 +
tests/qemu-iotests/210 | 1 +
tests/qemu-iotests/211 | 1 +
tests/qemu-iotests/212 | 1 +
tests/qemu-iotests/213 | 1 +
tests/qemu-iotests/214 | 1 +
tests/qemu-iotests/215 | 1 +
tests/qemu-iotests/216 | 1 +
tests/qemu-iotests/217 | 1 +
tests/qemu-iotests/218 | 1 +
tests/qemu-iotests/219 | 1 +
tests/qemu-iotests/220 | 1 +
tests/qemu-iotests/221 | 1 +
tests/qemu-iotests/222 | 1 +
tests/qemu-iotests/223 | 1 +
tests/qemu-iotests/224 | 1 +
tests/qemu-iotests/225 | 1 +
tests/qemu-iotests/226 | 1 +
tests/qemu-iotests/227 | 1 +
tests/qemu-iotests/228 | 1 +
tests/qemu-iotests/229 | 1 +
tests/qemu-iotests/231 | 1 +
tests/qemu-iotests/232 | 1 +
tests/qemu-iotests/233 | 1 +
tests/qemu-iotests/234 | 1 +
tests/qemu-iotests/235 | 1 +
tests/qemu-iotests/236 | 1 +
tests/qemu-iotests/237 | 1 +
tests/qemu-iotests/238 | 1 +
tests/qemu-iotests/239 | 1 +
tests/qemu-iotests/240 | 1 +
tests/qemu-iotests/241 | 1 +
tests/qemu-iotests/242 | 1 +
tests/qemu-iotests/243 | 1 +
tests/qemu-iotests/244 | 1 +
tests/qemu-iotests/245 | 1 +
tests/qemu-iotests/246 | 1 +
tests/qemu-iotests/247 | 1 +
tests/qemu-iotests/248 | 1 +
tests/qemu-iotests/249 | 1 +
tests/qemu-iotests/250 | 1 +
tests/qemu-iotests/251 | 1 +
tests/qemu-iotests/252 | 1 +
tests/qemu-iotests/253 | 1 +
tests/qemu-iotests/254 | 1 +
tests/qemu-iotests/255 | 1 +
tests/qemu-iotests/256 | 1 +
tests/qemu-iotests/257 | 1 +
tests/qemu-iotests/258 | 1 +
tests/qemu-iotests/259 | 1 +
tests/qemu-iotests/260 | 1 +
tests/qemu-iotests/261 | 1 +
tests/qemu-iotests/262 | 1 +
tests/qemu-iotests/263 | 1 +
tests/qemu-iotests/264 | 1 +
tests/qemu-iotests/265 | 1 +
tests/qemu-iotests/266 | 1 +
tests/qemu-iotests/267 | 1 +
tests/qemu-iotests/268 | 1 +
tests/qemu-iotests/270 | 1 +
tests/qemu-iotests/271 | 1 +
tests/qemu-iotests/272 | 1 +
tests/qemu-iotests/273 | 1 +
tests/qemu-iotests/274 | 1 +
tests/qemu-iotests/277 | 1 +
tests/qemu-iotests/279 | 1 +
tests/qemu-iotests/280 | 1 +
tests/qemu-iotests/281 | 1 +
tests/qemu-iotests/282 | 1 +
tests/qemu-iotests/283 | 1 +
tests/qemu-iotests/284 | 1 +
tests/qemu-iotests/286 | 1 +
tests/qemu-iotests/287 | 1 +
tests/qemu-iotests/288 | 1 +
tests/qemu-iotests/289 | 1 +
tests/qemu-iotests/290 | 1 +
tests/qemu-iotests/291 | 1 +
tests/qemu-iotests/292 | 1 +
tests/qemu-iotests/293 | 1 +
tests/qemu-iotests/294 | 1 +
tests/qemu-iotests/295 | 1 +
tests/qemu-iotests/296 | 1 +
tests/qemu-iotests/297 | 1 +
tests/qemu-iotests/299 | 1 +
tests/qemu-iotests/300 | 1 +
tests/qemu-iotests/301 | 1 +
tests/qemu-iotests/302 | 1 +
tests/qemu-iotests/303 | 1 +
tests/qemu-iotests/304 | 1 +
tests/qemu-iotests/305 | 1 +
tests/qemu-iotests/307 | 1 +
tests/qemu-iotests/308 | 1 +
tests/qemu-iotests/309 | 1 +
tests/qemu-iotests/312 | 1 +
291 files changed, 291 insertions(+)
diff --git a/tests/qemu-iotests/001 b/tests/qemu-iotests/001
index 696726e45f..6f980fd34d 100755
--- a/tests/qemu-iotests/001
+++ b/tests/qemu-iotests/001
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test simple read/write using plain bdrv_pread/bdrv_pwrite
#
diff --git a/tests/qemu-iotests/002 b/tests/qemu-iotests/002
index 1a0d411df5..5ce1647531 100755
--- a/tests/qemu-iotests/002
+++ b/tests/qemu-iotests/002
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test simple read/write using plain bdrv_pread/bdrv_pwrite
#
diff --git a/tests/qemu-iotests/003 b/tests/qemu-iotests/003
index 33eeade0de..03f902a83c 100755
--- a/tests/qemu-iotests/003
+++ b/tests/qemu-iotests/003
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto
#
# Test simple read/write using bdrv_aio_readv/bdrv_aio_writev
#
diff --git a/tests/qemu-iotests/004 b/tests/qemu-iotests/004
index d308dc4b49..e955579a67 100755
--- a/tests/qemu-iotests/004
+++ b/tests/qemu-iotests/004
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Make sure we can't read and write outside of the image size.
#
diff --git a/tests/qemu-iotests/005 b/tests/qemu-iotests/005
index b6d03ac37d..40e64a9a8f 100755
--- a/tests/qemu-iotests/005
+++ b/tests/qemu-iotests/005
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: img auto quick
#
# Make sure qemu-img can create 5TB images
#
diff --git a/tests/qemu-iotests/007 b/tests/qemu-iotests/007
index 160683adf8..936d3f14fb 100755
--- a/tests/qemu-iotests/007
+++ b/tests/qemu-iotests/007
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: snapshot auto
#
# Check for one possible case of qcow2 refcount corruption.
#
diff --git a/tests/qemu-iotests/008 b/tests/qemu-iotests/008
index 2b81b119bf..fa4990b513 100755
--- a/tests/qemu-iotests/008
+++ b/tests/qemu-iotests/008
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test simple asynchronous read/write operations.
#
diff --git a/tests/qemu-iotests/009 b/tests/qemu-iotests/009
index 4dc7d210f9..efa852bad3 100755
--- a/tests/qemu-iotests/009
+++ b/tests/qemu-iotests/009
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Nolan I qcow2 corruption - incorrectly reports free clusters
#
diff --git a/tests/qemu-iotests/010 b/tests/qemu-iotests/010
index df809b3088..4ae9027b47 100755
--- a/tests/qemu-iotests/010
+++ b/tests/qemu-iotests/010
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Nolan II qcow2 corruption - wrong used cluster
#
diff --git a/tests/qemu-iotests/011 b/tests/qemu-iotests/011
index 57b99ae4a9..5c99ac987f 100755
--- a/tests/qemu-iotests/011
+++ b/tests/qemu-iotests/011
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test for AIO allocation on the same cluster
#
diff --git a/tests/qemu-iotests/012 b/tests/qemu-iotests/012
index 12957285b3..3a24d2ca8d 100755
--- a/tests/qemu-iotests/012
+++ b/tests/qemu-iotests/012
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: auto quick
#
# Make sure we can open read-only images
#
diff --git a/tests/qemu-iotests/013 b/tests/qemu-iotests/013
index 5cb9032f16..d39d0cd88b 100755
--- a/tests/qemu-iotests/013
+++ b/tests/qemu-iotests/013
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto
#
# qcow2 pattern test, empty and compressed image - 4k cluster patterns
#
diff --git a/tests/qemu-iotests/014 b/tests/qemu-iotests/014
index e1221c0fff..2d23469332 100755
--- a/tests/qemu-iotests/014
+++ b/tests/qemu-iotests/014
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw
#
# qcow2 pattern test, complex patterns including compression and snapshots
# Using patterns for 4k cluster size.
diff --git a/tests/qemu-iotests/015 b/tests/qemu-iotests/015
index 4d8effd0ae..40c23235a6 100755
--- a/tests/qemu-iotests/015
+++ b/tests/qemu-iotests/015
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw snapshot
#
# Combined test to grow the refcount table and test snapshots.
#
diff --git a/tests/qemu-iotests/017 b/tests/qemu-iotests/017
index 3413e34f27..2024b85e79 100755
--- a/tests/qemu-iotests/017
+++ b/tests/qemu-iotests/017
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw backing auto quick
#
# Simple backing file reads
#
diff --git a/tests/qemu-iotests/018 b/tests/qemu-iotests/018
index 191b461a4d..6fcebbb40e 100755
--- a/tests/qemu-iotests/018
+++ b/tests/qemu-iotests/018
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw backing auto quick
#
# Merge backing file into test image when converting the image
#
diff --git a/tests/qemu-iotests/019 b/tests/qemu-iotests/019
index d3c11256dc..fa4458fd27 100755
--- a/tests/qemu-iotests/019
+++ b/tests/qemu-iotests/019
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw backing auto quick
#
# When using a backing file for the output image in qemu-img convert,
# the backing file clusters must not copied. The data must still be
diff --git a/tests/qemu-iotests/020 b/tests/qemu-iotests/020
index 596505be2d..60c672e17b 100755
--- a/tests/qemu-iotests/020
+++ b/tests/qemu-iotests/020
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw backing auto quick
#
# Commit changes to backing file
#
diff --git a/tests/qemu-iotests/021 b/tests/qemu-iotests/021
index f888269fd4..0fc89df2fe 100755
--- a/tests/qemu-iotests/021
+++ b/tests/qemu-iotests/021
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: io auto quick
#
# Test handling of invalid patterns arguments to qemu-io
#
diff --git a/tests/qemu-iotests/022 b/tests/qemu-iotests/022
index 99eb08f57f..a116cfe255 100755
--- a/tests/qemu-iotests/022
+++ b/tests/qemu-iotests/022
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw snapshot auto
#
# Test bdrv_load/save_vmstate using the usual patterns
#
diff --git a/tests/qemu-iotests/023 b/tests/qemu-iotests/023
index 02ed047820..d19d13ff5d 100755
--- a/tests/qemu-iotests/023
+++ b/tests/qemu-iotests/023
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw
#
# qcow2 pattern test with various cluster sizes
#
diff --git a/tests/qemu-iotests/024 b/tests/qemu-iotests/024
index 12aceb2d41..25a564a150 100755
--- a/tests/qemu-iotests/024
+++ b/tests/qemu-iotests/024
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw backing auto quick
#
# Rebasing COW images
#
diff --git a/tests/qemu-iotests/025 b/tests/qemu-iotests/025
index 1569d912f4..da77ed3154 100755
--- a/tests/qemu-iotests/025
+++ b/tests/qemu-iotests/025
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Resizing images
#
diff --git a/tests/qemu-iotests/026 b/tests/qemu-iotests/026
index 9ecc5880b1..d37e266dad 100755
--- a/tests/qemu-iotests/026
+++ b/tests/qemu-iotests/026
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw blkdbg
#
# qcow2 error path testing
#
diff --git a/tests/qemu-iotests/027 b/tests/qemu-iotests/027
index 494be0921f..b279c88f33 100755
--- a/tests/qemu-iotests/027
+++ b/tests/qemu-iotests/027
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test that sub-cluster allocating writes zero the rest of the cluster
#
diff --git a/tests/qemu-iotests/028 b/tests/qemu-iotests/028
index 57d34aae99..8c391f2adc 100755
--- a/tests/qemu-iotests/028
+++ b/tests/qemu-iotests/028
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw backing quick
#
# Test that backing files can be smaller than the image
#
diff --git a/tests/qemu-iotests/029 b/tests/qemu-iotests/029
index 61d78c00a4..bd71dd2f22 100755
--- a/tests/qemu-iotests/029
+++ b/tests/qemu-iotests/029
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# qcow2 internal snapshots/VM state tests
#
diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030
index dcb4b5d6a6..890784b116 100755
--- a/tests/qemu-iotests/030
+++ b/tests/qemu-iotests/030
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw auto backing
#
# Tests for image streaming.
#
diff --git a/tests/qemu-iotests/031 b/tests/qemu-iotests/031
index 2bcbc5886e..58b57a0ef2 100755
--- a/tests/qemu-iotests/031
+++ b/tests/qemu-iotests/031
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test that all qcow2 header extensions survive a header rewrite
#
diff --git a/tests/qemu-iotests/032 b/tests/qemu-iotests/032
index 8337a4d825..ebbe7cb0ba 100755
--- a/tests/qemu-iotests/032
+++ b/tests/qemu-iotests/032
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test that AIO requests are drained before an image is closed. This used
# to segfault because the request coroutine kept running even after the
diff --git a/tests/qemu-iotests/033 b/tests/qemu-iotests/033
index 8b40991d55..da9133c44b 100755
--- a/tests/qemu-iotests/033
+++ b/tests/qemu-iotests/033
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test aligned and misaligned write zeroes operations.
#
diff --git a/tests/qemu-iotests/034 b/tests/qemu-iotests/034
index 08f7aea6d5..ac1af8f646 100755
--- a/tests/qemu-iotests/034
+++ b/tests/qemu-iotests/034
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto backing quick
#
# Test bdrv_pwrite_zeroes with backing files (see also 154)
#
diff --git a/tests/qemu-iotests/035 b/tests/qemu-iotests/035
index d950a0dd1e..0c0c4fdd42 100755
--- a/tests/qemu-iotests/035
+++ b/tests/qemu-iotests/035
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Let a few AIO requests run in parallel and have them access different L2
# tables so that the cache has a chance to get used up.
diff --git a/tests/qemu-iotests/036 b/tests/qemu-iotests/036
index 6b82638080..5e567012a8 100755
--- a/tests/qemu-iotests/036
+++ b/tests/qemu-iotests/036
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test qcow2 feature bits
#
diff --git a/tests/qemu-iotests/037 b/tests/qemu-iotests/037
index bb893c43dc..85b1015056 100755
--- a/tests/qemu-iotests/037
+++ b/tests/qemu-iotests/037
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto backing quick
#
# Test COW from backing files
#
diff --git a/tests/qemu-iotests/038 b/tests/qemu-iotests/038
index 30f1f73c25..65bf7a753e 100755
--- a/tests/qemu-iotests/038
+++ b/tests/qemu-iotests/038
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto backing quick
#
# Test COW from backing files with AIO
#
diff --git a/tests/qemu-iotests/039 b/tests/qemu-iotests/039
index ad3867c3fc..12b2c7fa7b 100755
--- a/tests/qemu-iotests/039
+++ b/tests/qemu-iotests/039
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test qcow2 lazy refcounts
#
diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040
index dc6069edc0..7ebc9ed825 100755
--- a/tests/qemu-iotests/040
+++ b/tests/qemu-iotests/040
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw auto
#
# Tests for image block commit.
#
diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041
index a7780853cd..5cc02b24fc 100755
--- a/tests/qemu-iotests/041
+++ b/tests/qemu-iotests/041
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw auto backing
#
# Tests for image mirroring.
#
diff --git a/tests/qemu-iotests/042 b/tests/qemu-iotests/042
index e8f23a174c..411e54ae04 100755
--- a/tests/qemu-iotests/042
+++ b/tests/qemu-iotests/042
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test qemu-img operation on zero size images
#
diff --git a/tests/qemu-iotests/043 b/tests/qemu-iotests/043
index 3271737f69..f8ce3288db 100755
--- a/tests/qemu-iotests/043
+++ b/tests/qemu-iotests/043
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto backing
#
# Test that qemu-img info --backing-chain detects infinite loops
#
diff --git a/tests/qemu-iotests/044 b/tests/qemu-iotests/044
index 7e99ea7c68..64b18eb7c8 100755
--- a/tests/qemu-iotests/044
+++ b/tests/qemu-iotests/044
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw
#
# Tests growing a large refcount table.
#
diff --git a/tests/qemu-iotests/045 b/tests/qemu-iotests/045
index 5acc89099c..45eb239baa 100755
--- a/tests/qemu-iotests/045
+++ b/tests/qemu-iotests/045
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Tests for fdsets and getfd.
#
diff --git a/tests/qemu-iotests/046 b/tests/qemu-iotests/046
index ed6fae3529..50b0678f60 100755
--- a/tests/qemu-iotests/046
+++ b/tests/qemu-iotests/046
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto aio quick
#
# Test concurrent cluster allocations
#
diff --git a/tests/qemu-iotests/047 b/tests/qemu-iotests/047
index 4528465fb0..8dd21e0a81 100755
--- a/tests/qemu-iotests/047
+++ b/tests/qemu-iotests/047
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Regression test for commit b7ab0fea (which was a corruption fix,
# despite the commit message claiming otherwise)
diff --git a/tests/qemu-iotests/048 b/tests/qemu-iotests/048
index 2af6b74b41..bf8e4bf528 100755
--- a/tests/qemu-iotests/048
+++ b/tests/qemu-iotests/048
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: img auto quick
##
## qemu-img compare test
##
diff --git a/tests/qemu-iotests/049 b/tests/qemu-iotests/049
index 82b1e6c202..ed12fa49d7 100755
--- a/tests/qemu-iotests/049
+++ b/tests/qemu-iotests/049
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto
#
# Check qemu-img option parsing
#
diff --git a/tests/qemu-iotests/050 b/tests/qemu-iotests/050
index 741bdb610e..1de01c312e 100755
--- a/tests/qemu-iotests/050
+++ b/tests/qemu-iotests/050
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto backing quick
#
# Test qemu-img rebase with zero clusters
#
diff --git a/tests/qemu-iotests/051 b/tests/qemu-iotests/051
index bee26075b2..7cbd1415ce 100755
--- a/tests/qemu-iotests/051
+++ b/tests/qemu-iotests/051
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw
#
# Test command line configuration of block devices and driver-specific options
#
diff --git a/tests/qemu-iotests/052 b/tests/qemu-iotests/052
index 8d5c10601f..2f23ac9b65 100755
--- a/tests/qemu-iotests/052
+++ b/tests/qemu-iotests/052
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto backing quick
#
# Test bdrv_pread/bdrv_pwrite using BDRV_O_SNAPSHOT
#
diff --git a/tests/qemu-iotests/053 b/tests/qemu-iotests/053
index 71d299c4f9..9a2958d42d 100755
--- a/tests/qemu-iotests/053
+++ b/tests/qemu-iotests/053
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test qemu-img convert when image length is not a multiple of cluster size
#
diff --git a/tests/qemu-iotests/054 b/tests/qemu-iotests/054
index 40922db2b1..ea147012c3 100755
--- a/tests/qemu-iotests/054
+++ b/tests/qemu-iotests/054
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test huge qcow2 images
#
diff --git a/tests/qemu-iotests/055 b/tests/qemu-iotests/055
index 4d3744b0d3..5d6b607051 100755
--- a/tests/qemu-iotests/055
+++ b/tests/qemu-iotests/055
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw
#
# Tests for drive-backup and blockdev-backup
#
diff --git a/tests/qemu-iotests/056 b/tests/qemu-iotests/056
index 052456aa00..0e6b8591e7 100755
--- a/tests/qemu-iotests/056
+++ b/tests/qemu-iotests/056
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw backing
#
# Tests for drive-backup
#
diff --git a/tests/qemu-iotests/057 b/tests/qemu-iotests/057
index a8b4bb60e0..b0d431999e 100755
--- a/tests/qemu-iotests/057
+++ b/tests/qemu-iotests/057
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw
#
# Tests for internal snapshot.
#
diff --git a/tests/qemu-iotests/058 b/tests/qemu-iotests/058
index d84740ed9f..ce35ff4ee0 100755
--- a/tests/qemu-iotests/058
+++ b/tests/qemu-iotests/058
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test export internal snapshot by qemu-nbd, convert it by qemu-img.
#
diff --git a/tests/qemu-iotests/059 b/tests/qemu-iotests/059
index dcc442be9f..65c0c32b26 100755
--- a/tests/qemu-iotests/059
+++ b/tests/qemu-iotests/059
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test case for vmdk
#
diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060
index 4b81d1aa51..db26c6b246 100755
--- a/tests/qemu-iotests/060
+++ b/tests/qemu-iotests/060
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test case for image corruption (overlapping data structures) in qcow2
#
diff --git a/tests/qemu-iotests/061 b/tests/qemu-iotests/061
index 5747beb7ed..e26d94a0df 100755
--- a/tests/qemu-iotests/061
+++ b/tests/qemu-iotests/061
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto
#
# Test case for image option amendment in qcow2.
#
diff --git a/tests/qemu-iotests/062 b/tests/qemu-iotests/062
index f26b88df9d..321252298d 100755
--- a/tests/qemu-iotests/062
+++ b/tests/qemu-iotests/062
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test case for snapshotting images with unallocated zero clusters in
# qcow2
diff --git a/tests/qemu-iotests/063 b/tests/qemu-iotests/063
index c750b3806e..3a44758053 100755
--- a/tests/qemu-iotests/063
+++ b/tests/qemu-iotests/063
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# test of qemu-img convert -n - convert without creation
#
diff --git a/tests/qemu-iotests/064 b/tests/qemu-iotests/064
index 90673186ec..71fc575b21 100755
--- a/tests/qemu-iotests/064
+++ b/tests/qemu-iotests/064
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test VHDX read/write from a sample image created with Hyper-V
#
diff --git a/tests/qemu-iotests/065 b/tests/qemu-iotests/065
index 29a7f7ad60..3c2ca27627 100755
--- a/tests/qemu-iotests/065
+++ b/tests/qemu-iotests/065
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Test for additional information emitted by qemu-img info on qcow2
# images
diff --git a/tests/qemu-iotests/066 b/tests/qemu-iotests/066
index a4ac613f8e..a780ed7ab5 100755
--- a/tests/qemu-iotests/066
+++ b/tests/qemu-iotests/066
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test case for preallocated zero clusters in qcow2
#
diff --git a/tests/qemu-iotests/068 b/tests/qemu-iotests/068
index ccd1a9f1db..03e03508a6 100755
--- a/tests/qemu-iotests/068
+++ b/tests/qemu-iotests/068
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test case for loading a saved VM state from a qcow2 image
#
diff --git a/tests/qemu-iotests/069 b/tests/qemu-iotests/069
index a4da83b2d9..222dcba741 100755
--- a/tests/qemu-iotests/069
+++ b/tests/qemu-iotests/069
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test case for deleting a backing file
#
diff --git a/tests/qemu-iotests/070 b/tests/qemu-iotests/070
index cb0f927c16..b181e00f9b 100755
--- a/tests/qemu-iotests/070
+++ b/tests/qemu-iotests/070
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test VHDX log replay from an image with a journal that needs to be
# replayed
diff --git a/tests/qemu-iotests/071 b/tests/qemu-iotests/071
index 49faae6684..d99cef5a42 100755
--- a/tests/qemu-iotests/071
+++ b/tests/qemu-iotests/071
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test case for the QMP blkdebug and blkverify interfaces
#
diff --git a/tests/qemu-iotests/072 b/tests/qemu-iotests/072
index f0b73e7e65..c492ab8a78 100755
--- a/tests/qemu-iotests/072
+++ b/tests/qemu-iotests/072
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test case for nested image formats
#
diff --git a/tests/qemu-iotests/073 b/tests/qemu-iotests/073
index 68517821e8..90afd420bd 100755
--- a/tests/qemu-iotests/073
+++ b/tests/qemu-iotests/073
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test count_contiguous_clusters in qcow2
#
diff --git a/tests/qemu-iotests/074 b/tests/qemu-iotests/074
index db03edf0b0..c32c94b50d 100755
--- a/tests/qemu-iotests/074
+++ b/tests/qemu-iotests/074
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
##
## qemu-img compare test (qcow2 only ones)
##
diff --git a/tests/qemu-iotests/075 b/tests/qemu-iotests/075
index 389d5675fa..ca2ed2a05c 100755
--- a/tests/qemu-iotests/075
+++ b/tests/qemu-iotests/075
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# cloop format input validation tests
#
diff --git a/tests/qemu-iotests/076 b/tests/qemu-iotests/076
index 0d405ef3f2..1a8927d765 100755
--- a/tests/qemu-iotests/076
+++ b/tests/qemu-iotests/076
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: io
#
# parallels format input validation tests
#
diff --git a/tests/qemu-iotests/077 b/tests/qemu-iotests/077
index c284952082..fbb90d8036 100755
--- a/tests/qemu-iotests/077
+++ b/tests/qemu-iotests/077
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test concurrent pread/pwrite
#
diff --git a/tests/qemu-iotests/078 b/tests/qemu-iotests/078
index 54fc654d8e..0b48b7f137 100755
--- a/tests/qemu-iotests/078
+++ b/tests/qemu-iotests/078
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# bochs format input validation tests
#
diff --git a/tests/qemu-iotests/079 b/tests/qemu-iotests/079
index 0f0d94a2ac..793e1f9d08 100755
--- a/tests/qemu-iotests/079
+++ b/tests/qemu-iotests/079
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto
#
# Test qcow2 preallocation with different cluster_sizes
#
diff --git a/tests/qemu-iotests/080 b/tests/qemu-iotests/080
index bda8617c38..3306500683 100755
--- a/tests/qemu-iotests/080
+++ b/tests/qemu-iotests/080
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto
#
# qcow2 format input validation tests
#
diff --git a/tests/qemu-iotests/081 b/tests/qemu-iotests/081
index 4e19972931..1ac66f197e 100755
--- a/tests/qemu-iotests/081
+++ b/tests/qemu-iotests/081
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test Quorum block driver
#
diff --git a/tests/qemu-iotests/082 b/tests/qemu-iotests/082
index 2135581867..021b9bef06 100755
--- a/tests/qemu-iotests/082
+++ b/tests/qemu-iotests/082
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test qemu-img command line parsing
#
diff --git a/tests/qemu-iotests/083 b/tests/qemu-iotests/083
index 10fdfc8ebb..bc32b537b2 100755
--- a/tests/qemu-iotests/083
+++ b/tests/qemu-iotests/083
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw
#
# Test NBD client unexpected disconnect
#
diff --git a/tests/qemu-iotests/084 b/tests/qemu-iotests/084
index c29d7395e9..e51e91a7c8 100755
--- a/tests/qemu-iotests/084
+++ b/tests/qemu-iotests/084
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: img quick
#
# Test case for VDI header corruption; image too large, and too many blocks.
# Also simple test for creating dynamic and static VDI images.
diff --git a/tests/qemu-iotests/085 b/tests/qemu-iotests/085
index e99eb44581..d557522943 100755
--- a/tests/qemu-iotests/085
+++ b/tests/qemu-iotests/085
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw
#
# Live snapshot tests
#
diff --git a/tests/qemu-iotests/086 b/tests/qemu-iotests/086
index fea1a7bd8a..c055e7bfe1 100755
--- a/tests/qemu-iotests/086
+++ b/tests/qemu-iotests/086
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test qemu-img progress output
#
diff --git a/tests/qemu-iotests/087 b/tests/qemu-iotests/087
index 678e748c58..edd43f1a28 100755
--- a/tests/qemu-iotests/087
+++ b/tests/qemu-iotests/087
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test unsupported blockdev-add cases
#
diff --git a/tests/qemu-iotests/088 b/tests/qemu-iotests/088
index ef1163346c..e3102fe888 100755
--- a/tests/qemu-iotests/088
+++ b/tests/qemu-iotests/088
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# vpc (VHD) format input validation tests
#
diff --git a/tests/qemu-iotests/089 b/tests/qemu-iotests/089
index f0929b64c0..48bdc42e42 100755
--- a/tests/qemu-iotests/089
+++ b/tests/qemu-iotests/089
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test case for support of JSON filenames
#
diff --git a/tests/qemu-iotests/090 b/tests/qemu-iotests/090
index 87e872ebf4..2044c09e9b 100755
--- a/tests/qemu-iotests/090
+++ b/tests/qemu-iotests/090
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test for discarding compressed clusters on qcow2 images
#
diff --git a/tests/qemu-iotests/091 b/tests/qemu-iotests/091
index 8dee168bf6..9d144b9439 100755
--- a/tests/qemu-iotests/091
+++ b/tests/qemu-iotests/091
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw migration quick
#
# Live migration test
#
diff --git a/tests/qemu-iotests/092 b/tests/qemu-iotests/092
index 40ec62b6f1..bfa116d191 100755
--- a/tests/qemu-iotests/092
+++ b/tests/qemu-iotests/092
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# qcow1 format input validation tests
#
diff --git a/tests/qemu-iotests/093 b/tests/qemu-iotests/093
index 32ded11430..7745cb04b6 100755
--- a/tests/qemu-iotests/093
+++ b/tests/qemu-iotests/093
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: throttle
#
# Tests for IO throttling
#
diff --git a/tests/qemu-iotests/094 b/tests/qemu-iotests/094
index 2d3e1004d3..a295fb20ef 100755
--- a/tests/qemu-iotests/094
+++ b/tests/qemu-iotests/094
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test case for drive-mirror to NBD
#
diff --git a/tests/qemu-iotests/095 b/tests/qemu-iotests/095
index 7604ae6966..20b5f9bf61 100755
--- a/tests/qemu-iotests/095
+++ b/tests/qemu-iotests/095
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test for commit of larger active layer
#
diff --git a/tests/qemu-iotests/096 b/tests/qemu-iotests/096
index 5915f92786..b5d7636bdc 100755
--- a/tests/qemu-iotests/096
+++ b/tests/qemu-iotests/096
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Test that snapshots move the throttling configuration to the active
# layer
diff --git a/tests/qemu-iotests/097 b/tests/qemu-iotests/097
index 1837d4e8e0..30313f8867 100755
--- a/tests/qemu-iotests/097
+++ b/tests/qemu-iotests/097
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto backing
#
# Commit changes into backing chains and empty the top image if the
# backing image is not explicitly specified
diff --git a/tests/qemu-iotests/098 b/tests/qemu-iotests/098
index a35ce7205e..4c37eb0cf5 100755
--- a/tests/qemu-iotests/098
+++ b/tests/qemu-iotests/098
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto backing quick
#
# Test qcow2's bdrv_make_empty for images without internal snapshots
#
diff --git a/tests/qemu-iotests/099 b/tests/qemu-iotests/099
index 65e8e92572..2f1199ce04 100755
--- a/tests/qemu-iotests/099
+++ b/tests/qemu-iotests/099
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test valid filenames for blkdebug and blkverify representatively for
# other protocols (such as NBD) when queried
diff --git a/tests/qemu-iotests/101 b/tests/qemu-iotests/101
index a4c1b6366a..4c4a8cea11 100755
--- a/tests/qemu-iotests/101
+++ b/tests/qemu-iotests/101
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test short file I/O
#
diff --git a/tests/qemu-iotests/102 b/tests/qemu-iotests/102
index 9d747c7bbf..8b4c4c905f 100755
--- a/tests/qemu-iotests/102
+++ b/tests/qemu-iotests/102
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test case for qemu-io -c map and qemu-img map
#
diff --git a/tests/qemu-iotests/103 b/tests/qemu-iotests/103
index 220481db4c..726f8313ef 100755
--- a/tests/qemu-iotests/103
+++ b/tests/qemu-iotests/103
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test case for qcow2 metadata cache size specification
#
diff --git a/tests/qemu-iotests/104 b/tests/qemu-iotests/104
index c70f28a9a1..3ebb74cf6e 100755
--- a/tests/qemu-iotests/104
+++ b/tests/qemu-iotests/104
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto
#
# Test image creation with aligned and unaligned sizes
#
diff --git a/tests/qemu-iotests/105 b/tests/qemu-iotests/105
index 4d55a2d3ef..d804685110 100755
--- a/tests/qemu-iotests/105
+++ b/tests/qemu-iotests/105
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Create, read, write big image
#
diff --git a/tests/qemu-iotests/106 b/tests/qemu-iotests/106
index 20ad7bd5a2..333144502c 100755
--- a/tests/qemu-iotests/106
+++ b/tests/qemu-iotests/106
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test preallocated resize of raw images
#
diff --git a/tests/qemu-iotests/107 b/tests/qemu-iotests/107
index d24829ccf9..e68f1e07c7 100755
--- a/tests/qemu-iotests/107
+++ b/tests/qemu-iotests/107
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Tests updates of the qcow2 L1 table
#
diff --git a/tests/qemu-iotests/108 b/tests/qemu-iotests/108
index ba67748bdf..8eaef0b8bf 100755
--- a/tests/qemu-iotests/108
+++ b/tests/qemu-iotests/108
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test case for repairing qcow2 images which cannot be repaired using
# the on-disk refcount structures
diff --git a/tests/qemu-iotests/109 b/tests/qemu-iotests/109
index 3ffeaf3c55..e207a555f3 100755
--- a/tests/qemu-iotests/109
+++ b/tests/qemu-iotests/109
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw
#
# Test writing image headers of other formats into raw images
#
diff --git a/tests/qemu-iotests/110 b/tests/qemu-iotests/110
index f1813d0dfb..1fa36ccdb7 100755
--- a/tests/qemu-iotests/110
+++ b/tests/qemu-iotests/110
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto backing quick
#
# Test case for relative backing file names in complex BDS trees
#
diff --git a/tests/qemu-iotests/111 b/tests/qemu-iotests/111
index bd839a39f4..3ba25f6161 100755
--- a/tests/qemu-iotests/111
+++ b/tests/qemu-iotests/111
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test case for non-existing backing file when creating a qcow2 image
# and not specifying the size
diff --git a/tests/qemu-iotests/112 b/tests/qemu-iotests/112
index 6e413f5651..07ac74fb2c 100755
--- a/tests/qemu-iotests/112
+++ b/tests/qemu-iotests/112
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw
#
# Test cases for different refcount_bits values
#
diff --git a/tests/qemu-iotests/113 b/tests/qemu-iotests/113
index 71a65de2e7..ee59b9a4b8 100755
--- a/tests/qemu-iotests/113
+++ b/tests/qemu-iotests/113
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test case for accessing creation options on image formats and
# protocols not supporting image creation
diff --git a/tests/qemu-iotests/114 b/tests/qemu-iotests/114
index 80e5e5e591..43cb0bc6c3 100755
--- a/tests/qemu-iotests/114
+++ b/tests/qemu-iotests/114
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test invalid backing file format in qcow2 images
#
diff --git a/tests/qemu-iotests/115 b/tests/qemu-iotests/115
index 7f53987d1b..26dd37dd6d 100755
--- a/tests/qemu-iotests/115
+++ b/tests/qemu-iotests/115
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw
#
# Test case for non-self-referential qcow2 refcount blocks
#
diff --git a/tests/qemu-iotests/116 b/tests/qemu-iotests/116
index 941b07a1a9..4f40fcb3d2 100755
--- a/tests/qemu-iotests/116
+++ b/tests/qemu-iotests/116
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test error code paths for invalid QED images
#
diff --git a/tests/qemu-iotests/117 b/tests/qemu-iotests/117
index 9039555ac4..48ebc012b1 100755
--- a/tests/qemu-iotests/117
+++ b/tests/qemu-iotests/117
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto
#
# Test case for shared BDS between backend trees
#
diff --git a/tests/qemu-iotests/118 b/tests/qemu-iotests/118
index 2350929fd8..1a2e219057 100755
--- a/tests/qemu-iotests/118
+++ b/tests/qemu-iotests/118
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw
#
# Test case for the QMP 'change' command and all other associated
# commands
diff --git a/tests/qemu-iotests/119 b/tests/qemu-iotests/119
index ea6770a484..5770b50045 100755
--- a/tests/qemu-iotests/119
+++ b/tests/qemu-iotests/119
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# NBD test case for overriding BDRV_O_PROTOCOL by explicitly specifying
# a driver
diff --git a/tests/qemu-iotests/120 b/tests/qemu-iotests/120
index 45c55c1c01..7187731253 100755
--- a/tests/qemu-iotests/120
+++ b/tests/qemu-iotests/120
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Non-NBD test cases for overriding BDRV_O_PROTOCOL by explicitly
# specifying a driver
diff --git a/tests/qemu-iotests/121 b/tests/qemu-iotests/121
index 8357ce089a..ba3d8d9377 100755
--- a/tests/qemu-iotests/121
+++ b/tests/qemu-iotests/121
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw
#
# Test cases for qcow2 refcount table growth
#
diff --git a/tests/qemu-iotests/122 b/tests/qemu-iotests/122
index 0f3d4ca851..5d550ed13e 100755
--- a/tests/qemu-iotests/122
+++ b/tests/qemu-iotests/122
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw
#
# Test some qemu-img convert cases
#
diff --git a/tests/qemu-iotests/123 b/tests/qemu-iotests/123
index 01b771c76e..e19111f70d 100755
--- a/tests/qemu-iotests/123
+++ b/tests/qemu-iotests/123
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test case for qemu-img convert to NBD
#
diff --git a/tests/qemu-iotests/124 b/tests/qemu-iotests/124
index 3705cbb6b3..3b21bc497f 100755
--- a/tests/qemu-iotests/124
+++ b/tests/qemu-iotests/124
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw backing
#
# Tests for incremental drive-backup
#
diff --git a/tests/qemu-iotests/125 b/tests/qemu-iotests/125
index 5720e86dce..bd390b3a99 100755
--- a/tests/qemu-iotests/125
+++ b/tests/qemu-iotests/125
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw
#
# Test preallocated growth of qcow2 images
#
diff --git a/tests/qemu-iotests/126 b/tests/qemu-iotests/126
index dd5a211227..92c0547746 100755
--- a/tests/qemu-iotests/126
+++ b/tests/qemu-iotests/126
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto backing
#
# Tests handling of colons in filenames (which may be confused with protocol
# prefixes)
diff --git a/tests/qemu-iotests/127 b/tests/qemu-iotests/127
index 77fdfd0205..98e8e82a82 100755
--- a/tests/qemu-iotests/127
+++ b/tests/qemu-iotests/127
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto backing quick
#
# Test case for mirroring with dataplane
#
diff --git a/tests/qemu-iotests/128 b/tests/qemu-iotests/128
index 3606c41760..d0e00d24b1 100755
--- a/tests/qemu-iotests/128
+++ b/tests/qemu-iotests/128
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test that opening O_DIRECT succeeds when image file I/O produces EIO
#
diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129
index 0e13244d85..f57a2e19f6 100755
--- a/tests/qemu-iotests/129
+++ b/tests/qemu-iotests/129
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Tests that "bdrv_drain_all" doesn't drain block jobs
#
diff --git a/tests/qemu-iotests/130 b/tests/qemu-iotests/130
index a7b365701c..7257f09677 100755
--- a/tests/qemu-iotests/130
+++ b/tests/qemu-iotests/130
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test that temporary backing file overrides (on the command line or in
# blockdev-add) don't replace the original path stored in the image during
diff --git a/tests/qemu-iotests/131 b/tests/qemu-iotests/131
index 27870231cf..d7456cae5b 100755
--- a/tests/qemu-iotests/131
+++ b/tests/qemu-iotests/131
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# parallels format validation tests (created by QEMU)
#
diff --git a/tests/qemu-iotests/132 b/tests/qemu-iotests/132
index 39ea43067e..367ea08036 100755
--- a/tests/qemu-iotests/132
+++ b/tests/qemu-iotests/132
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Test mirror with unmap
#
diff --git a/tests/qemu-iotests/133 b/tests/qemu-iotests/133
index bc82d8ebd7..d997db1685 100755
--- a/tests/qemu-iotests/133
+++ b/tests/qemu-iotests/133
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: auto quick
#
# Test for reopen
#
diff --git a/tests/qemu-iotests/134 b/tests/qemu-iotests/134
index 17fe1d6ed4..ded153c0b9 100755
--- a/tests/qemu-iotests/134
+++ b/tests/qemu-iotests/134
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test encrypted read/write using plain bdrv_pread/bdrv_pwrite
#
diff --git a/tests/qemu-iotests/135 b/tests/qemu-iotests/135
index 3b3d1dc2a5..299075b4c9 100755
--- a/tests/qemu-iotests/135
+++ b/tests/qemu-iotests/135
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw
#
# Test VPC open of image with large Max Table Entries value.
#
diff --git a/tests/qemu-iotests/136 b/tests/qemu-iotests/136
index d59400c9fc..8fce88bd67 100755
--- a/tests/qemu-iotests/136
+++ b/tests/qemu-iotests/136
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw
#
# Tests for block device statistics
#
diff --git a/tests/qemu-iotests/137 b/tests/qemu-iotests/137
index de555a91c9..4680d5df3d 100755
--- a/tests/qemu-iotests/137
+++ b/tests/qemu-iotests/137
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto
#
# Test qcow2 reopen
#
diff --git a/tests/qemu-iotests/138 b/tests/qemu-iotests/138
index e87a64eb89..951cfa67d4 100755
--- a/tests/qemu-iotests/138
+++ b/tests/qemu-iotests/138
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# General test case for qcow2's image check
#
diff --git a/tests/qemu-iotests/139 b/tests/qemu-iotests/139
index 1452fd24b3..e79b3c21fd 100755
--- a/tests/qemu-iotests/139
+++ b/tests/qemu-iotests/139
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Test cases for the QMP 'blockdev-del' command
#
diff --git a/tests/qemu-iotests/140 b/tests/qemu-iotests/140
index ff6b904fa0..91e08c30d4 100755
--- a/tests/qemu-iotests/140
+++ b/tests/qemu-iotests/140
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test case for ejecting a BlockBackend with an NBD server attached to it
#
diff --git a/tests/qemu-iotests/141 b/tests/qemu-iotests/141
index 21aa0b42d8..115cc1691e 100755
--- a/tests/qemu-iotests/141
+++ b/tests/qemu-iotests/141
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test case for ejecting BDSs with block jobs still running on them
#
diff --git a/tests/qemu-iotests/143 b/tests/qemu-iotests/143
index d2349903b1..72151acf27 100755
--- a/tests/qemu-iotests/143
+++ b/tests/qemu-iotests/143
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: auto quick
#
# Test case for connecting to a non-existing NBD export name
#
diff --git a/tests/qemu-iotests/144 b/tests/qemu-iotests/144
index 4569ac0b4b..60e9ddd75f 100755
--- a/tests/qemu-iotests/144
+++ b/tests/qemu-iotests/144
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
# Check live snapshot, followed by active commit, and another snapshot.
#
# This test is to catch the error case of BZ #1300209:
diff --git a/tests/qemu-iotests/145 b/tests/qemu-iotests/145
index 9427549651..a2ce92516d 100755
--- a/tests/qemu-iotests/145
+++ b/tests/qemu-iotests/145
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: quick
#
# Test the combination of -incoming and snapshot=on
#
diff --git a/tests/qemu-iotests/146 b/tests/qemu-iotests/146
index ddc3c1fd80..98aca96732 100755
--- a/tests/qemu-iotests/146
+++ b/tests/qemu-iotests/146
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: quick
#
# Test VHD image format creator detection and override
#
diff --git a/tests/qemu-iotests/147 b/tests/qemu-iotests/147
index d7a9f31089..47dfa62e6b 100755
--- a/tests/qemu-iotests/147
+++ b/tests/qemu-iotests/147
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: img
#
# Test case for NBD's blockdev-add interface
#
diff --git a/tests/qemu-iotests/148 b/tests/qemu-iotests/148
index 5e14a455b1..7ccbde4633 100755
--- a/tests/qemu-iotests/148
+++ b/tests/qemu-iotests/148
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Test the rate limit of QMP events
#
diff --git a/tests/qemu-iotests/149 b/tests/qemu-iotests/149
index 852768f80a..328fd05a4c 100755
--- a/tests/qemu-iotests/149
+++ b/tests/qemu-iotests/149
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw sudo
#
# Copyright (C) 2016 Red Hat, Inc.
#
diff --git a/tests/qemu-iotests/150 b/tests/qemu-iotests/150
index 3b1f32197a..ac6930ae20 100755
--- a/tests/qemu-iotests/150
+++ b/tests/qemu-iotests/150
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test that qemu-img convert -S 0 fully allocates the target image
#
diff --git a/tests/qemu-iotests/151 b/tests/qemu-iotests/151
index f2df72c29c..182f6b5321 100755
--- a/tests/qemu-iotests/151
+++ b/tests/qemu-iotests/151
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw
#
# Tests for active mirroring
#
diff --git a/tests/qemu-iotests/152 b/tests/qemu-iotests/152
index cc2ea09654..4e179c340f 100755
--- a/tests/qemu-iotests/152
+++ b/tests/qemu-iotests/152
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Tests for drive-mirror with source size unaligned to granularity
#
diff --git a/tests/qemu-iotests/153 b/tests/qemu-iotests/153
index 34045ea3cf..607af59091 100755
--- a/tests/qemu-iotests/153
+++ b/tests/qemu-iotests/153
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test image locking
#
diff --git a/tests/qemu-iotests/154 b/tests/qemu-iotests/154
index 34a1c051b6..24e29ae2ff 100755
--- a/tests/qemu-iotests/154
+++ b/tests/qemu-iotests/154
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto backing quick
#
# qcow2 specific bdrv_pwrite_zeroes tests with backing files (complements 034)
#
diff --git a/tests/qemu-iotests/155 b/tests/qemu-iotests/155
index 988f986144..bafef9dd9a 100755
--- a/tests/qemu-iotests/155
+++ b/tests/qemu-iotests/155
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw
#
# Test whether the backing BDSs are correct after completion of a
# mirror block job; in "existing" modes (drive-mirror with
diff --git a/tests/qemu-iotests/156 b/tests/qemu-iotests/156
index 9c7878dd2d..65dcedd493 100755
--- a/tests/qemu-iotests/156
+++ b/tests/qemu-iotests/156
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Tests oVirt-like storage migration:
# - Create snapshot
diff --git a/tests/qemu-iotests/157 b/tests/qemu-iotests/157
index 7cbac38099..0dc9ba68d2 100755
--- a/tests/qemu-iotests/157
+++ b/tests/qemu-iotests/157
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: quick
#
# Test command line configuration of block devices with qdev
#
diff --git a/tests/qemu-iotests/158 b/tests/qemu-iotests/158
index cf23742c59..a95878e4ce 100755
--- a/tests/qemu-iotests/158
+++ b/tests/qemu-iotests/158
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test encrypted read/write using backing files
#
diff --git a/tests/qemu-iotests/159 b/tests/qemu-iotests/159
index f9690053a2..4eb476d3a8 100755
--- a/tests/qemu-iotests/159
+++ b/tests/qemu-iotests/159
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# qemu-img dd test with different block sizes
#
diff --git a/tests/qemu-iotests/160 b/tests/qemu-iotests/160
index 0572b5ae9a..7984a9c6f7 100755
--- a/tests/qemu-iotests/160
+++ b/tests/qemu-iotests/160
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# qemu-img dd test for the skip option
#
diff --git a/tests/qemu-iotests/161 b/tests/qemu-iotests/161
index 4fb7d0cbf0..f25effab93 100755
--- a/tests/qemu-iotests/161
+++ b/tests/qemu-iotests/161
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test reopening a backing image after block-stream and block-commit
#
diff --git a/tests/qemu-iotests/162 b/tests/qemu-iotests/162
index c0053ed975..cf17f494d8 100755
--- a/tests/qemu-iotests/162
+++ b/tests/qemu-iotests/162
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: quick
#
# Test case for specifying runtime options of the wrong type to some
# block drivers
diff --git a/tests/qemu-iotests/163 b/tests/qemu-iotests/163
index 5a3cc840a5..dedce8ef43 100755
--- a/tests/qemu-iotests/163
+++ b/tests/qemu-iotests/163
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw
#
# Tests for shrinking images
#
diff --git a/tests/qemu-iotests/165 b/tests/qemu-iotests/165
index fb56a769b4..abc4ffadd5 100755
--- a/tests/qemu-iotests/165
+++ b/tests/qemu-iotests/165
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Tests for persistent dirty bitmaps.
#
diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169
index 40afb15299..a5c7bc83e0 100755
--- a/tests/qemu-iotests/169
+++ b/tests/qemu-iotests/169
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw migration
#
# Tests for dirty bitmaps migration.
#
diff --git a/tests/qemu-iotests/170 b/tests/qemu-iotests/170
index 6c8f0e8085..41387e4d66 100755
--- a/tests/qemu-iotests/170
+++ b/tests/qemu-iotests/170
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# qemu-img dd test
#
diff --git a/tests/qemu-iotests/171 b/tests/qemu-iotests/171
index f3582edb10..d1d77f7013 100755
--- a/tests/qemu-iotests/171
+++ b/tests/qemu-iotests/171
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test 'offset' and 'size' options of the raw driver. Make sure we can't
# (or can) read and write outside of the image size.
diff --git a/tests/qemu-iotests/172 b/tests/qemu-iotests/172
index b45782e8db..0ac942a6c5 100755
--- a/tests/qemu-iotests/172
+++ b/tests/qemu-iotests/172
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: auto
#
# Test floppy configuration
#
diff --git a/tests/qemu-iotests/173 b/tests/qemu-iotests/173
index ec6d1705e5..9594f3c5ea 100755
--- a/tests/qemu-iotests/173
+++ b/tests/qemu-iotests/173
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw
#
# Test QAPI commands looking up protocol based images with relative
# filename backing strings
diff --git a/tests/qemu-iotests/174 b/tests/qemu-iotests/174
index 1b0dd2e8b7..d4cecb5756 100755
--- a/tests/qemu-iotests/174
+++ b/tests/qemu-iotests/174
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: auto
#
# Test that qemu-io fail with non-zero exit code
#
diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175
index 21a77a2bf5..f74f053b71 100755
--- a/tests/qemu-iotests/175
+++ b/tests/qemu-iotests/175
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: quick
#
# Test creating raw image preallocation mode
#
diff --git a/tests/qemu-iotests/176 b/tests/qemu-iotests/176
index 5ce3b27069..27ac25467f 100755
--- a/tests/qemu-iotests/176
+++ b/tests/qemu-iotests/176
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto backing
#
# Commit changes into backing chains and empty the top image if the
# backing image is not explicitly specified.
diff --git a/tests/qemu-iotests/177 b/tests/qemu-iotests/177
index 595bfd4236..8d8745b29a 100755
--- a/tests/qemu-iotests/177
+++ b/tests/qemu-iotests/177
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test corner cases with unusual block geometries
#
diff --git a/tests/qemu-iotests/178 b/tests/qemu-iotests/178
index f09b27caac..3b1a7adce4 100755
--- a/tests/qemu-iotests/178
+++ b/tests/qemu-iotests/178
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: img
#
# qemu-img measure sub-command tests
#
diff --git a/tests/qemu-iotests/179 b/tests/qemu-iotests/179
index 7ada04c641..09447b5610 100755
--- a/tests/qemu-iotests/179
+++ b/tests/qemu-iotests/179
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test case for write zeroes with unmap
#
diff --git a/tests/qemu-iotests/181 b/tests/qemu-iotests/181
index 438c2dcd80..820c53ef35 100755
--- a/tests/qemu-iotests/181
+++ b/tests/qemu-iotests/181
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto migration quick
#
# Test postcopy live migration with shared storage
#
diff --git a/tests/qemu-iotests/182 b/tests/qemu-iotests/182
index 56a2dd58e6..55a0384c08 100755
--- a/tests/qemu-iotests/182
+++ b/tests/qemu-iotests/182
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test image locking for POSIX locks
#
diff --git a/tests/qemu-iotests/183 b/tests/qemu-iotests/183
index d889a3b19c..ee62939e72 100755
--- a/tests/qemu-iotests/183
+++ b/tests/qemu-iotests/183
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw migration quick
#
# Test old-style block migration (migrate -b)
#
diff --git a/tests/qemu-iotests/184 b/tests/qemu-iotests/184
index eebb53faed..513d167098 100755
--- a/tests/qemu-iotests/184
+++ b/tests/qemu-iotests/184
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test I/O throttle block filter driver interface
#
diff --git a/tests/qemu-iotests/185 b/tests/qemu-iotests/185
index fd5e6ebe11..7bc8fe5767 100755
--- a/tests/qemu-iotests/185
+++ b/tests/qemu-iotests/185
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw
#
# Test exiting qemu while jobs are still running
#
diff --git a/tests/qemu-iotests/186 b/tests/qemu-iotests/186
index 0db25b0e68..072e54e62b 100755
--- a/tests/qemu-iotests/186
+++ b/tests/qemu-iotests/186
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto
#
# Test 'info block' with all kinds of configurations
#
diff --git a/tests/qemu-iotests/187 b/tests/qemu-iotests/187
index f262d83e3a..70b74b033c 100755
--- a/tests/qemu-iotests/187
+++ b/tests/qemu-iotests/187
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto
#
# Test switching between read-only and read-write
#
diff --git a/tests/qemu-iotests/188 b/tests/qemu-iotests/188
index 13b225fded..ce087d1873 100755
--- a/tests/qemu-iotests/188
+++ b/tests/qemu-iotests/188
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test encrypted read/write using plain bdrv_pread/bdrv_pwrite
#
diff --git a/tests/qemu-iotests/189 b/tests/qemu-iotests/189
index 3e5ded14c6..4e463385b2 100755
--- a/tests/qemu-iotests/189
+++ b/tests/qemu-iotests/189
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw
#
# Test encrypted read/write using backing files
#
diff --git a/tests/qemu-iotests/190 b/tests/qemu-iotests/190
index c22d8d64f9..7fb8447354 100755
--- a/tests/qemu-iotests/190
+++ b/tests/qemu-iotests/190
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# qemu-img measure sub-command tests on huge qcow2 files
#
diff --git a/tests/qemu-iotests/191 b/tests/qemu-iotests/191
index 95a891350d..ce695b95c2 100755
--- a/tests/qemu-iotests/191
+++ b/tests/qemu-iotests/191
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto
#
# Test commit block job where top has two parents
#
diff --git a/tests/qemu-iotests/192 b/tests/qemu-iotests/192
index d2ba55dd90..d809187fca 100755
--- a/tests/qemu-iotests/192
+++ b/tests/qemu-iotests/192
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test NBD export with -incoming (non-shared storage migration use case from
# libvirt)
diff --git a/tests/qemu-iotests/194 b/tests/qemu-iotests/194
index 7a4863ab18..3889266afa 100755
--- a/tests/qemu-iotests/194
+++ b/tests/qemu-iotests/194
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw migration quick
#
# Copyright (C) 2017 Red Hat, Inc.
#
diff --git a/tests/qemu-iotests/195 b/tests/qemu-iotests/195
index 967af5b7b5..f1df69079f 100755
--- a/tests/qemu-iotests/195
+++ b/tests/qemu-iotests/195
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test change-backing-file command
#
diff --git a/tests/qemu-iotests/196 b/tests/qemu-iotests/196
index e8fcf37273..2451515094 100755
--- a/tests/qemu-iotests/196
+++ b/tests/qemu-iotests/196
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick migration
#
# Test clearing unknown autoclear_features flag by qcow2 after
# migration. This test mimics migration to older qemu.
diff --git a/tests/qemu-iotests/197 b/tests/qemu-iotests/197
index a161c89816..a2547bc280 100755
--- a/tests/qemu-iotests/197
+++ b/tests/qemu-iotests/197
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test case for copy-on-read into qcow2
#
diff --git a/tests/qemu-iotests/198 b/tests/qemu-iotests/198
index 46f0c54537..b333a8f281 100755
--- a/tests/qemu-iotests/198
+++ b/tests/qemu-iotests/198
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw
#
# Test commit of encrypted qcow2 files
#
diff --git a/tests/qemu-iotests/199 b/tests/qemu-iotests/199
index 58fad872a1..dbf10e58d3 100755
--- a/tests/qemu-iotests/199
+++ b/tests/qemu-iotests/199
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw migration
#
# Tests for dirty bitmaps postcopy migration.
#
diff --git a/tests/qemu-iotests/200 b/tests/qemu-iotests/200
index 046539154f..f80517e342 100755
--- a/tests/qemu-iotests/200
+++ b/tests/qemu-iotests/200
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw
#
# Block job co-routine race condition test.
#
diff --git a/tests/qemu-iotests/201 b/tests/qemu-iotests/201
index 483eb189c5..1b8eb51d8f 100755
--- a/tests/qemu-iotests/201
+++ b/tests/qemu-iotests/201
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw migration quick
#
# Test savevm and loadvm after live migration with postcopy flag
#
diff --git a/tests/qemu-iotests/202 b/tests/qemu-iotests/202
index e3900a44d1..8eb5f32d15 100755
--- a/tests/qemu-iotests/202
+++ b/tests/qemu-iotests/202
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Copyright (C) 2017 Red Hat, Inc.
#
diff --git a/tests/qemu-iotests/203 b/tests/qemu-iotests/203
index 4b4bd3307d..ea30e50497 100755
--- a/tests/qemu-iotests/203
+++ b/tests/qemu-iotests/203
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw auto migration quick
#
# Copyright (C) 2017 Red Hat, Inc.
#
diff --git a/tests/qemu-iotests/204 b/tests/qemu-iotests/204
index 536bb8b534..ab68b6d75c 100755
--- a/tests/qemu-iotests/204
+++ b/tests/qemu-iotests/204
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test corner cases with unusual block geometries
#
diff --git a/tests/qemu-iotests/205 b/tests/qemu-iotests/205
index 43432cb599..c0e107328f 100755
--- a/tests/qemu-iotests/205
+++ b/tests/qemu-iotests/205
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Tests for qmp command nbd-server-remove.
#
diff --git a/tests/qemu-iotests/206 b/tests/qemu-iotests/206
index d12d7cb566..c3cdad4ce4 100755
--- a/tests/qemu-iotests/206
+++ b/tests/qemu-iotests/206
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw
#
# Test qcow2 and file image creation
#
diff --git a/tests/qemu-iotests/207 b/tests/qemu-iotests/207
index a6621410da..f9f3fd7131 100755
--- a/tests/qemu-iotests/207
+++ b/tests/qemu-iotests/207
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw
#
# Test ssh image creation
#
diff --git a/tests/qemu-iotests/208 b/tests/qemu-iotests/208
index 54aa4be273..6117f165fa 100755
--- a/tests/qemu-iotests/208
+++ b/tests/qemu-iotests/208
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Copyright (C) 2018 Red Hat, Inc.
#
diff --git a/tests/qemu-iotests/209 b/tests/qemu-iotests/209
index 8c804f4a30..ff7efea11b 100755
--- a/tests/qemu-iotests/209
+++ b/tests/qemu-iotests/209
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Tests for NBD BLOCK_STATUS extension
#
diff --git a/tests/qemu-iotests/210 b/tests/qemu-iotests/210
index 7bf591f313..5a62ed4dd1 100755
--- a/tests/qemu-iotests/210
+++ b/tests/qemu-iotests/210
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw
#
# Test luks and file image creation
#
diff --git a/tests/qemu-iotests/211 b/tests/qemu-iotests/211
index 4969edb00c..f52cadade1 100755
--- a/tests/qemu-iotests/211
+++ b/tests/qemu-iotests/211
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Test VDI and file image creation
#
diff --git a/tests/qemu-iotests/212 b/tests/qemu-iotests/212
index 45d08842bb..d4af0c4ac8 100755
--- a/tests/qemu-iotests/212
+++ b/tests/qemu-iotests/212
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Test parallels and file image creation
#
diff --git a/tests/qemu-iotests/213 b/tests/qemu-iotests/213
index cf638eb927..78d839ab64 100755
--- a/tests/qemu-iotests/213
+++ b/tests/qemu-iotests/213
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Test vhdx and file image creation
#
diff --git a/tests/qemu-iotests/214 b/tests/qemu-iotests/214
index 75ae7a14b5..0889089d81 100755
--- a/tests/qemu-iotests/214
+++ b/tests/qemu-iotests/214
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto
#
# Test qcow2 image compression
#
diff --git a/tests/qemu-iotests/215 b/tests/qemu-iotests/215
index f99bae78c7..d464596f14 100755
--- a/tests/qemu-iotests/215
+++ b/tests/qemu-iotests/215
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test case for copy-on-read into qcow2, using the COR filter driver
#
diff --git a/tests/qemu-iotests/216 b/tests/qemu-iotests/216
index f93c61aad6..c02f8d2880 100755
--- a/tests/qemu-iotests/216
+++ b/tests/qemu-iotests/216
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Copy-on-read tests using a COR filter node
#
diff --git a/tests/qemu-iotests/217 b/tests/qemu-iotests/217
index 7385342498..e693f326a3 100755
--- a/tests/qemu-iotests/217
+++ b/tests/qemu-iotests/217
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# I/O errors when working with internal qcow2 snapshots, and repairing
# the result
diff --git a/tests/qemu-iotests/218 b/tests/qemu-iotests/218
index 5586870456..ae7c4fb187 100755
--- a/tests/qemu-iotests/218
+++ b/tests/qemu-iotests/218
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# This test covers what happens when a mirror block job is cancelled
# in various phases of its existence.
diff --git a/tests/qemu-iotests/219 b/tests/qemu-iotests/219
index db272c5249..16c3ca7fff 100755
--- a/tests/qemu-iotests/219
+++ b/tests/qemu-iotests/219
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw
#
# Copyright (C) 2018 Red Hat, Inc.
#
diff --git a/tests/qemu-iotests/220 b/tests/qemu-iotests/220
index 9ba3b3fdcb..7d08b9b040 100755
--- a/tests/qemu-iotests/220
+++ b/tests/qemu-iotests/220
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto
#
# max limits on compression in huge qcow2 files
#
diff --git a/tests/qemu-iotests/221 b/tests/qemu-iotests/221
index 7e6086b205..c463fd4b11 100755
--- a/tests/qemu-iotests/221
+++ b/tests/qemu-iotests/221
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test qemu-img vs. unaligned images
# (See also 253, which is the O_DIRECT version)
diff --git a/tests/qemu-iotests/222 b/tests/qemu-iotests/222
index 14d67c875b..b48afe623e 100755
--- a/tests/qemu-iotests/222
+++ b/tests/qemu-iotests/222
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# This test covers the basic fleecing workflow, which provides a
# point-in-time snapshot of a node that can be queried over NBD.
diff --git a/tests/qemu-iotests/223 b/tests/qemu-iotests/223
index d68bc3cb6f..da87f2f4a2 100755
--- a/tests/qemu-iotests/223
+++ b/tests/qemu-iotests/223
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test reading dirty bitmap over NBD
#
diff --git a/tests/qemu-iotests/224 b/tests/qemu-iotests/224
index 017b0685ba..38dd153625 100755
--- a/tests/qemu-iotests/224
+++ b/tests/qemu-iotests/224
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Test json:{} filenames with qemu-internal BDSs
# (the one of commit, to be precise)
diff --git a/tests/qemu-iotests/225 b/tests/qemu-iotests/225
index 0186ec8156..c0053790db 100755
--- a/tests/qemu-iotests/225
+++ b/tests/qemu-iotests/225
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test vmdk backing file correlation
#
diff --git a/tests/qemu-iotests/226 b/tests/qemu-iotests/226
index c1e1fb2b1c..6a9adb4a0b 100755
--- a/tests/qemu-iotests/226
+++ b/tests/qemu-iotests/226
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: auto quick
#
# This test covers expected filetypes for the file, host_cdrom and
# host_device drivers.
diff --git a/tests/qemu-iotests/227 b/tests/qemu-iotests/227
index 637d7c3726..7e45a47ac6 100755
--- a/tests/qemu-iotests/227
+++ b/tests/qemu-iotests/227
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: quick
#
# Test query-blockstats with different ways to create a BB
#
diff --git a/tests/qemu-iotests/228 b/tests/qemu-iotests/228
index 266fce6cda..a5eda2e149 100755
--- a/tests/qemu-iotests/228
+++ b/tests/qemu-iotests/228
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Test for when a backing file is considered overridden (thus, a
# json:{} filename is generated for the overlay) and when it is not
diff --git a/tests/qemu-iotests/229 b/tests/qemu-iotests/229
index 273ac2472d..4bc99390b5 100755
--- a/tests/qemu-iotests/229
+++ b/tests/qemu-iotests/229
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: auto quick
#
# Test for force canceling a running blockjob that is paused in
# an error state.
diff --git a/tests/qemu-iotests/231 b/tests/qemu-iotests/231
index c0b053ac30..0f66d0ca36 100755
--- a/tests/qemu-iotests/231
+++ b/tests/qemu-iotests/231
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: quick
#
# Test legacy and modern option parsing for rbd/ceph. This will not
# actually connect to a ceph server, but rather looks for the appropriate
diff --git a/tests/qemu-iotests/232 b/tests/qemu-iotests/232
index 685356ac3b..b30faaa218 100755
--- a/tests/qemu-iotests/232
+++ b/tests/qemu-iotests/232
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: quick
#
# Test for auto-read-only
#
diff --git a/tests/qemu-iotests/233 b/tests/qemu-iotests/233
index a5c17c3963..7ce5764903 100755
--- a/tests/qemu-iotests/233
+++ b/tests/qemu-iotests/233
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: quick
#
# Test NBD TLS certificate / authorization integration
#
diff --git a/tests/qemu-iotests/234 b/tests/qemu-iotests/234
index 73c899ddae..cb5f1753e0 100755
--- a/tests/qemu-iotests/234
+++ b/tests/qemu-iotests/234
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: quick migration
#
# Copyright (C) 2018 Red Hat, Inc.
#
diff --git a/tests/qemu-iotests/235 b/tests/qemu-iotests/235
index d1b10ac36b..20d16dbf38 100755
--- a/tests/qemu-iotests/235
+++ b/tests/qemu-iotests/235
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: quick
#
# Simple mirror test
#
diff --git a/tests/qemu-iotests/236 b/tests/qemu-iotests/236
index 6f5cee2444..f6c44517d6 100755
--- a/tests/qemu-iotests/236
+++ b/tests/qemu-iotests/236
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: quick
#
# Test bitmap merges.
#
diff --git a/tests/qemu-iotests/237 b/tests/qemu-iotests/237
index 5b21ad3509..43dfd3bd40 100755
--- a/tests/qemu-iotests/237
+++ b/tests/qemu-iotests/237
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Test vmdk and file image creation
#
diff --git a/tests/qemu-iotests/238 b/tests/qemu-iotests/238
index b8fcf15a1f..8a10af57f1 100755
--- a/tests/qemu-iotests/238
+++ b/tests/qemu-iotests/238
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: quick
#
# Regression test for throttle group member unregister segfault with iothread
#
diff --git a/tests/qemu-iotests/239 b/tests/qemu-iotests/239
index b0991ffe59..4f0037148d 100755
--- a/tests/qemu-iotests/239
+++ b/tests/qemu-iotests/239
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test case for dmg
#
diff --git a/tests/qemu-iotests/240 b/tests/qemu-iotests/240
index c0f71f0461..ab077f4ceb 100755
--- a/tests/qemu-iotests/240
+++ b/tests/qemu-iotests/240
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: quick
# Test hot plugging and unplugging with iothreads
#
diff --git a/tests/qemu-iotests/241 b/tests/qemu-iotests/241
index 8dae8d39e4..c962c8b607 100755
--- a/tests/qemu-iotests/241
+++ b/tests/qemu-iotests/241
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test qemu-nbd vs. unaligned images
#
diff --git a/tests/qemu-iotests/242 b/tests/qemu-iotests/242
index a16de3085f..a9b27668c2 100755
--- a/tests/qemu-iotests/242
+++ b/tests/qemu-iotests/242
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Test for qcow2 bitmap printed information
#
diff --git a/tests/qemu-iotests/243 b/tests/qemu-iotests/243
index 17388a4644..8bbb510120 100755
--- a/tests/qemu-iotests/243
+++ b/tests/qemu-iotests/243
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test qcow2 preallocation
#
diff --git a/tests/qemu-iotests/244 b/tests/qemu-iotests/244
index f2b2dddf1c..a46b441627 100755
--- a/tests/qemu-iotests/244
+++ b/tests/qemu-iotests/244
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test qcow2 with external data files
#
diff --git a/tests/qemu-iotests/245 b/tests/qemu-iotests/245
index e60c8326d3..86f00f290f 100755
--- a/tests/qemu-iotests/245
+++ b/tests/qemu-iotests/245
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw
#
# Test cases for the QMP 'x-blockdev-reopen' command
#
diff --git a/tests/qemu-iotests/246 b/tests/qemu-iotests/246
index 58a479cc1f..fa3102c546 100755
--- a/tests/qemu-iotests/246
+++ b/tests/qemu-iotests/246
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Test persistent bitmap resizing.
#
diff --git a/tests/qemu-iotests/247 b/tests/qemu-iotests/247
index 6cf2679750..ace6dba052 100755
--- a/tests/qemu-iotests/247
+++ b/tests/qemu-iotests/247
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test for auto-read-only with commit block job
#
diff --git a/tests/qemu-iotests/248 b/tests/qemu-iotests/248
index 18ba03467e..4daaed1530 100755
--- a/tests/qemu-iotests/248
+++ b/tests/qemu-iotests/248
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Test resume mirror after auto pause on ENOSPC
#
diff --git a/tests/qemu-iotests/249 b/tests/qemu-iotests/249
index 29453b8c90..28bffd4d57 100755
--- a/tests/qemu-iotests/249
+++ b/tests/qemu-iotests/249
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test that a backing image is put back in read-only mode after
# block-commit (both when it fails and when it succeeds).
diff --git a/tests/qemu-iotests/250 b/tests/qemu-iotests/250
index 3df275c76b..f069ca1759 100755
--- a/tests/qemu-iotests/250
+++ b/tests/qemu-iotests/250
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test big discard in qcow2 shrink
#
diff --git a/tests/qemu-iotests/251 b/tests/qemu-iotests/251
index 294773bdc1..8bdec37d32 100755
--- a/tests/qemu-iotests/251
+++ b/tests/qemu-iotests/251
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test qemu-img convert --salvage
#
diff --git a/tests/qemu-iotests/252 b/tests/qemu-iotests/252
index 1d74afff99..2134b9993a 100755
--- a/tests/qemu-iotests/252
+++ b/tests/qemu-iotests/252
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto backing quick
#
# Tests for rebasing COW images that require zero cluster support
#
diff --git a/tests/qemu-iotests/253 b/tests/qemu-iotests/253
index d88d5afa45..35039d20a8 100755
--- a/tests/qemu-iotests/253
+++ b/tests/qemu-iotests/253
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test qemu-img vs. unaligned images; O_DIRECT version
# (Originates from 221)
diff --git a/tests/qemu-iotests/254 b/tests/qemu-iotests/254
index 150e58be8e..49da948407 100755
--- a/tests/qemu-iotests/254
+++ b/tests/qemu-iotests/254
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw backing quick
#
# Test external snapshot with bitmap copying and moving.
#
diff --git a/tests/qemu-iotests/255 b/tests/qemu-iotests/255
index 8f08f741da..c43aa9c67a 100755
--- a/tests/qemu-iotests/255
+++ b/tests/qemu-iotests/255
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Test commit job graph modifications while requests are active
#
diff --git a/tests/qemu-iotests/256 b/tests/qemu-iotests/256
index db8d6f31cf..8d82a1dd86 100755
--- a/tests/qemu-iotests/256
+++ b/tests/qemu-iotests/256
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw auto quick
#
# Test incremental/backup across iothread contexts
#
diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
index c80e06ae28..a2f4b5afe6 100755
--- a/tests/qemu-iotests/257
+++ b/tests/qemu-iotests/257
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw
#
# Test bitmap-sync backups (incremental, differential, and partials)
#
diff --git a/tests/qemu-iotests/258 b/tests/qemu-iotests/258
index e305a1502f..9a2d33ae5e 100755
--- a/tests/qemu-iotests/258
+++ b/tests/qemu-iotests/258
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Very specific tests for adjacent commit/stream block jobs
#
diff --git a/tests/qemu-iotests/259 b/tests/qemu-iotests/259
index 62e29af05f..76cde429c4 100755
--- a/tests/qemu-iotests/259
+++ b/tests/qemu-iotests/259
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test generic image creation fallback (by using NBD)
#
diff --git a/tests/qemu-iotests/260 b/tests/qemu-iotests/260
index 804a7addb9..a35cb7b61f 100755
--- a/tests/qemu-iotests/260
+++ b/tests/qemu-iotests/260
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Tests for temporary external snapshot when we have bitmaps.
#
diff --git a/tests/qemu-iotests/261 b/tests/qemu-iotests/261
index 847b4c6a37..d1c8037ab1 100755
--- a/tests/qemu-iotests/261
+++ b/tests/qemu-iotests/261
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw
#
# Test case for qcow2's handling of extra data in snapshot table entries
# (and more generally, how certain cases of broken snapshot tables are
diff --git a/tests/qemu-iotests/262 b/tests/qemu-iotests/262
index 03af061f94..32d69988ef 100755
--- a/tests/qemu-iotests/262
+++ b/tests/qemu-iotests/262
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick migration
#
# Copyright (C) 2019 Red Hat, Inc.
#
diff --git a/tests/qemu-iotests/263 b/tests/qemu-iotests/263
index f598a12899..ec09b41405 100755
--- a/tests/qemu-iotests/263
+++ b/tests/qemu-iotests/263
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test encrypted write that crosses cluster boundary of two unallocated clusters
# Based on 188
diff --git a/tests/qemu-iotests/264 b/tests/qemu-iotests/264
index 666f164ed8..960f0449a4 100755
--- a/tests/qemu-iotests/264
+++ b/tests/qemu-iotests/264
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw
#
# Test nbd reconnect
#
diff --git a/tests/qemu-iotests/265 b/tests/qemu-iotests/265
index 0e800fb524..4b3b52c6e6 100755
--- a/tests/qemu-iotests/265
+++ b/tests/qemu-iotests/265
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test reverse-ordered qcow2 writes on a sub-cluster level
#
diff --git a/tests/qemu-iotests/266 b/tests/qemu-iotests/266
index 91bdf8729e..71ce81d0df 100755
--- a/tests/qemu-iotests/266
+++ b/tests/qemu-iotests/266
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Test VPC and file image creation
#
diff --git a/tests/qemu-iotests/267 b/tests/qemu-iotests/267
index e44be49c77..2e2afdad9c 100755
--- a/tests/qemu-iotests/267
+++ b/tests/qemu-iotests/267
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick snapshot
#
# Test which nodes are involved in internal snapshots
#
diff --git a/tests/qemu-iotests/268 b/tests/qemu-iotests/268
index ddf4312284..9a8a563760 100755
--- a/tests/qemu-iotests/268
+++ b/tests/qemu-iotests/268
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test write request with required alignment larger than the cluster size
#
diff --git a/tests/qemu-iotests/270 b/tests/qemu-iotests/270
index 00339c0f78..74352342db 100755
--- a/tests/qemu-iotests/270
+++ b/tests/qemu-iotests/270
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw backing quick
#
# Test large write to a qcow2 image
#
diff --git a/tests/qemu-iotests/271 b/tests/qemu-iotests/271
index e242b28b58..599b849cc6 100755
--- a/tests/qemu-iotests/271
+++ b/tests/qemu-iotests/271
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto
#
# Test qcow2 images with extended L2 entries
#
diff --git a/tests/qemu-iotests/272 b/tests/qemu-iotests/272
index de475bf6f0..4bcf410d81 100755
--- a/tests/qemu-iotests/272
+++ b/tests/qemu-iotests/272
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw
#
# Test compressed write to a qcow2 image at an offset above 4 GB
#
diff --git a/tests/qemu-iotests/273 b/tests/qemu-iotests/273
index 79b4ab4b05..6c33305b4d 100755
--- a/tests/qemu-iotests/273
+++ b/tests/qemu-iotests/273
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: backing quick
#
# Test multiple blockdev-snapshot calls with 'backing': null
#
diff --git a/tests/qemu-iotests/274 b/tests/qemu-iotests/274
index 76b1ba6a52..caab008e07 100755
--- a/tests/qemu-iotests/274
+++ b/tests/qemu-iotests/274
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw backing
#
# Copyright (C) 2019 Red Hat, Inc.
#
diff --git a/tests/qemu-iotests/277 b/tests/qemu-iotests/277
index a39ce2d873..24833e7eb6 100755
--- a/tests/qemu-iotests/277
+++ b/tests/qemu-iotests/277
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Test NBD client reconnection
#
diff --git a/tests/qemu-iotests/279 b/tests/qemu-iotests/279
index 2a6315cf17..6afef78bc8 100755
--- a/tests/qemu-iotests/279
+++ b/tests/qemu-iotests/279
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw backing quick
#
# Test qemu-img --backing-chain --image-opts
#
diff --git a/tests/qemu-iotests/280 b/tests/qemu-iotests/280
index f594bb9ed2..628f3c33ca 100755
--- a/tests/qemu-iotests/280
+++ b/tests/qemu-iotests/280
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw migration quick
#
# Copyright (C) 2019 Red Hat, Inc.
#
diff --git a/tests/qemu-iotests/281 b/tests/qemu-iotests/281
index 0bf973bca6..956698083f 100755
--- a/tests/qemu-iotests/281
+++ b/tests/qemu-iotests/281
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Test cases for blockdev + IOThread interactions
#
diff --git a/tests/qemu-iotests/282 b/tests/qemu-iotests/282
index 27da2a0023..3140445989 100755
--- a/tests/qemu-iotests/282
+++ b/tests/qemu-iotests/282
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw img quick
#
# Test qemu-img file cleanup for LUKS when using a non-UTF8 secret
#
diff --git a/tests/qemu-iotests/283 b/tests/qemu-iotests/283
index 383797ed62..79643e375b 100755
--- a/tests/qemu-iotests/283
+++ b/tests/qemu-iotests/283
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: auto quick
#
# Test for backup-top filter permission activation failure
#
diff --git a/tests/qemu-iotests/284 b/tests/qemu-iotests/284
index 9f6c29a79c..5a82639e7f 100755
--- a/tests/qemu-iotests/284
+++ b/tests/qemu-iotests/284
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw
#
# Test ref count checks on encrypted images
#
diff --git a/tests/qemu-iotests/286 b/tests/qemu-iotests/286
index f64e0eccea..120a8375b7 100755
--- a/tests/qemu-iotests/286
+++ b/tests/qemu-iotests/286
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test qemu-img snapshot -l
#
diff --git a/tests/qemu-iotests/287 b/tests/qemu-iotests/287
index 3bb383fd4b..22ce9ff0e4 100755
--- a/tests/qemu-iotests/287
+++ b/tests/qemu-iotests/287
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: auto quick
#
# Test case for an image using zstd compression
#
diff --git a/tests/qemu-iotests/288 b/tests/qemu-iotests/288
index 6c62065aef..47aca6592a 100755
--- a/tests/qemu-iotests/288
+++ b/tests/qemu-iotests/288
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: quick
#
# qemu-img measure tests for LUKS images
#
diff --git a/tests/qemu-iotests/289 b/tests/qemu-iotests/289
index fe69bde1eb..5dd6ec62db 100755
--- a/tests/qemu-iotests/289
+++ b/tests/qemu-iotests/289
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# qcow2 v3-exclusive error path testing
# (026 tests paths common to v2 and v3)
diff --git a/tests/qemu-iotests/290 b/tests/qemu-iotests/290
index 35c38d4f80..ed80da2685 100755
--- a/tests/qemu-iotests/290
+++ b/tests/qemu-iotests/290
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test how 'qemu-io -c discard' behaves on v2 and v3 qcow2 images
#
diff --git a/tests/qemu-iotests/291 b/tests/qemu-iotests/291
index bc70d5e389..20efb080a6 100755
--- a/tests/qemu-iotests/291
+++ b/tests/qemu-iotests/291
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test qemu-img bitmap handling
#
diff --git a/tests/qemu-iotests/292 b/tests/qemu-iotests/292
index 3ae2772e3b..73cbb9364a 100755
--- a/tests/qemu-iotests/292
+++ b/tests/qemu-iotests/292
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test resizing a qcow2 image with a backing file
#
diff --git a/tests/qemu-iotests/293 b/tests/qemu-iotests/293
index 3363bf07f0..37294487b1 100755
--- a/tests/qemu-iotests/293
+++ b/tests/qemu-iotests/293
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw
#
# Test encryption key management with luks
# Based on 134
diff --git a/tests/qemu-iotests/294 b/tests/qemu-iotests/294
index 4c375ed609..9059eb26b3 100755
--- a/tests/qemu-iotests/294
+++ b/tests/qemu-iotests/294
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Copyright (C) 2019 Red Hat, Inc.
#
diff --git a/tests/qemu-iotests/295 b/tests/qemu-iotests/295
index 59e674fa85..01a6c0b31f 100755
--- a/tests/qemu-iotests/295
+++ b/tests/qemu-iotests/295
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw
#
# Test case QMP's encrypted key management
#
diff --git a/tests/qemu-iotests/296 b/tests/qemu-iotests/296
index fb7dec88aa..0bc3c6c7d7 100755
--- a/tests/qemu-iotests/296
+++ b/tests/qemu-iotests/296
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw
#
# Test case for encryption key management versus image sharing
#
diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297
index 5c5420712b..85bc1c0c85 100755
--- a/tests/qemu-iotests/297
+++ b/tests/qemu-iotests/297
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: meta
#
# Copyright (C) 2020 Red Hat, Inc.
#
diff --git a/tests/qemu-iotests/299 b/tests/qemu-iotests/299
index e129c7f7cb..a7122941fd 100755
--- a/tests/qemu-iotests/299
+++ b/tests/qemu-iotests/299
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: auto quick
#
# Test shutdown when bitmap is exported through NBD server
#
diff --git a/tests/qemu-iotests/300 b/tests/qemu-iotests/300
index 5b75121b84..23aca59d9c 100755
--- a/tests/qemu-iotests/300
+++ b/tests/qemu-iotests/300
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: migration
#
# Copyright (C) 2020 Red Hat, Inc.
#
diff --git a/tests/qemu-iotests/301 b/tests/qemu-iotests/301
index 3823e95617..9f943cadbe 100755
--- a/tests/qemu-iotests/301
+++ b/tests/qemu-iotests/301
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: backing quick
#
# Test qcow backing file warnings
#
diff --git a/tests/qemu-iotests/302 b/tests/qemu-iotests/302
index a8506bda15..5695af4914 100755
--- a/tests/qemu-iotests/302
+++ b/tests/qemu-iotests/302
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: quick
#
# Tests converting qcow2 compressed to NBD
#
diff --git a/tests/qemu-iotests/303 b/tests/qemu-iotests/303
index 11cd9eeb26..425544c064 100755
--- a/tests/qemu-iotests/303
+++ b/tests/qemu-iotests/303
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Test for dumping of qcow2 image metadata
#
diff --git a/tests/qemu-iotests/304 b/tests/qemu-iotests/304
index aaf9e14617..198f282087 100755
--- a/tests/qemu-iotests/304
+++ b/tests/qemu-iotests/304
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick
#
# Tests dirty-bitmap backup with unaligned bitmap granularity
#
diff --git a/tests/qemu-iotests/305 b/tests/qemu-iotests/305
index 5a415eb2a4..8b26156923 100755
--- a/tests/qemu-iotests/305
+++ b/tests/qemu-iotests/305
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test the handling of errors in write requests with multiple allocations
#
diff --git a/tests/qemu-iotests/307 b/tests/qemu-iotests/307
index de7c25fcfc..9008974346 100755
--- a/tests/qemu-iotests/307
+++ b/tests/qemu-iotests/307
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# group: rw quick export
#
# Copyright (C) 2020 Red Hat, Inc.
#
diff --git a/tests/qemu-iotests/308 b/tests/qemu-iotests/308
index b30f4400f6..f122065d0f 100755
--- a/tests/qemu-iotests/308
+++ b/tests/qemu-iotests/308
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw
#
# Test FUSE exports (in ways that are not captured by the generic
# tests)
diff --git a/tests/qemu-iotests/309 b/tests/qemu-iotests/309
index fb61157c2e..b90b279994 100755
--- a/tests/qemu-iotests/309
+++ b/tests/qemu-iotests/309
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw auto quick
#
# Test qemu-nbd -A
#
diff --git a/tests/qemu-iotests/312 b/tests/qemu-iotests/312
index 41340494b0..4139745f0e 100755
--- a/tests/qemu-iotests/312
+++ b/tests/qemu-iotests/312
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
+# group: rw quick
#
# Test drive-mirror with quorum
#
--
2.29.2
^ permalink raw reply related [flat|nested] 43+ messages in thread* Re: [PATCH v6 06/11] iotests: define group in each iotest
2021-01-09 12:26 ` [PATCH v6 06/11] iotests: define group in each iotest Vladimir Sementsov-Ogievskiy
@ 2021-01-13 23:01 ` Eric Blake
0 siblings, 0 replies; 43+ messages in thread
From: Eric Blake @ 2021-01-13 23:01 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, den, jsnow, qemu-devel, mreitz
On 1/9/21 6:26 AM, Vladimir Sementsov-Ogievskiy wrote:
> We are going to drop group file. Define group in tests as a preparatory
> step.
>
> The patch is generated by
>
> cd tests/qemu-iotests
>
> grep '^[0-9]\{3\} ' group | while read line; do
> file=$(awk '{print $1}' <<< "$line");
> groups=$(sed -e 's/^... //' <<< "$line");
> awk "NR==2{print \"# group: $groups\"}1" $file > tmp;
> cat tmp > $file;
> done
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> 291 files changed, 291 insertions(+)
May need rebasing, but your reproduction formula works (requires bash,
but that's fine since the tests themselves do as well).
> +++ b/tests/qemu-iotests/312
> @@ -1,4 +1,5 @@
> #!/usr/bin/env bash
> +# group: rw quick
> #
> # Test drive-mirror with quorum
> #
I like how it looks.
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 43+ messages in thread
* [PATCH v6 07/11] iotests: add findtests.py
2021-01-09 12:26 [PATCH v6 00/11] Rework iotests/check Vladimir Sementsov-Ogievskiy
` (5 preceding siblings ...)
2021-01-09 12:26 ` [PATCH v6 06/11] iotests: define group in each iotest Vladimir Sementsov-Ogievskiy
@ 2021-01-09 12:26 ` Vladimir Sementsov-Ogievskiy
2021-01-12 16:42 ` Kevin Wolf
2021-01-13 23:10 ` Eric Blake
2021-01-09 12:26 ` [PATCH v6 08/11] iotests: add testenv.py Vladimir Sementsov-Ogievskiy
` (3 subsequent siblings)
10 siblings, 2 replies; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-09 12:26 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, vsementsov, jsnow, qemu-devel, mreitz, den
Add python script with new logic of searching for tests:
Current ./check behavior:
- tests are named [0-9][0-9][0-9]
- tests must be registered in group file (even if test doesn't belong
to any group, like 142)
Behavior of findtests.py:
- group file is dropped
- tests are all files in tests/ subdirectory (except for .out files),
so it's not needed more to "register the test", just create it with
appropriate name in tests/ subdirectory. Old names like
[0-9][0-9][0-9] (in root iotests directory) are supported too, but
not recommended for new tests
- groups are parsed from '# group: ' line inside test files
- optional file group.local may be used to define some additional
groups for downstreams
- 'disabled' group is used to temporary disable tests. So instead of
commenting tests in old 'group' file you now can add them to
disabled group with help of 'group.local' file
- selecting test ranges like 5-15 are not supported more
(to support restarting failed ./check command from the middle of the
process, new argument is added: --start-from)
Benefits:
- no rebase conflicts in group file on patch porting from branch to
branch
- no conflicts in upstream, when different series want to occupy same
test number
- meaningful names for test files
For example, with digital number, when some person wants to add some
test about block-stream, he most probably will just create a new
test. But if there would be test-block-stream test already, he will
at first look at it and may be just add a test-case into it.
And anyway meaningful names are better.
This commit don't update check behavior (which will be don in further
commit), still, the documentation changed like new behavior is already
here. Let's live with this small inconsistency for the following few
commits, until final change.
The file findtests.py is self-executable and may be used for debugging
purposes.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
docs/devel/testing.rst | 50 ++++++-
tests/qemu-iotests/findtests.py | 229 ++++++++++++++++++++++++++++++++
2 files changed, 278 insertions(+), 1 deletion(-)
create mode 100755 tests/qemu-iotests/findtests.py
diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 0aa7a13bba..bfb6b65edc 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -111,7 +111,7 @@ check-block
-----------
``make check-block`` runs a subset of the block layer iotests (the tests that
-are in the "auto" group in ``tests/qemu-iotests/group``).
+are in the "auto" group).
See the "QEMU iotests" section below for more information.
GCC gcov support
@@ -224,6 +224,54 @@ another application on the host may have locked the file, possibly leading to a
test failure. If using such devices are explicitly desired, consider adding
``locking=off`` option to disable image locking.
+Test case groups
+----------------
+
+Test may belong to some groups, you may define it in the comment inside the
+test. By convention, test groups are listed in the second line of the test
+file, after "#!/..." line, like this:
+
+.. code::
+
+ #!/usr/bin/env python3
+ # group: auto quick
+ #
+ ...
+
+Additional way of defining groups is creating tests/qemu-iotests/group.local
+file. This should be used only for downstream (this file should never appear
+in upstream). This file may be used for defining some downstream test groups
+or for temporary disable tests, like this:
+
+.. code::
+
+ # groups for some company downstream process
+ #
+ # ci - tests to run on build
+ # down - our downstream tests, not for upstream
+ #
+ # Format of each line is:
+ # TEST_NAME TEST_GROUP [TEST_GROUP ]...
+
+ 013 ci
+ 210 disabled
+ 215 disabled
+ our-ugly-workaround-test down ci
+
+The (not exhaustive) list of groups:
+
+- quick : Tests in this group should finish within some few seconds.
+
+- auto : Tests in this group are used during "make check" and should be
+ runnable in any case. That means they should run with every QEMU binary
+ (also non-x86), with every QEMU configuration (i.e. must not fail if
+ an optional feature is not compiled in - but reporting a "skip" is ok),
+ work at least with the qcow2 file format, work with all kind of host
+ filesystems and users (e.g. "nobody" or "root") and must not take too
+ much memory and disk space (since CI pipelines tend to fail otherwise).
+
+- disabled : Tests in this group are disabled and ignored by check.
+
.. _docker-ref:
Docker based tests
diff --git a/tests/qemu-iotests/findtests.py b/tests/qemu-iotests/findtests.py
new file mode 100755
index 0000000000..b053db48e8
--- /dev/null
+++ b/tests/qemu-iotests/findtests.py
@@ -0,0 +1,229 @@
+#!/usr/bin/env python3
+#
+# Parse command line options to define set of tests to run.
+#
+# Copyright (c) 2020 Virtuozzo International GmbH
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+import os
+import sys
+import glob
+import argparse
+import re
+from collections import defaultdict
+from contextlib import contextmanager
+from typing import Optional, List, Tuple, Iterator, Set
+
+
+@contextmanager
+def chdir(path: Optional[str] = None) -> Iterator:
+ if path is None:
+ yield
+ return
+
+ saved_dir = os.getcwd()
+ os.chdir(path)
+ try:
+ yield
+ finally:
+ os.chdir(saved_dir)
+
+
+class TestFinder:
+ _argparser = None
+ @classmethod
+ def get_argparser(cls) -> argparse.ArgumentParser:
+ if cls._argparser is not None:
+ return cls._argparser
+
+ p = argparse.ArgumentParser(description="= test selecting options =",
+ add_help=False, usage=argparse.SUPPRESS)
+
+ p.add_argument('-g', '--groups', metavar='group1,...',
+ help='include tests from these groups')
+ p.add_argument('-x', '--exclude-groups', metavar='group1,...',
+ help='exclude tests from these groups')
+ p.add_argument('--start-from', metavar='TEST',
+ help='Start from specified test: make sorted sequence '
+ 'of tests as usual and then drop tests from the first '
+ 'one to TEST (not inclusive). This may be used to '
+ 'rerun failed ./check command, starting from the '
+ 'middle of the process.')
+ p.add_argument('tests', metavar='TEST_FILES', nargs='*',
+ help='tests to run')
+
+ cls._argparser = p
+ return p
+
+ def __init__(self, test_dir: Optional[str] = None) -> None:
+ self.groups = defaultdict(set)
+
+ with chdir(test_dir):
+ self.all_tests = glob.glob('[0-9][0-9][0-9]')
+ self.all_tests += [f for f in glob.iglob('tests/*')
+ if not f.endswith('.out') and
+ os.path.isfile(f + '.out')]
+
+ for t in self.all_tests:
+ with open(t) as f:
+ for line in f:
+ if line.startswith('# group: '):
+ for g in line.split()[2:]:
+ self.groups[g].add(t)
+
+ def add_group_file(self, fname: str):
+ with open(fname) as f:
+ for line in f:
+ line = line.strip()
+
+ if (not line) or line[0] == '#':
+ continue
+
+ words = line.split()
+ test_file = words[0]
+ groups = words[1:]
+
+ if test_file not in self.all_tests:
+ print(f'Warning: {fname}: "{test_file}" test is not found.'
+ ' Skip.')
+ continue
+
+ for g in groups:
+ self.groups[g].add(test_file)
+
+ def parse_test_name(self, name: str) -> str:
+ if '/' in name:
+ raise ValueError('Paths are unsupported for test selecting, '
+ f'requiring "{name}" is wrong')
+
+ if re.fullmatch(r'\d+', name):
+ # Numbered tests are old naming convetion. We should convert them
+ # to three-digit-length, like 1 --> 001.
+ name = f'{int(name):03}'
+ else:
+ # Named tests all should be in tests/ subdirectory
+ name = os.path.join('tests', name)
+
+ if name not in self.all_tests:
+ raise ValueError(f'Test "{name}" is not found')
+
+ return name
+
+ def find_tests(self, groups: Optional[List[str]] = None,
+ exclude_groups: Optional[List[str]] = None,
+ tests: Optional[List[str]] = None,
+ start_from: Optional[str] = None) -> List[str]:
+ """Find tests
+
+ Algorithm:
+
+ 1. a. if some @groups specified
+ a.1 Take all tests from @groups
+ a.2 Drop tests, which are in at least one of @exclude_groups or in
+ 'disabled' group (if 'disabled' is not listed in @groups)
+ a.3 Add tests from @tests (don't exclude anything from them)
+
+ b. else, if some @tests specified:
+ b.1 exclude_groups must be not specified, so just take @tests
+
+ c. else (only @exclude_groups list is non-empty):
+ c.1 Take all tests
+ c.2 Drop tests, which are in at least one of @exclude_groups or in
+ 'disabled' group
+
+ 2. sort
+
+ 3. If start_from specified, drop tests from first one to @start_from
+ (not inclusive)
+ """
+ if groups is None:
+ groups = []
+ if exclude_groups is None:
+ exclude_groups = []
+ if tests is None:
+ tests = []
+
+ res: Set[str] = set()
+ if groups:
+ # Some groups specified. exclude_groups supported, additionally
+ # selecting some individual tests supported as well.
+ res.update(*(self.groups[g] for g in groups))
+ elif tests:
+ # Some individual tests specified, but no groups. In this case
+ # we don't support exclude_groups.
+ if exclude_groups:
+ raise ValueError("Can't exclude from individually specified "
+ "tests.")
+ else:
+ # No tests no groups: start from all tests, exclude_groups
+ # supported.
+ res.update(self.all_tests)
+
+ if 'disabled' not in groups and 'disabled' not in exclude_groups:
+ exclude_groups.append('disabled')
+
+ res = res.difference(*(self.groups[g] for g in exclude_groups))
+
+ # We want to add @tests. But for compatibility with old test names,
+ # we should convert any number < 100 to number padded by
+ # leading zeroes, like 1 -> 001 and 23 -> 023.
+ for t in tests:
+ res.add(self.parse_test_name(t))
+
+ sequence = sorted(res)
+
+ if start_from is not None:
+ del sequence[:sequence.index(self.parse_test_name(start_from))]
+
+ return sequence
+
+ def find_tests_argv(self, argv: List[str]) -> Tuple[List[str], List[str]]:
+ """Returns tuple of tests list and remaining arguments list"""
+ args, remaining = self.get_argparser().parse_known_args(argv)
+
+ if args.groups is not None:
+ args.groups = args.groups.split(',')
+
+ if args.exclude_groups is not None:
+ args.exclude_groups = args.exclude_groups.split(',')
+
+ return self.find_tests(**vars(args)), remaining
+
+
+def find_tests(argv, test_dir: Optional[str] = None) -> Tuple[List[str],
+ List[str]]:
+ """Returns tuple of tests list and remaining arguments list"""
+ tf = TestFinder(test_dir)
+
+ if test_dir is None:
+ group_local = 'group.local'
+ else:
+ group_local = os.path.join(test_dir, 'group.local')
+ if os.path.isfile(group_local):
+ tf.add_group_file(group_local)
+
+ return tf.find_tests_argv(argv)
+
+
+if __name__ == '__main__':
+ if len(sys.argv) == 2 and sys.argv[1] in ['-h', '--help']:
+ TestFinder.get_argparser().print_help()
+ exit()
+
+ tests, remaining_argv = find_tests(sys.argv[1:])
+ print('\n'.join(tests))
+ if remaining_argv:
+ print('\nUnhandled options: ', remaining_argv)
--
2.29.2
^ permalink raw reply related [flat|nested] 43+ messages in thread* Re: [PATCH v6 07/11] iotests: add findtests.py
2021-01-09 12:26 ` [PATCH v6 07/11] iotests: add findtests.py Vladimir Sementsov-Ogievskiy
@ 2021-01-12 16:42 ` Kevin Wolf
2021-01-13 10:37 ` Vladimir Sementsov-Ogievskiy
2021-01-14 7:38 ` Vladimir Sementsov-Ogievskiy
2021-01-13 23:10 ` Eric Blake
1 sibling, 2 replies; 43+ messages in thread
From: Kevin Wolf @ 2021-01-12 16:42 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy; +Cc: qemu-block, qemu-devel, mreitz, den, jsnow
Am 09.01.2021 um 13:26 hat Vladimir Sementsov-Ogievskiy geschrieben:
> Add python script with new logic of searching for tests:
>
> Current ./check behavior:
> - tests are named [0-9][0-9][0-9]
> - tests must be registered in group file (even if test doesn't belong
> to any group, like 142)
>
> Behavior of findtests.py:
> - group file is dropped
> - tests are all files in tests/ subdirectory (except for .out files),
> so it's not needed more to "register the test", just create it with
> appropriate name in tests/ subdirectory. Old names like
> [0-9][0-9][0-9] (in root iotests directory) are supported too, but
> not recommended for new tests
> - groups are parsed from '# group: ' line inside test files
> - optional file group.local may be used to define some additional
> groups for downstreams
> - 'disabled' group is used to temporary disable tests. So instead of
> commenting tests in old 'group' file you now can add them to
> disabled group with help of 'group.local' file
> - selecting test ranges like 5-15 are not supported more
> (to support restarting failed ./check command from the middle of the
> process, new argument is added: --start-from)
>
> Benefits:
> - no rebase conflicts in group file on patch porting from branch to
> branch
> - no conflicts in upstream, when different series want to occupy same
> test number
> - meaningful names for test files
> For example, with digital number, when some person wants to add some
> test about block-stream, he most probably will just create a new
> test. But if there would be test-block-stream test already, he will
> at first look at it and may be just add a test-case into it.
> And anyway meaningful names are better.
>
> This commit don't update check behavior (which will be don in further
> commit), still, the documentation changed like new behavior is already
> here. Let's live with this small inconsistency for the following few
> commits, until final change.
>
> The file findtests.py is self-executable and may be used for debugging
> purposes.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> docs/devel/testing.rst | 50 ++++++-
> tests/qemu-iotests/findtests.py | 229 ++++++++++++++++++++++++++++++++
I extended 297 so that it also checks the newly added Python file, and
pylint and mypy report some errors:
+************* Module findtests
+findtests.py:127:19: W0621: Redefining name 'tests' from outer scope (line 226) (redefined-outer-name)
+findtests.py:224:8: R1722: Consider using sys.exit() (consider-using-sys-exit)
+findtests.py:32: error: Missing type parameters for generic type "Iterator"
+findtests.py:87: error: Function is missing a return type annotation
+findtests.py:206: error: Function is missing a type annotation for one or more arguments
I would suggest including a final patch in this series that adds all new
Python files to the checking in 297.
> diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
> index 0aa7a13bba..bfb6b65edc 100644
> --- a/docs/devel/testing.rst
> +++ b/docs/devel/testing.rst
> @@ -111,7 +111,7 @@ check-block
> -----------
>
> ``make check-block`` runs a subset of the block layer iotests (the tests that
> -are in the "auto" group in ``tests/qemu-iotests/group``).
> +are in the "auto" group).
> See the "QEMU iotests" section below for more information.
>
> GCC gcov support
> @@ -224,6 +224,54 @@ another application on the host may have locked the file, possibly leading to a
> test failure. If using such devices are explicitly desired, consider adding
> ``locking=off`` option to disable image locking.
>
> +Test case groups
> +----------------
> +
> +Test may belong to some groups, you may define it in the comment inside the
> +test.
Maybe: "Tests may belong to one or more test groups, which are defined
in the form of a comment in the test source file."
> By convention, test groups are listed in the second line of the test
> +file, after "#!/..." line, like this:
"after the "#!/..." line"
> +
> +.. code::
> +
> + #!/usr/bin/env python3
> + # group: auto quick
> + #
> + ...
> +
> +Additional way of defining groups is creating tests/qemu-iotests/group.local
"An additional" or "Another".
"creating the ... file"
> +file. This should be used only for downstream (this file should never appear
> +in upstream). This file may be used for defining some downstream test groups
> +or for temporary disable tests, like this:
"disabling"
> +
> +.. code::
> +
> + # groups for some company downstream process
> + #
> + # ci - tests to run on build
> + # down - our downstream tests, not for upstream
> + #
> + # Format of each line is:
> + # TEST_NAME TEST_GROUP [TEST_GROUP ]...
> +
> + 013 ci
> + 210 disabled
> + 215 disabled
> + our-ugly-workaround-test down ci
> +
> +The (not exhaustive) list of groups:
Maybe just something like this?
"Note that the following group names have a special meaning:"
> +
> +- quick : Tests in this group should finish within some few seconds.
> +
> +- auto : Tests in this group are used during "make check" and should be
> + runnable in any case. That means they should run with every QEMU binary
> + (also non-x86), with every QEMU configuration (i.e. must not fail if
> + an optional feature is not compiled in - but reporting a "skip" is ok),
> + work at least with the qcow2 file format, work with all kind of host
> + filesystems and users (e.g. "nobody" or "root") and must not take too
> + much memory and disk space (since CI pipelines tend to fail otherwise).
> +
> +- disabled : Tests in this group are disabled and ignored by check.
> +
> .. _docker-ref:
>
> Docker based tests
> diff --git a/tests/qemu-iotests/findtests.py b/tests/qemu-iotests/findtests.py
> new file mode 100755
> index 0000000000..b053db48e8
> --- /dev/null
> +++ b/tests/qemu-iotests/findtests.py
> @@ -0,0 +1,229 @@
> +#!/usr/bin/env python3
> +#
> +# Parse command line options to define set of tests to run.
> +#
> +# Copyright (c) 2020 Virtuozzo International GmbH
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; either version 2 of the License, or
> +# (at your option) any later version.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program. If not, see <http://www.gnu.org/licenses/>.
> +#
> +
> +import os
> +import sys
> +import glob
> +import argparse
> +import re
> +from collections import defaultdict
> +from contextlib import contextmanager
> +from typing import Optional, List, Tuple, Iterator, Set
> +
> +
> +@contextmanager
> +def chdir(path: Optional[str] = None) -> Iterator:
As indicated by mypy, this should be Interator[None].
> + if path is None:
> + yield
> + return
> +
> + saved_dir = os.getcwd()
> + os.chdir(path)
> + try:
> + yield
> + finally:
> + os.chdir(saved_dir)
> +
> +
> +class TestFinder:
> + _argparser = None
> + @classmethod
> + def get_argparser(cls) -> argparse.ArgumentParser:
> + if cls._argparser is not None:
> + return cls._argparser
> +
> + p = argparse.ArgumentParser(description="= test selecting options =",
> + add_help=False, usage=argparse.SUPPRESS)
> +
> + p.add_argument('-g', '--groups', metavar='group1,...',
> + help='include tests from these groups')
> + p.add_argument('-x', '--exclude-groups', metavar='group1,...',
> + help='exclude tests from these groups')
> + p.add_argument('--start-from', metavar='TEST',
> + help='Start from specified test: make sorted sequence '
> + 'of tests as usual and then drop tests from the first '
> + 'one to TEST (not inclusive). This may be used to '
> + 'rerun failed ./check command, starting from the '
> + 'middle of the process.')
> + p.add_argument('tests', metavar='TEST_FILES', nargs='*',
> + help='tests to run')
> +
> + cls._argparser = p
> + return p
> +
> + def __init__(self, test_dir: Optional[str] = None) -> None:
> + self.groups = defaultdict(set)
> +
> + with chdir(test_dir):
> + self.all_tests = glob.glob('[0-9][0-9][0-9]')
> + self.all_tests += [f for f in glob.iglob('tests/*')
> + if not f.endswith('.out') and
> + os.path.isfile(f + '.out')]
> +
> + for t in self.all_tests:
> + with open(t) as f:
> + for line in f:
> + if line.startswith('# group: '):
> + for g in line.split()[2:]:
> + self.groups[g].add(t)
Do we need to allow more than one group comment in a test or could we
return early here, avoiding to read the whole file?
> +
> + def add_group_file(self, fname: str):
> + with open(fname) as f:
> + for line in f:
> + line = line.strip()
> +
> + if (not line) or line[0] == '#':
> + continue
> +
> + words = line.split()
> + test_file = words[0]
> + groups = words[1:]
> +
> + if test_file not in self.all_tests:
> + print(f'Warning: {fname}: "{test_file}" test is not found.'
> + ' Skip.')
> + continue
Should test_file be passed through parse_test_name()? I noticed that I
have to explicitly specify a tests/ prefix in group.local, whereas
prefixing this on the command line seems to be forbidden.
> + for g in groups:
> + self.groups[g].add(test_file)
> +
> + def parse_test_name(self, name: str) -> str:
> + if '/' in name:
> + raise ValueError('Paths are unsupported for test selecting, '
> + f'requiring "{name}" is wrong')
> +
> + if re.fullmatch(r'\d+', name):
> + # Numbered tests are old naming convetion. We should convert them
> + # to three-digit-length, like 1 --> 001.
> + name = f'{int(name):03}'
> + else:
> + # Named tests all should be in tests/ subdirectory
> + name = os.path.join('tests', name)
> +
> + if name not in self.all_tests:
> + raise ValueError(f'Test "{name}" is not found')
> +
> + return name
check should probably catch these ValueError exceptions. Currently, you
get a stack trace, which does include the exception message, but it
doesn't look very nice.
> + def find_tests(self, groups: Optional[List[str]] = None,
> + exclude_groups: Optional[List[str]] = None,
> + tests: Optional[List[str]] = None,
group and tests seem to be read-only, so this can be simplified to
'groups: Sequence[str] = ()' etc. without the explicit handling of None
below.
Maybe exclude_groups should then be treated the same for consistency.
This would mean creating a new list instead of calling .append().
> + start_from: Optional[str] = None) -> List[str]:
> + """Find tests
> +
> + Algorithm:
> +
> + 1. a. if some @groups specified
> + a.1 Take all tests from @groups
> + a.2 Drop tests, which are in at least one of @exclude_groups or in
> + 'disabled' group (if 'disabled' is not listed in @groups)
> + a.3 Add tests from @tests (don't exclude anything from them)
> +
> + b. else, if some @tests specified:
> + b.1 exclude_groups must be not specified, so just take @tests
> +
> + c. else (only @exclude_groups list is non-empty):
> + c.1 Take all tests
> + c.2 Drop tests, which are in at least one of @exclude_groups or in
> + 'disabled' group
> +
> + 2. sort
> +
> + 3. If start_from specified, drop tests from first one to @start_from
> + (not inclusive)
> + """
> + if groups is None:
> + groups = []
> + if exclude_groups is None:
> + exclude_groups = []
> + if tests is None:
> + tests = []
> +
> + res: Set[str] = set()
> + if groups:
> + # Some groups specified. exclude_groups supported, additionally
> + # selecting some individual tests supported as well.
> + res.update(*(self.groups[g] for g in groups))
> + elif tests:
> + # Some individual tests specified, but no groups. In this case
> + # we don't support exclude_groups.
> + if exclude_groups:
> + raise ValueError("Can't exclude from individually specified "
> + "tests.")
> + else:
> + # No tests no groups: start from all tests, exclude_groups
> + # supported.
> + res.update(self.all_tests)
> +
> + if 'disabled' not in groups and 'disabled' not in exclude_groups:
> + exclude_groups.append('disabled')
> +
> + res = res.difference(*(self.groups[g] for g in exclude_groups))
> +
> + # We want to add @tests. But for compatibility with old test names,
> + # we should convert any number < 100 to number padded by
> + # leading zeroes, like 1 -> 001 and 23 -> 023.
> + for t in tests:
> + res.add(self.parse_test_name(t))
> +
> + sequence = sorted(res)
> +
> + if start_from is not None:
> + del sequence[:sequence.index(self.parse_test_name(start_from))]
> +
> + return sequence
> +
> + def find_tests_argv(self, argv: List[str]) -> Tuple[List[str], List[str]]:
> + """Returns tuple of tests list and remaining arguments list"""
> + args, remaining = self.get_argparser().parse_known_args(argv)
> +
> + if args.groups is not None:
> + args.groups = args.groups.split(',')
> +
> + if args.exclude_groups is not None:
> + args.exclude_groups = args.exclude_groups.split(',')
> +
> + return self.find_tests(**vars(args)), remaining
> +
> +
> +def find_tests(argv, test_dir: Optional[str] = None) -> Tuple[List[str],
> + List[str]]:
> + """Returns tuple of tests list and remaining arguments list"""
> + tf = TestFinder(test_dir)
> +
> + if test_dir is None:
> + group_local = 'group.local'
> + else:
> + group_local = os.path.join(test_dir, 'group.local')
> + if os.path.isfile(group_local):
> + tf.add_group_file(group_local)
> +
> + return tf.find_tests_argv(argv)
> +
> +
> +if __name__ == '__main__':
> + if len(sys.argv) == 2 and sys.argv[1] in ['-h', '--help']:
> + TestFinder.get_argparser().print_help()
> + exit()
> +
> + tests, remaining_argv = find_tests(sys.argv[1:])
> + print('\n'.join(tests))
> + if remaining_argv:
> + print('\nUnhandled options: ', remaining_argv)
Kevin
^ permalink raw reply [flat|nested] 43+ messages in thread* Re: [PATCH v6 07/11] iotests: add findtests.py
2021-01-12 16:42 ` Kevin Wolf
@ 2021-01-13 10:37 ` Vladimir Sementsov-Ogievskiy
2021-01-13 11:18 ` Kevin Wolf
2021-01-14 7:38 ` Vladimir Sementsov-Ogievskiy
1 sibling, 1 reply; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-13 10:37 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-block, qemu-devel, mreitz, den, jsnow
12.01.2021 19:42, Kevin Wolf wrote:
> Am 09.01.2021 um 13:26 hat Vladimir Sementsov-Ogievskiy geschrieben:
>> Add python script with new logic of searching for tests:
>>
>> Current ./check behavior:
>> - tests are named [0-9][0-9][0-9]
>> - tests must be registered in group file (even if test doesn't belong
>> to any group, like 142)
>>
>> Behavior of findtests.py:
>> - group file is dropped
>> - tests are all files in tests/ subdirectory (except for .out files),
>> so it's not needed more to "register the test", just create it with
>> appropriate name in tests/ subdirectory. Old names like
>> [0-9][0-9][0-9] (in root iotests directory) are supported too, but
>> not recommended for new tests
>> - groups are parsed from '# group: ' line inside test files
>> - optional file group.local may be used to define some additional
>> groups for downstreams
>> - 'disabled' group is used to temporary disable tests. So instead of
>> commenting tests in old 'group' file you now can add them to
>> disabled group with help of 'group.local' file
>> - selecting test ranges like 5-15 are not supported more
>> (to support restarting failed ./check command from the middle of the
>> process, new argument is added: --start-from)
>>
>> Benefits:
>> - no rebase conflicts in group file on patch porting from branch to
>> branch
>> - no conflicts in upstream, when different series want to occupy same
>> test number
>> - meaningful names for test files
>> For example, with digital number, when some person wants to add some
>> test about block-stream, he most probably will just create a new
>> test. But if there would be test-block-stream test already, he will
>> at first look at it and may be just add a test-case into it.
>> And anyway meaningful names are better.
>>
>> This commit don't update check behavior (which will be don in further
>> commit), still, the documentation changed like new behavior is already
>> here. Let's live with this small inconsistency for the following few
>> commits, until final change.
>>
>> The file findtests.py is self-executable and may be used for debugging
>> purposes.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>> docs/devel/testing.rst | 50 ++++++-
>> tests/qemu-iotests/findtests.py | 229 ++++++++++++++++++++++++++++++++
>
> I extended 297 so that it also checks the newly added Python file, and
> pylint and mypy report some errors:
>
> +************* Module findtests
> +findtests.py:127:19: W0621: Redefining name 'tests' from outer scope (line 226) (redefined-outer-name)
> +findtests.py:224:8: R1722: Consider using sys.exit() (consider-using-sys-exit)
> +findtests.py:32: error: Missing type parameters for generic type "Iterator"
> +findtests.py:87: error: Function is missing a return type annotation
> +findtests.py:206: error: Function is missing a type annotation for one or more arguments
>
> I would suggest including a final patch in this series that adds all new
> Python files to the checking in 297.
>
>> diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
>> index 0aa7a13bba..bfb6b65edc 100644
>> --- a/docs/devel/testing.rst
>> +++ b/docs/devel/testing.rst
>> @@ -111,7 +111,7 @@ check-block
>> -----------
>>
>> ``make check-block`` runs a subset of the block layer iotests (the tests that
>> -are in the "auto" group in ``tests/qemu-iotests/group``).
>> +are in the "auto" group).
>> See the "QEMU iotests" section below for more information.
>>
>> GCC gcov support
>> @@ -224,6 +224,54 @@ another application on the host may have locked the file, possibly leading to a
>> test failure. If using such devices are explicitly desired, consider adding
>> ``locking=off`` option to disable image locking.
>>
>> +Test case groups
>> +----------------
>> +
>> +Test may belong to some groups, you may define it in the comment inside the
>> +test.
>
> Maybe: "Tests may belong to one or more test groups, which are defined
> in the form of a comment in the test source file."
>
>> By convention, test groups are listed in the second line of the test
>> +file, after "#!/..." line, like this:
>
> "after the "#!/..." line"
>
>> +
>> +.. code::
>> +
>> + #!/usr/bin/env python3
>> + # group: auto quick
>> + #
>> + ...
>> +
>> +Additional way of defining groups is creating tests/qemu-iotests/group.local
>
> "An additional" or "Another".
> "creating the ... file"
>
>> +file. This should be used only for downstream (this file should never appear
>> +in upstream). This file may be used for defining some downstream test groups
>> +or for temporary disable tests, like this:
>
> "disabling"
>
>> +
>> +.. code::
>> +
>> + # groups for some company downstream process
>> + #
>> + # ci - tests to run on build
>> + # down - our downstream tests, not for upstream
>> + #
>> + # Format of each line is:
>> + # TEST_NAME TEST_GROUP [TEST_GROUP ]...
>> +
>> + 013 ci
>> + 210 disabled
>> + 215 disabled
>> + our-ugly-workaround-test down ci
>> +
>> +The (not exhaustive) list of groups:
>
> Maybe just something like this?
>
> "Note that the following group names have a special meaning:"
>
>> +
>> +- quick : Tests in this group should finish within some few seconds.
>> +
>> +- auto : Tests in this group are used during "make check" and should be
>> + runnable in any case. That means they should run with every QEMU binary
>> + (also non-x86), with every QEMU configuration (i.e. must not fail if
>> + an optional feature is not compiled in - but reporting a "skip" is ok),
>> + work at least with the qcow2 file format, work with all kind of host
>> + filesystems and users (e.g. "nobody" or "root") and must not take too
>> + much memory and disk space (since CI pipelines tend to fail otherwise).
>> +
>> +- disabled : Tests in this group are disabled and ignored by check.
>> +
>> .. _docker-ref:
>>
>> Docker based tests
>> diff --git a/tests/qemu-iotests/findtests.py b/tests/qemu-iotests/findtests.py
>> new file mode 100755
>> index 0000000000..b053db48e8
>> --- /dev/null
>> +++ b/tests/qemu-iotests/findtests.py
>> @@ -0,0 +1,229 @@
>> +#!/usr/bin/env python3
>> +#
>> +# Parse command line options to define set of tests to run.
>> +#
>> +# Copyright (c) 2020 Virtuozzo International GmbH
>> +#
>> +# This program is free software; you can redistribute it and/or modify
>> +# it under the terms of the GNU General Public License as published by
>> +# the Free Software Foundation; either version 2 of the License, or
>> +# (at your option) any later version.
>> +#
>> +# This program is distributed in the hope that it will be useful,
>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> +# GNU General Public License for more details.
>> +#
>> +# You should have received a copy of the GNU General Public License
>> +# along with this program. If not, see <http://www.gnu.org/licenses/>.
>> +#
>> +
>> +import os
>> +import sys
>> +import glob
>> +import argparse
>> +import re
>> +from collections import defaultdict
>> +from contextlib import contextmanager
>> +from typing import Optional, List, Tuple, Iterator, Set
>> +
>> +
>> +@contextmanager
>> +def chdir(path: Optional[str] = None) -> Iterator:
>
> As indicated by mypy, this should be Interator[None].
>
>> + if path is None:
>> + yield
>> + return
>> +
>> + saved_dir = os.getcwd()
>> + os.chdir(path)
>> + try:
>> + yield
>> + finally:
>> + os.chdir(saved_dir)
>> +
>> +
>> +class TestFinder:
>> + _argparser = None
>> + @classmethod
>> + def get_argparser(cls) -> argparse.ArgumentParser:
>> + if cls._argparser is not None:
>> + return cls._argparser
>> +
>> + p = argparse.ArgumentParser(description="= test selecting options =",
>> + add_help=False, usage=argparse.SUPPRESS)
>> +
>> + p.add_argument('-g', '--groups', metavar='group1,...',
>> + help='include tests from these groups')
>> + p.add_argument('-x', '--exclude-groups', metavar='group1,...',
>> + help='exclude tests from these groups')
>> + p.add_argument('--start-from', metavar='TEST',
>> + help='Start from specified test: make sorted sequence '
>> + 'of tests as usual and then drop tests from the first '
>> + 'one to TEST (not inclusive). This may be used to '
>> + 'rerun failed ./check command, starting from the '
>> + 'middle of the process.')
>> + p.add_argument('tests', metavar='TEST_FILES', nargs='*',
>> + help='tests to run')
>> +
>> + cls._argparser = p
>> + return p
>> +
>> + def __init__(self, test_dir: Optional[str] = None) -> None:
>> + self.groups = defaultdict(set)
>> +
>> + with chdir(test_dir):
>> + self.all_tests = glob.glob('[0-9][0-9][0-9]')
>> + self.all_tests += [f for f in glob.iglob('tests/*')
>> + if not f.endswith('.out') and
>> + os.path.isfile(f + '.out')]
>> +
>> + for t in self.all_tests:
>> + with open(t) as f:
>> + for line in f:
>> + if line.startswith('# group: '):
>> + for g in line.split()[2:]:
>> + self.groups[g].add(t)
>
> Do we need to allow more than one group comment in a test or could we
> return early here, avoiding to read the whole file?
Hmm, if we defined it's as a "convention", I think we can just check only the second line of the file.
>
>> +
>> + def add_group_file(self, fname: str):
>> + with open(fname) as f:
>> + for line in f:
>> + line = line.strip()
>> +
>> + if (not line) or line[0] == '#':
>> + continue
>> +
>> + words = line.split()
>> + test_file = words[0]
>> + groups = words[1:]
>> +
>> + if test_file not in self.all_tests:
>> + print(f'Warning: {fname}: "{test_file}" test is not found.'
>> + ' Skip.')
>> + continue
>
> Should test_file be passed through parse_test_name()? I noticed that I
> have to explicitly specify a tests/ prefix in group.local, whereas
> prefixing this on the command line seems to be forbidden.
agree.
>
>> + for g in groups:
>> + self.groups[g].add(test_file)
>> +
>> + def parse_test_name(self, name: str) -> str:
>> + if '/' in name:
>> + raise ValueError('Paths are unsupported for test selecting, '
>> + f'requiring "{name}" is wrong')
>> +
>> + if re.fullmatch(r'\d+', name):
>> + # Numbered tests are old naming convetion. We should convert them
>> + # to three-digit-length, like 1 --> 001.
>> + name = f'{int(name):03}'
>> + else:
>> + # Named tests all should be in tests/ subdirectory
>> + name = os.path.join('tests', name)
>> +
>> + if name not in self.all_tests:
>> + raise ValueError(f'Test "{name}" is not found')
>> +
>> + return name
>
> check should probably catch these ValueError exceptions. Currently, you
> get a stack trace, which does include the exception message, but it
> doesn't look very nice.
>
>> + def find_tests(self, groups: Optional[List[str]] = None,
>> + exclude_groups: Optional[List[str]] = None,
>> + tests: Optional[List[str]] = None,
>
> group and tests seem to be read-only, so this can be simplified to
> 'groups: Sequence[str] = ()' etc. without the explicit handling of None
> below.
>
> Maybe exclude_groups should then be treated the same for consistency.
> This would mean creating a new list instead of calling .append().
will try.
thanks, I'll take all the suggestions.
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 43+ messages in thread* Re: [PATCH v6 07/11] iotests: add findtests.py
2021-01-13 10:37 ` Vladimir Sementsov-Ogievskiy
@ 2021-01-13 11:18 ` Kevin Wolf
0 siblings, 0 replies; 43+ messages in thread
From: Kevin Wolf @ 2021-01-13 11:18 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy; +Cc: qemu-block, qemu-devel, mreitz, den, jsnow
Am 13.01.2021 um 11:37 hat Vladimir Sementsov-Ogievskiy geschrieben:
> 12.01.2021 19:42, Kevin Wolf wrote:
> > Am 09.01.2021 um 13:26 hat Vladimir Sementsov-Ogievskiy geschrieben:
> > > + def __init__(self, test_dir: Optional[str] = None) -> None:
> > > + self.groups = defaultdict(set)
> > > +
> > > + with chdir(test_dir):
> > > + self.all_tests = glob.glob('[0-9][0-9][0-9]')
> > > + self.all_tests += [f for f in glob.iglob('tests/*')
> > > + if not f.endswith('.out') and
> > > + os.path.isfile(f + '.out')]
> > > +
> > > + for t in self.all_tests:
> > > + with open(t) as f:
> > > + for line in f:
> > > + if line.startswith('# group: '):
> > > + for g in line.split()[2:]:
> > > + self.groups[g].add(t)
> >
> > Do we need to allow more than one group comment in a test or could we
> > return early here, avoiding to read the whole file?
>
> Hmm, if we defined it's as a "convention", I think we can just check
> only the second line of the file.
I was actually just thinking of a break after finding the "# group: "
line. It would still be recognised as the third line, tenth line or
anywhere else, but only one line in the file would be recognised.
But if you think only looking at the second line is even easier, we can
do that. However, then I wouldn't call it "as a convention" any more in
the documentation because then it wouldn't work any other way any more.
Kevin
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v6 07/11] iotests: add findtests.py
2021-01-12 16:42 ` Kevin Wolf
2021-01-13 10:37 ` Vladimir Sementsov-Ogievskiy
@ 2021-01-14 7:38 ` Vladimir Sementsov-Ogievskiy
2021-01-14 10:48 ` Kevin Wolf
1 sibling, 1 reply; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-14 7:38 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-block, qemu-devel, mreitz, den, jsnow
12.01.2021 19:42, Kevin Wolf wrote:
>> + def find_tests(self, groups: Optional[List[str]] = None,
>> + exclude_groups: Optional[List[str]] = None,
>> + tests: Optional[List[str]] = None,
> group and tests seem to be read-only, so this can be simplified to
> 'groups: Sequence[str] = ()' etc. without the explicit handling of None
> below.
None comes when cmdline argument is not set, will sequence as you propose I get
Traceback (most recent call last):
File "/work/src/qemu/up-refactor-iotests/build/tests/qemu-iotests/./check", line 37, in <module>
tests, remaining_argv = find_tests(env.remaining_argv,
File "/work/src/qemu/up-refactor-iotests/tests/qemu-iotests/findtests.py", line 208, in find_tests
return tf.find_tests_argv(argv)
File "/work/src/qemu/up-refactor-iotests/tests/qemu-iotests/findtests.py", line 193, in find_tests_argv
return self.find_tests(**vars(args)), remaining
File "/work/src/qemu/up-refactor-iotests/tests/qemu-iotests/findtests.py", line 165, in find_tests
if 'disabled' not in groups and 'disabled' not in exclude_groups:
TypeError: argument of type 'NoneType' is not iterable
So, seems simpler to keep all as is than modifying find_tests_argv to not pass None arguments.
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 43+ messages in thread* Re: [PATCH v6 07/11] iotests: add findtests.py
2021-01-14 7:38 ` Vladimir Sementsov-Ogievskiy
@ 2021-01-14 10:48 ` Kevin Wolf
0 siblings, 0 replies; 43+ messages in thread
From: Kevin Wolf @ 2021-01-14 10:48 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy; +Cc: qemu-block, qemu-devel, mreitz, den, jsnow
Am 14.01.2021 um 08:38 hat Vladimir Sementsov-Ogievskiy geschrieben:
> 12.01.2021 19:42, Kevin Wolf wrote:
> > > + def find_tests(self, groups: Optional[List[str]] = None,
> > > + exclude_groups: Optional[List[str]] = None,
> > > + tests: Optional[List[str]] = None,
> > group and tests seem to be read-only, so this can be simplified to
> > 'groups: Sequence[str] = ()' etc. without the explicit handling of None
> > below.
>
> None comes when cmdline argument is not set, will sequence as you propose I get
>
> Traceback (most recent call last):
> File "/work/src/qemu/up-refactor-iotests/build/tests/qemu-iotests/./check", line 37, in <module>
> tests, remaining_argv = find_tests(env.remaining_argv,
> File "/work/src/qemu/up-refactor-iotests/tests/qemu-iotests/findtests.py", line 208, in find_tests
> return tf.find_tests_argv(argv)
> File "/work/src/qemu/up-refactor-iotests/tests/qemu-iotests/findtests.py", line 193, in find_tests_argv
> return self.find_tests(**vars(args)), remaining
> File "/work/src/qemu/up-refactor-iotests/tests/qemu-iotests/findtests.py", line 165, in find_tests
> if 'disabled' not in groups and 'disabled' not in exclude_groups:
> TypeError: argument of type 'NoneType' is not iterable
>
> So, seems simpler to keep all as is than modifying find_tests_argv to not pass None arguments.
Ah, I wasn't aware that you would get explicit None. Fair enough.
Kevin
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v6 07/11] iotests: add findtests.py
2021-01-09 12:26 ` [PATCH v6 07/11] iotests: add findtests.py Vladimir Sementsov-Ogievskiy
2021-01-12 16:42 ` Kevin Wolf
@ 2021-01-13 23:10 ` Eric Blake
1 sibling, 0 replies; 43+ messages in thread
From: Eric Blake @ 2021-01-13 23:10 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, den, jsnow, qemu-devel, mreitz
On 1/9/21 6:26 AM, Vladimir Sementsov-Ogievskiy wrote:
> Add python script with new logic of searching for tests:
>
> Current ./check behavior:
> - tests are named [0-9][0-9][0-9]
> - tests must be registered in group file (even if test doesn't belong
> to any group, like 142)
>
>
> This commit don't update check behavior (which will be don in further
done
> commit), still, the documentation changed like new behavior is already
> here. Let's live with this small inconsistency for the following few
> commits, until final change.
>
> The file findtests.py is self-executable and may be used for debugging
> purposes.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> docs/devel/testing.rst | 50 ++++++-
> tests/qemu-iotests/findtests.py | 229 ++++++++++++++++++++++++++++++++
> 2 files changed, 278 insertions(+), 1 deletion(-)
> create mode 100755 tests/qemu-iotests/findtests.py
In addition to Kevin's review,
> +Additional way of defining groups is creating tests/qemu-iotests/group.local
> +file. This should be used only for downstream (this file should never appear
> +in upstream). This file may be used for defining some downstream test groups
> +or for temporary disable tests, like this:
Kevin suggested s/disable/disabling/; I'd go one step further:
/temporary disable/temporarily disabling/
> +
> +- quick : Tests in this group should finish within some few seconds.
s/some/a/
Also, for each group name, the space before ':' looks odd.
> +++ b/tests/qemu-iotests/findtests.py
> @@ -0,0 +1,229 @@
> +#!/usr/bin/env python3
> +#
> +# Parse command line options to define set of tests to run.
> +#
> +# Copyright (c) 2020 Virtuozzo International GmbH
Worth claiming 2021 as well?
Otherwise, my python review is weak, but I didn't spot anything
blatantly wrong.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 43+ messages in thread
* [PATCH v6 08/11] iotests: add testenv.py
2021-01-09 12:26 [PATCH v6 00/11] Rework iotests/check Vladimir Sementsov-Ogievskiy
` (6 preceding siblings ...)
2021-01-09 12:26 ` [PATCH v6 07/11] iotests: add findtests.py Vladimir Sementsov-Ogievskiy
@ 2021-01-09 12:26 ` Vladimir Sementsov-Ogievskiy
2021-01-12 17:36 ` Kevin Wolf
` (2 more replies)
2021-01-09 12:26 ` [PATCH v6 09/11] iotests: add testrunner.py Vladimir Sementsov-Ogievskiy
` (2 subsequent siblings)
10 siblings, 3 replies; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-09 12:26 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, vsementsov, jsnow, qemu-devel, mreitz, den
Add TestEnv class, which will handle test environment in a new python
iotests running framework.
Difference with current ./check interface:
- -v (verbose) option dropped, as it is unused
- -xdiff option is dropped, until somebody complains that it is needed
- same for -n option
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
tests/qemu-iotests/testenv.py | 328 ++++++++++++++++++++++++++++++++++
1 file changed, 328 insertions(+)
create mode 100755 tests/qemu-iotests/testenv.py
diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py
new file mode 100755
index 0000000000..ecaf76fb85
--- /dev/null
+++ b/tests/qemu-iotests/testenv.py
@@ -0,0 +1,328 @@
+#!/usr/bin/env python3
+#
+# Parse command line options to manage test environment variables.
+#
+# Copyright (c) 2020 Virtuozzo International GmbH
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+import os
+import sys
+import tempfile
+from pathlib import Path
+import shutil
+import collections
+import subprocess
+import argparse
+from typing import List, Dict
+
+
+def get_default_machine(qemu_prog: str) -> str:
+ outp = subprocess.run([qemu_prog, '-machine', 'help'], check=True,
+ text=True, stdout=subprocess.PIPE).stdout
+
+ machines = outp.split('\n')
+ default_machine = next(m for m in machines if m.endswith(' (default)'))
+ default_machine = default_machine.split(' ', 1)[0]
+
+ alias_suf = ' (alias of {})'.format(default_machine)
+ alias = next((m for m in machines if m.endswith(alias_suf)), None)
+ if alias is not None:
+ default_machine = alias.split(' ', 1)[0]
+
+ return default_machine
+
+
+class TestEnv:
+ """
+ Manage system environment for running tests
+
+ The following variables are supported/provided. They are represented by
+ lower-cased TestEnv attributes.
+ """
+ env_variables = ['PYTHONPATH', 'TEST_DIR', 'SOCK_DIR', 'SAMPLE_IMG_DIR',
+ 'OUTPUT_DIR', 'PYTHON', 'QEMU_PROG', 'QEMU_IMG_PROG',
+ 'QEMU_IO_PROG', 'QEMU_NBD_PROG', 'QSD_PROG',
+ 'SOCKET_SCM_HELPER', 'QEMU_OPTIONS', 'QEMU_IMG_OPTIONS',
+ 'QEMU_IO_OPTIONS', 'QEMU_NBD_OPTIONS', 'IMGOPTS',
+ 'IMGFMT', 'IMGPROTO', 'AIOMODE', 'CACHEMODE',
+ 'VALGRIND_QEMU', 'CACHEMODE_IS_DEFAULT', 'IMGFMT_GENERIC',
+ 'IMGOPTSSYNTAX', 'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE']
+
+ def get_env(self) -> Dict[str, str]:
+ env = {}
+ for v in self.env_variables:
+ val = getattr(self, v.lower(), None)
+ if val is not None:
+ env[v] = val
+
+ return env
+
+ _argparser = None
+ @classmethod
+ def get_argparser(cls) -> argparse.ArgumentParser:
+ if cls._argparser is not None:
+ return cls._argparser
+
+ p = argparse.ArgumentParser(description="= test environment options =",
+ add_help=False, usage=argparse.SUPPRESS)
+
+ p.add_argument('-d', dest='debug', action='store_true', help='debug')
+ p.add_argument('-misalign', action='store_true',
+ help='misalign memory allocations')
+
+ p.set_defaults(imgfmt='raw', imgproto='file')
+
+ format_list = ['raw', 'bochs', 'cloop', 'parallels', 'qcow', 'qcow2',
+ 'qed', 'vdi', 'vpc', 'vhdx', 'vmdk', 'luks', 'dmg']
+ g = p.add_argument_group(
+ 'image format options',
+ 'The following options sets IMGFMT environment variable. '
+ 'At most one chose is allowed, default is "raw"')
+ g = g.add_mutually_exclusive_group()
+ for fmt in format_list:
+ g.add_argument('-' + fmt, dest='imgfmt', action='store_const',
+ const=fmt)
+
+ protocol_list = ['file', 'rbd', 'sheepdoc', 'nbd', 'ssh', 'nfs',
+ 'fuse']
+ g = p.add_argument_group(
+ 'image protocol options',
+ 'The following options sets IMGPROTO environment variably. '
+ 'At most one chose is allowed, default is "file"')
+ g = g.add_mutually_exclusive_group()
+ for prt in protocol_list:
+ g.add_argument('-' + prt, dest='imgproto', action='store_const',
+ const=prt)
+
+ g = p.add_mutually_exclusive_group()
+ # We don't set default for cachemode, as we need to distinguish dafult
+ # from user input later.
+ g.add_argument('-nocache', dest='cachemode', action='store_const',
+ const='none', help='set cache mode "none" (O_DIRECT), '
+ 'sets CACHEMODE environment variable')
+ g.add_argument('-c', dest='cachemode',
+ help='sets CACHEMODE environment variable')
+
+ p.add_argument('-i', dest='aiomode', default='threads',
+ help='sets AIOMODE environment variable')
+
+ g = p.add_argument_group('bash tests options',
+ 'The following options are ignored by '
+ 'python tests. TODO: support them in '
+ 'iotests.py')
+ g.add_argument('-o', dest='imgopts',
+ help='options to pass to qemu-img create/convert, sets '
+ 'IMGOPTS environment variable')
+ p.add_argument('-valgrind', dest='VALGRIND_QEMU', action='store_const',
+ const='y', help='use valgrind, sets VALGRIND_QEMU '
+ 'environment variable')
+
+ cls._argparser = p
+ return p
+
+ def init_handle_argv(self, argv: List[str]) -> None:
+
+ # Hints for mypy, about arguments which will be set by argparse
+ args, self.remaining_argv = self.get_argparser().parse_known_args(argv)
+ self.imgfmt = args.imgfmt
+ self.imgproto = args.imgproto
+ self.aiomode = args.aiomode
+ self.imgopts = args.imgopts
+ self.misalign = args.misalign
+ self.debug = args.debug
+
+ if args.cachemode is None:
+ self.cachemode_is_default = 'true'
+ self.cachemode = 'writeback'
+ else:
+ self.cachemode_is_default = 'false'
+ self.cachemode = args.cachemode
+
+ def init_directories(self):
+ """Init directory variables:
+ PYTHONPATH
+ TEST_DIR
+ SOCK_DIR
+ SAMPLE_IMG_DIR
+ OUTPUT_DIR
+ """
+ self.pythonpath = os.getenv('PYTHONPATH')
+ if self.pythonpath:
+ self.pythonpath = self.source_iotests + os.pathsep + \
+ self.pythonpath
+ else:
+ self.pythonpath = self.source_iotests
+
+ self.test_dir = os.getenv('TEST_DIR',
+ os.path.join(os.getcwd(), 'scratch'))
+ Path(self.test_dir).mkdir(parents=True, exist_ok=True)
+
+ self.sock_dir = os.getenv('SOCK_DIR')
+ self.tmp_sock_dir = False
+ if self.sock_dir:
+ Path(self.test_dir).mkdir(parents=True, exist_ok=True)
+ else:
+ self.sock_dir = tempfile.mkdtemp()
+ self.tmp_sock_dir = True
+
+ self.sample_img_dir = os.getenv('SAMPLE_IMG_DIR',
+ os.path.join(self.source_iotests,
+ 'sample_images'))
+
+ self.output_dir = os.getcwd() # OUTPUT_DIR
+
+ def init_binaries(self):
+ """Init binary path variables:
+ PYTHON (for bash tests)
+ QEMU_PROG, QEMU_IMG_PROG, QEMU_IO_PROG, QEMU_NBD_PROG, QSD_PROG
+ SOCKET_SCM_HELPER
+ """
+ self.python = '/usr/bin/python3 -B'
+
+ def root(*names):
+ return os.path.join(self.build_root, *names)
+
+ arch = os.uname().machine
+ if 'ppc64' in arch:
+ arch = 'ppc64'
+
+ self.qemu_prog = os.getenv('QEMU_PROG', root(f'qemu-system-{arch}'))
+ self.qemu_img_prog = os.getenv('QEMU_IMG_PROG', root('qemu-img'))
+ self.qemu_io_prog = os.getenv('QEMU_IO_PROG', root('qemu-io'))
+ self.qemu_nbd_prog = os.getenv('QEMU_NBD_PROG', root('qemu-nbd'))
+ self.qsd_prog = os.getenv('QSD_PROG', root('storage-daemon',
+ 'qemu-storage-daemon'))
+
+ for b in [self.qemu_img_prog, self.qemu_io_prog, self.qemu_nbd_prog,
+ self.qemu_prog, self.qsd_prog]:
+ if not os.path.exists(b):
+ exit('Not such file: ' + b)
+ if not os.access(b, os.X_OK):
+ exit('Not executable: ' + b)
+
+ helper_path = os.path.join(self.build_iotests, 'socket_scm_helper')
+ if os.access(helper_path, os.X_OK):
+ self.socket_scm_helper = helper_path # SOCKET_SCM_HELPER
+
+ def __init__(self, argv: List[str]) -> None:
+ """Parse args and environment"""
+
+ # Initialize generic paths: build_root, build_iotests, source_iotests,
+ # which are needed to initialize some environment variables. They are
+ # used by init_*() functions as well.
+
+
+ if os.path.islink(sys.argv[0]):
+ # called from the build tree
+ self.source_iotests = os.path.dirname(os.readlink(sys.argv[0]))
+ self.build_iotests = os.path.dirname(os.path.abspath(sys.argv[0]))
+ else:
+ # called from the source tree
+ self.source_iotests = os.getcwd()
+ self.build_iotests = self.source_iotests
+
+ self.build_root = os.path.join(self.build_iotests, '..', '..')
+
+ self.init_handle_argv(argv)
+ self.init_directories()
+ self.init_binaries()
+
+ # QEMU_OPTIONS
+ self.qemu_options = '-nodefaults -display none -accel qtest'
+ machine_map = (
+ (('arm', 'aarch64'), 'virt'),
+ ('avr', 'mega2560'),
+ ('rx', 'gdbsim-r5f562n8'),
+ ('tricore', 'tricore_testboard')
+ )
+ for suffix, machine in machine_map:
+ if self.qemu_prog.endswith(f'qemu-system-{suffix}'):
+ self.qemu_options += f' -machine {machine}'
+
+ # QEMU_DEFAULT_MACHINE
+ self.qemu_default_machine = get_default_machine(self.qemu_prog)
+
+ self.qemu_img_options = os.getenv('QEMU_IMG_OPTIONS')
+ self.qemu_nbd_options = os.getenv('QEMU_NBD_OPTIONS')
+
+ is_generic = self.imgfmt not in ['bochs', 'cloop', 'dmg']
+ self.imgfmt_generic = 'true' if is_generic else 'false'
+
+ self.qemu_io_options = f'--cache {self.cachemode} --aio {self.aiomode}'
+ if self.misalign:
+ self.qemu_io_options += ' --misalign'
+
+ self.qemu_io_options_no_fmt = self.qemu_io_options
+
+ if self.imgfmt == 'luks':
+ self.imgoptssyntax = 'true'
+ self.imgkeysecret = '123456'
+ if not self.imgopts:
+ self.imgopts = 'iter-time=10'
+ elif 'iter-time=' not in self.imgopts:
+ self.imgopts += ',iter-time=10'
+ else:
+ self.imgoptssyntax = 'false'
+ self.qemu_io_options += ' -f ' + self.imgfmt
+
+ if self.imgfmt == 'vmkd':
+ if not self.imgopts:
+ self.imgopts = 'zeroed_grain=on'
+ elif 'zeroed_grain=' not in self.imgopts:
+ self.imgopts += ',zeroed_grain=on'
+
+ def close(self) -> None:
+ if self.tmp_sock_dir:
+ shutil.rmtree(self.sock_dir)
+
+ def __enter__(self) -> 'TestEnv':
+ return self
+
+ def __exit__(self, *args) -> None:
+ self.close()
+
+ def print_env(self) -> None:
+ template = """\
+QEMU -- "{QEMU_PROG}" {QEMU_OPTIONS}
+QEMU_IMG -- "{QEMU_IMG_PROG}" {QEMU_IMG_OPTIONS}
+QEMU_IO -- "{QEMU_IO_PROG}" {QEMU_IO_OPTIONS}
+QEMU_NBD -- "{QEMU_NBD_PROG}" {QEMU_NBD_OPTIONS}
+IMGFMT -- {IMGFMT}{imgopts}
+IMGPROTO -- {IMGPROTO}
+PLATFORM -- {platform}
+TEST_DIR -- {TEST_DIR}
+SOCK_DIR -- {SOCK_DIR}
+SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER}"""
+
+ args = collections.defaultdict(str, self.get_env())
+
+ if 'IMGOPTS' in args:
+ args['imgopts'] = f" ({args['IMGOPTS']})"
+
+ u = os.uname()
+ args['platform'] = f'{u.sysname}/{u.machine} {u.nodename} {u.release}'
+
+ print(template.format_map(args))
+
+
+if __name__ == '__main__':
+ if len(sys.argv) == 2 and sys.argv[1] in ['-h', '--help']:
+ TestEnv.get_argparser().print_help()
+ exit()
+
+ with TestEnv(sys.argv) as te:
+ te.print_env()
+ print('\nUnhandled options: ', te.remaining_argv)
--
2.29.2
^ permalink raw reply related [flat|nested] 43+ messages in thread* Re: [PATCH v6 08/11] iotests: add testenv.py
2021-01-09 12:26 ` [PATCH v6 08/11] iotests: add testenv.py Vladimir Sementsov-Ogievskiy
@ 2021-01-12 17:36 ` Kevin Wolf
2021-01-14 6:14 ` Vladimir Sementsov-Ogievskiy
2021-01-14 4:28 ` Vladimir Sementsov-Ogievskiy
2021-01-15 11:18 ` Kevin Wolf
2 siblings, 1 reply; 43+ messages in thread
From: Kevin Wolf @ 2021-01-12 17:36 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy; +Cc: qemu-block, qemu-devel, mreitz, den, jsnow
Am 09.01.2021 um 13:26 hat Vladimir Sementsov-Ogievskiy geschrieben:
> Add TestEnv class, which will handle test environment in a new python
> iotests running framework.
>
> Difference with current ./check interface:
> - -v (verbose) option dropped, as it is unused
>
> - -xdiff option is dropped, until somebody complains that it is needed
> - same for -n option
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+************* Module testenv
+testenv.py:48:0: R0902: Too many instance attributes (34/7) (too-many-instance-attributes)
+testenv.py:212:16: R1722: Consider using sys.exit() (consider-using-sys-exit)
+testenv.py:214:16: R1722: Consider using sys.exit() (consider-using-sys-exit)
+testenv.py:324:8: R1722: Consider using sys.exit() (consider-using-sys-exit)
+testenv.py:1:0: R0801: Similar lines in 2 files
+==findtests:45
+==testenv:72
+ _argparser = None
+ @classmethod
+ def get_argparser(cls) -> argparse.ArgumentParser:
+ if cls._argparser is not None:
+ return cls._argparser
+ (duplicate-code)
+testenv.py:294: error: Function is missing a type annotation for one or more arguments
I wonder whether we should just disable the "similar lines" check? The
instance attributes one looks kind of arbitrary, too. The rest looks
valid enough.
Kevin
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v6 08/11] iotests: add testenv.py
2021-01-12 17:36 ` Kevin Wolf
@ 2021-01-14 6:14 ` Vladimir Sementsov-Ogievskiy
2021-01-14 6:16 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-14 6:14 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-block, qemu-devel, mreitz, den, jsnow
12.01.2021 20:36, Kevin Wolf wrote:
> Am 09.01.2021 um 13:26 hat Vladimir Sementsov-Ogievskiy geschrieben:
>> Add TestEnv class, which will handle test environment in a new python
>> iotests running framework.
>>
>> Difference with current ./check interface:
>> - -v (verbose) option dropped, as it is unused
>>
>> - -xdiff option is dropped, until somebody complains that it is needed
>> - same for -n option
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>
> +************* Module testenv
> +testenv.py:48:0: R0902: Too many instance attributes (34/7) (too-many-instance-attributes)
> +testenv.py:212:16: R1722: Consider using sys.exit() (consider-using-sys-exit)
> +testenv.py:214:16: R1722: Consider using sys.exit() (consider-using-sys-exit)
> +testenv.py:324:8: R1722: Consider using sys.exit() (consider-using-sys-exit)
> +testenv.py:1:0: R0801: Similar lines in 2 files
> +==findtests:45
> +==testenv:72
> + _argparser = None
> + @classmethod
> + def get_argparser(cls) -> argparse.ArgumentParser:
> + if cls._argparser is not None:
> + return cls._argparser
> + (duplicate-code)
> +testenv.py:294: error: Function is missing a type annotation for one or more arguments
Interesting, I don't see "similar lines" error in the output..
[root@kvm qemu-iotests]# pylint-3 --score=n findtests.py testenv.py
************* Module testenv
testenv.py:48:0: R0902: Too many instance attributes (34/7) (too-many-instance-attributes)
testenv.py:212:16: R1722: Consider using sys.exit() (consider-using-sys-exit)
testenv.py:214:16: R1722: Consider using sys.exit() (consider-using-sys-exit)
testenv.py:324:8: R1722: Consider using sys.exit() (consider-using-sys-exit)
[root@kvm qemu-iotests]# pylint --score=n findtests.py testenv.py
************* Module testenv
testenv.py:48:0: R0902: Too many instance attributes (34/7) (too-many-instance-attributes)
testenv.py:212:16: R1722: Consider using sys.exit() (consider-using-sys-exit)
testenv.py:214:16: R1722: Consider using sys.exit() (consider-using-sys-exit)
testenv.py:324:8: R1722: Consider using sys.exit() (consider-using-sys-exit)
[root@kvm qemu-iotests]# pylint-3 --version
pylint 2.6.0
astroid 2.4.2
Python 3.9.1 (default, Dec 8 2020, 00:00:00)
[GCC 10.2.1 20201125 (Red Hat 10.2.1-9)]
[root@kvm qemu-iotests]# pylint --version
pylint 2.6.0
astroid 2.4.2
Python 3.9.1 (default, Dec 8 2020, 00:00:00)
[GCC 10.2.1 20201125 (Red Hat 10.2.1-9)]
[root@kvm qemu-iotests]# which pylint-3
/usr/bin/pylint-3
[root@kvm qemu-iotests]# rpm -qf /usr/bin/pylint-3
python3-pylint-2.6.0-2.fc33.noarch
>
> I wonder whether we should just disable the "similar lines" check? The
> instance attributes one looks kind of arbitrary, too. The rest looks
> valid enough.
>
> Kevin
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v6 08/11] iotests: add testenv.py
2021-01-14 6:14 ` Vladimir Sementsov-Ogievskiy
@ 2021-01-14 6:16 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-14 6:16 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-block, qemu-devel, mreitz, den, jsnow
14.01.2021 09:14, Vladimir Sementsov-Ogievskiy wrote:
> 12.01.2021 20:36, Kevin Wolf wrote:
>> Am 09.01.2021 um 13:26 hat Vladimir Sementsov-Ogievskiy geschrieben:
>>> Add TestEnv class, which will handle test environment in a new python
>>> iotests running framework.
>>>
>>> Difference with current ./check interface:
>>> - -v (verbose) option dropped, as it is unused
>>>
>>> - -xdiff option is dropped, until somebody complains that it is needed
>>> - same for -n option
>>>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>
>> +************* Module testenv
>> +testenv.py:48:0: R0902: Too many instance attributes (34/7) (too-many-instance-attributes)
>> +testenv.py:212:16: R1722: Consider using sys.exit() (consider-using-sys-exit)
>> +testenv.py:214:16: R1722: Consider using sys.exit() (consider-using-sys-exit)
>> +testenv.py:324:8: R1722: Consider using sys.exit() (consider-using-sys-exit)
>> +testenv.py:1:0: R0801: Similar lines in 2 files
>> +==findtests:45
>> +==testenv:72
>> + _argparser = None
>> + @classmethod
>> + def get_argparser(cls) -> argparse.ArgumentParser:
>> + if cls._argparser is not None:
>> + return cls._argparser
>> + (duplicate-code)
>> +testenv.py:294: error: Function is missing a type annotation for one or more arguments
>
> Interesting, I don't see "similar lines" error in the output..
>
Ah, that's because I added a newline in findtests.py. Add it in testenv.py and the error comes back.
>
>>
>> I wonder whether we should just disable the "similar lines" check? The
>> instance attributes one looks kind of arbitrary, too. The rest looks
>> valid enough.
>>
>> Kevin
>>
>
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v6 08/11] iotests: add testenv.py
2021-01-09 12:26 ` [PATCH v6 08/11] iotests: add testenv.py Vladimir Sementsov-Ogievskiy
2021-01-12 17:36 ` Kevin Wolf
@ 2021-01-14 4:28 ` Vladimir Sementsov-Ogievskiy
2021-01-14 11:14 ` Kevin Wolf
2021-01-15 11:18 ` Kevin Wolf
2 siblings, 1 reply; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-14 4:28 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel, mreitz, den, jsnow
09.01.2021 15:26, Vladimir Sementsov-Ogievskiy wrote:
> Add TestEnv class, which will handle test environment in a new python
> iotests running framework.
>
> Difference with current ./check interface:
> - -v (verbose) option dropped, as it is unused
>
> - -xdiff option is dropped, until somebody complains that it is needed
> - same for -n option
One more thing dropped is looking for binaries in $build_iotests directory.
Do someone use this feature? Or we can drop it, and reimplement when someone report the degradation?
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v6 08/11] iotests: add testenv.py
2021-01-14 4:28 ` Vladimir Sementsov-Ogievskiy
@ 2021-01-14 11:14 ` Kevin Wolf
2021-01-14 11:26 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 43+ messages in thread
From: Kevin Wolf @ 2021-01-14 11:14 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy; +Cc: qemu-block, qemu-devel, mreitz, den, jsnow
Am 14.01.2021 um 05:28 hat Vladimir Sementsov-Ogievskiy geschrieben:
> 09.01.2021 15:26, Vladimir Sementsov-Ogievskiy wrote:
> > Add TestEnv class, which will handle test environment in a new python
> > iotests running framework.
> >
> > Difference with current ./check interface:
> > - -v (verbose) option dropped, as it is unused
> >
> > - -xdiff option is dropped, until somebody complains that it is needed
> > - same for -n option
>
> One more thing dropped is looking for binaries in $build_iotests
> directory. Do someone use this feature? Or we can drop it, and
> reimplement when someone report the degradation?
I seem to have socket_scm_helper only there, but you use
self.build_iotests for that, so I suppose this is different from what
you mean?
My tools and system emulator binaries are in the standard location in
the build directory, not in the iotests build directory, so they don't
need it.
Kevin
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v6 08/11] iotests: add testenv.py
2021-01-14 11:14 ` Kevin Wolf
@ 2021-01-14 11:26 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-14 11:26 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-block, qemu-devel, mreitz, den, jsnow
14.01.2021 14:14, Kevin Wolf wrote:
> Am 14.01.2021 um 05:28 hat Vladimir Sementsov-Ogievskiy geschrieben:
>> 09.01.2021 15:26, Vladimir Sementsov-Ogievskiy wrote:
>>> Add TestEnv class, which will handle test environment in a new python
>>> iotests running framework.
>>>
>>> Difference with current ./check interface:
>>> - -v (verbose) option dropped, as it is unused
>>>
>>> - -xdiff option is dropped, until somebody complains that it is needed
>>> - same for -n option
>>
>> One more thing dropped is looking for binaries in $build_iotests
>> directory. Do someone use this feature? Or we can drop it, and
>> reimplement when someone report the degradation?
>
> I seem to have socket_scm_helper only there, but you use
> self.build_iotests for that, so I suppose this is different from what
> you mean?
Yes socket_scm_helper is searched in build_iotests both pre- and after- my patches.
But for other tools (qemu-img, qemu-io, qemu-nbd, qemu, qemu-storage-daemon) old check script first search[1] in build_iotests, and then in standard location.. In this commit I don't do the step [1].
>
> My tools and system emulator binaries are in the standard location in
> the build directory, not in the iotests build directory, so they don't
> need it.
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v6 08/11] iotests: add testenv.py
2021-01-09 12:26 ` [PATCH v6 08/11] iotests: add testenv.py Vladimir Sementsov-Ogievskiy
2021-01-12 17:36 ` Kevin Wolf
2021-01-14 4:28 ` Vladimir Sementsov-Ogievskiy
@ 2021-01-15 11:18 ` Kevin Wolf
2021-01-15 12:19 ` Vladimir Sementsov-Ogievskiy
2 siblings, 1 reply; 43+ messages in thread
From: Kevin Wolf @ 2021-01-15 11:18 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy; +Cc: qemu-block, qemu-devel, mreitz, den, jsnow
Am 09.01.2021 um 13:26 hat Vladimir Sementsov-Ogievskiy geschrieben:
> Add TestEnv class, which will handle test environment in a new python
> iotests running framework.
>
> Difference with current ./check interface:
> - -v (verbose) option dropped, as it is unused
>
> - -xdiff option is dropped, until somebody complains that it is needed
> - same for -n option
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> tests/qemu-iotests/testenv.py | 328 ++++++++++++++++++++++++++++++++++
> 1 file changed, 328 insertions(+)
> create mode 100755 tests/qemu-iotests/testenv.py
>
> diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py
> new file mode 100755
> index 0000000000..ecaf76fb85
> --- /dev/null
> +++ b/tests/qemu-iotests/testenv.py
> @@ -0,0 +1,328 @@
> +#!/usr/bin/env python3
> +#
> +# Parse command line options to manage test environment variables.
> +#
> +# Copyright (c) 2020 Virtuozzo International GmbH
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; either version 2 of the License, or
> +# (at your option) any later version.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program. If not, see <http://www.gnu.org/licenses/>.
> +#
> +
> +import os
> +import sys
> +import tempfile
> +from pathlib import Path
> +import shutil
> +import collections
> +import subprocess
> +import argparse
> +from typing import List, Dict
> +
> +
> +def get_default_machine(qemu_prog: str) -> str:
> + outp = subprocess.run([qemu_prog, '-machine', 'help'], check=True,
> + text=True, stdout=subprocess.PIPE).stdout
> +
> + machines = outp.split('\n')
> + default_machine = next(m for m in machines if m.endswith(' (default)'))
> + default_machine = default_machine.split(' ', 1)[0]
> +
> + alias_suf = ' (alias of {})'.format(default_machine)
> + alias = next((m for m in machines if m.endswith(alias_suf)), None)
> + if alias is not None:
> + default_machine = alias.split(' ', 1)[0]
> +
> + return default_machine
> +
> +
> +class TestEnv:
> + """
> + Manage system environment for running tests
> +
> + The following variables are supported/provided. They are represented by
> + lower-cased TestEnv attributes.
> + """
> + env_variables = ['PYTHONPATH', 'TEST_DIR', 'SOCK_DIR', 'SAMPLE_IMG_DIR',
> + 'OUTPUT_DIR', 'PYTHON', 'QEMU_PROG', 'QEMU_IMG_PROG',
> + 'QEMU_IO_PROG', 'QEMU_NBD_PROG', 'QSD_PROG',
> + 'SOCKET_SCM_HELPER', 'QEMU_OPTIONS', 'QEMU_IMG_OPTIONS',
> + 'QEMU_IO_OPTIONS', 'QEMU_NBD_OPTIONS', 'IMGOPTS',
> + 'IMGFMT', 'IMGPROTO', 'AIOMODE', 'CACHEMODE',
> + 'VALGRIND_QEMU', 'CACHEMODE_IS_DEFAULT', 'IMGFMT_GENERIC',
> + 'IMGOPTSSYNTAX', 'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE']
> +
> + def get_env(self) -> Dict[str, str]:
> + env = {}
> + for v in self.env_variables:
> + val = getattr(self, v.lower(), None)
> + if val is not None:
> + env[v] = val
> +
> + return env
> +
> + _argparser = None
> + @classmethod
> + def get_argparser(cls) -> argparse.ArgumentParser:
> + if cls._argparser is not None:
> + return cls._argparser
> +
> + p = argparse.ArgumentParser(description="= test environment options =",
> + add_help=False, usage=argparse.SUPPRESS)
> +
> + p.add_argument('-d', dest='debug', action='store_true', help='debug')
> + p.add_argument('-misalign', action='store_true',
> + help='misalign memory allocations')
> +
> + p.set_defaults(imgfmt='raw', imgproto='file')
> +
> + format_list = ['raw', 'bochs', 'cloop', 'parallels', 'qcow', 'qcow2',
> + 'qed', 'vdi', 'vpc', 'vhdx', 'vmdk', 'luks', 'dmg']
> + g = p.add_argument_group(
> + 'image format options',
> + 'The following options sets IMGFMT environment variable. '
s/sets/set the/
> + 'At most one chose is allowed, default is "raw"')
s/chose/choice/
> + g = g.add_mutually_exclusive_group()
> + for fmt in format_list:
> + g.add_argument('-' + fmt, dest='imgfmt', action='store_const',
> + const=fmt)
> +
> + protocol_list = ['file', 'rbd', 'sheepdoc', 'nbd', 'ssh', 'nfs',
> + 'fuse']
> + g = p.add_argument_group(
> + 'image protocol options',
> + 'The following options sets IMGPROTO environment variably. '
> + 'At most one chose is allowed, default is "file"')
Same as above, but also s/variably/variable/.
Do we consider these environment variables user interfaces? So far I
thought of them as implementation details, but I guess if we want to
allow users to execute test scripts manually, they are some kind of user
interface.
However, shouldn't the variables themselves then be documented
somewhere? As it is, this feels like documenting thing X to be the same
as thing Y, without ever saying what Y is.
That said...
> + g = g.add_mutually_exclusive_group()
> + for prt in protocol_list:
> + g.add_argument('-' + prt, dest='imgproto', action='store_const',
> + const=prt)
...maybe we should just have help=f"test {fmt/prt}" here to match the
old help text. Then this documents the option and the help above
actually documents the environment variable.
> +
> + g = p.add_mutually_exclusive_group()
> + # We don't set default for cachemode, as we need to distinguish dafult
> + # from user input later.
> + g.add_argument('-nocache', dest='cachemode', action='store_const',
> + const='none', help='set cache mode "none" (O_DIRECT), '
> + 'sets CACHEMODE environment variable')
> + g.add_argument('-c', dest='cachemode',
> + help='sets CACHEMODE environment variable')
> +
> + p.add_argument('-i', dest='aiomode', default='threads',
> + help='sets AIOMODE environment variable')
> +
> + g = p.add_argument_group('bash tests options',
> + 'The following options are ignored by '
> + 'python tests. TODO: support them in '
> + 'iotests.py')
Let's not print TODO comments to the user, but just make it a comment in
the code. That makes it stand out better with syntax highlighting
anyway.
> + g.add_argument('-o', dest='imgopts',
> + help='options to pass to qemu-img create/convert, sets '
> + 'IMGOPTS environment variable')
> + p.add_argument('-valgrind', dest='VALGRIND_QEMU', action='store_const',
> + const='y', help='use valgrind, sets VALGRIND_QEMU '
> + 'environment variable')
> +
> + cls._argparser = p
> + return p
> +
> + def init_handle_argv(self, argv: List[str]) -> None:
> +
> + # Hints for mypy, about arguments which will be set by argparse
I don't understand what this comment wants to tell me.
> + args, self.remaining_argv = self.get_argparser().parse_known_args(argv)
> + self.imgfmt = args.imgfmt
> + self.imgproto = args.imgproto
> + self.aiomode = args.aiomode
> + self.imgopts = args.imgopts
> + self.misalign = args.misalign
> + self.debug = args.debug
> +
> + if args.cachemode is None:
> + self.cachemode_is_default = 'true'
> + self.cachemode = 'writeback'
> + else:
> + self.cachemode_is_default = 'false'
> + self.cachemode = args.cachemode
> +
> + def init_directories(self):
> + """Init directory variables:
> + PYTHONPATH
> + TEST_DIR
> + SOCK_DIR
> + SAMPLE_IMG_DIR
> + OUTPUT_DIR
> + """
> + self.pythonpath = os.getenv('PYTHONPATH')
> + if self.pythonpath:
> + self.pythonpath = self.source_iotests + os.pathsep + \
> + self.pythonpath
> + else:
> + self.pythonpath = self.source_iotests
> +
> + self.test_dir = os.getenv('TEST_DIR',
> + os.path.join(os.getcwd(), 'scratch'))
> + Path(self.test_dir).mkdir(parents=True, exist_ok=True)
> +
> + self.sock_dir = os.getenv('SOCK_DIR')
> + self.tmp_sock_dir = False
> + if self.sock_dir:
> + Path(self.test_dir).mkdir(parents=True, exist_ok=True)
> + else:
> + self.sock_dir = tempfile.mkdtemp()
> + self.tmp_sock_dir = True
> +
> + self.sample_img_dir = os.getenv('SAMPLE_IMG_DIR',
> + os.path.join(self.source_iotests,
> + 'sample_images'))
> +
> + self.output_dir = os.getcwd() # OUTPUT_DIR
> +
> + def init_binaries(self):
> + """Init binary path variables:
> + PYTHON (for bash tests)
> + QEMU_PROG, QEMU_IMG_PROG, QEMU_IO_PROG, QEMU_NBD_PROG, QSD_PROG
> + SOCKET_SCM_HELPER
> + """
> + self.python = '/usr/bin/python3 -B'
This doesn't look right, we need to respect the Python binary set in
configure (which I think we get from common.env)
> + def root(*names):
> + return os.path.join(self.build_root, *names)
> +
> + arch = os.uname().machine
> + if 'ppc64' in arch:
> + arch = 'ppc64'
> +
> + self.qemu_prog = os.getenv('QEMU_PROG', root(f'qemu-system-{arch}'))
> + self.qemu_img_prog = os.getenv('QEMU_IMG_PROG', root('qemu-img'))
> + self.qemu_io_prog = os.getenv('QEMU_IO_PROG', root('qemu-io'))
> + self.qemu_nbd_prog = os.getenv('QEMU_NBD_PROG', root('qemu-nbd'))
> + self.qsd_prog = os.getenv('QSD_PROG', root('storage-daemon',
> + 'qemu-storage-daemon'))
> +
> + for b in [self.qemu_img_prog, self.qemu_io_prog, self.qemu_nbd_prog,
> + self.qemu_prog, self.qsd_prog]:
> + if not os.path.exists(b):
> + exit('Not such file: ' + b)
> + if not os.access(b, os.X_OK):
> + exit('Not executable: ' + b)
> +
> + helper_path = os.path.join(self.build_iotests, 'socket_scm_helper')
> + if os.access(helper_path, os.X_OK):
> + self.socket_scm_helper = helper_path # SOCKET_SCM_HELPER
> +
> + def __init__(self, argv: List[str]) -> None:
> + """Parse args and environment"""
> +
> + # Initialize generic paths: build_root, build_iotests, source_iotests,
> + # which are needed to initialize some environment variables. They are
> + # used by init_*() functions as well.
> +
> +
> + if os.path.islink(sys.argv[0]):
> + # called from the build tree
> + self.source_iotests = os.path.dirname(os.readlink(sys.argv[0]))
> + self.build_iotests = os.path.dirname(os.path.abspath(sys.argv[0]))
> + else:
> + # called from the source tree
> + self.source_iotests = os.getcwd()
> + self.build_iotests = self.source_iotests
> +
> + self.build_root = os.path.join(self.build_iotests, '..', '..')
> +
> + self.init_handle_argv(argv)
> + self.init_directories()
> + self.init_binaries()
> +
> + # QEMU_OPTIONS
> + self.qemu_options = '-nodefaults -display none -accel qtest'
> + machine_map = (
> + (('arm', 'aarch64'), 'virt'),
How does this work? Won't we check for "qemu-system-('arm', 'aarch64')"
below, which we'll never find?
> + ('avr', 'mega2560'),
> + ('rx', 'gdbsim-r5f562n8'),
> + ('tricore', 'tricore_testboard')
> + )
> + for suffix, machine in machine_map:
> + if self.qemu_prog.endswith(f'qemu-system-{suffix}'):
> + self.qemu_options += f' -machine {machine}'
> +
> + # QEMU_DEFAULT_MACHINE
> + self.qemu_default_machine = get_default_machine(self.qemu_prog)
> +
> + self.qemu_img_options = os.getenv('QEMU_IMG_OPTIONS')
> + self.qemu_nbd_options = os.getenv('QEMU_NBD_OPTIONS')
> +
> + is_generic = self.imgfmt not in ['bochs', 'cloop', 'dmg']
> + self.imgfmt_generic = 'true' if is_generic else 'false'
> +
> + self.qemu_io_options = f'--cache {self.cachemode} --aio {self.aiomode}'
> + if self.misalign:
> + self.qemu_io_options += ' --misalign'
> +
> + self.qemu_io_options_no_fmt = self.qemu_io_options
> +
> + if self.imgfmt == 'luks':
> + self.imgoptssyntax = 'true'
> + self.imgkeysecret = '123456'
> + if not self.imgopts:
> + self.imgopts = 'iter-time=10'
> + elif 'iter-time=' not in self.imgopts:
> + self.imgopts += ',iter-time=10'
> + else:
> + self.imgoptssyntax = 'false'
> + self.qemu_io_options += ' -f ' + self.imgfmt
> +
> + if self.imgfmt == 'vmkd':
> + if not self.imgopts:
> + self.imgopts = 'zeroed_grain=on'
> + elif 'zeroed_grain=' not in self.imgopts:
> + self.imgopts += ',zeroed_grain=on'
> +
> + def close(self) -> None:
> + if self.tmp_sock_dir:
> + shutil.rmtree(self.sock_dir)
> +
> + def __enter__(self) -> 'TestEnv':
> + return self
> +
> + def __exit__(self, *args) -> None:
> + self.close()
> +
> + def print_env(self) -> None:
> + template = """\
> +QEMU -- "{QEMU_PROG}" {QEMU_OPTIONS}
> +QEMU_IMG -- "{QEMU_IMG_PROG}" {QEMU_IMG_OPTIONS}
> +QEMU_IO -- "{QEMU_IO_PROG}" {QEMU_IO_OPTIONS}
> +QEMU_NBD -- "{QEMU_NBD_PROG}" {QEMU_NBD_OPTIONS}
> +IMGFMT -- {IMGFMT}{imgopts}
> +IMGPROTO -- {IMGPROTO}
> +PLATFORM -- {platform}
> +TEST_DIR -- {TEST_DIR}
> +SOCK_DIR -- {SOCK_DIR}
> +SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER}"""
> +
> + args = collections.defaultdict(str, self.get_env())
> +
> + if 'IMGOPTS' in args:
> + args['imgopts'] = f" ({args['IMGOPTS']})"
> +
> + u = os.uname()
> + args['platform'] = f'{u.sysname}/{u.machine} {u.nodename} {u.release}'
> +
> + print(template.format_map(args))
> +
> +
> +if __name__ == '__main__':
> + if len(sys.argv) == 2 and sys.argv[1] in ['-h', '--help']:
> + TestEnv.get_argparser().print_help()
> + exit()
> +
> + with TestEnv(sys.argv) as te:
> + te.print_env()
> + print('\nUnhandled options: ', te.remaining_argv)
Kevin
^ permalink raw reply [flat|nested] 43+ messages in thread* Re: [PATCH v6 08/11] iotests: add testenv.py
2021-01-15 11:18 ` Kevin Wolf
@ 2021-01-15 12:19 ` Vladimir Sementsov-Ogievskiy
2021-01-15 12:45 ` Kevin Wolf
0 siblings, 1 reply; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-15 12:19 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-block, qemu-devel, mreitz, den, jsnow
15.01.2021 14:18, Kevin Wolf wrote:
> Am 09.01.2021 um 13:26 hat Vladimir Sementsov-Ogievskiy geschrieben:
>> Add TestEnv class, which will handle test environment in a new python
>> iotests running framework.
>>
>> Difference with current ./check interface:
>> - -v (verbose) option dropped, as it is unused
>>
>> - -xdiff option is dropped, until somebody complains that it is needed
>> - same for -n option
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>> tests/qemu-iotests/testenv.py | 328 ++++++++++++++++++++++++++++++++++
>> 1 file changed, 328 insertions(+)
>> create mode 100755 tests/qemu-iotests/testenv.py
>>
>> diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py
>> new file mode 100755
>> index 0000000000..ecaf76fb85
>> --- /dev/null
>> +++ b/tests/qemu-iotests/testenv.py
>> @@ -0,0 +1,328 @@
>> +#!/usr/bin/env python3
>> +#
>> +# Parse command line options to manage test environment variables.
>> +#
>> +# Copyright (c) 2020 Virtuozzo International GmbH
>> +#
>> +# This program is free software; you can redistribute it and/or modify
>> +# it under the terms of the GNU General Public License as published by
>> +# the Free Software Foundation; either version 2 of the License, or
>> +# (at your option) any later version.
>> +#
>> +# This program is distributed in the hope that it will be useful,
>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> +# GNU General Public License for more details.
>> +#
>> +# You should have received a copy of the GNU General Public License
>> +# along with this program. If not, see <http://www.gnu.org/licenses/>.
>> +#
>> +
>> +import os
>> +import sys
>> +import tempfile
>> +from pathlib import Path
>> +import shutil
>> +import collections
>> +import subprocess
>> +import argparse
>> +from typing import List, Dict
>> +
>> +
>> +def get_default_machine(qemu_prog: str) -> str:
>> + outp = subprocess.run([qemu_prog, '-machine', 'help'], check=True,
>> + text=True, stdout=subprocess.PIPE).stdout
>> +
>> + machines = outp.split('\n')
>> + default_machine = next(m for m in machines if m.endswith(' (default)'))
>> + default_machine = default_machine.split(' ', 1)[0]
>> +
>> + alias_suf = ' (alias of {})'.format(default_machine)
>> + alias = next((m for m in machines if m.endswith(alias_suf)), None)
>> + if alias is not None:
>> + default_machine = alias.split(' ', 1)[0]
>> +
>> + return default_machine
>> +
>> +
>> +class TestEnv:
>> + """
>> + Manage system environment for running tests
>> +
>> + The following variables are supported/provided. They are represented by
>> + lower-cased TestEnv attributes.
>> + """
>> + env_variables = ['PYTHONPATH', 'TEST_DIR', 'SOCK_DIR', 'SAMPLE_IMG_DIR',
>> + 'OUTPUT_DIR', 'PYTHON', 'QEMU_PROG', 'QEMU_IMG_PROG',
>> + 'QEMU_IO_PROG', 'QEMU_NBD_PROG', 'QSD_PROG',
>> + 'SOCKET_SCM_HELPER', 'QEMU_OPTIONS', 'QEMU_IMG_OPTIONS',
>> + 'QEMU_IO_OPTIONS', 'QEMU_NBD_OPTIONS', 'IMGOPTS',
>> + 'IMGFMT', 'IMGPROTO', 'AIOMODE', 'CACHEMODE',
>> + 'VALGRIND_QEMU', 'CACHEMODE_IS_DEFAULT', 'IMGFMT_GENERIC',
>> + 'IMGOPTSSYNTAX', 'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE']
>> +
>> + def get_env(self) -> Dict[str, str]:
>> + env = {}
>> + for v in self.env_variables:
>> + val = getattr(self, v.lower(), None)
>> + if val is not None:
>> + env[v] = val
>> +
>> + return env
>> +
>> + _argparser = None
>> + @classmethod
>> + def get_argparser(cls) -> argparse.ArgumentParser:
>> + if cls._argparser is not None:
>> + return cls._argparser
>> +
>> + p = argparse.ArgumentParser(description="= test environment options =",
>> + add_help=False, usage=argparse.SUPPRESS)
>> +
>> + p.add_argument('-d', dest='debug', action='store_true', help='debug')
>> + p.add_argument('-misalign', action='store_true',
>> + help='misalign memory allocations')
>> +
>> + p.set_defaults(imgfmt='raw', imgproto='file')
>> +
>> + format_list = ['raw', 'bochs', 'cloop', 'parallels', 'qcow', 'qcow2',
>> + 'qed', 'vdi', 'vpc', 'vhdx', 'vmdk', 'luks', 'dmg']
>> + g = p.add_argument_group(
>> + 'image format options',
>> + 'The following options sets IMGFMT environment variable. '
>
> s/sets/set the/
>
>> + 'At most one chose is allowed, default is "raw"')
>
> s/chose/choice/
>
>> + g = g.add_mutually_exclusive_group()
>> + for fmt in format_list:
>> + g.add_argument('-' + fmt, dest='imgfmt', action='store_const',
>> + const=fmt)
>> +
>> + protocol_list = ['file', 'rbd', 'sheepdoc', 'nbd', 'ssh', 'nfs',
>> + 'fuse']
>> + g = p.add_argument_group(
>> + 'image protocol options',
>> + 'The following options sets IMGPROTO environment variably. '
>> + 'At most one chose is allowed, default is "file"')
>
> Same as above, but also s/variably/variable/.
>
> Do we consider these environment variables user interfaces? So far I
> thought of them as implementation details, but I guess if we want to
> allow users to execute test scripts manually, they are some kind of user
> interface.
>
> However, shouldn't the variables themselves then be documented
> somewhere? As it is, this feels like documenting thing X to be the same
> as thing Y, without ever saying what Y is.
Yes, I do think that we lack some specification on the test interface, including
environment variables.. And then, probably some unification of the interface, to
be more clear and straightforward. But don't want to improve/refactor these things
in these series.
>
> That said...
>
>> + g = g.add_mutually_exclusive_group()
>> + for prt in protocol_list:
>> + g.add_argument('-' + prt, dest='imgproto', action='store_const',
>> + const=prt)
>
> ...maybe we should just have help=f"test {fmt/prt}" here to match the
> old help text. Then this documents the option and the help above
> actually documents the environment variable.
OK
>
>> +
>> + g = p.add_mutually_exclusive_group()
>> + # We don't set default for cachemode, as we need to distinguish dafult
>> + # from user input later.
>> + g.add_argument('-nocache', dest='cachemode', action='store_const',
>> + const='none', help='set cache mode "none" (O_DIRECT), '
>> + 'sets CACHEMODE environment variable')
>> + g.add_argument('-c', dest='cachemode',
>> + help='sets CACHEMODE environment variable')
>> +
>> + p.add_argument('-i', dest='aiomode', default='threads',
>> + help='sets AIOMODE environment variable')
>> +
>> + g = p.add_argument_group('bash tests options',
>> + 'The following options are ignored by '
>> + 'python tests. TODO: support them in '
>> + 'iotests.py')
>
> Let's not print TODO comments to the user, but just make it a comment in
> the code. That makes it stand out better with syntax highlighting
> anyway.
>
>> + g.add_argument('-o', dest='imgopts',
>> + help='options to pass to qemu-img create/convert, sets '
>> + 'IMGOPTS environment variable')
>> + p.add_argument('-valgrind', dest='VALGRIND_QEMU', action='store_const',
>> + const='y', help='use valgrind, sets VALGRIND_QEMU '
>> + 'environment variable')
>> +
>> + cls._argparser = p
>> + return p
>> +
>> + def init_handle_argv(self, argv: List[str]) -> None:
>> +
>> + # Hints for mypy, about arguments which will be set by argparse
>
> I don't understand what this comment wants to tell me.
I mean, I could automatically set self. attributes from args, but do it explictly to sutisfy mypy. But..
Actually, I'll move argv interface out of the file for v7.
I recently learned from another project, that merging cmdline interface (like it done her) into logic(model) class is a bad idea. (I just had to refactor it and split ligic from the interface, to reuse logic classes with another interface) [1]
Also, trying to fix pylint and mypy complains, I had to inherit classes from AbstractContextManager (for mypy).. And then, how to fix "testenv.py:1:0: R0801: Similar lines in 2 files" ? You wandered, should we disable it.. I think not. It's a good warning, showing bad design. True way of fixing is creating a base class with common functionality.. But than.. Multiple inheritance to sutisfy linters? Haha. This brings us to [1] again.
So, I decided to keep the classes with normal python interface (__init__ with normal arguments), and move the whole cmdline interface into check script itself. It looks a lot better.. Also, TestEnv is complicated enough even without argument parsing.
>
>> + args, self.remaining_argv = self.get_argparser().parse_known_args(argv)
>> + self.imgfmt = args.imgfmt
>> + self.imgproto = args.imgproto
>> + self.aiomode = args.aiomode
>> + self.imgopts = args.imgopts
>> + self.misalign = args.misalign
>> + self.debug = args.debug
>> +
>> + if args.cachemode is None:
>> + self.cachemode_is_default = 'true'
>> + self.cachemode = 'writeback'
>> + else:
>> + self.cachemode_is_default = 'false'
>> + self.cachemode = args.cachemode
>> +
>> + def init_directories(self):
>> + """Init directory variables:
>> + PYTHONPATH
>> + TEST_DIR
>> + SOCK_DIR
>> + SAMPLE_IMG_DIR
>> + OUTPUT_DIR
>> + """
>> + self.pythonpath = os.getenv('PYTHONPATH')
>> + if self.pythonpath:
>> + self.pythonpath = self.source_iotests + os.pathsep + \
>> + self.pythonpath
>> + else:
>> + self.pythonpath = self.source_iotests
>> +
>> + self.test_dir = os.getenv('TEST_DIR',
>> + os.path.join(os.getcwd(), 'scratch'))
>> + Path(self.test_dir).mkdir(parents=True, exist_ok=True)
>> +
>> + self.sock_dir = os.getenv('SOCK_DIR')
>> + self.tmp_sock_dir = False
>> + if self.sock_dir:
>> + Path(self.test_dir).mkdir(parents=True, exist_ok=True)
>> + else:
>> + self.sock_dir = tempfile.mkdtemp()
>> + self.tmp_sock_dir = True
>> +
>> + self.sample_img_dir = os.getenv('SAMPLE_IMG_DIR',
>> + os.path.join(self.source_iotests,
>> + 'sample_images'))
>> +
>> + self.output_dir = os.getcwd() # OUTPUT_DIR
>> +
>> + def init_binaries(self):
>> + """Init binary path variables:
>> + PYTHON (for bash tests)
>> + QEMU_PROG, QEMU_IMG_PROG, QEMU_IO_PROG, QEMU_NBD_PROG, QSD_PROG
>> + SOCKET_SCM_HELPER
>> + """
>> + self.python = '/usr/bin/python3 -B'
>
> This doesn't look right, we need to respect the Python binary set in
> configure (which I think we get from common.env)
Oh, I missed the change. Then I should just drop this self.python.
>
>> + def root(*names):
>> + return os.path.join(self.build_root, *names)
>> +
>> + arch = os.uname().machine
>> + if 'ppc64' in arch:
>> + arch = 'ppc64'
>> +
>> + self.qemu_prog = os.getenv('QEMU_PROG', root(f'qemu-system-{arch}'))
>> + self.qemu_img_prog = os.getenv('QEMU_IMG_PROG', root('qemu-img'))
>> + self.qemu_io_prog = os.getenv('QEMU_IO_PROG', root('qemu-io'))
>> + self.qemu_nbd_prog = os.getenv('QEMU_NBD_PROG', root('qemu-nbd'))
>> + self.qsd_prog = os.getenv('QSD_PROG', root('storage-daemon',
>> + 'qemu-storage-daemon'))
>> +
>> + for b in [self.qemu_img_prog, self.qemu_io_prog, self.qemu_nbd_prog,
>> + self.qemu_prog, self.qsd_prog]:
>> + if not os.path.exists(b):
>> + exit('Not such file: ' + b)
>> + if not os.access(b, os.X_OK):
>> + exit('Not executable: ' + b)
>> +
>> + helper_path = os.path.join(self.build_iotests, 'socket_scm_helper')
>> + if os.access(helper_path, os.X_OK):
>> + self.socket_scm_helper = helper_path # SOCKET_SCM_HELPER
>> +
>> + def __init__(self, argv: List[str]) -> None:
>> + """Parse args and environment"""
>> +
>> + # Initialize generic paths: build_root, build_iotests, source_iotests,
>> + # which are needed to initialize some environment variables. They are
>> + # used by init_*() functions as well.
>> +
>> +
>> + if os.path.islink(sys.argv[0]):
>> + # called from the build tree
>> + self.source_iotests = os.path.dirname(os.readlink(sys.argv[0]))
>> + self.build_iotests = os.path.dirname(os.path.abspath(sys.argv[0]))
>> + else:
>> + # called from the source tree
>> + self.source_iotests = os.getcwd()
>> + self.build_iotests = self.source_iotests
>> +
>> + self.build_root = os.path.join(self.build_iotests, '..', '..')
>> +
>> + self.init_handle_argv(argv)
>> + self.init_directories()
>> + self.init_binaries()
>> +
>> + # QEMU_OPTIONS
>> + self.qemu_options = '-nodefaults -display none -accel qtest'
>> + machine_map = (
>> + (('arm', 'aarch64'), 'virt'),
>
> How does this work? Won't we check for "qemu-system-('arm', 'aarch64')"
> below, which we'll never find?
Hmm. I just took existing logic from check:
[..]
case "$QEMU_PROG" in
*qemu-system-arm|*qemu-system-aarch64)
export QEMU_OPTIONS="$QEMU_OPTIONS -machine virt"
;;
[..]
>
>> + ('avr', 'mega2560'),
>> + ('rx', 'gdbsim-r5f562n8'),
>> + ('tricore', 'tricore_testboard')
>> + )
>> + for suffix, machine in machine_map:
>> + if self.qemu_prog.endswith(f'qemu-system-{suffix}'):
>> + self.qemu_options += f' -machine {machine}'
>> +
>> + # QEMU_DEFAULT_MACHINE
>> + self.qemu_default_machine = get_default_machine(self.qemu_prog)
>> +
>> + self.qemu_img_options = os.getenv('QEMU_IMG_OPTIONS')
>> + self.qemu_nbd_options = os.getenv('QEMU_NBD_OPTIONS')
>> +
>> + is_generic = self.imgfmt not in ['bochs', 'cloop', 'dmg']
>> + self.imgfmt_generic = 'true' if is_generic else 'false'
>> +
>> + self.qemu_io_options = f'--cache {self.cachemode} --aio {self.aiomode}'
>> + if self.misalign:
>> + self.qemu_io_options += ' --misalign'
>> +
>> + self.qemu_io_options_no_fmt = self.qemu_io_options
>> +
>> + if self.imgfmt == 'luks':
>> + self.imgoptssyntax = 'true'
>> + self.imgkeysecret = '123456'
>> + if not self.imgopts:
>> + self.imgopts = 'iter-time=10'
>> + elif 'iter-time=' not in self.imgopts:
>> + self.imgopts += ',iter-time=10'
>> + else:
>> + self.imgoptssyntax = 'false'
>> + self.qemu_io_options += ' -f ' + self.imgfmt
>> +
>> + if self.imgfmt == 'vmkd':
>> + if not self.imgopts:
>> + self.imgopts = 'zeroed_grain=on'
>> + elif 'zeroed_grain=' not in self.imgopts:
>> + self.imgopts += ',zeroed_grain=on'
>> +
>> + def close(self) -> None:
>> + if self.tmp_sock_dir:
>> + shutil.rmtree(self.sock_dir)
>> +
>> + def __enter__(self) -> 'TestEnv':
>> + return self
>> +
>> + def __exit__(self, *args) -> None:
>> + self.close()
>> +
>> + def print_env(self) -> None:
>> + template = """\
>> +QEMU -- "{QEMU_PROG}" {QEMU_OPTIONS}
>> +QEMU_IMG -- "{QEMU_IMG_PROG}" {QEMU_IMG_OPTIONS}
>> +QEMU_IO -- "{QEMU_IO_PROG}" {QEMU_IO_OPTIONS}
>> +QEMU_NBD -- "{QEMU_NBD_PROG}" {QEMU_NBD_OPTIONS}
>> +IMGFMT -- {IMGFMT}{imgopts}
>> +IMGPROTO -- {IMGPROTO}
>> +PLATFORM -- {platform}
>> +TEST_DIR -- {TEST_DIR}
>> +SOCK_DIR -- {SOCK_DIR}
>> +SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER}"""
>> +
>> + args = collections.defaultdict(str, self.get_env())
>> +
>> + if 'IMGOPTS' in args:
>> + args['imgopts'] = f" ({args['IMGOPTS']})"
>> +
>> + u = os.uname()
>> + args['platform'] = f'{u.sysname}/{u.machine} {u.nodename} {u.release}'
>> +
>> + print(template.format_map(args))
>> +
>> +
>> +if __name__ == '__main__':
>> + if len(sys.argv) == 2 and sys.argv[1] in ['-h', '--help']:
>> + TestEnv.get_argparser().print_help()
>> + exit()
>> +
>> + with TestEnv(sys.argv) as te:
>> + te.print_env()
>> + print('\nUnhandled options: ', te.remaining_argv)
>
> Kevin
>
Thanks for reviewing! I hope to post v7 today, with cmdline interface in 'check' script.
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 43+ messages in thread* Re: [PATCH v6 08/11] iotests: add testenv.py
2021-01-15 12:19 ` Vladimir Sementsov-Ogievskiy
@ 2021-01-15 12:45 ` Kevin Wolf
2021-01-15 13:10 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 43+ messages in thread
From: Kevin Wolf @ 2021-01-15 12:45 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy; +Cc: qemu-block, qemu-devel, mreitz, den, jsnow
Am 15.01.2021 um 13:19 hat Vladimir Sementsov-Ogievskiy geschrieben:
> 15.01.2021 14:18, Kevin Wolf wrote:
> > Am 09.01.2021 um 13:26 hat Vladimir Sementsov-Ogievskiy geschrieben:
> > > Add TestEnv class, which will handle test environment in a new python
> > > iotests running framework.
> > >
> > > Difference with current ./check interface:
> > > - -v (verbose) option dropped, as it is unused
> > >
> > > - -xdiff option is dropped, until somebody complains that it is needed
> > > - same for -n option
> > >
> > > Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> > > ---
> > > tests/qemu-iotests/testenv.py | 328 ++++++++++++++++++++++++++++++++++
> > > 1 file changed, 328 insertions(+)
> > > create mode 100755 tests/qemu-iotests/testenv.py
> > >
> > > diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py
> > > new file mode 100755
> > > index 0000000000..ecaf76fb85
> > > --- /dev/null
> > > +++ b/tests/qemu-iotests/testenv.py
> > > @@ -0,0 +1,328 @@
> > > +#!/usr/bin/env python3
> > > +#
> > > +# Parse command line options to manage test environment variables.
> > > +#
> > > +# Copyright (c) 2020 Virtuozzo International GmbH
> > > +#
> > > +# This program is free software; you can redistribute it and/or modify
> > > +# it under the terms of the GNU General Public License as published by
> > > +# the Free Software Foundation; either version 2 of the License, or
> > > +# (at your option) any later version.
> > > +#
> > > +# This program is distributed in the hope that it will be useful,
> > > +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > > +# GNU General Public License for more details.
> > > +#
> > > +# You should have received a copy of the GNU General Public License
> > > +# along with this program. If not, see <http://www.gnu.org/licenses/>.
> > > +#
> > > +
> > > +import os
> > > +import sys
> > > +import tempfile
> > > +from pathlib import Path
> > > +import shutil
> > > +import collections
> > > +import subprocess
> > > +import argparse
> > > +from typing import List, Dict
> > > +
> > > +
> > > +def get_default_machine(qemu_prog: str) -> str:
> > > + outp = subprocess.run([qemu_prog, '-machine', 'help'], check=True,
> > > + text=True, stdout=subprocess.PIPE).stdout
> > > +
> > > + machines = outp.split('\n')
> > > + default_machine = next(m for m in machines if m.endswith(' (default)'))
> > > + default_machine = default_machine.split(' ', 1)[0]
> > > +
> > > + alias_suf = ' (alias of {})'.format(default_machine)
> > > + alias = next((m for m in machines if m.endswith(alias_suf)), None)
> > > + if alias is not None:
> > > + default_machine = alias.split(' ', 1)[0]
> > > +
> > > + return default_machine
> > > +
> > > +
> > > +class TestEnv:
> > > + """
> > > + Manage system environment for running tests
> > > +
> > > + The following variables are supported/provided. They are represented by
> > > + lower-cased TestEnv attributes.
> > > + """
> > > + env_variables = ['PYTHONPATH', 'TEST_DIR', 'SOCK_DIR', 'SAMPLE_IMG_DIR',
> > > + 'OUTPUT_DIR', 'PYTHON', 'QEMU_PROG', 'QEMU_IMG_PROG',
> > > + 'QEMU_IO_PROG', 'QEMU_NBD_PROG', 'QSD_PROG',
> > > + 'SOCKET_SCM_HELPER', 'QEMU_OPTIONS', 'QEMU_IMG_OPTIONS',
> > > + 'QEMU_IO_OPTIONS', 'QEMU_NBD_OPTIONS', 'IMGOPTS',
> > > + 'IMGFMT', 'IMGPROTO', 'AIOMODE', 'CACHEMODE',
> > > + 'VALGRIND_QEMU', 'CACHEMODE_IS_DEFAULT', 'IMGFMT_GENERIC',
> > > + 'IMGOPTSSYNTAX', 'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE']
> > > +
> > > + def get_env(self) -> Dict[str, str]:
> > > + env = {}
> > > + for v in self.env_variables:
> > > + val = getattr(self, v.lower(), None)
> > > + if val is not None:
> > > + env[v] = val
> > > +
> > > + return env
> > > +
> > > + _argparser = None
> > > + @classmethod
> > > + def get_argparser(cls) -> argparse.ArgumentParser:
> > > + if cls._argparser is not None:
> > > + return cls._argparser
> > > +
> > > + p = argparse.ArgumentParser(description="= test environment options =",
> > > + add_help=False, usage=argparse.SUPPRESS)
> > > +
> > > + p.add_argument('-d', dest='debug', action='store_true', help='debug')
> > > + p.add_argument('-misalign', action='store_true',
> > > + help='misalign memory allocations')
> > > +
> > > + p.set_defaults(imgfmt='raw', imgproto='file')
> > > +
> > > + format_list = ['raw', 'bochs', 'cloop', 'parallels', 'qcow', 'qcow2',
> > > + 'qed', 'vdi', 'vpc', 'vhdx', 'vmdk', 'luks', 'dmg']
> > > + g = p.add_argument_group(
> > > + 'image format options',
> > > + 'The following options sets IMGFMT environment variable. '
> >
> > s/sets/set the/
> >
> > > + 'At most one chose is allowed, default is "raw"')
> >
> > s/chose/choice/
> >
> > > + g = g.add_mutually_exclusive_group()
> > > + for fmt in format_list:
> > > + g.add_argument('-' + fmt, dest='imgfmt', action='store_const',
> > > + const=fmt)
> > > +
> > > + protocol_list = ['file', 'rbd', 'sheepdoc', 'nbd', 'ssh', 'nfs',
> > > + 'fuse']
> > > + g = p.add_argument_group(
> > > + 'image protocol options',
> > > + 'The following options sets IMGPROTO environment variably. '
> > > + 'At most one chose is allowed, default is "file"')
> >
> > Same as above, but also s/variably/variable/.
> >
> > Do we consider these environment variables user interfaces? So far I
> > thought of them as implementation details, but I guess if we want to
> > allow users to execute test scripts manually, they are some kind of user
> > interface.
> >
> > However, shouldn't the variables themselves then be documented
> > somewhere? As it is, this feels like documenting thing X to be the same
> > as thing Y, without ever saying what Y is.
>
> Yes, I do think that we lack some specification on the test interface, including
> environment variables.. And then, probably some unification of the interface, to
> be more clear and straightforward. But don't want to improve/refactor these things
> in these series.
Yeah, that's fair. If the new help provides at least the same
information as the old one, that is good enough for me.
> >
> > That said...
> >
> > > + g = g.add_mutually_exclusive_group()
> > > + for prt in protocol_list:
> > > + g.add_argument('-' + prt, dest='imgproto', action='store_const',
> > > + const=prt)
> >
> > ...maybe we should just have help=f"test {fmt/prt}" here to match the
> > old help text. Then this documents the option and the help above
> > actually documents the environment variable.
>
> OK
>
> >
> > > +
> > > + g = p.add_mutually_exclusive_group()
> > > + # We don't set default for cachemode, as we need to distinguish dafult
> > > + # from user input later.
> > > + g.add_argument('-nocache', dest='cachemode', action='store_const',
> > > + const='none', help='set cache mode "none" (O_DIRECT), '
> > > + 'sets CACHEMODE environment variable')
> > > + g.add_argument('-c', dest='cachemode',
> > > + help='sets CACHEMODE environment variable')
> > > +
> > > + p.add_argument('-i', dest='aiomode', default='threads',
> > > + help='sets AIOMODE environment variable')
> > > +
> > > + g = p.add_argument_group('bash tests options',
> > > + 'The following options are ignored by '
> > > + 'python tests. TODO: support them in '
> > > + 'iotests.py')
> >
> > Let's not print TODO comments to the user, but just make it a comment in
> > the code. That makes it stand out better with syntax highlighting
> > anyway.
> >
> > > + g.add_argument('-o', dest='imgopts',
> > > + help='options to pass to qemu-img create/convert, sets '
> > > + 'IMGOPTS environment variable')
> > > + p.add_argument('-valgrind', dest='VALGRIND_QEMU', action='store_const',
> > > + const='y', help='use valgrind, sets VALGRIND_QEMU '
> > > + 'environment variable')
> > > +
> > > + cls._argparser = p
> > > + return p
> > > +
> > > + def init_handle_argv(self, argv: List[str]) -> None:
> > > +
> > > + # Hints for mypy, about arguments which will be set by argparse
> >
> > I don't understand what this comment wants to tell me.
>
> I mean, I could automatically set self. attributes from args, but do
> it explictly to sutisfy mypy. But..
>
> Actually, I'll move argv interface out of the file for v7.
>
> I recently learned from another project, that merging cmdline
> interface (like it done her) into logic(model) class is a bad idea. (I
> just had to refactor it and split ligic from the interface, to reuse
> logic classes with another interface) [1]
>
> Also, trying to fix pylint and mypy complains, I had to inherit
> classes from AbstractContextManager (for mypy).. And then, how to fix
> "testenv.py:1:0: R0801: Similar lines in 2 files" ? You wandered,
> should we disable it.. I think not. It's a good warning, showing bad
> design. True way of fixing is creating a base class with common
> functionality.. But than.. Multiple inheritance to sutisfy linters?
> Haha. This brings us to [1] again.
>
> So, I decided to keep the classes with normal python interface
> (__init__ with normal arguments), and move the whole cmdline interface
> into check script itself. It looks a lot better.. Also, TestEnv is
> complicated enough even without argument parsing.
Ok, then I'll wait for your new version to see what it looks like.
> >
> > > + args, self.remaining_argv = self.get_argparser().parse_known_args(argv)
> > > + self.imgfmt = args.imgfmt
> > > + self.imgproto = args.imgproto
> > > + self.aiomode = args.aiomode
> > > + self.imgopts = args.imgopts
> > > + self.misalign = args.misalign
> > > + self.debug = args.debug
> > > +
> > > + if args.cachemode is None:
> > > + self.cachemode_is_default = 'true'
> > > + self.cachemode = 'writeback'
> > > + else:
> > > + self.cachemode_is_default = 'false'
> > > + self.cachemode = args.cachemode
> > > +
> > > + def init_directories(self):
> > > + """Init directory variables:
> > > + PYTHONPATH
> > > + TEST_DIR
> > > + SOCK_DIR
> > > + SAMPLE_IMG_DIR
> > > + OUTPUT_DIR
> > > + """
> > > + self.pythonpath = os.getenv('PYTHONPATH')
> > > + if self.pythonpath:
> > > + self.pythonpath = self.source_iotests + os.pathsep + \
> > > + self.pythonpath
> > > + else:
> > > + self.pythonpath = self.source_iotests
> > > +
> > > + self.test_dir = os.getenv('TEST_DIR',
> > > + os.path.join(os.getcwd(), 'scratch'))
> > > + Path(self.test_dir).mkdir(parents=True, exist_ok=True)
> > > +
> > > + self.sock_dir = os.getenv('SOCK_DIR')
> > > + self.tmp_sock_dir = False
> > > + if self.sock_dir:
> > > + Path(self.test_dir).mkdir(parents=True, exist_ok=True)
> > > + else:
> > > + self.sock_dir = tempfile.mkdtemp()
> > > + self.tmp_sock_dir = True
> > > +
> > > + self.sample_img_dir = os.getenv('SAMPLE_IMG_DIR',
> > > + os.path.join(self.source_iotests,
> > > + 'sample_images'))
> > > +
> > > + self.output_dir = os.getcwd() # OUTPUT_DIR
> > > +
> > > + def init_binaries(self):
> > > + """Init binary path variables:
> > > + PYTHON (for bash tests)
> > > + QEMU_PROG, QEMU_IMG_PROG, QEMU_IO_PROG, QEMU_NBD_PROG, QSD_PROG
> > > + SOCKET_SCM_HELPER
> > > + """
> > > + self.python = '/usr/bin/python3 -B'
> >
> > This doesn't look right, we need to respect the Python binary set in
> > configure (which I think we get from common.env)
>
> Oh, I missed the change. Then I should just drop this self.python.
Do we still get the value from elsewhere or do we need to manually parse
common.env?
> >
> > > + def root(*names):
> > > + return os.path.join(self.build_root, *names)
> > > +
> > > + arch = os.uname().machine
> > > + if 'ppc64' in arch:
> > > + arch = 'ppc64'
> > > +
> > > + self.qemu_prog = os.getenv('QEMU_PROG', root(f'qemu-system-{arch}'))
> > > + self.qemu_img_prog = os.getenv('QEMU_IMG_PROG', root('qemu-img'))
> > > + self.qemu_io_prog = os.getenv('QEMU_IO_PROG', root('qemu-io'))
> > > + self.qemu_nbd_prog = os.getenv('QEMU_NBD_PROG', root('qemu-nbd'))
> > > + self.qsd_prog = os.getenv('QSD_PROG', root('storage-daemon',
> > > + 'qemu-storage-daemon'))
> > > +
> > > + for b in [self.qemu_img_prog, self.qemu_io_prog, self.qemu_nbd_prog,
> > > + self.qemu_prog, self.qsd_prog]:
> > > + if not os.path.exists(b):
> > > + exit('Not such file: ' + b)
> > > + if not os.access(b, os.X_OK):
> > > + exit('Not executable: ' + b)
> > > +
> > > + helper_path = os.path.join(self.build_iotests, 'socket_scm_helper')
> > > + if os.access(helper_path, os.X_OK):
> > > + self.socket_scm_helper = helper_path # SOCKET_SCM_HELPER
> > > +
> > > + def __init__(self, argv: List[str]) -> None:
> > > + """Parse args and environment"""
> > > +
> > > + # Initialize generic paths: build_root, build_iotests, source_iotests,
> > > + # which are needed to initialize some environment variables. They are
> > > + # used by init_*() functions as well.
> > > +
> > > +
> > > + if os.path.islink(sys.argv[0]):
> > > + # called from the build tree
> > > + self.source_iotests = os.path.dirname(os.readlink(sys.argv[0]))
> > > + self.build_iotests = os.path.dirname(os.path.abspath(sys.argv[0]))
> > > + else:
> > > + # called from the source tree
> > > + self.source_iotests = os.getcwd()
> > > + self.build_iotests = self.source_iotests
> > > +
> > > + self.build_root = os.path.join(self.build_iotests, '..', '..')
> > > +
> > > + self.init_handle_argv(argv)
> > > + self.init_directories()
> > > + self.init_binaries()
> > > +
> > > + # QEMU_OPTIONS
> > > + self.qemu_options = '-nodefaults -display none -accel qtest'
> > > + machine_map = (
> > > + (('arm', 'aarch64'), 'virt'),
> >
> > How does this work? Won't we check for "qemu-system-('arm', 'aarch64')"
> > below, which we'll never find?
>
> Hmm. I just took existing logic from check:
>
> [..]
> case "$QEMU_PROG" in
> *qemu-system-arm|*qemu-system-aarch64)
> export QEMU_OPTIONS="$QEMU_OPTIONS -machine virt"
> ;;
> [..]
What I mean is that the code below doesn't look like it's prepared to
interpret a tuple like ('arm', 'aarch64'), it expects a single string:
> >
> > > + ('avr', 'mega2560'),
> > > + ('rx', 'gdbsim-r5f562n8'),
> > > + ('tricore', 'tricore_testboard')
> > > + )
> > > + for suffix, machine in machine_map:
> > > + if self.qemu_prog.endswith(f'qemu-system-{suffix}'):
Here we get effectively:
suffix: Tuple[str, str] = ('arm', 'aarch64')
The formatted string uses str(suffix), which makes the result
"qemu-system-('arm', 'aarch64')".
Or am I misunderstanding something here?
> > > + self.qemu_options += f' -machine {machine}'
> > > +
> > > + # QEMU_DEFAULT_MACHINE
> > > + self.qemu_default_machine = get_default_machine(self.qemu_prog)
> > > +
> > > + self.qemu_img_options = os.getenv('QEMU_IMG_OPTIONS')
> > > + self.qemu_nbd_options = os.getenv('QEMU_NBD_OPTIONS')
> > > +
> > > + is_generic = self.imgfmt not in ['bochs', 'cloop', 'dmg']
> > > + self.imgfmt_generic = 'true' if is_generic else 'false'
> > > +
> > > + self.qemu_io_options = f'--cache {self.cachemode} --aio {self.aiomode}'
> > > + if self.misalign:
> > > + self.qemu_io_options += ' --misalign'
> > > +
> > > + self.qemu_io_options_no_fmt = self.qemu_io_options
> > > +
> > > + if self.imgfmt == 'luks':
> > > + self.imgoptssyntax = 'true'
> > > + self.imgkeysecret = '123456'
> > > + if not self.imgopts:
> > > + self.imgopts = 'iter-time=10'
> > > + elif 'iter-time=' not in self.imgopts:
> > > + self.imgopts += ',iter-time=10'
> > > + else:
> > > + self.imgoptssyntax = 'false'
> > > + self.qemu_io_options += ' -f ' + self.imgfmt
> > > +
> > > + if self.imgfmt == 'vmkd':
> > > + if not self.imgopts:
> > > + self.imgopts = 'zeroed_grain=on'
> > > + elif 'zeroed_grain=' not in self.imgopts:
> > > + self.imgopts += ',zeroed_grain=on'
> > > +
> > > + def close(self) -> None:
> > > + if self.tmp_sock_dir:
> > > + shutil.rmtree(self.sock_dir)
> > > +
> > > + def __enter__(self) -> 'TestEnv':
> > > + return self
> > > +
> > > + def __exit__(self, *args) -> None:
> > > + self.close()
> > > +
> > > + def print_env(self) -> None:
> > > + template = """\
> > > +QEMU -- "{QEMU_PROG}" {QEMU_OPTIONS}
> > > +QEMU_IMG -- "{QEMU_IMG_PROG}" {QEMU_IMG_OPTIONS}
> > > +QEMU_IO -- "{QEMU_IO_PROG}" {QEMU_IO_OPTIONS}
> > > +QEMU_NBD -- "{QEMU_NBD_PROG}" {QEMU_NBD_OPTIONS}
> > > +IMGFMT -- {IMGFMT}{imgopts}
> > > +IMGPROTO -- {IMGPROTO}
> > > +PLATFORM -- {platform}
> > > +TEST_DIR -- {TEST_DIR}
> > > +SOCK_DIR -- {SOCK_DIR}
> > > +SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER}"""
> > > +
> > > + args = collections.defaultdict(str, self.get_env())
> > > +
> > > + if 'IMGOPTS' in args:
> > > + args['imgopts'] = f" ({args['IMGOPTS']})"
> > > +
> > > + u = os.uname()
> > > + args['platform'] = f'{u.sysname}/{u.machine} {u.nodename} {u.release}'
> > > +
> > > + print(template.format_map(args))
> > > +
> > > +
> > > +if __name__ == '__main__':
> > > + if len(sys.argv) == 2 and sys.argv[1] in ['-h', '--help']:
> > > + TestEnv.get_argparser().print_help()
> > > + exit()
> > > +
> > > + with TestEnv(sys.argv) as te:
> > > + te.print_env()
> > > + print('\nUnhandled options: ', te.remaining_argv)
>
> Thanks for reviewing! I hope to post v7 today, with cmdline interface
> in 'check' script.
Sounds good. Then I'll continue the review when v7 is on the list.
Kevin
^ permalink raw reply [flat|nested] 43+ messages in thread* Re: [PATCH v6 08/11] iotests: add testenv.py
2021-01-15 12:45 ` Kevin Wolf
@ 2021-01-15 13:10 ` Vladimir Sementsov-Ogievskiy
2021-01-15 13:20 ` Kevin Wolf
0 siblings, 1 reply; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-15 13:10 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-block, qemu-devel, mreitz, den, jsnow
15.01.2021 15:45, Kevin Wolf wrote:
> Am 15.01.2021 um 13:19 hat Vladimir Sementsov-Ogievskiy geschrieben:
>> 15.01.2021 14:18, Kevin Wolf wrote:
>>> Am 09.01.2021 um 13:26 hat Vladimir Sementsov-Ogievskiy geschrieben:
>>>> Add TestEnv class, which will handle test environment in a new python
>>>> iotests running framework.
>>>>
>>>> Difference with current ./check interface:
>>>> - -v (verbose) option dropped, as it is unused
>>>>
>>>> - -xdiff option is dropped, until somebody complains that it is needed
>>>> - same for -n option
>>>>
>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>> ---
>>>> tests/qemu-iotests/testenv.py | 328 ++++++++++++++++++++++++++++++++++
>>>> 1 file changed, 328 insertions(+)
>>>> create mode 100755 tests/qemu-iotests/testenv.py
>>>>
[..]
>>>> + def init_binaries(self):
>>>> + """Init binary path variables:
>>>> + PYTHON (for bash tests)
>>>> + QEMU_PROG, QEMU_IMG_PROG, QEMU_IO_PROG, QEMU_NBD_PROG, QSD_PROG
>>>> + SOCKET_SCM_HELPER
>>>> + """
>>>> + self.python = '/usr/bin/python3 -B'
>>>
>>> This doesn't look right, we need to respect the Python binary set in
>>> configure (which I think we get from common.env)
>>
>> Oh, I missed the change. Then I should just drop this self.python.
>
> Do we still get the value from elsewhere or do we need to manually parse
> common.env?
Hmm.. Good question. We have either parse common.env, and still create self.python variable.
Or drop it, and include common.env directly to bash tests. For this we'll need to export
BUILD_IOTESTS, and do
. $BUILD_IOTESTS/common.env
in common.rc..
>
>>>
>>>> + def root(*names):
>>>> + return os.path.join(self.build_root, *names)
>>>> +
>>>> + arch = os.uname().machine
>>>> + if 'ppc64' in arch:
>>>> + arch = 'ppc64'
>>>> +
>>>> + self.qemu_prog = os.getenv('QEMU_PROG', root(f'qemu-system-{arch}'))
>>>> + self.qemu_img_prog = os.getenv('QEMU_IMG_PROG', root('qemu-img'))
>>>> + self.qemu_io_prog = os.getenv('QEMU_IO_PROG', root('qemu-io'))
>>>> + self.qemu_nbd_prog = os.getenv('QEMU_NBD_PROG', root('qemu-nbd'))
>>>> + self.qsd_prog = os.getenv('QSD_PROG', root('storage-daemon',
>>>> + 'qemu-storage-daemon'))
>>>> +
>>>> + for b in [self.qemu_img_prog, self.qemu_io_prog, self.qemu_nbd_prog,
>>>> + self.qemu_prog, self.qsd_prog]:
>>>> + if not os.path.exists(b):
>>>> + exit('Not such file: ' + b)
>>>> + if not os.access(b, os.X_OK):
>>>> + exit('Not executable: ' + b)
>>>> +
>>>> + helper_path = os.path.join(self.build_iotests, 'socket_scm_helper')
>>>> + if os.access(helper_path, os.X_OK):
>>>> + self.socket_scm_helper = helper_path # SOCKET_SCM_HELPER
>>>> +
>>>> + def __init__(self, argv: List[str]) -> None:
>>>> + """Parse args and environment"""
>>>> +
>>>> + # Initialize generic paths: build_root, build_iotests, source_iotests,
>>>> + # which are needed to initialize some environment variables. They are
>>>> + # used by init_*() functions as well.
>>>> +
>>>> +
>>>> + if os.path.islink(sys.argv[0]):
>>>> + # called from the build tree
>>>> + self.source_iotests = os.path.dirname(os.readlink(sys.argv[0]))
>>>> + self.build_iotests = os.path.dirname(os.path.abspath(sys.argv[0]))
>>>> + else:
>>>> + # called from the source tree
>>>> + self.source_iotests = os.getcwd()
>>>> + self.build_iotests = self.source_iotests
>>>> +
>>>> + self.build_root = os.path.join(self.build_iotests, '..', '..')
>>>> +
>>>> + self.init_handle_argv(argv)
>>>> + self.init_directories()
>>>> + self.init_binaries()
>>>> +
>>>> + # QEMU_OPTIONS
>>>> + self.qemu_options = '-nodefaults -display none -accel qtest'
>>>> + machine_map = (
>>>> + (('arm', 'aarch64'), 'virt'),
>>>
>>> How does this work? Won't we check for "qemu-system-('arm', 'aarch64')"
>>> below, which we'll never find?
>>
>> Hmm. I just took existing logic from check:
>>
>> [..]
>> case "$QEMU_PROG" in
>> *qemu-system-arm|*qemu-system-aarch64)
>> export QEMU_OPTIONS="$QEMU_OPTIONS -machine virt"
>> ;;
>> [..]
>
> What I mean is that the code below doesn't look like it's prepared to
> interpret a tuple like ('arm', 'aarch64'), it expects a single string:
>
>>>
>>>> + ('avr', 'mega2560'),
>>>> + ('rx', 'gdbsim-r5f562n8'),
>>>> + ('tricore', 'tricore_testboard')
>>>> + )
>>>> + for suffix, machine in machine_map:
>>>> + if self.qemu_prog.endswith(f'qemu-system-{suffix}'):
>
> Here we get effectively:
>
> suffix: Tuple[str, str] = ('arm', 'aarch64')
>
> The formatted string uses str(suffix), which makes the result
> "qemu-system-('arm', 'aarch64')".
>
> Or am I misunderstanding something here?
Ah, you are right:) Will fix. I interpreted your
Won't we check for "qemu-system-('arm', 'aarch64')"
as
Won't we check for "qemu-system-arm" or "qemu-system-aarch64"
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 43+ messages in thread* Re: [PATCH v6 08/11] iotests: add testenv.py
2021-01-15 13:10 ` Vladimir Sementsov-Ogievskiy
@ 2021-01-15 13:20 ` Kevin Wolf
2021-01-15 13:30 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 43+ messages in thread
From: Kevin Wolf @ 2021-01-15 13:20 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy; +Cc: qemu-block, qemu-devel, mreitz, den, jsnow
Am 15.01.2021 um 14:10 hat Vladimir Sementsov-Ogievskiy geschrieben:
> 15.01.2021 15:45, Kevin Wolf wrote:
> > Am 15.01.2021 um 13:19 hat Vladimir Sementsov-Ogievskiy geschrieben:
> > > 15.01.2021 14:18, Kevin Wolf wrote:
> > > > Am 09.01.2021 um 13:26 hat Vladimir Sementsov-Ogievskiy geschrieben:
> > > > > Add TestEnv class, which will handle test environment in a new python
> > > > > iotests running framework.
> > > > >
> > > > > Difference with current ./check interface:
> > > > > - -v (verbose) option dropped, as it is unused
> > > > >
> > > > > - -xdiff option is dropped, until somebody complains that it is needed
> > > > > - same for -n option
> > > > >
> > > > > Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> > > > > ---
> > > > > tests/qemu-iotests/testenv.py | 328 ++++++++++++++++++++++++++++++++++
> > > > > 1 file changed, 328 insertions(+)
> > > > > create mode 100755 tests/qemu-iotests/testenv.py
> > > > >
>
> [..]
>
> > > > > + def init_binaries(self):
> > > > > + """Init binary path variables:
> > > > > + PYTHON (for bash tests)
> > > > > + QEMU_PROG, QEMU_IMG_PROG, QEMU_IO_PROG, QEMU_NBD_PROG, QSD_PROG
> > > > > + SOCKET_SCM_HELPER
> > > > > + """
> > > > > + self.python = '/usr/bin/python3 -B'
> > > >
> > > > This doesn't look right, we need to respect the Python binary set in
> > > > configure (which I think we get from common.env)
> > >
> > > Oh, I missed the change. Then I should just drop this self.python.
> >
> > Do we still get the value from elsewhere or do we need to manually parse
> > common.env?
>
> Hmm.. Good question. We have either parse common.env, and still create self.python variable.
>
> Or drop it, and include common.env directly to bash tests. For this we'll need to export
>
> BUILD_IOTESTS, and do
> . $BUILD_IOTESTS/common.env
>
> in common.rc..
check uses it, too, for running Python test cases.
Kevin
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v6 08/11] iotests: add testenv.py
2021-01-15 13:20 ` Kevin Wolf
@ 2021-01-15 13:30 ` Vladimir Sementsov-Ogievskiy
2021-01-16 11:03 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-15 13:30 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-block, qemu-devel, mreitz, den, jsnow
15.01.2021 16:20, Kevin Wolf wrote:
> Am 15.01.2021 um 14:10 hat Vladimir Sementsov-Ogievskiy geschrieben:
>> 15.01.2021 15:45, Kevin Wolf wrote:
>>> Am 15.01.2021 um 13:19 hat Vladimir Sementsov-Ogievskiy geschrieben:
>>>> 15.01.2021 14:18, Kevin Wolf wrote:
>>>>> Am 09.01.2021 um 13:26 hat Vladimir Sementsov-Ogievskiy geschrieben:
>>>>>> Add TestEnv class, which will handle test environment in a new python
>>>>>> iotests running framework.
>>>>>>
>>>>>> Difference with current ./check interface:
>>>>>> - -v (verbose) option dropped, as it is unused
>>>>>>
>>>>>> - -xdiff option is dropped, until somebody complains that it is needed
>>>>>> - same for -n option
>>>>>>
>>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>>>> ---
>>>>>> tests/qemu-iotests/testenv.py | 328 ++++++++++++++++++++++++++++++++++
>>>>>> 1 file changed, 328 insertions(+)
>>>>>> create mode 100755 tests/qemu-iotests/testenv.py
>>>>>>
>>
>> [..]
>>
>>>>>> + def init_binaries(self):
>>>>>> + """Init binary path variables:
>>>>>> + PYTHON (for bash tests)
>>>>>> + QEMU_PROG, QEMU_IMG_PROG, QEMU_IO_PROG, QEMU_NBD_PROG, QSD_PROG
>>>>>> + SOCKET_SCM_HELPER
>>>>>> + """
>>>>>> + self.python = '/usr/bin/python3 -B'
>>>>>
>>>>> This doesn't look right, we need to respect the Python binary set in
>>>>> configure (which I think we get from common.env)
>>>>
>>>> Oh, I missed the change. Then I should just drop this self.python.
>>>
>>> Do we still get the value from elsewhere or do we need to manually parse
>>> common.env?
>>
>> Hmm.. Good question. We have either parse common.env, and still create self.python variable.
>>
>> Or drop it, and include common.env directly to bash tests. For this we'll need to export
>>
>> BUILD_IOTESTS, and do
>> . $BUILD_IOTESTS/common.env
>>
>> in common.rc..
>
> check uses it, too, for running Python test cases.
>
But new check (written in python) doesn't.. Should I keep bash check, which will have only one line to call check.py with help of PYTHON?
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v6 08/11] iotests: add testenv.py
2021-01-15 13:30 ` Vladimir Sementsov-Ogievskiy
@ 2021-01-16 11:03 ` Vladimir Sementsov-Ogievskiy
2021-01-16 11:19 ` Vladimir Sementsov-Ogievskiy
2021-01-18 9:59 ` Kevin Wolf
0 siblings, 2 replies; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-16 11:03 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-block, qemu-devel, mreitz, den, jsnow
15.01.2021 16:30, Vladimir Sementsov-Ogievskiy wrote:
> 15.01.2021 16:20, Kevin Wolf wrote:
>> Am 15.01.2021 um 14:10 hat Vladimir Sementsov-Ogievskiy geschrieben:
>>> 15.01.2021 15:45, Kevin Wolf wrote:
>>>> Am 15.01.2021 um 13:19 hat Vladimir Sementsov-Ogievskiy geschrieben:
>>>>> 15.01.2021 14:18, Kevin Wolf wrote:
>>>>>> Am 09.01.2021 um 13:26 hat Vladimir Sementsov-Ogievskiy geschrieben:
>>>>>>> Add TestEnv class, which will handle test environment in a new python
>>>>>>> iotests running framework.
>>>>>>>
>>>>>>> Difference with current ./check interface:
>>>>>>> - -v (verbose) option dropped, as it is unused
>>>>>>>
>>>>>>> - -xdiff option is dropped, until somebody complains that it is needed
>>>>>>> - same for -n option
>>>>>>>
>>>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>>>>> ---
>>>>>>> tests/qemu-iotests/testenv.py | 328 ++++++++++++++++++++++++++++++++++
>>>>>>> 1 file changed, 328 insertions(+)
>>>>>>> create mode 100755 tests/qemu-iotests/testenv.py
>>>>>>>
>>>
>>> [..]
>>>
>>>>>>> + def init_binaries(self):
>>>>>>> + """Init binary path variables:
>>>>>>> + PYTHON (for bash tests)
>>>>>>> + QEMU_PROG, QEMU_IMG_PROG, QEMU_IO_PROG, QEMU_NBD_PROG, QSD_PROG
>>>>>>> + SOCKET_SCM_HELPER
>>>>>>> + """
>>>>>>> + self.python = '/usr/bin/python3 -B'
>>>>>>
>>>>>> This doesn't look right, we need to respect the Python binary set in
>>>>>> configure (which I think we get from common.env)
>>>>>
>>>>> Oh, I missed the change. Then I should just drop this self.python.
>>>>
>>>> Do we still get the value from elsewhere or do we need to manually parse
>>>> common.env?
>>>
>>> Hmm.. Good question. We have either parse common.env, and still create self.python variable.
>>>
>>> Or drop it, and include common.env directly to bash tests. For this we'll need to export
>>>
>>> BUILD_IOTESTS, and do
>>> . $BUILD_IOTESTS/common.env
>>>
>>> in common.rc..
>>
>> check uses it, too, for running Python test cases.
>>
>
> But new check (written in python) doesn't.. Should I keep bash check, which will have only one line to call check.py with help of PYTHON?
>
>
Or finally, may be just drop it? Can we just use system python for tests, now when we are already in a python3 world?
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v6 08/11] iotests: add testenv.py
2021-01-16 11:03 ` Vladimir Sementsov-Ogievskiy
@ 2021-01-16 11:19 ` Vladimir Sementsov-Ogievskiy
2021-01-18 9:59 ` Kevin Wolf
1 sibling, 0 replies; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-16 11:19 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-block, qemu-devel, mreitz, den, jsnow
16.01.2021 14:03, Vladimir Sementsov-Ogievskiy wrote:
> 15.01.2021 16:30, Vladimir Sementsov-Ogievskiy wrote:
>> 15.01.2021 16:20, Kevin Wolf wrote:
>>> Am 15.01.2021 um 14:10 hat Vladimir Sementsov-Ogievskiy geschrieben:
>>>> 15.01.2021 15:45, Kevin Wolf wrote:
>>>>> Am 15.01.2021 um 13:19 hat Vladimir Sementsov-Ogievskiy geschrieben:
>>>>>> 15.01.2021 14:18, Kevin Wolf wrote:
>>>>>>> Am 09.01.2021 um 13:26 hat Vladimir Sementsov-Ogievskiy geschrieben:
>>>>>>>> Add TestEnv class, which will handle test environment in a new python
>>>>>>>> iotests running framework.
>>>>>>>>
>>>>>>>> Difference with current ./check interface:
>>>>>>>> - -v (verbose) option dropped, as it is unused
>>>>>>>>
>>>>>>>> - -xdiff option is dropped, until somebody complains that it is needed
>>>>>>>> - same for -n option
>>>>>>>>
>>>>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>>>>>> ---
>>>>>>>> tests/qemu-iotests/testenv.py | 328 ++++++++++++++++++++++++++++++++++
>>>>>>>> 1 file changed, 328 insertions(+)
>>>>>>>> create mode 100755 tests/qemu-iotests/testenv.py
>>>>>>>>
>>>>
>>>> [..]
>>>>
>>>>>>>> + def init_binaries(self):
>>>>>>>> + """Init binary path variables:
>>>>>>>> + PYTHON (for bash tests)
>>>>>>>> + QEMU_PROG, QEMU_IMG_PROG, QEMU_IO_PROG, QEMU_NBD_PROG, QSD_PROG
>>>>>>>> + SOCKET_SCM_HELPER
>>>>>>>> + """
>>>>>>>> + self.python = '/usr/bin/python3 -B'
>>>>>>>
>>>>>>> This doesn't look right, we need to respect the Python binary set in
>>>>>>> configure (which I think we get from common.env)
>>>>>>
>>>>>> Oh, I missed the change. Then I should just drop this self.python.
>>>>>
>>>>> Do we still get the value from elsewhere or do we need to manually parse
>>>>> common.env?
>>>>
>>>> Hmm.. Good question. We have either parse common.env, and still create self.python variable.
>>>>
>>>> Or drop it, and include common.env directly to bash tests. For this we'll need to export
>>>>
>>>> BUILD_IOTESTS, and do
>>>> . $BUILD_IOTESTS/common.env
>>>>
>>>> in common.rc..
>>>
>>> check uses it, too, for running Python test cases.
>>>
>>
>> But new check (written in python) doesn't.. Should I keep bash check, which will have only one line to call check.py with help of PYTHON?
>>
>>
>
> Or finally, may be just drop it? Can we just use system python for tests, now when we are already in a python3 world?
>
Oh, I know. We just need to use "self.python = sys.executable", so that tests uses the interpreter that runs check.
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v6 08/11] iotests: add testenv.py
2021-01-16 11:03 ` Vladimir Sementsov-Ogievskiy
2021-01-16 11:19 ` Vladimir Sementsov-Ogievskiy
@ 2021-01-18 9:59 ` Kevin Wolf
2021-01-18 17:00 ` Vladimir Sementsov-Ogievskiy
1 sibling, 1 reply; 43+ messages in thread
From: Kevin Wolf @ 2021-01-18 9:59 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy; +Cc: qemu-block, qemu-devel, mreitz, den, jsnow
Am 16.01.2021 um 12:03 hat Vladimir Sementsov-Ogievskiy geschrieben:
> 15.01.2021 16:30, Vladimir Sementsov-Ogievskiy wrote:
> > 15.01.2021 16:20, Kevin Wolf wrote:
> > > Am 15.01.2021 um 14:10 hat Vladimir Sementsov-Ogievskiy geschrieben:
> > > > 15.01.2021 15:45, Kevin Wolf wrote:
> > > > > Am 15.01.2021 um 13:19 hat Vladimir Sementsov-Ogievskiy geschrieben:
> > > > > > 15.01.2021 14:18, Kevin Wolf wrote:
> > > > > > > Am 09.01.2021 um 13:26 hat Vladimir Sementsov-Ogievskiy geschrieben:
> > > > > > > > Add TestEnv class, which will handle test environment in a new python
> > > > > > > > iotests running framework.
> > > > > > > >
> > > > > > > > Difference with current ./check interface:
> > > > > > > > - -v (verbose) option dropped, as it is unused
> > > > > > > >
> > > > > > > > - -xdiff option is dropped, until somebody complains that it is needed
> > > > > > > > - same for -n option
> > > > > > > >
> > > > > > > > Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> > > > > > > > ---
> > > > > > > > Â Â Â tests/qemu-iotests/testenv.py | 328 ++++++++++++++++++++++++++++++++++
> > > > > > > > Â Â Â 1 file changed, 328 insertions(+)
> > > > > > > > Â Â Â create mode 100755 tests/qemu-iotests/testenv.py
> > > > > > > >
> > > >
> > > > [..]
> > > >
> > > > > > > > +Â Â Â def init_binaries(self):
> > > > > > > > +Â Â Â Â Â Â Â """Init binary path variables:
> > > > > > > > +Â Â Â Â Â Â Â Â Â Â Â Â PYTHON (for bash tests)
> > > > > > > > +Â Â Â Â Â Â Â Â Â Â Â Â QEMU_PROG, QEMU_IMG_PROG, QEMU_IO_PROG, QEMU_NBD_PROG, QSD_PROG
> > > > > > > > +Â Â Â Â Â Â Â Â Â Â Â Â SOCKET_SCM_HELPER
> > > > > > > > +Â Â Â Â Â Â Â """
> > > > > > > > +Â Â Â Â Â Â Â self.python = '/usr/bin/python3 -B'
> > > > > > >
> > > > > > > This doesn't look right, we need to respect the Python binary set in
> > > > > > > configure (which I think we get from common.env)
> > > > > >
> > > > > > Oh, I missed the change. Then I should just drop this self.python.
> > > > >
> > > > > Do we still get the value from elsewhere or do we need to manually parse
> > > > > common.env?
> > > >
> > > > Hmm.. Good question. We have either parse common.env, and still create self.python variable.
> > > >
> > > > Or drop it, and include common.env directly to bash tests. For this we'll need to export
> > > >
> > > > BUILD_IOTESTS, and do
> > > > Â . $BUILD_IOTESTS/common.env
> > > >
> > > > in common.rc..
> > >
> > > check uses it, too, for running Python test cases.
> > >
> >
> > But new check (written in python) doesn't.. Should I keep bash check, which will have only one line to call check.py with help of PYTHON?
> >
> >
>
> Or finally, may be just drop it? Can we just use system python for
> tests, now when we are already in a python3 world?
You can only assume the Python 3 (or more specifically 3.6+) world if
you are using the Python interpreter that was passed to configure. This
will commonly be the same thing as the first python3 in PATH, but it
doesn't have to be.
So your solution of just using the same interpreter as for check might
be okay as long as we make sure that 'make check' calls check with the
configured Python interpreter instead of relying on the shebang line.
Kevin
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v6 08/11] iotests: add testenv.py
2021-01-18 9:59 ` Kevin Wolf
@ 2021-01-18 17:00 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-18 17:00 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-block, qemu-devel, mreitz, den, jsnow
18.01.2021 12:59, Kevin Wolf wrote:
> Am 16.01.2021 um 12:03 hat Vladimir Sementsov-Ogievskiy geschrieben:
>> 15.01.2021 16:30, Vladimir Sementsov-Ogievskiy wrote:
>>> 15.01.2021 16:20, Kevin Wolf wrote:
>>>> Am 15.01.2021 um 14:10 hat Vladimir Sementsov-Ogievskiy geschrieben:
>>>>> 15.01.2021 15:45, Kevin Wolf wrote:
>>>>>> Am 15.01.2021 um 13:19 hat Vladimir Sementsov-Ogievskiy geschrieben:
>>>>>>> 15.01.2021 14:18, Kevin Wolf wrote:
>>>>>>>> Am 09.01.2021 um 13:26 hat Vladimir Sementsov-Ogievskiy geschrieben:
>>>>>>>>> Add TestEnv class, which will handle test environment in a new python
>>>>>>>>> iotests running framework.
>>>>>>>>>
>>>>>>>>> Difference with current ./check interface:
>>>>>>>>> - -v (verbose) option dropped, as it is unused
>>>>>>>>>
>>>>>>>>> - -xdiff option is dropped, until somebody complains that it is needed
>>>>>>>>> - same for -n option
>>>>>>>>>
>>>>>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>>>>>>> ---
>>>>>>>>> Â Â Â tests/qemu-iotests/testenv.py | 328 ++++++++++++++++++++++++++++++++++
>>>>>>>>> Â Â Â 1 file changed, 328 insertions(+)
>>>>>>>>> Â Â Â create mode 100755 tests/qemu-iotests/testenv.py
>>>>>>>>>
>>>>>
>>>>> [..]
>>>>>
>>>>>>>>> +Â Â Â def init_binaries(self):
>>>>>>>>> +Â Â Â Â Â Â Â """Init binary path variables:
>>>>>>>>> +Â Â Â Â Â Â Â Â Â Â Â Â PYTHON (for bash tests)
>>>>>>>>> +Â Â Â Â Â Â Â Â Â Â Â Â QEMU_PROG, QEMU_IMG_PROG, QEMU_IO_PROG, QEMU_NBD_PROG, QSD_PROG
>>>>>>>>> +Â Â Â Â Â Â Â Â Â Â Â Â SOCKET_SCM_HELPER
>>>>>>>>> +Â Â Â Â Â Â Â """
>>>>>>>>> +Â Â Â Â Â Â Â self.python = '/usr/bin/python3 -B'
>>>>>>>>
>>>>>>>> This doesn't look right, we need to respect the Python binary set in
>>>>>>>> configure (which I think we get from common.env)
>>>>>>>
>>>>>>> Oh, I missed the change. Then I should just drop this self.python.
>>>>>>
>>>>>> Do we still get the value from elsewhere or do we need to manually parse
>>>>>> common.env?
>>>>>
>>>>> Hmm.. Good question. We have either parse common.env, and still create self.python variable.
>>>>>
>>>>> Or drop it, and include common.env directly to bash tests. For this we'll need to export
>>>>>
>>>>> BUILD_IOTESTS, and do
>>>>> Â . $BUILD_IOTESTS/common.env
>>>>>
>>>>> in common.rc..
>>>>
>>>> check uses it, too, for running Python test cases.
>>>>
>>>
>>> But new check (written in python) doesn't.. Should I keep bash check, which will have only one line to call check.py with help of PYTHON?
>>>
>>>
>>
>> Or finally, may be just drop it? Can we just use system python for
>> tests, now when we are already in a python3 world?
>
> You can only assume the Python 3 (or more specifically 3.6+) world if
> you are using the Python interpreter that was passed to configure. This
> will commonly be the same thing as the first python3 in PATH, but it
> doesn't have to be.
>
> So your solution of just using the same interpreter as for check might
> be okay as long as we make sure that 'make check' calls check with the
> configured Python interpreter instead of relying on the shebang line.
>
I've sent v7 with exactly this thing: use same interpreter as for check and adjust check-block.sh which is called from make check
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 43+ messages in thread
* [PATCH v6 09/11] iotests: add testrunner.py
2021-01-09 12:26 [PATCH v6 00/11] Rework iotests/check Vladimir Sementsov-Ogievskiy
` (7 preceding siblings ...)
2021-01-09 12:26 ` [PATCH v6 08/11] iotests: add testenv.py Vladimir Sementsov-Ogievskiy
@ 2021-01-09 12:26 ` Vladimir Sementsov-Ogievskiy
2021-01-09 12:26 ` [PATCH v6 10/11] iotests: rewrite check into python Vladimir Sementsov-Ogievskiy
2021-01-09 12:26 ` [PATCH v6 11/11] iotests: rename and move 169 and 199 tests Vladimir Sementsov-Ogievskiy
10 siblings, 0 replies; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-09 12:26 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, vsementsov, jsnow, qemu-devel, mreitz, den
Add TestRunner class, which will run tests in a new python iotests
running framework.
There are some differences with current ./check behavior, most
significant are:
- Consider all tests self-executable, just run them, don't run python
by hand.
- Elapsed time is cached in json file
- Elapsed time precision increased a bit
- use python difflib instead of "diff -w", to ignore spaces at line
ends strip lines by hand. Do not ignore other spaces.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
tests/qemu-iotests/testrunner.py | 351 +++++++++++++++++++++++++++++++
1 file changed, 351 insertions(+)
create mode 100644 tests/qemu-iotests/testrunner.py
diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py
new file mode 100644
index 0000000000..e395877882
--- /dev/null
+++ b/tests/qemu-iotests/testrunner.py
@@ -0,0 +1,351 @@
+# Class for actual tests running.
+#
+# Copyright (c) 2020 Virtuozzo International GmbH
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+import os
+import random
+from pathlib import Path
+import datetime
+import time
+import difflib
+import subprocess
+import collections
+import contextlib
+import json
+import argparse
+import termios
+import sys
+from contextlib import contextmanager
+from typing import List, Optional, Iterator
+
+from testenv import TestEnv
+
+
+def silent_unlink(path: Path) -> None:
+ try:
+ path.unlink()
+ except OSError:
+ pass
+
+
+def file_diff(file1: str, file2: str) -> List[str]:
+ with open(file1) as f1, open(file2) as f2:
+ # We want to ignore spaces at line ends. There are a lot of mess about
+ # it in iotests.
+ # TODO: fix all tests to not produce extra spaces, fix all .out files
+ # and use strict diff here!
+ seq1 = [line.rstrip() for line in f1]
+ seq2 = [line.rstrip() for line in f2]
+ return list(difflib.unified_diff(seq1, seq2, file1, file2))
+
+
+# We want to save current tty settings during test run,
+# since an aborting qemu call may leave things screwed up.
+@contextmanager
+def savetty() -> Iterator:
+ isterm = sys.stdin.isatty()
+ if isterm:
+ fd = sys.stdin.fileno()
+ attr = termios.tcgetattr(0)
+
+ try:
+ yield
+ finally:
+ if isterm:
+ termios.tcsetattr(fd, termios.TCSADRAIN, attr)
+
+
+class LastElapsedTime:
+ """ Cache for elapsed time for tests, to show it during new test run
+
+ Use get() in any time. But, if use update you should then call save(),
+ or use update() inside with-block.
+ """
+ def __init__(self, cache_file: str, env: TestEnv) -> None:
+ self.env = env
+ self.cache_file = cache_file
+
+ try:
+ with open(cache_file) as f:
+ self.cache = json.load(f)
+ except (OSError, ValueError):
+ self.cache = {}
+
+ def get(self, test: str,
+ default: Optional[float] = None) -> Optional[float]:
+ if test not in self.cache:
+ return default
+
+ if self.env.imgproto not in self.cache[test]:
+ return default
+
+ return self.cache[test][self.env.imgproto].get(self.env.imgfmt,
+ default)
+
+ def update(self, test: str, elapsed: float) -> None:
+ d = self.cache.setdefault(test, {})
+ d = d.setdefault(self.env.imgproto, {})
+ d[self.env.imgfmt] = elapsed
+
+ def save(self) -> None:
+ with open(self.cache_file, 'w') as f:
+ json.dump(self.cache, f)
+
+ def __enter__(self) -> 'LastElapsedTime':
+ return self
+
+ def __exit__(self, *args) -> None:
+ self.save()
+
+
+TestResult = collections.namedtuple(
+ 'TestResult',
+ ['status', 'description', 'elapsed', 'diff', 'casenotrun'],
+ defaults=('', '', '', ''))
+
+
+class TestRunner:
+ _argparser = None
+ @classmethod
+ def get_argparser(cls) -> argparse.ArgumentParser:
+ if cls._argparser is not None:
+ return cls._argparser
+
+ p = argparse.ArgumentParser(description="= test running options =",
+ add_help=False, usage=argparse.SUPPRESS)
+
+ p.add_argument('-makecheck', action='store_true',
+ help='pretty print output for make check')
+
+ cls._argparser = p
+ return p
+
+ def __init__(self, argv: List[str], env: TestEnv) -> None:
+ self.env = env
+ self.test_run_env = self.env.get_env()
+ if 'MALLOC_PERTURB_' not in os.environ and \
+ 'MALLOC_PERTURB_' not in self.test_run_env:
+ x = random.randrange(1, 255)
+ self.test_run_env['MALLOC_PERTURB_'] = str(x)
+
+ args, self.remaining_argv = self.get_argparser().parse_known_args(argv)
+ self.makecheck = args.makecheck
+
+ self.last_elapsed = LastElapsedTime('.last-elapsed-cache', env)
+
+ def __enter__(self) -> 'TestRunner':
+ self._stack = contextlib.ExitStack()
+ self._stack.enter_context(self.env)
+ self._stack.enter_context(self.last_elapsed)
+ self._stack.enter_context(savetty())
+ return self
+
+ def __exit__(self, *args) -> None:
+ self._stack.close()
+
+ def test_print_one_line(self, test: str, starttime: str,
+ endtime: Optional[str] = None, status: str = '...',
+ lasttime: Optional[float] = None,
+ thistime: Optional[str] = None,
+ description: str = '',
+ test_field_width: Optional[int] = None,
+ end: str = '\n') -> None:
+ """ Print short test info before/after test run """
+ test = os.path.basename(test)
+
+ if test_field_width is None:
+ test_field_width = 8
+
+ if self.makecheck and status != '...':
+ if status and status != 'pass':
+ status = f' [{status}]'
+ else:
+ status = ''
+
+ print(f' TEST iotest-{self.env.imgfmt}: {test}{status}')
+ return
+
+ if lasttime:
+ lasttime_s = f' (last: {lasttime}s)'
+ else:
+ lasttime_s = ''
+ if thistime:
+ thistime = f'{thistime:.1f}s'
+ else:
+ thistime = '...'
+
+ if endtime:
+ endtime = f'[{endtime}]'
+ else:
+ endtime = ''
+
+ if status == 'pass':
+ col = '\033[32m'
+ elif status == 'fail':
+ col = '\033[1m\033[31m'
+ elif status == 'not run':
+ col = '\033[33m'
+ else:
+ col = ''
+
+ col_end = '\033[0m'
+
+ print(f'{test:{test_field_width}} {col}{status:10}{col_end} '
+ f'[{starttime}] {endtime:13}{thistime:5} {lasttime_s:14} '
+ f'{description}', end=end)
+
+ def find_reference(self, test: str) -> str:
+ if self.env.cachemode == 'none':
+ ref = f'{test}.out.nocache'
+ if os.path.isfile(ref):
+ return ref
+
+ ref = f'{test}.out.{self.env.imgfmt}'
+ if os.path.isfile(ref):
+ return ref
+
+ ref = f'{test}.{self.env.qemu_default_machine}.out'
+ if os.path.isfile(ref):
+ return ref
+
+ return f'{test}.out'
+
+ def do_run_test(self, test: str) -> TestResult:
+ f_test = Path(test)
+ f_bad = Path(f_test.name + '.out.bad')
+ f_notrun = Path(f_test.name + '.notrun')
+ f_casenotrun = Path(f_test.name + '.casenotrun')
+ f_reference = Path(self.find_reference(test))
+
+ if not f_test.exists():
+ return TestResult(status='fail',
+ description=f'No such test file: {f_test}')
+
+ if not os.access(str(f_test), os.X_OK):
+ exit(f'Not executable: {f_test}')
+
+ if not f_reference.exists():
+ return TestResult(status='fail',
+ description='No qualified output '
+ f'(expected {f_reference})')
+
+ for p in (f_bad, f_notrun, f_casenotrun):
+ silent_unlink(p)
+
+ args = [str(f_test.resolve())]
+ if self.env.debug:
+ args.append('-d')
+
+ env = os.environ.copy()
+ env.update(self.test_run_env)
+
+ t0 = time.time()
+ with f_bad.open('w') as f:
+ ret = subprocess.run(args, cwd=str(f_test.parent), env=env,
+ stdout=f, stderr=subprocess.STDOUT).returncode
+ elapsed = round(time.time() - t0, 1)
+
+ if ret != 0:
+ return TestResult(status='fail', elapsed=elapsed,
+ description=f'failed, exit status {ret}',
+ diff=file_diff(str(f_reference), str(f_bad)))
+
+ if f_notrun.exists():
+ return TestResult(status='not run', elapsed=elapsed,
+ description=f_notrun.read_text().strip())
+
+ casenotrun = None
+ if f_casenotrun.exists():
+ casenotrun = f_casenotrun.read_text()
+
+ diff = file_diff(str(f_reference), str(f_bad))
+ if diff:
+ return TestResult(status='fail', elapsed=elapsed,
+ description='output mismatch (see {f_bad})',
+ diff=diff, casenotrun=casenotrun)
+ else:
+ f_bad.unlink()
+ self.last_elapsed.update(test, elapsed)
+ return TestResult(status='pass', elapsed=elapsed,
+ casenotrun=casenotrun)
+
+ def run_test(self, test: str,
+ test_field_width: Optional[int] = None) -> TestResult:
+ last_el = self.last_elapsed.get(test)
+ start = datetime.datetime.now().strftime('%H:%M:%S')
+
+ self.test_print_one_line(test=test, starttime=start, lasttime=last_el,
+ end='\r', test_field_width=test_field_width)
+
+ res = self.do_run_test(test)
+
+ end = datetime.datetime.now().strftime('%H:%M:%S')
+ self.test_print_one_line(test=test, status=res.status,
+ starttime=start, endtime=end,
+ lasttime=last_el, thistime=res.elapsed,
+ description=res.description,
+ test_field_width=test_field_width)
+
+ if res.casenotrun:
+ print(res.casenotrun)
+
+ return res
+
+ def run_tests(self, tests: List[str]) -> None:
+ n_run = 0
+ failed = []
+ notrun = []
+ casenotrun = []
+
+ if not self.makecheck:
+ self.env.print_env()
+ print()
+
+ test_field_width = max(len(os.path.basename(t)) for t in tests) + 2
+
+ for t in tests:
+ name = os.path.basename(t)
+ res = self.run_test(t, test_field_width=test_field_width)
+
+ assert res.status in ('pass', 'fail', 'not run')
+
+ if res.casenotrun:
+ casenotrun.append(t)
+
+ if res.status != 'not run':
+ n_run += 1
+
+ if res.status == 'fail':
+ failed.append(name)
+ if self.makecheck:
+ self.env.print_env()
+ if res.diff:
+ print('\n'.join(res.diff))
+ elif res.status == 'not run':
+ notrun.append(name)
+
+ if notrun:
+ print('Not run:', ' '.join(notrun))
+
+ if casenotrun:
+ print('Some cases not run in:', ' '.join(casenotrun))
+
+ if failed:
+ print('Failures:', ' '.join(failed))
+ print(f'Failed {len(failed)} of {n_run} iotests')
+ else:
+ print(f'Passed all {n_run} iotests')
--
2.29.2
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v6 10/11] iotests: rewrite check into python
2021-01-09 12:26 [PATCH v6 00/11] Rework iotests/check Vladimir Sementsov-Ogievskiy
` (8 preceding siblings ...)
2021-01-09 12:26 ` [PATCH v6 09/11] iotests: add testrunner.py Vladimir Sementsov-Ogievskiy
@ 2021-01-09 12:26 ` Vladimir Sementsov-Ogievskiy
2021-01-12 17:41 ` Kevin Wolf
2021-01-13 23:20 ` Eric Blake
2021-01-09 12:26 ` [PATCH v6 11/11] iotests: rename and move 169 and 199 tests Vladimir Sementsov-Ogievskiy
10 siblings, 2 replies; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-09 12:26 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, vsementsov, jsnow, qemu-devel, mreitz, den
Just use classes introduced in previous three commits. Behavior
difference is described in these three commits.
Drop group file, as it becomes unused.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
tests/qemu-iotests/check | 994 ++-------------------------------------
tests/qemu-iotests/group | 321 -------------
2 files changed, 28 insertions(+), 1287 deletions(-)
delete mode 100644 tests/qemu-iotests/group
diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
index 952762d5ed..48bb3128c3 100755
--- a/tests/qemu-iotests/check
+++ b/tests/qemu-iotests/check
@@ -1,7 +1,8 @@
-#!/usr/bin/env bash
+#!/usr/bin/env python3
#
-# Copyright (C) 2009 Red Hat, Inc.
-# Copyright (c) 2000-2002,2006 Silicon Graphics, Inc. All Rights Reserved.
+# Configure environment and run group of tests in it.
+#
+# Copyright (c) 2020 Virtuozzo International GmbH
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License as
@@ -14,967 +15,28 @@
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-#
-#
-# Control script for QA
-#
-
-status=0
-needwrap=true
-try=0
-n_bad=0
-bad=""
-notrun=""
-casenotrun=""
-interrupt=true
-makecheck=false
-
-_init_error()
-{
- echo "check: $1" >&2
- exit 1
-}
-
-if [ -L "$0" ]
-then
- # called from the build tree
- source_iotests=$(dirname "$(readlink "$0")")
- if [ -z "$source_iotests" ]
- then
- _init_error "failed to obtain source tree name from check symlink"
- fi
- source_iotests=$(cd "$source_iotests"; pwd) || _init_error "failed to enter source tree"
- build_iotests=$(cd "$(dirname "$0")"; pwd)
-else
- # called from the source tree
- source_iotests=$PWD
- # this may be an in-tree build (note that in the following code we may not
- # assume that it truly is and have to test whether the build results
- # actually exist)
- build_iotests=$PWD
-fi
-
-build_root="$build_iotests/../.."
-
-# we need common.env
-if ! . "$build_iotests/common.env"
-then
- _init_error "failed to source common.env (make sure the qemu-iotests are run from tests/qemu-iotests in the build tree)"
-fi
-
-# we need common.config
-if ! . "$source_iotests/common.config"
-then
- _init_error "failed to source common.config"
-fi
-
-_full_imgfmt_details()
-{
- if [ -n "$IMGOPTS" ]; then
- echo "$IMGFMT ($IMGOPTS)"
- else
- echo "$IMGFMT"
- fi
-}
-
-_full_platform_details()
-{
- os=$(uname -s)
- host=$(hostname -s)
- kernel=$(uname -r)
- platform=$(uname -m)
- echo "$os/$platform $host $kernel"
-}
-
-_full_env_details()
-{
- cat <<EOF
-QEMU -- "$QEMU_PROG" $QEMU_OPTIONS
-QEMU_IMG -- "$QEMU_IMG_PROG" $QEMU_IMG_OPTIONS
-QEMU_IO -- "$QEMU_IO_PROG" $QEMU_IO_OPTIONS
-QEMU_NBD -- "$QEMU_NBD_PROG" $QEMU_NBD_OPTIONS
-IMGFMT -- $FULL_IMGFMT_DETAILS
-IMGPROTO -- $IMGPROTO
-PLATFORM -- $FULL_HOST_DETAILS
-TEST_DIR -- $TEST_DIR
-SOCK_DIR -- $SOCK_DIR
-SOCKET_SCM_HELPER -- $SOCKET_SCM_HELPER
-
-EOF
-}
-
-# $1 = prog to look for
-set_prog_path()
-{
- p=$(command -v $1 2> /dev/null)
- if [ -n "$p" -a -x "$p" ]; then
- type -p "$p"
- else
- return 1
- fi
-}
-
-if [ -z "$TEST_DIR" ]; then
- TEST_DIR=$PWD/scratch
-fi
-mkdir -p "$TEST_DIR" || _init_error 'Failed to create TEST_DIR'
-
-tmp_sock_dir=false
-if [ -z "$SOCK_DIR" ]; then
- SOCK_DIR=$(mktemp -d)
- tmp_sock_dir=true
-fi
-mkdir -p "$SOCK_DIR" || _init_error 'Failed to create SOCK_DIR'
-
-diff="diff -u"
-verbose=false
-debug=false
-group=false
-xgroup=false
-imgopts=false
-showme=false
-sortme=false
-expunge=true
-have_test_arg=false
-cachemode=false
-aiomode=false
-
-tmp="${TEST_DIR}"/$$
-rm -f $tmp.list $tmp.tmp $tmp.sed
-
-export IMGFMT=raw
-export IMGFMT_GENERIC=true
-export IMGPROTO=file
-export IMGOPTS=""
-export CACHEMODE="writeback"
-export AIOMODE="threads"
-export QEMU_IO_OPTIONS=""
-export QEMU_IO_OPTIONS_NO_FMT=""
-export CACHEMODE_IS_DEFAULT=true
-export VALGRIND_QEMU=
-export IMGKEYSECRET=
-export IMGOPTSSYNTAX=false
-
-# Save current tty settings, since an aborting qemu call may leave things
-# screwed up
-STTY_RESTORE=
-if test -t 0; then
- STTY_RESTORE=$(stty -g)
-fi
-
-for r
-do
-
- if $group
- then
- # arg after -g
- group_list=$(sed -n <"$source_iotests/group" -e 's/$/ /' -e "/^[0-9][0-9][0-9].* $r /"'{
-s/ .*//p
-}')
- if [ -z "$group_list" ]
- then
- echo "Group \"$r\" is empty or not defined?"
- exit 1
- fi
- [ ! -s $tmp.list ] && touch $tmp.list
- for t in $group_list
- do
- if grep -s "^$t\$" $tmp.list >/dev/null
- then
- :
- else
- echo "$t" >>$tmp.list
- fi
- done
- group=false
- continue
-
- elif $xgroup
- then
- # arg after -x
- # Populate $tmp.list with all tests
- awk '/^[0-9]{3,}/ {print $1}' "${source_iotests}/group" > $tmp.list 2>/dev/null
- group_list=$(sed -n <"$source_iotests/group" -e 's/$/ /' -e "/^[0-9][0-9][0-9].* $r /"'{
-s/ .*//p
-}')
- if [ -z "$group_list" ]
- then
- echo "Group \"$r\" is empty or not defined?"
- exit 1
- fi
- numsed=0
- rm -f $tmp.sed
- for t in $group_list
- do
- if [ $numsed -gt 100 ]
- then
- sed -f $tmp.sed <$tmp.list >$tmp.tmp
- mv $tmp.tmp $tmp.list
- numsed=0
- rm -f $tmp.sed
- fi
- echo "/^$t\$/d" >>$tmp.sed
- numsed=$(expr $numsed + 1)
- done
- sed -f $tmp.sed <$tmp.list >$tmp.tmp
- mv $tmp.tmp $tmp.list
- xgroup=false
- continue
-
- elif $imgopts
- then
- IMGOPTS="$r"
- imgopts=false
- continue
- elif $cachemode
- then
- CACHEMODE="$r"
- CACHEMODE_IS_DEFAULT=false
- cachemode=false
- continue
- elif $aiomode
- then
- AIOMODE="$r"
- aiomode=false
- continue
- fi
-
- xpand=true
- case "$r"
- in
-
- -\? | -h | --help) # usage
- echo "Usage: $0 [options] [testlist]"'
-
-common options
- -v verbose
- -d debug
-
-image format options
- -raw test raw (default)
- -bochs test bochs
- -cloop test cloop
- -parallels test parallels
- -qcow test qcow
- -qcow2 test qcow2
- -qed test qed
- -vdi test vdi
- -vpc test vpc
- -vhdx test vhdx
- -vmdk test vmdk
- -luks test luks
- -dmg test dmg
-
-image protocol options
- -file test file (default)
- -rbd test rbd
- -sheepdog test sheepdog
- -nbd test nbd
- -fuse test fuse
- -ssh test ssh
- -nfs test nfs
-
-other options
- -xdiff graphical mode diff
- -nocache use O_DIRECT on backing file
- -misalign misalign memory allocations
- -n show me, do not run tests
- -o options -o options to pass to qemu-img create/convert
- -c mode cache mode
- -i mode AIO mode
- -makecheck pretty print output for make check
-
-testlist options
- -g group[,group...] include tests from these groups
- -x group[,group...] exclude tests from these groups
- NNN include test NNN
- NNN-NNN include test range (eg. 012-021)
-'
- exit 0
- ;;
-
- -raw)
- IMGFMT=raw
- xpand=false
- ;;
-
- -bochs)
- IMGFMT=bochs
- IMGFMT_GENERIC=false
- xpand=false
- ;;
-
- -cloop)
- IMGFMT=cloop
- IMGFMT_GENERIC=false
- xpand=false
- ;;
-
- -parallels)
- IMGFMT=parallels
- xpand=false
- ;;
-
- -qcow)
- IMGFMT=qcow
- xpand=false
- ;;
-
- -qcow2)
- IMGFMT=qcow2
- xpand=false
- ;;
-
- -luks)
- IMGOPTSSYNTAX=true
- IMGFMT=luks
- IMGKEYSECRET=123456
- xpand=false
- ;;
-
- -dmg)
- IMGFMT=dmg
- IMGFMT_GENERIC=false
- xpand=false
- ;;
-
- -qed)
- IMGFMT=qed
- xpand=false
- ;;
-
- -vdi)
- IMGFMT=vdi
- xpand=false
- ;;
-
- -vmdk)
- IMGFMT=vmdk
- xpand=false
- ;;
-
- -vpc)
- IMGFMT=vpc
- xpand=false
- ;;
-
- -vhdx)
- IMGFMT=vhdx
- xpand=false
- ;;
-
- -file)
- IMGPROTO=file
- xpand=false
- ;;
-
- -rbd)
- IMGPROTO=rbd
- xpand=false
- ;;
-
- -sheepdog)
- IMGPROTO=sheepdog
- xpand=false
- ;;
-
- -nbd)
- IMGPROTO=nbd
- xpand=false
- ;;
-
- -fuse)
- IMGPROTO=fuse
- xpand=false
- ;;
-
- -ssh)
- IMGPROTO=ssh
- xpand=false
- ;;
-
- -nfs)
- IMGPROTO=nfs
- xpand=false
- ;;
-
- -nocache)
- CACHEMODE="none"
- CACHEMODE_IS_DEFAULT=false
- xpand=false
- ;;
-
- -misalign)
- QEMU_IO_OPTIONS="$QEMU_IO_OPTIONS --misalign"
- xpand=false
- ;;
-
- -valgrind)
- VALGRIND_QEMU='y'
- xpand=false
- ;;
-
- -g) # -g group ... pick from group file
- group=true
- xpand=false
- ;;
-
- -xdiff) # graphical diff mode
- xpand=false
-
- if [ ! -z "$DISPLAY" ]
- then
- command -v xdiff >/dev/null 2>&1 && diff=xdiff
- command -v gdiff >/dev/null 2>&1 && diff=gdiff
- command -v tkdiff >/dev/null 2>&1 && diff=tkdiff
- command -v xxdiff >/dev/null 2>&1 && diff=xxdiff
- fi
- ;;
- -makecheck) # makecheck friendly output
- makecheck=true
- xpand=false
- ;;
- -n) # show me, don't do it
- showme=true
- xpand=false
- ;;
- -o)
- imgopts=true
- xpand=false
- ;;
- -c)
- cachemode=true
- xpand=false
- ;;
- -i)
- aiomode=true
- xpand=false
- ;;
- -T) # deprecated timestamp option
- xpand=false
- ;;
- -v)
- verbose=true
- xpand=false
- ;;
- -d)
- debug=true
- xpand=false
- ;;
- -x) # -x group ... exclude from group file
- xgroup=true
- xpand=false
- ;;
- '[0-9][0-9][0-9] [0-9][0-9][0-9][0-9]')
- echo "No tests?"
- status=1
- exit $status
- ;;
-
- [0-9]*-[0-9]*)
- eval $(echo $r | sed -e 's/^/start=/' -e 's/-/ end=/')
- ;;
-
- [0-9]*-)
- eval $(echo $r | sed -e 's/^/start=/' -e 's/-//')
- end=$(echo [0-9][0-9][0-9] [0-9][0-9][0-9][0-9] | sed -e 's/\[0-9]//g' -e 's/ *$//' -e 's/.* //')
- if [ -z "$end" ]
- then
- echo "No tests in range \"$r\"?"
- status=1
- exit $status
- fi
- ;;
-
- *)
- start=$r
- end=$r
- ;;
-
- esac
-
- # get rid of leading 0s as can be interpreted as octal
- start=$(echo $start | sed 's/^0*//')
- end=$(echo $end | sed 's/^0*//')
-
- if $xpand
- then
- have_test_arg=true
- awk </dev/null '
-BEGIN { for (t='$start'; t<='$end'; t++) printf "%03d\n",t }' \
- | while read id
- do
- if grep -s "^$id\( \|\$\)" "$source_iotests/group" >/dev/null
- then
- # in group file ... OK
- echo $id >>$tmp.list
- else
- if [ -f expunged ] && $expunge && egrep "^$id([ ]|\$)" expunged >/dev/null
- then
- # expunged ... will be reported, but not run, later
- echo $id >>$tmp.list
- else
- # oops
- if [ "$start" == "$end" -a "$id" == "$end" ]
- then
- echo "$id - unknown test"
- exit 1
- else
- echo "$id - unknown test, ignored"
- fi
- fi
- fi
- done || exit 1
- fi
-
-done
-
-# Set qemu-io cache mode with $CACHEMODE we have
-QEMU_IO_OPTIONS="$QEMU_IO_OPTIONS --cache $CACHEMODE"
-# Set qemu-io aio mode with $AIOMODE we have
-QEMU_IO_OPTIONS="$QEMU_IO_OPTIONS --aio $AIOMODE"
-
-QEMU_IO_OPTIONS_NO_FMT="$QEMU_IO_OPTIONS"
-if [ "$IMGOPTSSYNTAX" != "true" ]; then
- QEMU_IO_OPTIONS="$QEMU_IO_OPTIONS -f $IMGFMT"
-fi
-
-# Set default options for qemu-img create -o if they were not specified
-if [ "$IMGFMT" == "qcow2" ] && ! (echo "$IMGOPTS" | grep "compat=" > /dev/null); then
- IMGOPTS=$(_optstr_add "$IMGOPTS" "compat=1.1")
-fi
-if [ "$IMGFMT" == "luks" ] && ! (echo "$IMGOPTS" | grep "iter-time=" > /dev/null); then
- IMGOPTS=$(_optstr_add "$IMGOPTS" "iter-time=10")
-fi
-if [ "$IMGFMT" == "vmdk" ] && ! (echo "$IMGOPTS" | grep "zeroed_grain=" > /dev/null); then
- IMGOPTS=$(_optstr_add "$IMGOPTS" "zeroed_grain=on")
-fi
-
-if [ -z "$SAMPLE_IMG_DIR" ]; then
- SAMPLE_IMG_DIR="$source_iotests/sample_images"
-fi
-
-export TEST_DIR
-export SOCK_DIR
-export SAMPLE_IMG_DIR
-
-if [ -s $tmp.list ]
-then
- # found some valid test numbers ... this is good
- :
-else
- if $have_test_arg
- then
- # had test numbers, but none in group file ... do nothing
- touch $tmp.list
- else
- # no test numbers, do everything from group file
- sed -n -e '/^[0-9][0-9][0-9]*/s/^\([0-9]*\).*/\1/p' <"$source_iotests/group" >$tmp.list
- fi
-fi
-
-# should be sort -n, but this did not work for Linux when this
-# was ported from IRIX
-#
-list=$(sort $tmp.list)
-rm -f $tmp.list $tmp.tmp $tmp.sed
-
-if [ -z "$QEMU_PROG" ]
-then
- if [ -x "$build_iotests/qemu" ]; then
- export QEMU_PROG="$build_iotests/qemu"
- elif [ -x "$build_root/qemu-system-${qemu_arch}" ]; then
- export QEMU_PROG="$build_root/qemu-system-${qemu_arch}"
- else
- pushd "$build_root" > /dev/null
- for binary in qemu-system-*
- do
- if [ -x "$binary" ]
- then
- export QEMU_PROG="$build_root/$binary"
- break
- fi
- done
- popd > /dev/null
- [ "$QEMU_PROG" = "" ] && _init_error "qemu not found"
- fi
-fi
-export QEMU_PROG="$(type -p "$QEMU_PROG")"
-
-export QEMU_OPTIONS="-nodefaults -display none -accel qtest"
-case "$QEMU_PROG" in
- *qemu-system-arm|*qemu-system-aarch64)
- export QEMU_OPTIONS="$QEMU_OPTIONS -machine virt"
- ;;
- *qemu-system-avr)
- export QEMU_OPTIONS="$QEMU_OPTIONS -machine mega2560"
- ;;
- *qemu-system-rx)
- export QEMU_OPTIONS="$QEMU_OPTIONS -machine gdbsim-r5f562n8"
- ;;
- *qemu-system-tricore)
- export QEMU_OPTIONS="-$QEMU_OPTIONS -machine tricore_testboard"
- ;;
-esac
-
-if [ -z "$QEMU_IMG_PROG" ]; then
- if [ -x "$build_iotests/qemu-img" ]; then
- export QEMU_IMG_PROG="$build_iotests/qemu-img"
- elif [ -x "$build_root/qemu-img" ]; then
- export QEMU_IMG_PROG="$build_root/qemu-img"
- else
- _init_error "qemu-img not found"
- fi
-fi
-export QEMU_IMG_PROG="$(type -p "$QEMU_IMG_PROG")"
-
-if [ -z "$QEMU_IO_PROG" ]; then
- if [ -x "$build_iotests/qemu-io" ]; then
- export QEMU_IO_PROG="$build_iotests/qemu-io"
- elif [ -x "$build_root/qemu-io" ]; then
- export QEMU_IO_PROG="$build_root/qemu-io"
- else
- _init_error "qemu-io not found"
- fi
-fi
-export QEMU_IO_PROG="$(type -p "$QEMU_IO_PROG")"
-
-if [ -z $QEMU_NBD_PROG ]; then
- if [ -x "$build_iotests/qemu-nbd" ]; then
- export QEMU_NBD_PROG="$build_iotests/qemu-nbd"
- elif [ -x "$build_root/qemu-nbd" ]; then
- export QEMU_NBD_PROG="$build_root/qemu-nbd"
- else
- _init_error "qemu-nbd not found"
- fi
-fi
-export QEMU_NBD_PROG="$(type -p "$QEMU_NBD_PROG")"
-
-if [ -z "$QSD_PROG" ]; then
- if [ -x "$build_iotests/qemu-storage-daemon" ]; then
- export QSD_PROG="$build_iotests/qemu-storage-daemon"
- elif [ -x "$build_root/storage-daemon/qemu-storage-daemon" ]; then
- export QSD_PROG="$build_root/storage-daemon/qemu-storage-daemon"
- else
- _init_error "qemu-storage-daemon not found"
- fi
-fi
-export QSD_PROG="$(type -p "$QSD_PROG")"
-
-if [ -x "$build_iotests/socket_scm_helper" ]
-then
- export SOCKET_SCM_HELPER="$build_iotests/socket_scm_helper"
-fi
-
-python_usable=false
-if $PYTHON -c 'import sys; sys.exit(0 if sys.version_info >= (3,6) else 1)'
-then
- # Our python framework also requires virtio-blk
- if "$QEMU_PROG" -M none -device help | grep -q virtio-blk >/dev/null 2>&1
- then
- python_usable=true
- else
- python_unusable_because="Missing virtio-blk in QEMU binary"
- fi
-else
- python_unusable_because="Unsupported Python version"
-fi
-
-default_machine=$($QEMU_PROG -machine help | sed -n '/(default)/ s/ .*//p')
-default_alias_machine=$($QEMU_PROG -machine help | \
- sed -n "/(alias of $default_machine)/ { s/ .*//p; q; }")
-if [[ "$default_alias_machine" ]]; then
- default_machine="$default_alias_machine"
-fi
-
-export QEMU_DEFAULT_MACHINE="$default_machine"
-
-TIMESTAMP_FILE=check.time-$IMGPROTO-$IMGFMT
-
-_wallclock()
-{
- date "+%H %M %S" | awk '{ print $1*3600 + $2*60 + $3 }'
-}
-
-_wrapup()
-{
- if $showme
- then
- :
- elif $needwrap
- then
- if [ -f $TIMESTAMP_FILE -a -f $tmp.time ]
- then
- cat $TIMESTAMP_FILE $tmp.time \
- | awk '
- { t[$1] = $2 }
-END { if (NR > 0) {
- for (i in t) print i " " t[i]
- }
- }' \
- | sort -n >$tmp.out
- mv $tmp.out $TIMESTAMP_FILE
- fi
-
- if [ -f $tmp.expunged ]
- then
- notrun=$(wc -l <$tmp.expunged | sed -e 's/ *//g')
- try=$(expr $try - $notrun)
- list=$(echo "$list" | sed -f $tmp.expunged)
- fi
-
- echo "" >>check.log
- date >>check.log
- echo $list | fmt | sed -e 's/^/ /' >>check.log
- $interrupt && echo "Interrupted!" >>check.log
-
- if [ ! -z "$notrun" ]
- then
- echo "Not run:$notrun"
- echo "Not run:$notrun" >>check.log
- fi
- if [ ! -z "$casenotrun" ]
- then
- echo "Some cases not run in:$casenotrun"
- echo "Some cases not run in:$casenotrun" >>check.log
- fi
- if [ ! -z "$n_bad" -a $n_bad != 0 ]
- then
- echo "Failures:$bad"
- echo "Failed $n_bad of $try iotests"
- echo "Failures:$bad" | fmt >>check.log
- echo "Failed $n_bad of $try iotests" >>check.log
- else
- echo "Passed all $try iotests"
- echo "Passed all $try iotests" >>check.log
- fi
- needwrap=false
- fi
-
- if test -n "$STTY_RESTORE"; then
- stty $STTY_RESTORE
- fi
- rm -f "${TEST_DIR}"/*.out "${TEST_DIR}"/*.err "${TEST_DIR}"/*.time
- rm -f "${TEST_DIR}"/check.pid "${TEST_DIR}"/check.sts
- rm -f $tmp.*
-
- if $tmp_sock_dir
- then
- rm -rf "$SOCK_DIR"
- fi
-}
-
-trap "_wrapup; exit \$status" 0 1 2 3 15
-
-# Report the test start and results. For makecheck we want to pretty
-# print the whole report at the end of the execution.
-# args: $seq, $starttime, $lasttime
-_report_test_start()
-{
- if ! $makecheck; then
- if [ -n "$3" ]; then
- local lasttime=" (last: $3s)"
- fi
- printf "%-8s %-10s [%s] %4s%-14s\r" "$1" "..." "$2" "..." "$lasttime"
- fi
-}
-# args:$seq $status $starttime $lasttime $thistime $details
-_report_test_result()
-{
- local status lasttime thistime
- if $makecheck; then
- if [ -n "$2" ] && [ "$2" != "pass" ]; then
- status=" [$2]"
- fi
- printf " TEST iotest-$IMGFMT: %s%s\n" "$1" "$status"
- return
- fi
-
- if [ -n "$4" ]; then
- lasttime=" (last: $4s)"
- fi
- if [ -n "$5" ]; then
- thistime=" $5s"
- fi
- case "$2" in
- "pass") status=$(printf "\e[32m%-10s\e[0m" "$2") ;;
- "fail") status=$(printf "\e[1m\e[31m%-10s\e[0m" "$2") ;;
- "not run") status=$(printf "\e[33m%-10s\e[0m" "$2") ;;
- *) status=$(printf "%-10s" "$2") ;;
- esac
-
- printf "%-8s %s [%s] [%s] %4s%-14s %s\n" "$1" "$status" "$3" "$(date '+%T')" "$thistime" "$lasttime" "$6"
-}
-
-[ -f $TIMESTAMP_FILE ] || touch $TIMESTAMP_FILE
-
-FULL_IMGFMT_DETAILS=$(_full_imgfmt_details)
-FULL_HOST_DETAILS=$(_full_platform_details)
-
-if ! $makecheck; then
- _full_env_details
-fi
-
-seq="check"
-
-[ -n "$TESTS_REMAINING_LOG" ] && echo $list > $TESTS_REMAINING_LOG
-
-for seq in $list
-do
- err=false # error flag
- printdiff=false # show diff to reference output?
- status="" # test result summary
- results="" # test result details
- thistime="" # time the test took
-
- if [ -n "$TESTS_REMAINING_LOG" ] ; then
- sed -e "s/$seq//" -e 's/ / /' -e 's/^ *//' $TESTS_REMAINING_LOG > $TESTS_REMAINING_LOG.tmp
- mv $TESTS_REMAINING_LOG.tmp $TESTS_REMAINING_LOG
- sync
- fi
-
- lasttime=$(sed -n -e "/^$seq /s/.* //p" <$TIMESTAMP_FILE)
- starttime=$(date "+%T")
- _report_test_start $seq $starttime $lasttime
-
- if $showme
- then
- status="not run"
- elif [ -f expunged ] && $expunge && egrep "^$seq([ ]|\$)" expunged >/dev/null
- then
- status="not run"
- results="expunged"
- rm -f $seq.out.bad
- echo "/^$seq\$/d" >>$tmp.expunged
- elif [ ! -f "$source_iotests/$seq" ]
- then
- status="not run"
- results="no such test?"
- echo "/^$seq\$/d" >>$tmp.expunged
- else
- # really going to try and run this one
- #
- rm -f $seq.out.bad
- rm -f core $seq.notrun
- rm -f $seq.casenotrun
-
- start=$(_wallclock)
-
- if [ "$(head -n 1 "$source_iotests/$seq")" == "#!/usr/bin/env python3" ]; then
- if $python_usable; then
- run_command="$PYTHON $seq"
- else
- run_command="false"
- echo "$python_unusable_because" > $seq.notrun
- fi
- else
- run_command="./$seq"
- fi
- export OUTPUT_DIR=$PWD
- if $debug; then
- (cd "$source_iotests";
- MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(($RANDOM % 255 + 1))} \
- $run_command -d 2>&1 | tee $tmp.out)
- else
- (cd "$source_iotests";
- MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(($RANDOM % 255 + 1))} \
- $run_command >$tmp.out 2>&1)
- fi
- sts=$?
- stop=$(_wallclock)
-
- if [ -f core ]
- then
- mv core $seq.core
- status="fail"
- results="[dumped core] $seq.core"
- err=true
- fi
-
- if [ -f $seq.notrun ]
- then
- # overwrites timestamp output
- status="not run"
- results="$(cat $seq.notrun)"
- else
- if [ $sts -ne 0 ]
- then
- status="fail"
- results=$(printf %s "[failed, exit status $sts]")
- err=true
- fi
-
- reference="$source_iotests/$seq.out"
- reference_machine="$source_iotests/$seq.$QEMU_DEFAULT_MACHINE.out"
- if [ -f "$reference_machine" ]; then
- reference="$reference_machine"
- fi
-
- reference_format="$source_iotests/$seq.out.$IMGFMT"
- if [ -f "$reference_format" ]; then
- reference="$reference_format"
- fi
-
- if [ "$CACHEMODE" = "none" ]; then
- [ -f "$source_iotests/$seq.out.nocache" ] && reference="$source_iotests/$seq.out.nocache"
- fi
-
- if [ ! -f "$reference" ]
- then
- status="fail"
- results="no qualified output"
- err=true
- else
- if diff -w "$reference" $tmp.out >/dev/null 2>&1
- then
- if ! $err; then
- status="pass"
- thistime=$(expr $stop - $start)
- echo "$seq $thistime" >>$tmp.time
- fi
- else
- mv $tmp.out $seq.out.bad
- status="fail"
- results="output mismatch (see $seq.out.bad)"
- printdiff=true
- err=true
- fi
- fi
- fi
- if [ -f $seq.casenotrun ]
- then
- cat $seq.casenotrun
- casenotrun="$casenotrun $seq"
- fi
- fi
-
- # come here for each test, except when $showme is true
- #
- _report_test_result $seq "$status" "$starttime" "$lasttime" "$thistime" "$results"
- case "$status" in
- "pass")
- try=$(expr $try + 1)
- ;;
- "fail")
- try=$(expr $try + 1)
- if $makecheck; then
- _full_env_details
- fi
- if $printdiff; then
- $diff -w "$reference" "$PWD"/$seq.out.bad
- fi
- bad="$bad $seq"
- n_bad=$(expr $n_bad + 1)
- quick=false
- ;;
- "not run")
- notrun="$notrun $seq"
- ;;
- esac
-
- seq="after_$seq"
-done
-interrupt=false
-status=$(expr $n_bad)
-exit
+import sys
+import os
+from findtests import find_tests, TestFinder
+from testenv import TestEnv
+from testrunner import TestRunner
+
+if __name__ == '__main__':
+ if len(sys.argv) == 2 and sys.argv[1] in ['-h', '--help']:
+ print('Usage: ./check [options] [testlist]')
+ print()
+ TestFinder.get_argparser().print_help()
+ print()
+ TestEnv.get_argparser().print_help()
+ print()
+ TestRunner.get_argparser().print_help()
+ exit()
+
+ env = TestEnv(sys.argv[1:])
+ tests, remaining_argv = find_tests(env.remaining_argv,
+ test_dir=env.source_iotests)
+
+ with TestRunner(remaining_argv, env) as tr:
+ assert not tr.remaining_argv
+ tr.run_tests([os.path.join(env.source_iotests, t) for t in tests])
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
deleted file mode 100644
index bc5bc324fe..0000000000
--- a/tests/qemu-iotests/group
+++ /dev/null
@@ -1,321 +0,0 @@
-#
-# QA groups control file
-# Defines test groups
-#
-# Some notes about the groups:
-#
-# - do not start group names with a digit
-#
-# - quick : Tests in this group should finish within some few seconds.
-#
-# - img : Tests in this group can be used to excercise the qemu-img tool.
-#
-# - auto : Tests in this group are used during "make check" and should be
-# runnable in any case. That means they should run with every QEMU binary
-# (also non-x86), with every QEMU configuration (i.e. must not fail if
-# an optional feature is not compiled in - but reporting a "skip" is ok),
-# work at least with the qcow2 file format, work with all kind of host
-# filesystems and users (e.g. "nobody" or "root") and must not take too
-# much memory and disk space (since CI pipelines tend to fail otherwise).
-#
-
-#
-# test-group association ... one line per test
-#
-001 rw auto quick
-002 rw auto quick
-003 rw auto
-004 rw auto quick
-005 img auto quick
-# 006 was removed, do not reuse
-007 snapshot auto
-008 rw auto quick
-009 rw auto quick
-010 rw auto quick
-011 rw auto quick
-012 auto quick
-013 rw auto
-014 rw
-015 rw snapshot
-# 016 was removed, do not reuse
-017 rw backing auto quick
-018 rw backing auto quick
-019 rw backing auto quick
-020 rw backing auto quick
-021 io auto quick
-022 rw snapshot auto
-023 rw
-024 rw backing auto quick
-025 rw auto quick
-026 rw blkdbg
-027 rw auto quick
-028 rw backing quick
-029 rw auto quick
-030 rw auto backing
-031 rw auto quick
-032 rw auto quick
-033 rw auto quick
-034 rw auto backing quick
-035 rw auto quick
-036 rw auto quick
-037 rw auto backing quick
-038 rw auto backing quick
-039 rw auto quick
-040 rw auto
-041 rw auto backing
-042 rw auto quick
-043 rw auto backing
-044 rw
-045 rw quick
-046 rw auto aio quick
-047 rw auto quick
-048 img auto quick
-049 rw auto
-050 rw auto backing quick
-051 rw
-052 rw auto backing quick
-053 rw auto quick
-054 rw auto quick
-055 rw
-056 rw backing
-057 rw
-058 rw quick
-059 rw quick
-060 rw auto quick
-061 rw auto
-062 rw auto quick
-063 rw auto quick
-064 rw quick
-065 rw quick
-066 rw auto quick
-# 067 was removed, do not reuse
-068 rw quick
-069 rw auto quick
-070 rw quick
-071 rw auto quick
-072 rw auto quick
-073 rw auto quick
-074 rw auto quick
-075 rw quick
-076 io
-077 rw quick
-078 rw quick
-079 rw auto
-080 rw auto
-081 rw quick
-082 rw quick
-083 rw
-084 img quick
-085 rw
-086 rw auto quick
-087 rw quick
-088 rw quick
-089 rw auto quick
-090 rw auto quick
-091 rw migration quick
-092 rw quick
-093 throttle
-094 rw quick
-095 rw quick
-096 rw quick
-097 rw auto backing
-098 rw auto backing quick
-099 rw auto quick
-# 100 was removed, do not reuse
-101 rw quick
-102 rw quick
-103 rw auto quick
-104 rw auto
-105 rw auto quick
-106 rw quick
-107 rw auto quick
-108 rw auto quick
-109 rw
-110 rw auto backing quick
-111 rw auto quick
-112 rw
-113 rw quick
-114 rw auto quick
-115 rw
-116 rw quick
-117 rw auto
-118 rw
-119 rw quick
-120 rw auto quick
-121 rw
-122 rw
-123 rw quick
-124 rw backing
-125 rw
-126 rw auto backing
-127 rw auto backing quick
-128 rw quick
-129 rw quick
-130 rw quick
-131 rw quick
-132 rw quick
-133 auto quick
-134 rw auto quick
-135 rw
-136 rw
-137 rw auto
-138 rw auto quick
-139 rw quick
-140 rw auto quick
-141 rw auto quick
-142
-143 auto quick
-144 rw quick
-145 quick
-146 quick
-147 img
-148 rw quick
-149 rw sudo
-150 rw auto quick
-151 rw
-152 rw quick
-153 rw quick
-154 rw auto backing quick
-155 rw
-156 rw auto quick
-157 quick
-158 rw auto quick
-159 rw auto quick
-160 rw quick
-161 rw auto quick
-162 quick
-163 rw
-165 rw quick
-169 rw migration
-170 rw auto quick
-171 rw quick
-172 auto
-173 rw
-174 auto
-175 quick
-176 rw auto backing
-177 rw auto quick
-178 img
-179 rw auto quick
-181 rw auto migration quick
-182 rw quick
-183 rw migration quick
-184 rw auto quick
-185 rw
-186 rw auto
-187 rw auto
-188 rw quick
-189 rw
-190 rw auto quick
-191 rw auto
-192 rw auto quick
-194 rw migration quick
-195 rw auto quick
-196 rw quick migration
-197 rw quick
-198 rw
-199 rw migration
-200 rw
-201 rw migration quick
-202 rw quick
-203 rw auto migration quick
-204 rw quick
-205 rw quick
-206 rw
-207 rw
-208 rw quick
-209 rw quick
-210 rw
-211 rw quick
-212 rw quick
-213 rw quick
-214 rw auto
-215 rw quick
-216 rw quick
-217 rw auto quick
-218 rw quick
-219 rw
-220 rw auto
-221 rw quick
-222 rw quick
-223 rw quick
-224 rw quick
-225 rw quick
-226 auto quick
-227 quick
-228 rw quick
-229 auto quick
-231 quick
-232 quick
-233 quick
-234 quick migration
-235 quick
-236 quick
-237 rw quick
-238 quick
-239 rw quick
-240 quick
-241 rw quick
-242 rw quick
-243 rw quick
-244 rw auto quick
-245 rw
-246 rw quick
-247 rw quick
-248 rw quick
-249 rw auto quick
-250 rw quick
-251 rw auto quick
-252 rw auto backing quick
-253 rw quick
-254 rw backing quick
-255 rw quick
-256 rw auto quick
-257 rw
-258 rw quick
-259 rw auto quick
-260 rw quick
-261 rw
-262 rw quick migration
-263 rw quick
-264 rw
-265 rw auto quick
-266 rw quick
-267 rw auto quick snapshot
-268 rw auto quick
-270 rw backing quick
-271 rw auto
-272 rw
-273 backing quick
-274 rw backing
-277 rw quick
-279 rw backing quick
-280 rw migration quick
-281 rw quick
-282 rw img quick
-283 auto quick
-284 rw
-286 rw quick
-287 auto quick
-288 quick
-289 rw quick
-290 rw auto quick
-291 rw quick
-292 rw auto quick
-293 rw
-294 rw quick
-295 rw
-296 rw
-297 meta
-298
-299 auto quick
-300 migration
-301 backing quick
-302 quick
-303 rw quick
-304 rw quick
-305 rw quick
-307 rw quick export
-308 rw
-309 rw auto quick
-312 rw quick
--
2.29.2
^ permalink raw reply related [flat|nested] 43+ messages in thread* Re: [PATCH v6 10/11] iotests: rewrite check into python
2021-01-09 12:26 ` [PATCH v6 10/11] iotests: rewrite check into python Vladimir Sementsov-Ogievskiy
@ 2021-01-12 17:41 ` Kevin Wolf
2021-01-13 23:20 ` Eric Blake
1 sibling, 0 replies; 43+ messages in thread
From: Kevin Wolf @ 2021-01-12 17:41 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy; +Cc: qemu-block, qemu-devel, mreitz, den, jsnow
Am 09.01.2021 um 13:26 hat Vladimir Sementsov-Ogievskiy geschrieben:
> Just use classes introduced in previous three commits. Behavior
> difference is described in these three commits.
>
> Drop group file, as it becomes unused.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> tests/qemu-iotests/check | 994 ++-------------------------------------
> tests/qemu-iotests/group | 321 -------------
> 2 files changed, 28 insertions(+), 1287 deletions(-)
> delete mode 100644 tests/qemu-iotests/group
> +import sys
> +import os
> +from findtests import find_tests, TestFinder
> +from testenv import TestEnv
> +from testrunner import TestRunner
> +
> +if __name__ == '__main__':
> + if len(sys.argv) == 2 and sys.argv[1] in ['-h', '--help']:
> + print('Usage: ./check [options] [testlist]')
> + print()
> + TestFinder.get_argparser().print_help()
> + print()
> + TestEnv.get_argparser().print_help()
> + print()
> + TestRunner.get_argparser().print_help()
> + exit()
+check:34:8: R1722: Consider using sys.exit() (consider-using-sys-exit)
> +
> + env = TestEnv(sys.argv[1:])
> + tests, remaining_argv = find_tests(env.remaining_argv,
> + test_dir=env.source_iotests)
> +
> + with TestRunner(remaining_argv, env) as tr:
> + assert not tr.remaining_argv
> + tr.run_tests([os.path.join(env.source_iotests, t) for t in tests])
The assertion means that giving an unknown option results in an error
message like this:
$ build/check -T -raw
Traceback (most recent call last):
File "/home/kwolf/source/qemu/tests/qemu-iotests/build/check", line 41, in <module>
assert not tr.remaining_argv
AssertionError
I think this could be a bit friendlier (especially since this doesn't
even tell you which of your options was wrong).
Kevin
^ permalink raw reply [flat|nested] 43+ messages in thread* Re: [PATCH v6 10/11] iotests: rewrite check into python
2021-01-09 12:26 ` [PATCH v6 10/11] iotests: rewrite check into python Vladimir Sementsov-Ogievskiy
2021-01-12 17:41 ` Kevin Wolf
@ 2021-01-13 23:20 ` Eric Blake
2021-01-14 0:18 ` John Snow
1 sibling, 1 reply; 43+ messages in thread
From: Eric Blake @ 2021-01-13 23:20 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, den, jsnow, qemu-devel, mreitz
On 1/9/21 6:26 AM, Vladimir Sementsov-Ogievskiy wrote:
> Just use classes introduced in previous three commits. Behavior
> difference is described in these three commits.
>
> Drop group file, as it becomes unused.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> tests/qemu-iotests/check | 994 ++-------------------------------------
> tests/qemu-iotests/group | 321 -------------
> 2 files changed, 28 insertions(+), 1287 deletions(-)
> delete mode 100644 tests/qemu-iotests/group
The bulk of the work was done in the earlier patches, and my python
review is weak; but I can confirm that with this patch applied, my usual
attempts at running ./check still appeared to work for me. That's not
the same as proving you did 1:1 feature translation (and in fact, your
commit message documented dropping some features like -v), but if 'make
check' and CI tools still run, I'm okay leaving it up to developers to
complain about any other feature that they used but which go missing (or
to implement it).
Tested-by: Eric Blake <eblake@redhat.com>
>
> diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
> index 952762d5ed..48bb3128c3 100755
> --- a/tests/qemu-iotests/check
> +++ b/tests/qemu-iotests/check
> @@ -1,7 +1,8 @@
> -#!/usr/bin/env bash
> +#!/usr/bin/env python3
> #
> -# Copyright (C) 2009 Red Hat, Inc.
> -# Copyright (c) 2000-2002,2006 Silicon Graphics, Inc. All Rights Reserved.
> +# Configure environment and run group of tests in it.
> +#
> +# Copyright (c) 2020 Virtuozzo International GmbH
You may want to claim 2021 as well.
> +import sys
> +import os
> +from findtests import find_tests, TestFinder
> +from testenv import TestEnv
> +from testrunner import TestRunner
> +
> +if __name__ == '__main__':
> + if len(sys.argv) == 2 and sys.argv[1] in ['-h', '--help']:
> + print('Usage: ./check [options] [testlist]')
> + print()
> + TestFinder.get_argparser().print_help()
> + print()
> + TestEnv.get_argparser().print_help()
> + print()
> + TestRunner.get_argparser().print_help()
> + exit()
> +
> + env = TestEnv(sys.argv[1:])
> + tests, remaining_argv = find_tests(env.remaining_argv,
> + test_dir=env.source_iotests)
> +
> + with TestRunner(remaining_argv, env) as tr:
> + assert not tr.remaining_argv
> + tr.run_tests([os.path.join(env.source_iotests, t) for t in tests])
A lot shorter for the main engine ;)
> diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
> deleted file mode 100644
> index bc5bc324fe..0000000000
> --- a/tests/qemu-iotests/group
> +++ /dev/null
> @@ -1,321 +0,0 @@
> -#
> -# QA groups control file
> -# Defines test groups
Happy to see this conflict magnet go!
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 43+ messages in thread* Re: [PATCH v6 10/11] iotests: rewrite check into python
2021-01-13 23:20 ` Eric Blake
@ 2021-01-14 0:18 ` John Snow
0 siblings, 0 replies; 43+ messages in thread
From: John Snow @ 2021-01-14 0:18 UTC (permalink / raw)
To: Eric Blake, Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, den, qemu-devel, mreitz
On 1/13/21 6:20 PM, Eric Blake wrote:
> On 1/9/21 6:26 AM, Vladimir Sementsov-Ogievskiy wrote:
>> Just use classes introduced in previous three commits. Behavior
>> difference is described in these three commits.
>>
>> Drop group file, as it becomes unused.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>> tests/qemu-iotests/check | 994 ++-------------------------------------
>> tests/qemu-iotests/group | 321 -------------
>> 2 files changed, 28 insertions(+), 1287 deletions(-)
>> delete mode 100644 tests/qemu-iotests/group
>
> The bulk of the work was done in the earlier patches, and my python
> review is weak; but I can confirm that with this patch applied, my usual
> attempts at running ./check still appeared to work for me. That's not
> the same as proving you did 1:1 feature translation (and in fact, your
> commit message documented dropping some features like -v), but if 'make
> check' and CI tools still run, I'm okay leaving it up to developers to
> complain about any other feature that they used but which go missing (or
> to implement it).
>
> Tested-by: Eric Blake <eblake@redhat.com>
>
I will *try* to give this a look, but I am still buried under post-PTO
mail; and I feel comfortable with Kevin's Python review otherwise, so
don't hold your breath waiting to hear from me.
>>
>> diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
>> index 952762d5ed..48bb3128c3 100755
>> --- a/tests/qemu-iotests/check
>> +++ b/tests/qemu-iotests/check
>> @@ -1,7 +1,8 @@
>> -#!/usr/bin/env bash
>> +#!/usr/bin/env python3
>> #
>> -# Copyright (C) 2009 Red Hat, Inc.
>> -# Copyright (c) 2000-2002,2006 Silicon Graphics, Inc. All Rights Reserved.
>> +# Configure environment and run group of tests in it.
>> +#
>> +# Copyright (c) 2020 Virtuozzo International GmbH
>
> You may want to claim 2021 as well.
>
>> +import sys
>> +import os
>> +from findtests import find_tests, TestFinder
>> +from testenv import TestEnv
>> +from testrunner import TestRunner
>> +
>> +if __name__ == '__main__':
>> + if len(sys.argv) == 2 and sys.argv[1] in ['-h', '--help']:
>> + print('Usage: ./check [options] [testlist]')
>> + print()
>> + TestFinder.get_argparser().print_help()
>> + print()
>> + TestEnv.get_argparser().print_help()
>> + print()
>> + TestRunner.get_argparser().print_help()
>> + exit()
>> +
>> + env = TestEnv(sys.argv[1:])
>> + tests, remaining_argv = find_tests(env.remaining_argv,
>> + test_dir=env.source_iotests)
>> +
>> + with TestRunner(remaining_argv, env) as tr:
>> + assert not tr.remaining_argv
>> + tr.run_tests([os.path.join(env.source_iotests, t) for t in tests])
>
> A lot shorter for the main engine ;)
>
>> diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
>> deleted file mode 100644
>> index bc5bc324fe..0000000000
>> --- a/tests/qemu-iotests/group
>> +++ /dev/null
>> @@ -1,321 +0,0 @@
>> -#
>> -# QA groups control file
>> -# Defines test groups
>
> Happy to see this conflict magnet go!
>
This rules.
--js
^ permalink raw reply [flat|nested] 43+ messages in thread
* [PATCH v6 11/11] iotests: rename and move 169 and 199 tests
2021-01-09 12:26 [PATCH v6 00/11] Rework iotests/check Vladimir Sementsov-Ogievskiy
` (9 preceding siblings ...)
2021-01-09 12:26 ` [PATCH v6 10/11] iotests: rewrite check into python Vladimir Sementsov-Ogievskiy
@ 2021-01-09 12:26 ` Vladimir Sementsov-Ogievskiy
10 siblings, 0 replies; 43+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-09 12:26 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, vsementsov, jsnow, qemu-devel, mreitz, den
Rename bitmaps migration tests and move them to tests subdirectory to
demonstrate new human-friendly test naming.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
This patch is and RFC and may be postponed. Or may be applied as well..
tests/qemu-iotests/{199 => tests/migrate-bitmaps-postcopy-test} | 0
.../{199.out => tests/migrate-bitmaps-postcopy-test.out} | 0
tests/qemu-iotests/{169 => tests/migrate-bitmaps-test} | 0
tests/qemu-iotests/{169.out => tests/migrate-bitmaps-test.out} | 0
4 files changed, 0 insertions(+), 0 deletions(-)
rename tests/qemu-iotests/{199 => tests/migrate-bitmaps-postcopy-test} (100%)
rename tests/qemu-iotests/{199.out => tests/migrate-bitmaps-postcopy-test.out} (100%)
rename tests/qemu-iotests/{169 => tests/migrate-bitmaps-test} (100%)
rename tests/qemu-iotests/{169.out => tests/migrate-bitmaps-test.out} (100%)
diff --git a/tests/qemu-iotests/199 b/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test
similarity index 100%
rename from tests/qemu-iotests/199
rename to tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test
diff --git a/tests/qemu-iotests/199.out b/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test.out
similarity index 100%
rename from tests/qemu-iotests/199.out
rename to tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test.out
diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/tests/migrate-bitmaps-test
similarity index 100%
rename from tests/qemu-iotests/169
rename to tests/qemu-iotests/tests/migrate-bitmaps-test
diff --git a/tests/qemu-iotests/169.out b/tests/qemu-iotests/tests/migrate-bitmaps-test.out
similarity index 100%
rename from tests/qemu-iotests/169.out
rename to tests/qemu-iotests/tests/migrate-bitmaps-test.out
--
2.29.2
^ permalink raw reply [flat|nested] 43+ messages in thread