* [PATCH] qemu-iotests: move check-block back to Makefiles
@ 2020-09-02 8:00 Paolo Bonzini
2020-09-02 8:19 ` Daniel P. Berrangé
2020-09-02 16:51 ` Alex Bennée
0 siblings, 2 replies; 6+ messages in thread
From: Paolo Bonzini @ 2020-09-02 8:00 UTC (permalink / raw)
To: qemu-devel; +Cc: alex.bennee, qemu-block
check-block has its own test harness, unlike every other test. If
we capture its output, as is in general nicer to do without V=1,
there will be no sign of progress. So for lack of a better option
just move the invocation of the test back to Makefile rules.
As a side effect, this will also fix "make check" in --disable-tools
builds, as they were trying to run qemu-iotests without having
made qemu-img before.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
meson.build | 2 --
tests/Makefile.include | 15 ++++++++++++---
tests/qemu-iotests/meson.build | 4 ----
3 files changed, 12 insertions(+), 9 deletions(-)
diff --git a/meson.build b/meson.build
index 55c7d2318c..356af9142c 100644
--- a/meson.build
+++ b/meson.build
@@ -1095,11 +1095,9 @@ if have_tools
dependencies: [authz, block, crypto, io, qom, qemuutil], install: true)
qemu_io = executable('qemu-io', files('qemu-io.c'),
dependencies: [block, qemuutil], install: true)
- qemu_block_tools += [qemu_img, qemu_io]
if targetos != 'windows'
qemu_nbd = executable('qemu-nbd', files('qemu-nbd.c'),
dependencies: [block, qemuutil], install: true)
- qemu_block_tools += [qemu_nbd]
endif
subdir('storage-daemon')
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 9ac8f5b86a..08301f5bc9 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -468,7 +468,6 @@ check-tcg: $(RUN_TCG_TARGET_RULES)
.PHONY: clean-tcg
clean-tcg: $(CLEAN_TCG_TARGET_RULES)
-
# Python venv for running tests
.PHONY: check-venv check-acceptance
@@ -523,8 +522,18 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images
# Consolidated targets
.PHONY: check-block check-unit check check-clean get-vm-images
-check-block:
-check-build: build-unit
+check:
+
+ifeq ($(CONFIG_TOOLS)$(CONFIG_POSIX),yy)
+QEMU_IOTESTS_HELPERS-$(CONFIG_LINUX) = tests/qemu-iotests/socket_scm_helper$(EXESUF)
+check: check-block
+check-block: $(SRC_PATH)/tests/check-block.sh qemu-img$(EXESUF) \
+ qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y) \
+ $(patsubst %-softmmu,qemu-system-%,$(filter %-softmmu,$(TARGET_DIRS)))
+ @$<
+endif
+
+check-build: build-unit $(QEMU_IOTESTS_HELPERS-y)
check-clean:
rm -rf $(check-unit-y) tests/*.o tests/*/*.o $(QEMU_IOTESTS_HELPERS-y)
diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build
index 3de09fb8fa..60470936b4 100644
--- a/tests/qemu-iotests/meson.build
+++ b/tests/qemu-iotests/meson.build
@@ -4,7 +4,3 @@ if 'CONFIG_LINUX' in config_host
else
socket_scm_helper = []
endif
-test('qemu-iotests', sh, args: [files('../check-block.sh')],
- depends: [qemu_block_tools, emulators, socket_scm_helper],
- suite: 'block', timeout: 10000)
-
--
2.26.2
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] qemu-iotests: move check-block back to Makefiles
2020-09-02 8:00 [PATCH] qemu-iotests: move check-block back to Makefiles Paolo Bonzini
@ 2020-09-02 8:19 ` Daniel P. Berrangé
2020-09-02 8:37 ` Paolo Bonzini
2020-09-02 16:51 ` Alex Bennée
1 sibling, 1 reply; 6+ messages in thread
From: Daniel P. Berrangé @ 2020-09-02 8:19 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: alex.bennee, qemu-devel, qemu-block
On Wed, Sep 02, 2020 at 04:00:46AM -0400, Paolo Bonzini wrote:
> check-block has its own test harness, unlike every other test. If
> we capture its output, as is in general nicer to do without V=1,
> there will be no sign of progress. So for lack of a better option
> just move the invocation of the test back to Makefile rules.
I expect the correct long term solution here is to stop using the
check-block.sh script. Instead have code which sets up each
of the I/O tests as an explicit test target in meson. We could
use meson's test grouping features too.
>
> As a side effect, this will also fix "make check" in --disable-tools
> builds, as they were trying to run qemu-iotests without having
> made qemu-img before.
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
> meson.build | 2 --
> tests/Makefile.include | 15 ++++++++++++---
> tests/qemu-iotests/meson.build | 4 ----
> 3 files changed, 12 insertions(+), 9 deletions(-)
>
> diff --git a/meson.build b/meson.build
> index 55c7d2318c..356af9142c 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -1095,11 +1095,9 @@ if have_tools
> dependencies: [authz, block, crypto, io, qom, qemuutil], install: true)
> qemu_io = executable('qemu-io', files('qemu-io.c'),
> dependencies: [block, qemuutil], install: true)
> - qemu_block_tools += [qemu_img, qemu_io]
> if targetos != 'windows'
> qemu_nbd = executable('qemu-nbd', files('qemu-nbd.c'),
> dependencies: [block, qemuutil], install: true)
> - qemu_block_tools += [qemu_nbd]
> endif
>
> subdir('storage-daemon')
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 9ac8f5b86a..08301f5bc9 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -468,7 +468,6 @@ check-tcg: $(RUN_TCG_TARGET_RULES)
> .PHONY: clean-tcg
> clean-tcg: $(CLEAN_TCG_TARGET_RULES)
>
> -
> # Python venv for running tests
>
> .PHONY: check-venv check-acceptance
> @@ -523,8 +522,18 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images
> # Consolidated targets
>
> .PHONY: check-block check-unit check check-clean get-vm-images
> -check-block:
> -check-build: build-unit
> +check:
> +
> +ifeq ($(CONFIG_TOOLS)$(CONFIG_POSIX),yy)
> +QEMU_IOTESTS_HELPERS-$(CONFIG_LINUX) = tests/qemu-iotests/socket_scm_helper$(EXESUF)
> +check: check-block
> +check-block: $(SRC_PATH)/tests/check-block.sh qemu-img$(EXESUF) \
> + qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y) \
> + $(patsubst %-softmmu,qemu-system-%,$(filter %-softmmu,$(TARGET_DIRS)))
> + @$<
> +endif
> +
> +check-build: build-unit $(QEMU_IOTESTS_HELPERS-y)
>
> check-clean:
> rm -rf $(check-unit-y) tests/*.o tests/*/*.o $(QEMU_IOTESTS_HELPERS-y)
> diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build
> index 3de09fb8fa..60470936b4 100644
> --- a/tests/qemu-iotests/meson.build
> +++ b/tests/qemu-iotests/meson.build
> @@ -4,7 +4,3 @@ if 'CONFIG_LINUX' in config_host
> else
> socket_scm_helper = []
> endif
> -test('qemu-iotests', sh, args: [files('../check-block.sh')],
> - depends: [qemu_block_tools, emulators, socket_scm_helper],
> - suite: 'block', timeout: 10000)
> -
> --
> 2.26.2
>
>
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] qemu-iotests: move check-block back to Makefiles
2020-09-02 8:19 ` Daniel P. Berrangé
@ 2020-09-02 8:37 ` Paolo Bonzini
2020-09-02 10:21 ` Thomas Huth
0 siblings, 1 reply; 6+ messages in thread
From: Paolo Bonzini @ 2020-09-02 8:37 UTC (permalink / raw)
To: Daniel P. Berrangé; +Cc: alex.bennee, qemu-devel, qemu-block
On 02/09/20 10:19, Daniel P. Berrangé wrote:
> On Wed, Sep 02, 2020 at 04:00:46AM -0400, Paolo Bonzini wrote:
>> check-block has its own test harness, unlike every other test. If
>> we capture its output, as is in general nicer to do without V=1,
>> there will be no sign of progress. So for lack of a better option
>> just move the invocation of the test back to Makefile rules.
>
> I expect the correct long term solution here is to stop using the
> check-block.sh script. Instead have code which sets up each
> of the I/O tests as an explicit test target in meson. We could
> use meson's test grouping features too.
I'm not sure, "check-acceptance" will never be integrated in Meson, and
it may well be the same for "check-block". Actually I wonder if Avocado
would be a better check-block.sh than check-block.sh.
Paolo
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] qemu-iotests: move check-block back to Makefiles
2020-09-02 8:37 ` Paolo Bonzini
@ 2020-09-02 10:21 ` Thomas Huth
2020-09-03 15:51 ` Lukáš Doktor
0 siblings, 1 reply; 6+ messages in thread
From: Thomas Huth @ 2020-09-02 10:21 UTC (permalink / raw)
To: Paolo Bonzini, Daniel P. Berrangé
Cc: Lukas Doktor, qemu-block, qemu-devel, Wainer Moschetta,
Cleber Rosa, alex.bennee
On 02/09/2020 10.37, Paolo Bonzini wrote:
> On 02/09/20 10:19, Daniel P. Berrangé wrote:
>> On Wed, Sep 02, 2020 at 04:00:46AM -0400, Paolo Bonzini wrote:
>>> check-block has its own test harness, unlike every other test. If
>>> we capture its output, as is in general nicer to do without V=1,
>>> there will be no sign of progress. So for lack of a better option
>>> just move the invocation of the test back to Makefile rules.
>>
>> I expect the correct long term solution here is to stop using the
>> check-block.sh script. Instead have code which sets up each
>> of the I/O tests as an explicit test target in meson. We could
>> use meson's test grouping features too.
>
> I'm not sure, "check-acceptance" will never be integrated in Meson, and
> it may well be the same for "check-block". Actually I wonder if Avocado
> would be a better check-block.sh than check-block.sh.
Wasn't there even some support for the iotests in avocado (or
avocado-vt) at one point in time? ... not sure anymore, Cleber, Wainer,
do you remember?
Thomas
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] qemu-iotests: move check-block back to Makefiles
2020-09-02 8:00 [PATCH] qemu-iotests: move check-block back to Makefiles Paolo Bonzini
2020-09-02 8:19 ` Daniel P. Berrangé
@ 2020-09-02 16:51 ` Alex Bennée
1 sibling, 0 replies; 6+ messages in thread
From: Alex Bennée @ 2020-09-02 16:51 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: qemu-devel, qemu-block
Paolo Bonzini <pbonzini@redhat.com> writes:
> check-block has its own test harness, unlike every other test. If
> we capture its output, as is in general nicer to do without V=1,
> there will be no sign of progress. So for lack of a better option
> just move the invocation of the test back to Makefile rules.
>
> As a side effect, this will also fix "make check" in --disable-tools
> builds, as they were trying to run qemu-iotests without having
> made qemu-img before.
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
I've pulled it into my testing/next tree but feel free to include in
your next PR if you beat me to it.
> ---
> meson.build | 2 --
> tests/Makefile.include | 15 ++++++++++++---
> tests/qemu-iotests/meson.build | 4 ----
> 3 files changed, 12 insertions(+), 9 deletions(-)
>
> diff --git a/meson.build b/meson.build
> index 55c7d2318c..356af9142c 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -1095,11 +1095,9 @@ if have_tools
> dependencies: [authz, block, crypto, io, qom, qemuutil], install: true)
> qemu_io = executable('qemu-io', files('qemu-io.c'),
> dependencies: [block, qemuutil], install: true)
> - qemu_block_tools += [qemu_img, qemu_io]
> if targetos != 'windows'
> qemu_nbd = executable('qemu-nbd', files('qemu-nbd.c'),
> dependencies: [block, qemuutil], install: true)
> - qemu_block_tools += [qemu_nbd]
> endif
>
> subdir('storage-daemon')
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 9ac8f5b86a..08301f5bc9 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -468,7 +468,6 @@ check-tcg: $(RUN_TCG_TARGET_RULES)
> .PHONY: clean-tcg
> clean-tcg: $(CLEAN_TCG_TARGET_RULES)
>
> -
> # Python venv for running tests
>
> .PHONY: check-venv check-acceptance
> @@ -523,8 +522,18 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images
> # Consolidated targets
>
> .PHONY: check-block check-unit check check-clean get-vm-images
> -check-block:
> -check-build: build-unit
> +check:
> +
> +ifeq ($(CONFIG_TOOLS)$(CONFIG_POSIX),yy)
> +QEMU_IOTESTS_HELPERS-$(CONFIG_LINUX) = tests/qemu-iotests/socket_scm_helper$(EXESUF)
> +check: check-block
> +check-block: $(SRC_PATH)/tests/check-block.sh qemu-img$(EXESUF) \
> + qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y) \
> + $(patsubst %-softmmu,qemu-system-%,$(filter %-softmmu,$(TARGET_DIRS)))
> + @$<
> +endif
> +
> +check-build: build-unit $(QEMU_IOTESTS_HELPERS-y)
>
> check-clean:
> rm -rf $(check-unit-y) tests/*.o tests/*/*.o $(QEMU_IOTESTS_HELPERS-y)
> diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build
> index 3de09fb8fa..60470936b4 100644
> --- a/tests/qemu-iotests/meson.build
> +++ b/tests/qemu-iotests/meson.build
> @@ -4,7 +4,3 @@ if 'CONFIG_LINUX' in config_host
> else
> socket_scm_helper = []
> endif
> -test('qemu-iotests', sh, args: [files('../check-block.sh')],
> - depends: [qemu_block_tools, emulators, socket_scm_helper],
> - suite: 'block', timeout: 10000)
> -
--
Alex Bennée
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] qemu-iotests: move check-block back to Makefiles
2020-09-02 10:21 ` Thomas Huth
@ 2020-09-03 15:51 ` Lukáš Doktor
0 siblings, 0 replies; 6+ messages in thread
From: Lukáš Doktor @ 2020-09-03 15:51 UTC (permalink / raw)
To: Thomas Huth, Paolo Bonzini, Daniel P. Berrangé
Cc: Wainer Moschetta, alex.bennee, qemu-devel, qemu-block, Cleber Rosa
[-- Attachment #1.1: Type: text/plain, Size: 2261 bytes --]
Dne 02. 09. 20 v 12:21 Thomas Huth napsal(a):
> On 02/09/2020 10.37, Paolo Bonzini wrote:
>> On 02/09/20 10:19, Daniel P. Berrangé wrote:
>>> On Wed, Sep 02, 2020 at 04:00:46AM -0400, Paolo Bonzini wrote:
>>>> check-block has its own test harness, unlike every other test. If
>>>> we capture its output, as is in general nicer to do without V=1,
>>>> there will be no sign of progress. So for lack of a better option
>>>> just move the invocation of the test back to Makefile rules.
>>>
>>> I expect the correct long term solution here is to stop using the
>>> check-block.sh script. Instead have code which sets up each
>>> of the I/O tests as an explicit test target in meson. We could
>>> use meson's test grouping features too.
>>
>> I'm not sure, "check-acceptance" will never be integrated in Meson, and
>> it may well be the same for "check-block". Actually I wonder if Avocado
>> would be a better check-block.sh than check-block.sh.
>
> Wasn't there even some support for the iotests in avocado (or
> avocado-vt) at one point in time? ... not sure anymore, Cleber, Wainer,
> do you remember?
>
> Thomas
>
Avocado supports so called "external runner", which allows to specify a command to be executed and arguments as different variants so it was just a matter of coming up with the list of "./check" invocation like "./check -qcow2 -nbd 001". All of these were executed as a separate test and reported PASS/FAIL. Actually I created a wrapper, which also checked the output of the "./check" and allowed the test to result in "WARN" which I then used for reporting skipped tests.
Cleber, is there a better way of running qemu-iotests? I know Drew had a plan on adding kvm-unit-test runner which would support kvm-unit-tests out of the box including the proper setup, but I'm not aware of any such initiative for qemu-iotests.
Regards,
Lukáš
PS: Now I remembered that I actually contributed the kvm-unit-test wrapper upstream, it's slightly outdated compare to the downstream version but could work as an example (if you're interested I can try to synchronize it with the current downstream implementation): https://github.com/avocado-framework/avocado/blob/master/contrib/testsuites/run-kvm-unit-test.sh
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-09-03 15:54 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-02 8:00 [PATCH] qemu-iotests: move check-block back to Makefiles Paolo Bonzini
2020-09-02 8:19 ` Daniel P. Berrangé
2020-09-02 8:37 ` Paolo Bonzini
2020-09-02 10:21 ` Thomas Huth
2020-09-03 15:51 ` Lukáš Doktor
2020-09-02 16:51 ` Alex Bennée
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.