[PATCH] iotests/testrunner: Flush after run_test()

[PATCH] iotests/testrunner: Flush after run_test()

[Hanna Reitz]

When stdout is not a terminal, the buffer may not be flushed at each end
of line, so we should flush after each test is done.  This is especially
apparent when run by check-block, in two ways:

First, when running make check-block -jX with X > 1, progress indication
was missing, even though testrunner.py does theoretically print each
test's status once it has been run, even in multi-processing mode.
Flushing after each test restores this progress indication.

Second, sometimes make check-block failed altogether, with an error
message that "too few tests [were] run".  I presume that's because one
worker process in the job pool did not get to flush its stdout before
the main process exited, and so meson did not get to see that worker's
test results.  In any case, by flushing at the end of run_test(), the
problem has disappeared for me.

Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 tests/qemu-iotests/testrunner.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py
index aae70a8341..10d9e8ef27 100644
--- a/tests/qemu-iotests/testrunner.py
+++ b/tests/qemu-iotests/testrunner.py
@@ -378,6 +378,7 @@ def run_test(self, test: str,
             else:
                 print(res.casenotrun)
 
+        sys.stdout.flush()
         return res
 
     def run_tests(self, tests: List[str], jobs: int = 1) -> bool:
-- 
2.35.1

Re: [PATCH] iotests/testrunner: Flush after run_test()

[Eric Blake]

On Fri, May 06, 2022 at 03:42:15PM +0200, Hanna Reitz wrote:
> When stdout is not a terminal, the buffer may not be flushed at each end
> of line, so we should flush after each test is done.  This is especially
> apparent when run by check-block, in two ways:
> 
> First, when running make check-block -jX with X > 1, progress indication
> was missing, even though testrunner.py does theoretically print each
> test's status once it has been run, even in multi-processing mode.
> Flushing after each test restores this progress indication.
> 
> Second, sometimes make check-block failed altogether, with an error
> message that "too few tests [were] run".  I presume that's because one
> worker process in the job pool did not get to flush its stdout before
> the main process exited, and so meson did not get to see that worker's
> test results.  In any case, by flushing at the end of run_test(), the
> problem has disappeared for me.
> 
> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
> ---
>  tests/qemu-iotests/testrunner.py | 1 +
>  1 file changed, 1 insertion(+)

Reviewed-by: Eric Blake <eblake@redhat.com>

> 
> diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py
> index aae70a8341..10d9e8ef27 100644
> --- a/tests/qemu-iotests/testrunner.py
> +++ b/tests/qemu-iotests/testrunner.py
> @@ -378,6 +378,7 @@ def run_test(self, test: str,
>              else:
>                  print(res.casenotrun)
>  
> +        sys.stdout.flush()
>          return res
>  
>      def run_tests(self, tests: List[str], jobs: int = 1) -> bool:
> -- 
> 2.35.1
> 
> 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [PATCH] iotests/testrunner: Flush after run_test()

[Kevin Wolf]

Am 06.05.2022 um 15:42 hat Hanna Reitz geschrieben:
> When stdout is not a terminal, the buffer may not be flushed at each end
> of line, so we should flush after each test is done.  This is especially
> apparent when run by check-block, in two ways:
> 
> First, when running make check-block -jX with X > 1, progress indication
> was missing, even though testrunner.py does theoretically print each
> test's status once it has been run, even in multi-processing mode.
> Flushing after each test restores this progress indication.
> 
> Second, sometimes make check-block failed altogether, with an error
> message that "too few tests [were] run".  I presume that's because one
> worker process in the job pool did not get to flush its stdout before
> the main process exited, and so meson did not get to see that worker's
> test results.  In any case, by flushing at the end of run_test(), the
> problem has disappeared for me.
> 
> Signed-off-by: Hanna Reitz <hreitz@redhat.com>

Thanks, applied to the block branch.

Kevin