[Qemu-devel] iotest 223 is flaky

[Qemu-devel] iotest 223 is flaky

[Alberto Garcia]

I can make it fail by running it a few times:

223 1s ... - output mismatch (see 223.out.bad)
--- tests/qemu-iotests/223.out       2019-02-08 17:25:15.584387100 +0200
+++ tests/qemu-iotests/223.out.bad     2019-03-05 18:05:16.855223976 +0200
@@ -92,9 +92,10 @@
 
 === Use qemu-nbd as server ===
 
-[{ "start": 0, "length": 65536, "depth": 0, "zero": false, "data": false},
-{ "start": 65536, "length": 2031616, "depth": 0, "zero": false, "data": true},
-{ "start": 2097152, "length": 2097152, "depth": 0, "zero": false, "data": false}]
+qemu-nbd: Failed to blk_new_open 'tests/qemu-iotests/scratch/t.qcow2': Failed to get shared "write" lock
+Is another process using the image [tests/qemu-iotests/scratch/t.qcow2]?
+qemu-img: Could not open 'driver=nbd,server.type=unix,server.path=tests/qemu-iotests/scratch/qemu-nbd.sock,x-dirty-bitmap=qemu:dirty-bitmap:b': Failed to connect socket tests/qemu-iotests/scratch/qemu-nbd.sock: Connection refused
+./common.nbd: line 33: kill: (11122) - No such process
 [{ "start": 0, "length": 512, "depth": 0, "zero": false, "data": true},
 { "start": 512, "length": 512, "depth": 0, "zero": false, "data": false},
 { "start": 1024, "length": 2096128, "depth": 0, "zero": false, "data": true},

Berto

Re: [Qemu-devel] iotest 223 is flaky

[Eric Blake]

On 3/5/19 10:10 AM, Alberto Garcia wrote:
> I can make it fail by running it a few times:
> 
> 223 1s ... - output mismatch (see 223.out.bad)
> --- tests/qemu-iotests/223.out       2019-02-08 17:25:15.584387100 +0200
> +++ tests/qemu-iotests/223.out.bad     2019-03-05 18:05:16.855223976 +0200
> @@ -92,9 +92,10 @@
>   
>   === Use qemu-nbd as server ===
>   
> -[{ "start": 0, "length": 65536, "depth": 0, "zero": false, "data": false},
> -{ "start": 65536, "length": 2031616, "depth": 0, "zero": false, "data": true},
> -{ "start": 2097152, "length": 2097152, "depth": 0, "zero": false, "data": false}]
> +qemu-nbd: Failed to blk_new_open 'tests/qemu-iotests/scratch/t.qcow2': Failed to get shared "write" lock
> +Is another process using the image [tests/qemu-iotests/scratch/t.qcow2]?

I haven't been able to reproduce it yet, but see what might be the problem:

_send_qemu_cmd $QEMU_HANDLE '{"execute":"quit"}' "return"

echo
echo "=== Use qemu-nbd as server ==="
echo

nbd_server_start_unix_socket -r -f $IMGFMT -B b "$TEST_IMG"

There was nothing in the qemu process to let go of t.qcow2 prior to the 
quit request. And although the quit command may have had time to reply 
over the socket that the command was accepted, there may still be a 
delay before the process actually dies, where under sufficient load, the 
qemu-nbd process can start up during that window.

Patch coming up; since I couldn't reproduce the failure, I'd appreciate 
if you could confirm if it seems to help in your setup.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org