* migration-test random intermittent failure, openbsd VM
@ 2024-02-08 17:09 Peter Maydell
2024-02-08 18:04 ` Fabiano Rosas
0 siblings, 1 reply; 5+ messages in thread
From: Peter Maydell @ 2024-02-08 17:09 UTC (permalink / raw)
To: QEMU Developers; +Cc: Peter Xu, Fabiano Rosas
Random intermittent in migration-test when running the tests
in the openbsd VM (i.e. what you get from 'make -C build vm-build-openbsd')
Any ideas?
106/847 qemu:qtest+qtest-x86_64 / qtest-x86_64/migration-test
ERROR 157.51s killed by signal 6 SIGABRT
――――――――――――――――――――――――――――――――――――― ✀ ―――――――――――――――――――――――――――――――――――――
stderr:
qemu-system-x86_64: multifd_send_sync_main: channel 12 has already quit
qemu-system-x86_64: TLS handshake failed: The TLS connection was
non-properly terminated.
qemu-system-x86_64: TLS handshake failed: Error in the push function.
qemu-system-x86_64: Failed to connect to '127.0.0.1:23083': Address
already in use
qemu-system-x86_64: TLS handshake failed: Error in the push function.
qemu-system-x86_64: TLS handshake failed: Error in the push function.
qemu-system-x86_64: TLS handshake failed: Error in the push function.
qemu-system-x86_64: TLS handshake failed: Error in the push function.
**
ERROR:../src/tests/qtest/migration-helpers.c:180:check_migration_status:
assertion failed (current_status != "failed"): ("failed" != "failed")
qemu-system-x86_64: TLS handshake failed: Error in the push function.
(test program exited with status code -6)
thanks
-- PMM
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: migration-test random intermittent failure, openbsd VM
2024-02-08 17:09 migration-test random intermittent failure, openbsd VM Peter Maydell
@ 2024-02-08 18:04 ` Fabiano Rosas
2024-02-08 18:14 ` Daniel P. Berrangé
2024-02-08 20:09 ` Peter Maydell
0 siblings, 2 replies; 5+ messages in thread
From: Fabiano Rosas @ 2024-02-08 18:04 UTC (permalink / raw)
To: Peter Maydell, QEMU Developers; +Cc: Peter Xu, Daniel P. Berrangé
Peter Maydell <peter.maydell@linaro.org> writes:
+cc Daniel.
> Random intermittent in migration-test when running the tests
> in the openbsd VM (i.e. what you get from 'make -C build vm-build-openbsd')
> Any ideas?
Where's your HEAD at?
>
> 106/847 qemu:qtest+qtest-x86_64 / qtest-x86_64/migration-test
> ERROR 157.51s killed by signal 6 SIGABRT
> ――――――――――――――――――――――――――――――――――――― ✀ ―――――――――――――――――――――――――――――――――――――
> stderr:
> qemu-system-x86_64: multifd_send_sync_main: channel 12 has already quit
> qemu-system-x86_64: TLS handshake failed: The TLS connection was
> non-properly terminated.
> qemu-system-x86_64: TLS handshake failed: Error in the push function.
> qemu-system-x86_64: Failed to connect to '127.0.0.1:23083': Address
> already in use
Looks like a legitimate port clash here when doing the TLS
connection. It seems the test is not prepared to deal with it.
> qemu-system-x86_64: TLS handshake failed: Error in the push function.
> qemu-system-x86_64: TLS handshake failed: Error in the push function.
> qemu-system-x86_64: TLS handshake failed: Error in the push function.
> qemu-system-x86_64: TLS handshake failed: Error in the push function.
> **
> ERROR:../src/tests/qtest/migration-helpers.c:180:check_migration_status:
> assertion failed (current_status != "failed"): ("failed" != "failed")
> qemu-system-x86_64: TLS handshake failed: Error in the push function.
>
> (test program exited with status code -6)
>
>
> thanks
> -- PMM
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: migration-test random intermittent failure, openbsd VM
2024-02-08 18:04 ` Fabiano Rosas
@ 2024-02-08 18:14 ` Daniel P. Berrangé
2024-02-08 20:09 ` Peter Maydell
1 sibling, 0 replies; 5+ messages in thread
From: Daniel P. Berrangé @ 2024-02-08 18:14 UTC (permalink / raw)
To: Fabiano Rosas; +Cc: Peter Maydell, QEMU Developers, Peter Xu
On Thu, Feb 08, 2024 at 03:04:23PM -0300, Fabiano Rosas wrote:
> Peter Maydell <peter.maydell@linaro.org> writes:
>
> +cc Daniel.
>
> > Random intermittent in migration-test when running the tests
> > in the openbsd VM (i.e. what you get from 'make -C build vm-build-openbsd')
> > Any ideas?
>
> Where's your HEAD at?
>
> >
> > 106/847 qemu:qtest+qtest-x86_64 / qtest-x86_64/migration-test
> > ERROR 157.51s killed by signal 6 SIGABRT
> > ――――――――――――――――――――――――――――――――――――― ✀ ―――――――――――――――――――――――――――――――――――――
> > stderr:
> > qemu-system-x86_64: multifd_send_sync_main: channel 12 has already quit
> > qemu-system-x86_64: TLS handshake failed: The TLS connection was
> > non-properly terminated.
> > qemu-system-x86_64: TLS handshake failed: Error in the push function.
> > qemu-system-x86_64: Failed to connect to '127.0.0.1:23083': Address
> > already in use
>
> Looks like a legitimate port clash here when doing the TLS
> connection. It seems the test is not prepared to deal with it.
The migration-test.c code always says to listen on ':0' so the
kernel dynamically allocates an address, we when query that
port and connect to it. So we should never fail on listen/bind
server side.
The error message though is about the client side as it says
"failed to connect".
I've no idea how on earth you trigger "Address already in use" as
a client !
Every outbound connection does require a port, but QEMU always
leave the port on 0, so the kenrel dynamically allocates an
outbound port. Somehow this is failing, but I can't see how
it is QEMU's fault ?!?!?
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: migration-test random intermittent failure, openbsd VM
2024-02-08 18:04 ` Fabiano Rosas
2024-02-08 18:14 ` Daniel P. Berrangé
@ 2024-02-08 20:09 ` Peter Maydell
2024-02-15 16:53 ` Fabiano Rosas
1 sibling, 1 reply; 5+ messages in thread
From: Peter Maydell @ 2024-02-08 20:09 UTC (permalink / raw)
To: Fabiano Rosas; +Cc: QEMU Developers, Peter Xu, Daniel P. Berrangé
On Thu, 8 Feb 2024 at 18:04, Fabiano Rosas <farosas@suse.de> wrote:
>
> Peter Maydell <peter.maydell@linaro.org> writes:
>
> +cc Daniel.
>
> > Random intermittent in migration-test when running the tests
> > in the openbsd VM (i.e. what you get from 'make -C build vm-build-openbsd')
> > Any ideas?
>
> Where's your HEAD at?
03e4bc0bc02 in this case.
thanks
-- PMM
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: migration-test random intermittent failure, openbsd VM
2024-02-08 20:09 ` Peter Maydell
@ 2024-02-15 16:53 ` Fabiano Rosas
0 siblings, 0 replies; 5+ messages in thread
From: Fabiano Rosas @ 2024-02-15 16:53 UTC (permalink / raw)
To: Peter Maydell; +Cc: QEMU Developers, Peter Xu, Daniel P. Berrangé
Peter Maydell <peter.maydell@linaro.org> writes:
> On Thu, 8 Feb 2024 at 18:04, Fabiano Rosas <farosas@suse.de> wrote:
>>
>> Peter Maydell <peter.maydell@linaro.org> writes:
>>
>> +cc Daniel.
>>
>> > Random intermittent in migration-test when running the tests
>> > in the openbsd VM (i.e. what you get from 'make -C build vm-build-openbsd')
>> > Any ideas?
>>
>> Where's your HEAD at?
>
> 03e4bc0bc02 in this case.
>
> thanks
> -- PMM
I spent some time on this today, it reproduces inconsistently and I
couldn't find much.
The only thing I noticed so far is that when the tests fail, netstat
shows several ports at TIME_WAIT TCP state, including the one that just
got rejected. Maybe some issue with how we're closing these sockets
vs. the openbsd network stack implementation. But that's a long shot
without more investigation.
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2024-02-15 16:54 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-02-08 17:09 migration-test random intermittent failure, openbsd VM Peter Maydell
2024-02-08 18:04 ` Fabiano Rosas
2024-02-08 18:14 ` Daniel P. Berrangé
2024-02-08 20:09 ` Peter Maydell
2024-02-15 16:53 ` Fabiano Rosas
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.