* Re: random crash in migration-test, i386 guest, x86-64 host:
2021-07-19 18:56 ` Peter Maydell
@ 2021-07-20 8:08 ` Dr. David Alan Gilbert
2021-07-20 10:33 ` Dr. David Alan Gilbert
1 sibling, 0 replies; 4+ messages in thread
From: Dr. David Alan Gilbert @ 2021-07-20 8:08 UTC (permalink / raw)
To: Peter Maydell; +Cc: QEMU Developers, Juan Quintela
* Peter Maydell (peter.maydell@linaro.org) wrote:
> On Mon, 19 Jul 2021 at 13:43, Peter Maydell <peter.maydell@linaro.org> wrote:
> >
> > Random crash in migration-tests:
> >
> > MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}
> > QTEST_QEMU_IMG=./qemu-img G_TEST_DBUS_DAEMON=/mnt/nvmedisk/linaro/qem
> > u-for-merges/tests/dbus-vmstate-daemon.sh
> > QTEST_QEMU_BINARY=./qemu-system-i386
> > QTEST_QEMU_STORAGE_DAEMON_BINARY=./storage-daemon/qemu-
> > storage-daemon tests/qtest/migration-test --tap -k
> > PASS 1 qtest-i386/migration-test /i386/migration/bad_dest
> > PASS 2 qtest-i386/migration-test /i386/migration/fd_proto
> > PASS 3 qtest-i386/migration-test /i386/migration/validate_uuid
> > PASS 4 qtest-i386/migration-test /i386/migration/validate_uuid_error
> > PASS 5 qtest-i386/migration-test /i386/migration/validate_uuid_src_not_set
> > PASS 6 qtest-i386/migration-test /i386/migration/validate_uuid_dst_not_set
> > PASS 7 qtest-i386/migration-test /i386/migration/auto_converge
> > PASS 8 qtest-i386/migration-test /i386/migration/postcopy/unix
> > Broken pipe
> > ../../tests/qtest/libqtest.c:166: kill_qemu() detected QEMU death from
> > signal 6 (Aborted) (core dumped)
> > ERROR qtest-i386/migration-test - too few tests run (expected 15, got 8)
> > Makefile.mtest:280: recipe for target 'run-test-33' failed
> > make: *** [run-test-33] Error 1
>
> And again, different guest architecture, still x86-64 host:
>
> MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}
> QTEST_QEMU_IMG=./qemu-img G_TEST_DBUS_DAEMON=/mnt/nvmedisk/linaro/qem
> u-for-merges/tests/dbus-vmstate-daemon.sh
> QTEST_QEMU_BINARY=./qemu-system-aarch64
> QTEST_QEMU_STORAGE_DAEMON_BINARY=./storage-daemon/qe
> mu-storage-daemon tests/qtest/migration-test --tap -k
> PASS 1 qtest-aarch64/migration-test /aarch64/migration/bad_dest
> PASS 2 qtest-aarch64/migration-test /aarch64/migration/fd_proto
> PASS 3 qtest-aarch64/migration-test /aarch64/migration/validate_uuid
> PASS 4 qtest-aarch64/migration-test /aarch64/migration/validate_uuid_error
> PASS 5 qtest-aarch64/migration-test /aarch64/migration/validate_uuid_src_not_set
> PASS 6 qtest-aarch64/migration-test /aarch64/migration/validate_uuid_dst_not_set
> PASS 7 qtest-aarch64/migration-test /aarch64/migration/auto_converge
> PASS 8 qtest-aarch64/migration-test /aarch64/migration/postcopy/unix
> Broken pipe
> ../../tests/qtest/libqtest.c:166: kill_qemu() detected QEMU death from
> signal 6 (Aborted) (core dumped)
Thanks; I'll sit one in a loop and see if I can get it to fail.
Dave
> ERROR qtest-aarch64/migration-test - too few tests run (expected 15, got 8)
> Makefile.mtest:264: recipe for target 'run-test-31' failed
>
> -- PMM
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: random crash in migration-test, i386 guest, x86-64 host:
2021-07-19 18:56 ` Peter Maydell
2021-07-20 8:08 ` Dr. David Alan Gilbert
@ 2021-07-20 10:33 ` Dr. David Alan Gilbert
1 sibling, 0 replies; 4+ messages in thread
From: Dr. David Alan Gilbert @ 2021-07-20 10:33 UTC (permalink / raw)
To: Peter Maydell, peterx; +Cc: QEMU Developers, Juan Quintela
* Peter Maydell (peter.maydell@linaro.org) wrote:
> On Mon, 19 Jul 2021 at 13:43, Peter Maydell <peter.maydell@linaro.org> wrote:
> >
> > Random crash in migration-tests:
> >
> > MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}
> > QTEST_QEMU_IMG=./qemu-img G_TEST_DBUS_DAEMON=/mnt/nvmedisk/linaro/qem
> > u-for-merges/tests/dbus-vmstate-daemon.sh
> > QTEST_QEMU_BINARY=./qemu-system-i386
> > QTEST_QEMU_STORAGE_DAEMON_BINARY=./storage-daemon/qemu-
> > storage-daemon tests/qtest/migration-test --tap -k
> > PASS 1 qtest-i386/migration-test /i386/migration/bad_dest
> > PASS 2 qtest-i386/migration-test /i386/migration/fd_proto
> > PASS 3 qtest-i386/migration-test /i386/migration/validate_uuid
> > PASS 4 qtest-i386/migration-test /i386/migration/validate_uuid_error
> > PASS 5 qtest-i386/migration-test /i386/migration/validate_uuid_src_not_set
> > PASS 6 qtest-i386/migration-test /i386/migration/validate_uuid_dst_not_set
> > PASS 7 qtest-i386/migration-test /i386/migration/auto_converge
> > PASS 8 qtest-i386/migration-test /i386/migration/postcopy/unix
> > Broken pipe
> > ../../tests/qtest/libqtest.c:166: kill_qemu() detected QEMU death from
> > signal 6 (Aborted) (core dumped)
> > ERROR qtest-i386/migration-test - too few tests run (expected 15, got 8)
> > Makefile.mtest:280: recipe for target 'run-test-33' failed
> > make: *** [run-test-33] Error 1
>
> And again, different guest architecture, still x86-64 host:
>
> MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}
> QTEST_QEMU_IMG=./qemu-img G_TEST_DBUS_DAEMON=/mnt/nvmedisk/linaro/qem
> u-for-merges/tests/dbus-vmstate-daemon.sh
> QTEST_QEMU_BINARY=./qemu-system-aarch64
> QTEST_QEMU_STORAGE_DAEMON_BINARY=./storage-daemon/qe
> mu-storage-daemon tests/qtest/migration-test --tap -k
> PASS 1 qtest-aarch64/migration-test /aarch64/migration/bad_dest
> PASS 2 qtest-aarch64/migration-test /aarch64/migration/fd_proto
> PASS 3 qtest-aarch64/migration-test /aarch64/migration/validate_uuid
> PASS 4 qtest-aarch64/migration-test /aarch64/migration/validate_uuid_error
> PASS 5 qtest-aarch64/migration-test /aarch64/migration/validate_uuid_src_not_set
> PASS 6 qtest-aarch64/migration-test /aarch64/migration/validate_uuid_dst_not_set
> PASS 7 qtest-aarch64/migration-test /aarch64/migration/auto_converge
> PASS 8 qtest-aarch64/migration-test /aarch64/migration/postcopy/unix
> Broken pipe
> ../../tests/qtest/libqtest.c:166: kill_qemu() detected QEMU death from
> signal 6 (Aborted) (core dumped)
> ERROR qtest-aarch64/migration-test - too few tests run (expected 15, got 8)
> Makefile.mtest:264: recipe for target 'run-test-31' failed
>
> -- PMM
It looks like we've got another yank_unregister problem; Peter's patches
in the last pull did try and fix some, perhaps it just moved around:
/aarch64/migration/postcopy/recovery: Broken pipe
../qemu/tests/qtest/libqtest.c:166: kill_qemu() detected QEMU death from signal 6 (Aborted) (core dumped)
Aborted (core dumped)
Tue 2021-07-20 11:08:38 BST 1186997 0 0 6 present /home/dgilbert/try/tests/qtest/migration-test
Tue 2021-07-20 11:08:38 BST 1187097 0 0 6 present /home/dgilbert/try/qemu-system-aarch64
#4 0x000055a56d80593d in yank_unregister_instance (instance=instance@entry=0x7ffe5e32a9b0) at ../qemu/util/yank.c:107
#5 0x000055a56d0b4cdc in migrate_fd_cleanup (s=s@entry=0x55a570556400) at ../qemu/migration/migration.c:1830
#6 0x000055a56d0b4e09 in migrate_fd_cleanup_bh (opaque=0x55a570556400) at ../qemu/migration/migration.c:1846
#7 0x000055a56d7da9a6 in aio_bh_poll (ctx=ctx@entry=0x55a5703098d0) at ../qemu/util/async.c:169
#8 0x000055a56d7e09ce in aio_dispatch (ctx=0x55a5703098d0) at ../qemu/util/aio-posix.c:381
#9 0x000055a56d7da87e in aio_ctx_dispatch (source=<optimized out>, callback=<optimized out>, user_data=<optimized out>) at ../qemu/util/async.c:311
#10 0x00007f870018067d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
#11 0x000055a56d7ec0a8 in glib_pollfds_poll () at ../qemu/util/main-loop.c:232
#12 0x000055a56d7ec0a8 in os_host_main_loop_wait (timeout=<optimized out>) at ../qemu/util/main-loop.c:255
#13 0x000055a56d7ec0a8 in main_loop_wait (nonblocking=nonblocking@entry=0) at ../qemu/util/main-loop.c:531
#14 0x000055a56d5b0411 in qemu_main_loop () at ../qemu/softmmu/runstate.c:726
#15 0x000055a56d02f05e in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at ../qemu/softmmu/main.c:50
---
#5 0x00007fa419a1f7ff in raise () from /lib64/libc.so.6
#6 0x00007fa419a09c35 in abort () from /lib64/libc.so.6
#7 0x000055c24aa8c3c9 in qmp_fd_receive (fd=4) at ../qemu/tests/qtest/libqtest.c:618
#8 0x000055c24aa8c408 in qtest_qmp_receive_dict (s=s@entry=0x55c24c692600) at ../qemu/tests/qtest/libqtest.c:646
#9 0x000055c24aa8c43e in qtest_qmp_receive (s=s@entry=0x55c24c692600) at ../qemu/tests/qtest/libqtest.c:634
#10 0x000055c24aa8cc32 in qtest_vqmp (s=s@entry=0x55c24c692600, fmt=<optimized out>, ap=ap@entry=0x7ffd15fd8130) at ../qemu/tests/qtest/libqtest.c:747
#11 0x000055c24aa8ac62 in wait_command (who=0x55c24c692600, who@entry=0x55c24c66e740,
command=command@entry=0x55c24aabc508 "{ 'execute': 'query-migrate' }") at ../qemu/tests/qtest/migration-helpers.c:63
#12 0x000055c24aa8ad73 in migrate_query (who=who@entry=0x55c24c66e740) at ../qemu/tests/qtest/migration-helpers.c:116
#13 migrate_query_status (who=who@entry=0x55c24c692600) at ../qemu/tests/qtest/migration-helpers.c:116
#14 0x000055c24aa8afd2 in check_migration_status (ungoals=0x0, goal=0x55c24aac54ca "completed", who=0x55c24c692600)
at ../qemu/tests/qtest/migration-helpers.c:156
#15 wait_for_migration_status (who=0x55c24c692600, goal=<optimized out>, ungoals=0x0) at ../qemu/tests/qtest/migration-helpers.c:156
#16 0x000055c24aa8953f in migrate_postcopy_complete (from=0x55c24c692600, to=0x55c24c6955c0) at ../qemu/tests/qtest/migration-test.c:697
695 static void migrate_postcopy_complete(QTestState *from, QTestState *to)
696 {
697 wait_for_migration_complete(from);
#17 0x000055c24aa897e4 in test_postcopy_recovery () at ../qemu/tests/qtest/migration-test.c:780
780 migrate_postcopy_complete(from, to);
781 }
#18 0x00007fa41a3c1a6a in g_test_run_suite_internal () from /lib64/libglib-2.0.so.0
#19 0x00007fa41a3c1924 in g_test_run_suite_internal () from /lib64/libglib-2.0.so.0
#20 0x00007fa41a3c1924 in g_test_run_suite_internal () from /lib64/libglib-2.0.so.0
#21 0x00007fa41a3c1924 in g_test_run_suite_internal () from /lib64/libglib-2.0.so.0
#22 0x00007fa41a3c1d22 in g_test_run_suite () from /lib64/libglib-2.0.so.0
#23 0x00007fa41a3c1d45 in g_test_run () from /lib64/libglib-2.0.so.0
#24 0x000055c24aa87843 in main (argc=<optimized out>, argv=<optimized out>) at ../qemu/tests/qtest/migration-test.c:1494
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 4+ messages in thread