All of lore.kernel.org
 help / color / mirror / Atom feed
From: Peter Maydell <peter.maydell@linaro.org>
To: Paolo Bonzini <pbonzini@redhat.com>
Cc: "Alex Bennée" <alex.bennee@linaro.org>,
	"QEMU Developers" <qemu-devel@nongnu.org>,
	"Stefan Hajnoczi" <stefanha@redhat.com>
Subject: Re: test-aio-multithread assertion
Date: Tue, 19 Nov 2019 18:01:43 +0000	[thread overview]
Message-ID: <CAFEAcA8V-vbfoEcreX=-8W+ZX1mcvOo_VspAh4Ea9+iUOjcDwQ@mail.gmail.com> (raw)
In-Reply-To: <CAFEAcA-aq-0_QuHyiqpjWMKcxSaadyyfgcvg5bKPUChpSbGmvA@mail.gmail.com>

On Tue, 19 Nov 2019 at 17:55, Peter Maydell <peter.maydell@linaro.org> wrote:
> Here's a gdb backtrace from as assertion failure under NetBSD
> when running ./tests/test-aio-multithread -p /aio/multi/mutex/contended
> (which is pretty replicable for me); does it look like the same
> thing as the commit 2362a28ea1 issue? (Apologies for all the
> "optimized out" stuff in the backtraces; I'll try to rerun with
> a --enable-debug build.)

...that was easier than I thought it might be. Here's the
--enable-debug version:

Thread 7 (LWP 2):
#0  0x000078b302099c0a in ___lwp_park60 () from /usr/lib/libc.so.12
#1  0x000078b302409ee3 in pthread_cond_timedwait () from
/usr/lib/libpthread.so.1
#2  0x0000000000519388 in qemu_futex_wait (ev=0x7d26e0
<rcu_call_ready_event>, val=4294967295)
    at /home/qemu/qemu-test.0WfUNX/src/util/qemu-thread-posix.c:364
#3  0x000000000051959d in qemu_event_wait (ev=0x7d26e0 <rcu_call_ready_event>)
    at /home/qemu/qemu-test.0WfUNX/src/util/qemu-thread-posix.c:459
#4  0x00000000005305db in call_rcu_thread (opaque=0x0)
    at /home/qemu/qemu-test.0WfUNX/src/util/rcu.c:260
#5  0x000000000051969f in qemu_thread_start (args=0x78b305722c00)
    at /home/qemu/qemu-test.0WfUNX/src/util/qemu-thread-posix.c:519
#6  0x000078b30240b8b5 in ?? () from /usr/lib/libpthread.so.1
#7  0x000078b302081e20 in ?? () from /usr/lib/libc.so.12
#8  0x00010102464c457f in ?? ()
#9  0x0000000000000000 in ?? ()

Thread 6 (LWP 3):
#0  0x000078b30203e02a in write () from /usr/lib/libc.so.12
#1  0x000078b302407561 in write () from /usr/lib/libpthread.so.1
#2  0x0000000000516e26 in event_notifier_set (e=0x78b304701284)
    at /home/qemu/qemu-test.0WfUNX/src/util/event_notifier-posix.c:100
#3  0x0000000000511ccb in aio_notify (ctx=0x78b304701180)
    at /home/qemu/qemu-test.0WfUNX/src/util/async.c:350
#4  0x000000000051183d in qemu_bh_schedule (bh=0x78b304717050)
    at /home/qemu/qemu-test.0WfUNX/src/util/async.c:167
#5  0x0000000000512100 in aio_co_schedule (ctx=0x78b304701180,
co=0x78b303701280)
    at /home/qemu/qemu-test.0WfUNX/src/util/async.c:473
#6  0x0000000000512180 in aio_co_enter (ctx=0x78b304701180, co=0x78b303701280)
    at /home/qemu/qemu-test.0WfUNX/src/util/async.c:494
#7  0x000000000051214f in aio_co_wake (co=0x78b303701280)
    at /home/qemu/qemu-test.0WfUNX/src/util/async.c:488
#8  0x0000000000531b3e in qemu_co_mutex_wake (mutex=0x7d1700
<comutex>, co=0x78b303701280)
    at /home/qemu/qemu-test.0WfUNX/src/util/qemu-coroutine-lock.c:198
#9  0x0000000000531eb2 in qemu_co_mutex_unlock (mutex=0x7d1700 <comutex>)
    at /home/qemu/qemu-test.0WfUNX/src/util/qemu-coroutine-lock.c:301
#10 0x000000000040b8b5 in test_multi_co_mutex_entry (opaque=0x0)
    at /home/qemu/qemu-test.0WfUNX/src/tests/test-aio-multithread.c:208
#11 0x0000000000532bc1 in coroutine_trampoline (i0=57676000, i1=30899)
    at /home/qemu/qemu-test.0WfUNX/src/util/coroutine-ucontext.c:115
#12 0x000078b30205dba0 in ?? () from /usr/lib/libc.so.12
#13 0x00010102464c457f in ?? ()
#14 0x0000000000000000 in ?? ()

Thread 5 (LWP 5):
#0  0x000078b30203e02a in write () from /usr/lib/libc.so.12
#1  0x000078b302407561 in write () from /usr/lib/libpthread.so.1
#2  0x0000000000516e26 in event_notifier_set (e=0x78b305316284)
    at /home/qemu/qemu-test.0WfUNX/src/util/event_notifier-posix.c:100
#3  0x0000000000511ccb in aio_notify (ctx=0x78b305316180)
    at /home/qemu/qemu-test.0WfUNX/src/util/async.c:350
#4  0x000000000051183d in qemu_bh_schedule (bh=0x78b305301080)
    at /home/qemu/qemu-test.0WfUNX/src/util/async.c:167
#5  0x0000000000512100 in aio_co_schedule (ctx=0x78b305316180,
co=0x78b303701350)
    at /home/qemu/qemu-test.0WfUNX/src/util/async.c:473
#6  0x0000000000512180 in aio_co_enter (ctx=0x78b305316180, co=0x78b303701350)
    at /home/qemu/qemu-test.0WfUNX/src/util/async.c:494
#7  0x000000000051214f in aio_co_wake (co=0x78b303701350)
    at /home/qemu/qemu-test.0WfUNX/src/util/async.c:488
#8  0x0000000000531b3e in qemu_co_mutex_wake (mutex=0x7d1700
<comutex>, co=0x78b303701350)
    at /home/qemu/qemu-test.0WfUNX/src/util/qemu-coroutine-lock.c:198
#9  0x0000000000531eb2 in qemu_co_mutex_unlock (mutex=0x7d1700 <comutex>)
    at /home/qemu/qemu-test.0WfUNX/src/util/qemu-coroutine-lock.c:301
#10 0x000000000040b8b5 in test_multi_co_mutex_entry (opaque=0x0)
    at /home/qemu/qemu-test.0WfUNX/src/tests/test-aio-multithread.c:208
#11 0x0000000000532bc1 in coroutine_trampoline (i0=57676416, i1=30899)
    at /home/qemu/qemu-test.0WfUNX/src/util/coroutine-ucontext.c:115

#12 0x000078b30205dba0 in ?? () from /usr/lib/libc.so.12
#13 0x00010102464c457f in ?? ()
#14 0x0000000000000000 in ?? ()

Thread 4 (LWP 6):
#0  0x000078b30203e28a in poll () from /usr/lib/libc.so.12
#1  0x000078b3024072d0 in poll () from /usr/lib/libpthread.so.1
#2  0x0000000000513bf9 in qemu_poll_ns (fds=0x78b3053050c0, nfds=1, timeout=-1)
    at /home/qemu/qemu-test.0WfUNX/src/util/qemu-timer.c:351
#3  0x000000000051665a in aio_poll (ctx=0x78b305316180, blocking=true)
    at /home/qemu/qemu-test.0WfUNX/src/util/aio-posix.c:669
#4  0x00000000004ddf5c in iothread_run (opaque=0x78b30530b270)
    at /home/qemu/qemu-test.0WfUNX/src/tests/iothread.c:51
#5  0x000000000051969f in qemu_thread_start (args=0x78b305303080)
    at /home/qemu/qemu-test.0WfUNX/src/util/qemu-thread-posix.c:519
#6  0x000078b30240b8b5 in ?? () from /usr/lib/libpthread.so.1
#7  0x000078b302081e20 in ?? () from /usr/lib/libc.so.12
#8  0x0000000000000000 in ?? ()

Thread 3 (LWP 7):
#0  0x000078b30203e28a in poll () from /usr/lib/libc.so.12
#1  0x000078b3024072d0 in poll () from /usr/lib/libpthread.so.1
#2  0x0000000000513bf9 in qemu_poll_ns (fds=0x78b303f22080, nfds=1, timeout=-1)
    at /home/qemu/qemu-test.0WfUNX/src/util/qemu-timer.c:351
#3  0x000000000051665a in aio_poll (ctx=0x78b303f01180, blocking=true)
    at /home/qemu/qemu-test.0WfUNX/src/util/aio-posix.c:669
#4  0x00000000004ddf5c in iothread_run (opaque=0x78b30530b300)
    at /home/qemu/qemu-test.0WfUNX/src/tests/iothread.c:51
#5  0x000000000051969f in qemu_thread_start (args=0x78b305303080)
    at /home/qemu/qemu-test.0WfUNX/src/util/qemu-thread-posix.c:519
#6  0x000078b30240b8b5 in ?? () from /usr/lib/libpthread.so.1
#7  0x000078b302081e20 in ?? () from /usr/lib/libc.so.12
#8  0x0000000000000000 in ?? ()

Thread 2 (LWP 4):
#0  0x000078b3020fe7ea in _lwp_kill () from /usr/lib/libc.so.12
#1  0x000078b3020fe092 in abort () from /usr/lib/libc.so.12
#2  0x000078b30209fd57 in __assert13 () from /usr/lib/libc.so.12
#3  0x0000000000511ae8 in aio_ctx_finalize (source=0x78b305316180)
    at /home/qemu/qemu-test.0WfUNX/src/util/async.c:279
#4  0x000078b304045596 in g_source_unref_internal () from
/usr/pkg/lib/libglib-2.0.so.0
#5  0x0000000000512247 in aio_context_unref (ctx=0x78b305316180)
    at /home/qemu/qemu-test.0WfUNX/src/util/async.c:516
#6  0x000000000051210c in aio_co_schedule (ctx=0x78b305316180,
co=0x78b303701350)
    at /home/qemu/qemu-test.0WfUNX/src/util/async.c:475
#7  0x0000000000512180 in aio_co_enter (ctx=0x78b305316180, co=0x78b303701350)
    at /home/qemu/qemu-test.0WfUNX/src/util/async.c:494
#8  0x000000000051214f in aio_co_wake (co=0x78b303701350)
    at /home/qemu/qemu-test.0WfUNX/src/util/async.c:488
#9  0x0000000000531b3e in qemu_co_mutex_wake (mutex=0x7d1700
<comutex>, co=0x78b303701350)
    at /home/qemu/qemu-test.0WfUNX/src/util/qemu-coroutine-lock.c:198
#10 0x0000000000531eb2 in qemu_co_mutex_unlock (mutex=0x7d1700 <comutex>)
    at /home/qemu/qemu-test.0WfUNX/src/util/qemu-coroutine-lock.c:301
#11 0x000000000040b8b5 in test_multi_co_mutex_entry (opaque=0x0)
    at /home/qemu/qemu-test.0WfUNX/src/tests/test-aio-multithread.c:208
#12 0x0000000000532bc1 in coroutine_trampoline (i0=57676208, i1=30899)
    at /home/qemu/qemu-test.0WfUNX/src/util/coroutine-ucontext.c:115
#13 0x000078b30205dba0 in ?? () from /usr/lib/libc.so.12
#14 0x00010102464c457f in ?? ()
#15 0x0000000000000000 in ?? ()

Thread 1 (LWP 1):
#0  0x000078b30203e2ea in _sys___nanosleep50 () from /usr/lib/libc.so.12
#1  0x000078b302407287 in __nanosleep50 () from /usr/lib/libpthread.so.1
#2  0x000078b30406e511 in g_usleep () from /usr/pkg/lib/libglib-2.0.so.0
#3  0x000000000040b9ba in test_multi_co_mutex (threads=5, seconds=1)
    at /home/qemu/qemu-test.0WfUNX/src/tests/test-aio-multithread.c:237
#4  0x000000000040ba75 in test_multi_co_mutex_1 ()
    at /home/qemu/qemu-test.0WfUNX/src/tests/test-aio-multithread.c:255
#5  0x000078b30406bd15 in g_test_run_suite_internal () from
/usr/pkg/lib/libglib-2.0.so.0
#6  0x000078b30406bc3e in g_test_run_suite_internal () from
/usr/pkg/lib/libglib-2.0.so.0
#7  0x000078b30406bc3e in g_test_run_suite_internal () from
/usr/pkg/lib/libglib-2.0.so.0
#8  0x000078b30406bc3e in g_test_run_suite_internal () from
/usr/pkg/lib/libglib-2.0.so.0
#9  0x000078b30406c1a0 in g_test_run_suite () from /usr/pkg/lib/libglib-2.0.so.0
#10 0x000078b30406c1e8 in g_test_run () from /usr/pkg/lib/libglib-2.0.so.0
#11 0x000000000040bdc3 in main (argc=1, argv=0x7f7fffc34528)
    at /home/qemu/qemu-test.0WfUNX/src/tests/test-aio-multithread.c:459

thanks
-- PMM


  reply	other threads:[~2019-11-19 18:04 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-09-04 11:50 [Qemu-devel] test-aio-multithread assertion Peter Maydell
2019-09-05 13:17 ` Peter Maydell
2019-10-14 10:26   ` Peter Maydell
2019-10-14 10:50     ` Paolo Bonzini
2019-11-19 17:55       ` Peter Maydell
2019-11-19 18:01         ` Peter Maydell [this message]
2019-09-16 15:33 ` [Qemu-devel] " Stefan Hajnoczi

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAFEAcA8V-vbfoEcreX=-8W+ZX1mcvOo_VspAh4Ea9+iUOjcDwQ@mail.gmail.com' \
    --to=peter.maydell@linaro.org \
    --cc=alex.bennee@linaro.org \
    --cc=pbonzini@redhat.com \
    --cc=qemu-devel@nongnu.org \
    --cc=stefanha@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.