All of lore.kernel.org
 help / color / mirror / Atom feed
* [Qemu-devel] [Bug 1729623] [NEW] test-aio-multithread fails with 'Co-routine re-entered recursively'
@ 2017-11-02 15:11 R.Nageswara Sastry
  2017-11-02 15:14 ` [Qemu-devel] [Bug 1729623] " R.Nageswara Sastry
                   ` (3 more replies)
  0 siblings, 4 replies; 6+ messages in thread
From: R.Nageswara Sastry @ 2017-11-02 15:11 UTC (permalink / raw)
  To: qemu-devel

Public bug reported:

git head is at fa73e146250181852c0915aa65df8d54d35485fa

configure with the following

./configure --enable-attr --enable-bsd-user --enable-cap-ng\
 --enable-coroutine-pool  --enable-crypto-afalg --enable-curl\
 --enable-curses --enable-debug --enable-debug-info\
 --enable-debug-tcg      --enable-fdt      --enable-gcrypt \
 --enable-gnutls      --enable-gprof      --enable-gtk  \
 --enable-guest-agent      --enable-kvm      --enable-libiscsi \
 --enable-libssh2      --enable-linux-aio      --enable-linux-user \
 --enable-live-block-migration      --enable-modules   \
 --enable-numa      --enable-pie      --enable-profiler \
 --enable-qom-cast-debug      --enable-rbd      --enable-replication  \
 --enable-seccomp      --enable-smartcard      --enable-stack-protector \
 --enable-system      --enable-tcg      --enable-tcg-interpreter  \
 --enable-tools      --enable-tpm      --enable-trace-backend=ftrace \
 --enable-user      --enable-vhost-net      --enable-vhost-scsi  \
 --enable-vhost-user      --enable-vhost-vsock      --enable-virtfs  \ 
 --enable-vnc      --enable-tpm      --enable-vnc-png   \
 --enable-vnc-sasl      --enable-werror      --enable-xfsctl \
 --enable-gcov --enable-debug-stack-usage

make -j 32

make test-aio-multithread V=1

...
File '/home/nasastry/qemu/include/qapi/qmp/qobject.h'
No executable lines

MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))} gtester -k --verbose -m=quick tests/test-aio-multithread
TEST: tests/test-aio-multithread... (pid=86877)
  /aio/multi/lifecycle:                                                OK
  /aio/multi/schedule:                                                 Co-routine re-entered recursively
FAIL
GTester: last random seed: R02S681209ce87fc22715b41223212d9f6f0
(pid=86891)
  /aio/multi/mutex/contended:                                          OK
  /aio/multi/mutex/handoff:                                            OK
  /aio/multi/mutex/mcs:                                                OK
  /aio/multi/mutex/pthread:                                            OK
FAIL: tests/test-aio-multithread
make: *** [check-tests/test-aio-multithread] Error 1

Full log will be attached.

** Affects: qemu
     Importance: Undecided
         Status: New

** Attachment added: "full run log configure, make, make check"
   https://bugs.launchpad.net/bugs/1729623/+attachment/5002488/+files/configure_most_run_tests.txt

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1729623

Title:
  test-aio-multithread fails with 'Co-routine re-entered recursively'

Status in QEMU:
  New

Bug description:
  git head is at fa73e146250181852c0915aa65df8d54d35485fa

  configure with the following

  ./configure --enable-attr --enable-bsd-user --enable-cap-ng\
   --enable-coroutine-pool  --enable-crypto-afalg --enable-curl\
   --enable-curses --enable-debug --enable-debug-info\
   --enable-debug-tcg      --enable-fdt      --enable-gcrypt \
   --enable-gnutls      --enable-gprof      --enable-gtk  \
   --enable-guest-agent      --enable-kvm      --enable-libiscsi \
   --enable-libssh2      --enable-linux-aio      --enable-linux-user \
   --enable-live-block-migration      --enable-modules   \
   --enable-numa      --enable-pie      --enable-profiler \
   --enable-qom-cast-debug      --enable-rbd      --enable-replication  \
   --enable-seccomp      --enable-smartcard      --enable-stack-protector \
   --enable-system      --enable-tcg      --enable-tcg-interpreter  \
   --enable-tools      --enable-tpm      --enable-trace-backend=ftrace \
   --enable-user      --enable-vhost-net      --enable-vhost-scsi  \
   --enable-vhost-user      --enable-vhost-vsock      --enable-virtfs  \ 
   --enable-vnc      --enable-tpm      --enable-vnc-png   \
   --enable-vnc-sasl      --enable-werror      --enable-xfsctl \
   --enable-gcov --enable-debug-stack-usage

  make -j 32

  make test-aio-multithread V=1

  ...
  File '/home/nasastry/qemu/include/qapi/qmp/qobject.h'
  No executable lines

  MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))} gtester -k --verbose -m=quick tests/test-aio-multithread
  TEST: tests/test-aio-multithread... (pid=86877)
    /aio/multi/lifecycle:                                                OK
    /aio/multi/schedule:                                                 Co-routine re-entered recursively
  FAIL
  GTester: last random seed: R02S681209ce87fc22715b41223212d9f6f0
  (pid=86891)
    /aio/multi/mutex/contended:                                          OK
    /aio/multi/mutex/handoff:                                            OK
    /aio/multi/mutex/mcs:                                                OK
    /aio/multi/mutex/pthread:                                            OK
  FAIL: tests/test-aio-multithread
  make: *** [check-tests/test-aio-multithread] Error 1

  Full log will be attached.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1729623/+subscriptions

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [Qemu-devel] [Bug 1729623] Re: test-aio-multithread fails with 'Co-routine re-entered recursively'
  2017-11-02 15:11 [Qemu-devel] [Bug 1729623] [NEW] test-aio-multithread fails with 'Co-routine re-entered recursively' R.Nageswara Sastry
@ 2017-11-02 15:14 ` R.Nageswara Sastry
  2017-11-03  9:48 ` [Qemu-devel] [Bug 1729623] [NEW] " Stefan Hajnoczi
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 6+ messages in thread
From: R.Nageswara Sastry @ 2017-11-02 15:14 UTC (permalink / raw)
  To: qemu-devel

This try is inspiration from tests/docker/test-full.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1729623

Title:
  test-aio-multithread fails with 'Co-routine re-entered recursively'

Status in QEMU:
  New

Bug description:
  git head is at fa73e146250181852c0915aa65df8d54d35485fa

  configure with the following

  ./configure --enable-attr --enable-bsd-user --enable-cap-ng\
   --enable-coroutine-pool  --enable-crypto-afalg --enable-curl\
   --enable-curses --enable-debug --enable-debug-info\
   --enable-debug-tcg      --enable-fdt      --enable-gcrypt \
   --enable-gnutls      --enable-gprof      --enable-gtk  \
   --enable-guest-agent      --enable-kvm      --enable-libiscsi \
   --enable-libssh2      --enable-linux-aio      --enable-linux-user \
   --enable-live-block-migration      --enable-modules   \
   --enable-numa      --enable-pie      --enable-profiler \
   --enable-qom-cast-debug      --enable-rbd      --enable-replication  \
   --enable-seccomp      --enable-smartcard      --enable-stack-protector \
   --enable-system      --enable-tcg      --enable-tcg-interpreter  \
   --enable-tools      --enable-tpm      --enable-trace-backend=ftrace \
   --enable-user      --enable-vhost-net      --enable-vhost-scsi  \
   --enable-vhost-user      --enable-vhost-vsock      --enable-virtfs  \ 
   --enable-vnc      --enable-tpm      --enable-vnc-png   \
   --enable-vnc-sasl      --enable-werror      --enable-xfsctl \
   --enable-gcov --enable-debug-stack-usage

  make -j 32

  make test-aio-multithread V=1

  ...
  File '/home/nasastry/qemu/include/qapi/qmp/qobject.h'
  No executable lines

  MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))} gtester -k --verbose -m=quick tests/test-aio-multithread
  TEST: tests/test-aio-multithread... (pid=86877)
    /aio/multi/lifecycle:                                                OK
    /aio/multi/schedule:                                                 Co-routine re-entered recursively
  FAIL
  GTester: last random seed: R02S681209ce87fc22715b41223212d9f6f0
  (pid=86891)
    /aio/multi/mutex/contended:                                          OK
    /aio/multi/mutex/handoff:                                            OK
    /aio/multi/mutex/mcs:                                                OK
    /aio/multi/mutex/pthread:                                            OK
  FAIL: tests/test-aio-multithread
  make: *** [check-tests/test-aio-multithread] Error 1

  Full log will be attached.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1729623/+subscriptions

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [Qemu-devel] [Bug 1729623] [NEW] test-aio-multithread fails with 'Co-routine re-entered recursively'
  2017-11-02 15:11 [Qemu-devel] [Bug 1729623] [NEW] test-aio-multithread fails with 'Co-routine re-entered recursively' R.Nageswara Sastry
  2017-11-02 15:14 ` [Qemu-devel] [Bug 1729623] " R.Nageswara Sastry
@ 2017-11-03  9:48 ` Stefan Hajnoczi
  2017-11-06  4:57 ` [Qemu-devel] [Bug 1729623] " R.Nageswara Sastry
  2018-01-03 17:15 ` Murilo Opsfelder Araújo
  3 siblings, 0 replies; 6+ messages in thread
From: Stefan Hajnoczi @ 2017-11-03  9:48 UTC (permalink / raw)
  To: Bug 1729623; +Cc: qemu-devel, Paolo Bonzini

[-- Attachment #1: Type: text/plain, Size: 2129 bytes --]

On Thu, Nov 02, 2017 at 03:11:30PM -0000, R.Nageswara Sastry wrote:
> Public bug reported:
> 
> git head is at fa73e146250181852c0915aa65df8d54d35485fa
> 
> configure with the following
> 
> ./configure --enable-attr --enable-bsd-user --enable-cap-ng\
>  --enable-coroutine-pool  --enable-crypto-afalg --enable-curl\
>  --enable-curses --enable-debug --enable-debug-info\
>  --enable-debug-tcg      --enable-fdt      --enable-gcrypt \
>  --enable-gnutls      --enable-gprof      --enable-gtk  \
>  --enable-guest-agent      --enable-kvm      --enable-libiscsi \
>  --enable-libssh2      --enable-linux-aio      --enable-linux-user \
>  --enable-live-block-migration      --enable-modules   \
>  --enable-numa      --enable-pie      --enable-profiler \
>  --enable-qom-cast-debug      --enable-rbd      --enable-replication  \
>  --enable-seccomp      --enable-smartcard      --enable-stack-protector \
>  --enable-system      --enable-tcg      --enable-tcg-interpreter  \
>  --enable-tools      --enable-tpm      --enable-trace-backend=ftrace \
>  --enable-user      --enable-vhost-net      --enable-vhost-scsi  \
>  --enable-vhost-user      --enable-vhost-vsock      --enable-virtfs  \ 
>  --enable-vnc      --enable-tpm      --enable-vnc-png   \
>  --enable-vnc-sasl      --enable-werror      --enable-xfsctl \
>  --enable-gcov --enable-debug-stack-usage
> 
> make -j 32
> 
> make test-aio-multithread V=1
> 
> ...
> File '/home/nasastry/qemu/include/qapi/qmp/qobject.h'
> No executable lines
> 
> MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))} gtester -k --verbose -m=quick tests/test-aio-multithread
> TEST: tests/test-aio-multithread... (pid=86877)
>   /aio/multi/lifecycle:                                                OK
>   /aio/multi/schedule:                                                 Co-routine re-entered recursively
> FAIL

I cannot reproduce it locally.

Please run with gdb tests/test-aio-multithread and post the backtrace
when the "Co-routine re-entered recursively" message is printed using
the "thread apply all bt" gdb command.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [Qemu-devel] [Bug 1729623] Re: test-aio-multithread fails with 'Co-routine re-entered recursively'
  2017-11-02 15:11 [Qemu-devel] [Bug 1729623] [NEW] test-aio-multithread fails with 'Co-routine re-entered recursively' R.Nageswara Sastry
  2017-11-02 15:14 ` [Qemu-devel] [Bug 1729623] " R.Nageswara Sastry
  2017-11-03  9:48 ` [Qemu-devel] [Bug 1729623] [NEW] " Stefan Hajnoczi
@ 2017-11-06  4:57 ` R.Nageswara Sastry
  2017-11-06 19:03   ` Stefan Hajnoczi
  2018-01-03 17:15 ` Murilo Opsfelder Araújo
  3 siblings, 1 reply; 6+ messages in thread
From: R.Nageswara Sastry @ 2017-11-06  4:57 UTC (permalink / raw)
  To: qemu-devel

I could re-produce using the following command,

# MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))} gtester -k --verbose -m=quick tests/test-aio-multithread
TEST: tests/test-aio-multithread... (pid=116074)
  /aio/multi/lifecycle:                                                OK
  /aio/multi/schedule:                                                 Co-routine re-entered recursively
FAIL
GTester: last random seed: R02Seb8ebc3a9c10406e7725cdff556c14e4
(pid=116088)
  /aio/multi/mutex/contended:                                          OK
  /aio/multi/mutex/handoff:                                            OK
  /aio/multi/mutex/mcs:                                                OK
  /aio/multi/mutex/pthread:                                            OK
FAIL: tests/test-aio-multithread
[root@zzfp365-lp1 qemu]# echo $?
1

No important dmesg messages.

Trying with gdb

When I ran only the test case seeing a segfault.
[root@zzfp365-lp1 qemu]# tests/test-aio-multithread
/aio/multi/lifecycle: OK
/aio/multi/schedule: Segmentation fault

from dmesg:
test-aio-multit[117118]: unhandled signal 11 at 00007fff88a2fe88 nip 00007fff8c08ed24 lr 00007fff8c08ebb0 code 30001

from gdb:
# gdb tests/test-aio-multithread core.117109
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-100.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "ppc64le-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/nasastry/qemu/tests/test-aio-multithread...done.

warning: core file may not match specified executable file.
[New LWP 117118]
[New LWP 117116]
[New LWP 117109]
[New LWP 117120]
[New LWP 117119]
[New LWP 117117]
[New LWP 117110]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `tests/test-aio-multithread '.
Program terminated with signal 11, Segmentation fault.
#0  0x00007fff8c08ed24 in __vmx__longjmp () from /lib64/libc.so.6
warning: File "/home/nasastry/qemu/.gdbinit" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load:/usr/bin/mono-gdb.py:/usr/lib/golang/src/pkg/runtime/runtime-gdb.py".
To enable execution of this file add
	add-auto-load-safe-path /home/nasastry/qemu/.gdbinit
line to your configuration file "/root/.gdbinit".
To completely disable this security protection add
	set auto-load safe-path /
line to your configuration file "/root/.gdbinit".
For more information about this security protection see the
"Auto-loading safe path" section in the GDB manual.  E.g., run from the shell:
	info "(gdb)Auto-loading safe path"
Missing separate debuginfos, use: debuginfo-install glib2-2.50.3-3.el7.ppc64le glibc-2.17-196.el7.ppc64le gmp-6.0.0-15.el7.ppc64le gnutls-3.3.26-9.el7.ppc64le libaio-0.3.109-13.el7.ppc64le libcap-ng-0.7.5-4.el7.ppc64le libffi-3.0.13-18.el7.ppc64le libgcrypt-1.5.3-14.el7.ppc64le libgpg-error-1.12-3.el7.ppc64le libtasn1-4.10-1.el7.ppc64le nettle-2.7.1-8.el7.ppc64le p11-kit-0.23.5-3.el7.ppc64le pcre-8.32-17.el7.ppc64le zlib-1.2.7-17.el7.ppc64le
(gdb) bt
#0  0x00007fff8c08ed24 in __vmx__longjmp () from /lib64/libc.so.6
#1  0x00007fff8c08ebb0 in siglongjmp@@GLIBC_2.17 () from /lib64/libc.so.6
Cannot access memory at address 0x7fff88a2feb0
(gdb) bt full
#0  0x00007fff8c08ed24 in __vmx__longjmp () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fff8c08ebb0 in siglongjmp@@GLIBC_2.17 () from /lib64/libc.so.6
No symbol table info available.
(gdb) thread apply all bt

Thread 7 (Thread 0x7fff8bc4edc0 (LWP 117110)):
#0  0x00007fff8c16cdd4 in syscall () from /lib64/libc.so.6
#1  0x00000001121f2868 in qemu_futex_wait (f=0x11273d74c <rcu_call_ready_event>, val=4294967295) at /home/nasastry/qemu/include/qemu/futex.h:29
#2  0x00000001121f2e60 in qemu_event_wait (ev=0x11273d74c <rcu_call_ready_event>) at util/qemu-thread-posix.c:442
#3  0x0000000112229a98 in call_rcu_thread (opaque=0x0) at util/rcu.c:249
#4  0x00007fff8c248af4 in start_thread () from /lib64/libpthread.so.0
#5  0x00007fff8c174ef4 in clone () from /lib64/libc.so.6

Thread 6 (Thread 0x7fff89c4edc0 (LWP 117117)):
#0  0x00007fff8c1644b0 in ppoll () from /lib64/libc.so.6
#1  0x00000001121e3414 in qemu_poll_ns (fds=0x7fff7400a850, nfds=1, timeout=-1) at util/qemu-timer.c:322
#2  0x00000001121eb5f8 in aio_poll (ctx=0x7fff740008c0, blocking=true) at util/aio-posix.c:629
#3  0x000000011210efa0 in iothread_run (opaque=0x125335600) at tests/iothread.c:51
#4  0x00007fff8c248af4 in start_thread () from /lib64/libpthread.so.0
#5  0x00007fff8c174ef4 in clone () from /lib64/libc.so.6

Thread 5 (Thread 0x7fff8ac4edc0 (LWP 117119)):
#0  0x00000001121f01a0 in qemu_free_stack (stack=0x7fff88810000, sz=1114112) at util/oslib-posix.c:574
#1  0x000000011222f738 in qemu_coroutine_delete (co_=0x125336e30) at util/coroutine-ucontext.c:153
#2  0x000000011222b04c in coroutine_delete (co=0x125336e30) at util/qemu-coroutine.c:102
#3  0x000000011222b310 in qemu_aio_coroutine_enter (ctx=0x7fff7c0008c0, co=0x125336e30) at util/qemu-coroutine.c:140
#4  0x000000011222b3ec in qemu_coroutine_enter (co=0x125336e30) at util/qemu-coroutine.c:149
#5  0x00000001121df09c in co_schedule_bh_cb (opaque=0x7fff7c0008c0) at util/async.c:391
#6  0x00000001121dd7c8 in aio_bh_call (bh=0x7fff7c008400) at util/async.c:90
#7  0x00000001121dd990 in aio_bh_poll (ctx=0x7fff7c0008c0) at util/async.c:118
#8  0x00000001121eba60 in aio_poll (ctx=0x7fff7c0008c0, blocking=true) at util/aio-posix.c:689
#9  0x000000011210efa0 in iothread_run (opaque=0x125335c00) at tests/iothread.c:51
#10 0x00007fff8c248af4 in start_thread () from /lib64/libpthread.so.0
#11 0x00007fff8c174ef4 in clone () from /lib64/libc.so.6

Thread 4 (Thread 0x7fff8b44edc0 (LWP 117120)):
#0  0x00000001121f01e0 in qemu_free_stack (stack=0x7fff88700000, sz=1114112) at util/oslib-posix.c:573
#1  0x000000011222f738 in qemu_coroutine_delete (co_=0x125337130) at util/coroutine-ucontext.c:153
#2  0x000000011222b04c in coroutine_delete (co=0x125337130) at util/qemu-coroutine.c:102
#3  0x000000011222b310 in qemu_aio_coroutine_enter (ctx=0x7fff840008c0, co=0x125337130) at util/qemu-coroutine.c:140
#4  0x000000011222b3ec in qemu_coroutine_enter (co=0x125337130) at util/qemu-coroutine.c:149
#5  0x00000001121df09c in co_schedule_bh_cb (opaque=0x7fff840008c0) at util/async.c:391
#6  0x00000001121dd7c8 in aio_bh_call (bh=0x7fff84008400) at util/async.c:90
#7  0x00000001121dd990 in aio_bh_poll (ctx=0x7fff840008c0) at util/async.c:118
#8  0x00000001121eba60 in aio_poll (ctx=0x7fff840008c0, blocking=true) at util/aio-posix.c:689
#9  0x000000011210efa0 in iothread_run (opaque=0x125335f00) at tests/iothread.c:51
#10 0x00007fff8c248af4 in start_thread () from /lib64/libpthread.so.0
#11 0x00007fff8c174ef4 in clone () from /lib64/libc.so.6

Thread 3 (Thread 0x7fff8cb4b8b0 (LWP 117109)):
#0  0x00007fff8c24a074 in pthread_join () from /lib64/libpthread.so.0
#1  0x00000001121f3898 in qemu_thread_join (thread=0x125336208) at util/qemu-thread-posix.c:547
#2  0x000000011210f0ac in iothread_join (iothread=0x125336200) at tests/iothread.c:62
#3  0x0000000111f5a408 in join_aio_contexts () at tests/test-aio-multithread.c:93
#4  0x0000000111f5adc0 in test_multi_co_schedule (seconds=1) at tests/test-aio-multithread.c:183
#5  0x0000000111f5aee8 in test_multi_co_schedule_1 () at tests/test-aio-multithread.c:191
#6  0x00007fff8c94a3d4 in g_test_run_suite_internal () from /lib64/libglib-2.0.so.0
#7  0x00007fff8c94a1a8 in g_test_run_suite_internal () from /lib64/libglib-2.0.so.0
#8  0x00007fff8c94a1a8 in g_test_run_suite_internal () from /lib64/libglib-2.0.so.0
#9  0x00007fff8c94a788 in g_test_run_suite () from /lib64/libglib-2.0.so.0
#10 0x00007fff8c94a7c0 in g_test_run () from /lib64/libglib-2.0.so.0
#11 0x0000000111f5ceb0 in main (argc=1, argv=0x7ffff82b45a8) at tests/test-aio-multithread.c:462

Thread 2 (Thread 0x7fff8944edc0 (LWP 117116)):
#0  0x00007fff8c977bb8 in g_mutex_lock () from /lib64/libglib-2.0.so.0
#1  0x00007fff8c93c090 in private_thread_memory_cleanup () from /lib64/libglib-2.0.so.0
#2  0x00007fff8c248924 in __nptl_deallocate_tsd () from /lib64/libpthread.so.0
#3  0x00007fff8c248b00 in start_thread () from /lib64/libpthread.so.0
#4  0x00007fff8c174ef4 in clone () from /lib64/libc.so.6

Thread 1 (Thread 0x7fff8a44edc0 (LWP 117118)):
#0  0x00007fff8c08ed24 in __vmx__longjmp () from /lib64/libc.so.6
---Type <return> to continue, or q <return> to quit---
#1  0x00007fff8c08ebb0 in siglongjmp@@GLIBC_2.17 () from /lib64/libc.so.6
Cannot access memory at address 0x7fff88a2feb0


So the following thread crashed with segfault: As the core file extension matching with this thread  -- core.117109

Thread 3 (Thread 0x7fff8cb4b8b0 (LWP 117109)):
#0  0x00007fff8c24a074 in pthread_join () from /lib64/libpthread.so.0
#1  0x00000001121f3898 in qemu_thread_join (thread=0x125336208) at util/qemu-thread-posix.c:547
#2  0x000000011210f0ac in iothread_join (iothread=0x125336200) at tests/iothread.c:62
#3  0x0000000111f5a408 in join_aio_contexts () at tests/test-aio-multithread.c:93
#4  0x0000000111f5adc0 in test_multi_co_schedule (seconds=1) at tests/test-aio-multithread.c:183
#5  0x0000000111f5aee8 in test_multi_co_schedule_1 () at tests/test-aio-multithread.c:191
#6  0x00007fff8c94a3d4 in g_test_run_suite_internal () from /lib64/libglib-2.0.so.0
#7  0x00007fff8c94a1a8 in g_test_run_suite_internal () from /lib64/libglib-2.0.so.0
#8  0x00007fff8c94a1a8 in g_test_run_suite_internal () from /lib64/libglib-2.0.so.0
#9  0x00007fff8c94a788 in g_test_run_suite () from /lib64/libglib-2.0.so.0
#10 0x00007fff8c94a7c0 in g_test_run () from /lib64/libglib-2.0.so.0
#11 0x0000000111f5ceb0 in main (argc=1, argv=0x7ffff82b45a8) at tests/test-aio-multithread.c:462


But can't run:
gdb --args MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))} gtester -k --verbose -m=quick tests/test-aio-multithread
...
gdb) r
Starting program:  gtester -k --verbose -m=quick tests/test-aio-multithread
No executable file specified.
Use the "file" or "exec-file" command.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1729623

Title:
  test-aio-multithread fails with 'Co-routine re-entered recursively'

Status in QEMU:
  New

Bug description:
  git head is at fa73e146250181852c0915aa65df8d54d35485fa

  configure with the following

  ./configure --enable-attr --enable-bsd-user --enable-cap-ng\
   --enable-coroutine-pool  --enable-crypto-afalg --enable-curl\
   --enable-curses --enable-debug --enable-debug-info\
   --enable-debug-tcg      --enable-fdt      --enable-gcrypt \
   --enable-gnutls      --enable-gprof      --enable-gtk  \
   --enable-guest-agent      --enable-kvm      --enable-libiscsi \
   --enable-libssh2      --enable-linux-aio      --enable-linux-user \
   --enable-live-block-migration      --enable-modules   \
   --enable-numa      --enable-pie      --enable-profiler \
   --enable-qom-cast-debug      --enable-rbd      --enable-replication  \
   --enable-seccomp      --enable-smartcard      --enable-stack-protector \
   --enable-system      --enable-tcg      --enable-tcg-interpreter  \
   --enable-tools      --enable-tpm      --enable-trace-backend=ftrace \
   --enable-user      --enable-vhost-net      --enable-vhost-scsi  \
   --enable-vhost-user      --enable-vhost-vsock      --enable-virtfs  \ 
   --enable-vnc      --enable-tpm      --enable-vnc-png   \
   --enable-vnc-sasl      --enable-werror      --enable-xfsctl \
   --enable-gcov --enable-debug-stack-usage

  make -j 32

  make test-aio-multithread V=1

  ...
  File '/home/nasastry/qemu/include/qapi/qmp/qobject.h'
  No executable lines

  MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))} gtester -k --verbose -m=quick tests/test-aio-multithread
  TEST: tests/test-aio-multithread... (pid=86877)
    /aio/multi/lifecycle:                                                OK
    /aio/multi/schedule:                                                 Co-routine re-entered recursively
  FAIL
  GTester: last random seed: R02S681209ce87fc22715b41223212d9f6f0
  (pid=86891)
    /aio/multi/mutex/contended:                                          OK
    /aio/multi/mutex/handoff:                                            OK
    /aio/multi/mutex/mcs:                                                OK
    /aio/multi/mutex/pthread:                                            OK
  FAIL: tests/test-aio-multithread
  make: *** [check-tests/test-aio-multithread] Error 1

  Full log will be attached.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1729623/+subscriptions

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [Qemu-devel] [Bug 1729623] Re: test-aio-multithread fails with 'Co-routine re-entered recursively'
  2017-11-06  4:57 ` [Qemu-devel] [Bug 1729623] " R.Nageswara Sastry
@ 2017-11-06 19:03   ` Stefan Hajnoczi
  0 siblings, 0 replies; 6+ messages in thread
From: Stefan Hajnoczi @ 2017-11-06 19:03 UTC (permalink / raw)
  To: Bug 1729623; +Cc: qemu-devel

[-- Attachment #1: Type: text/plain, Size: 1105 bytes --]

On Mon, Nov 06, 2017 at 04:57:13AM -0000, R.Nageswara Sastry wrote:
> I could re-produce using the following command,
> 
> # MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))} gtester -k --verbose -m=quick tests/test-aio-multithread
> TEST: tests/test-aio-multithread... (pid=116074)
>   /aio/multi/lifecycle:                                                OK
>   /aio/multi/schedule:                                                 Co-routine re-entered recursively
> FAIL
> GTester: last random seed: R02Seb8ebc3a9c10406e7725cdff556c14e4
> (pid=116088)
>   /aio/multi/mutex/contended:                                          OK
>   /aio/multi/mutex/handoff:                                            OK
>   /aio/multi/mutex/mcs:                                                OK
>   /aio/multi/mutex/pthread:                                            OK
> FAIL: tests/test-aio-multithread
> [root@zzfp365-lp1 qemu]# echo $?
> 1
> 
> No important dmesg messages.
> 
> Trying with gdb

Thanks, the backtrace was useful.  I have sent a patch and CCed you.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [Qemu-devel] [Bug 1729623] Re: test-aio-multithread fails with 'Co-routine re-entered recursively'
  2017-11-02 15:11 [Qemu-devel] [Bug 1729623] [NEW] test-aio-multithread fails with 'Co-routine re-entered recursively' R.Nageswara Sastry
                   ` (2 preceding siblings ...)
  2017-11-06  4:57 ` [Qemu-devel] [Bug 1729623] " R.Nageswara Sastry
@ 2018-01-03 17:15 ` Murilo Opsfelder Araújo
  3 siblings, 0 replies; 6+ messages in thread
From: Murilo Opsfelder Araújo @ 2018-01-03 17:15 UTC (permalink / raw)
  To: qemu-devel

I confirmed with Stefan and this bug was fixed by
https://git.qemu.org/?p=qemu.git;a=commitdiff;h=fb0c43f34eed8b18678c6e1f481d8564b35c99ed

commit fb0c43f34eed8b18678c6e1f481d8564b35c99ed
Author: Stefan Hajnoczi <stefanha@redhat.com>
Date:   Mon Nov 6 19:02:33 2017 +0000

    tests-aio-multithread: fix /aio/multi/schedule race condition

** Changed in: qemu
       Status: New => Fix Released

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1729623

Title:
  test-aio-multithread fails with 'Co-routine re-entered recursively'

Status in QEMU:
  Fix Released

Bug description:
  git head is at fa73e146250181852c0915aa65df8d54d35485fa

  configure with the following

  ./configure --enable-attr --enable-bsd-user --enable-cap-ng\
   --enable-coroutine-pool  --enable-crypto-afalg --enable-curl\
   --enable-curses --enable-debug --enable-debug-info\
   --enable-debug-tcg      --enable-fdt      --enable-gcrypt \
   --enable-gnutls      --enable-gprof      --enable-gtk  \
   --enable-guest-agent      --enable-kvm      --enable-libiscsi \
   --enable-libssh2      --enable-linux-aio      --enable-linux-user \
   --enable-live-block-migration      --enable-modules   \
   --enable-numa      --enable-pie      --enable-profiler \
   --enable-qom-cast-debug      --enable-rbd      --enable-replication  \
   --enable-seccomp      --enable-smartcard      --enable-stack-protector \
   --enable-system      --enable-tcg      --enable-tcg-interpreter  \
   --enable-tools      --enable-tpm      --enable-trace-backend=ftrace \
   --enable-user      --enable-vhost-net      --enable-vhost-scsi  \
   --enable-vhost-user      --enable-vhost-vsock      --enable-virtfs  \ 
   --enable-vnc      --enable-tpm      --enable-vnc-png   \
   --enable-vnc-sasl      --enable-werror      --enable-xfsctl \
   --enable-gcov --enable-debug-stack-usage

  make -j 32

  make test-aio-multithread V=1

  ...
  File '/home/nasastry/qemu/include/qapi/qmp/qobject.h'
  No executable lines

  MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))} gtester -k --verbose -m=quick tests/test-aio-multithread
  TEST: tests/test-aio-multithread... (pid=86877)
    /aio/multi/lifecycle:                                                OK
    /aio/multi/schedule:                                                 Co-routine re-entered recursively
  FAIL
  GTester: last random seed: R02S681209ce87fc22715b41223212d9f6f0
  (pid=86891)
    /aio/multi/mutex/contended:                                          OK
    /aio/multi/mutex/handoff:                                            OK
    /aio/multi/mutex/mcs:                                                OK
    /aio/multi/mutex/pthread:                                            OK
  FAIL: tests/test-aio-multithread
  make: *** [check-tests/test-aio-multithread] Error 1

  Full log will be attached.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1729623/+subscriptions

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2018-01-03 17:21 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-11-02 15:11 [Qemu-devel] [Bug 1729623] [NEW] test-aio-multithread fails with 'Co-routine re-entered recursively' R.Nageswara Sastry
2017-11-02 15:14 ` [Qemu-devel] [Bug 1729623] " R.Nageswara Sastry
2017-11-03  9:48 ` [Qemu-devel] [Bug 1729623] [NEW] " Stefan Hajnoczi
2017-11-06  4:57 ` [Qemu-devel] [Bug 1729623] " R.Nageswara Sastry
2017-11-06 19:03   ` Stefan Hajnoczi
2018-01-03 17:15 ` Murilo Opsfelder Araújo

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.