[Qemu-devel] Freeze / spin in virtio blk...flatview do translate

[Qemu-devel] Freeze / spin in virtio blk...flatview do translate

[Frank Yang]

Hi qemu-devel,

So we're using QEMU 2.12 for recent Android Emulator canaryies, and we're
seeing a lot of hangs on mac in flatview_translate in qemu 2.12.

What would be some pointers for diagnosing excessive I/O?

Especially, metrics to see if a system is on the verge of getting into main
loop spins.

We have not reproduced this hang so far, this is from user crash reports
that triggered our hang detector (where 15+ seconds pass without main loop
/ VCPU threads being able to go back and ping their loopers in main loop /
vcpu threads.

0x00000001024e9fcb(qemu-system-x86_64 -exec.c:511)flatview_translate
0x00000001024f2390(qemu-system-x86_64
-memory.h:1865)address_space_lduw_internal_cached
0x000000010246ff11(qemu-system-x86_64
-virtio-access.h:166)virtio_queue_set_notification
0x00000001024fa2c9(qemu-system-x86_64+ 0x000a72c9)virtio_blk_handle_vq
0x00000001024746ee(qemu-system-x86_64
-virtio.c:1521)virtio_queue_host_notifier_aio_read
0x0000000103a5ed8a(qemu-system-x86_64 -aio-posix.c:406)aio_dispatch_handlers
0x0000000103a5ecc8(qemu-system-x86_64 -aio-posix.c:437)aio_dispatch
0x0000000103a5c158(qemu-system-x86_64 -async.c:261)aio_ctx_dispatch
0x0000000103a92103(qemu-system-x86_64 -gmain.c:3072)g_main_context_dispatch
0x0000000103a5e4ad(qemu-system-x86_64 -main-loop.c:224)main_loop_wait
0x0000000102468ab8(qemu-system-x86_64 -vl.c:2172)main_impl
0x0000000102461a3a(qemu-system-x86_64 -vl.c:3332)run_qemu_main
0x000000010246eef3(qemu-system-x86_64
-main.cpp:577)enter_qemu_main_loop(int, char**)
0x00000001062b63a9(libQt5Core.5.dylib
-qthread_unix.cpp:344)QThreadPrivate::start(void*)
0x00007fff65118660
0x00007fff6511850c
0x00007fff65117bf8
0x00000001062b623f(libQt5Core.5.dylib+ 0x0002623f)

Thanks,

Frank

Re: [Qemu-devel] Freeze / spin in virtio blk...flatview do translate

[Frank Yang]

I notice at least two commits in upstream QEMU that might impact this:

https://github.com/qemu/qemu/commit/ce3a9eaff4e5f29514dba35a001894cb7a238e07#diff-8bfe2ea13d8c6dab17a555f300ac2f66
https://github.com/qemu/qemu/commit/45641dba38f6f44c3ea44c2d1c37b31619276ce3#diff-a9288ea1a561573c7d3036de7d7048e8


On Sat, Sep 15, 2018 at 11:41 AM Frank Yang <lfy@google.com> wrote:

> Hi qemu-devel,
>
> So we're using QEMU 2.12 for recent Android Emulator canaryies, and we're
> seeing a lot of hangs on mac in flatview_translate in qemu 2.12.
>
> What would be some pointers for diagnosing excessive I/O?
>
> Especially, metrics to see if a system is on the verge of getting into
> main loop spins.
>
> We have not reproduced this hang so far, this is from user crash reports
> that triggered our hang detector (where 15+ seconds pass without main loop
> / VCPU threads being able to go back and ping their loopers in main loop /
> vcpu threads.
>
> 0x00000001024e9fcb(qemu-system-x86_64 -exec.c:511)flatview_translate
> 0x00000001024f2390(qemu-system-x86_64
> -memory.h:1865)address_space_lduw_internal_cached
> 0x000000010246ff11(qemu-system-x86_64
> -virtio-access.h:166)virtio_queue_set_notification
> 0x00000001024fa2c9(qemu-system-x86_64+ 0x000a72c9)virtio_blk_handle_vq
> 0x00000001024746ee(qemu-system-x86_64
> -virtio.c:1521)virtio_queue_host_notifier_aio_read
> 0x0000000103a5ed8a(qemu-system-x86_64
> -aio-posix.c:406)aio_dispatch_handlers
> 0x0000000103a5ecc8(qemu-system-x86_64 -aio-posix.c:437)aio_dispatch
> 0x0000000103a5c158(qemu-system-x86_64 -async.c:261)aio_ctx_dispatch
> 0x0000000103a92103(qemu-system-x86_64 -gmain.c:3072)g_main_context_dispatch
> 0x0000000103a5e4ad(qemu-system-x86_64 -main-loop.c:224)main_loop_wait
> 0x0000000102468ab8(qemu-system-x86_64 -vl.c:2172)main_impl
> 0x0000000102461a3a(qemu-system-x86_64 -vl.c:3332)run_qemu_main
> 0x000000010246eef3(qemu-system-x86_64
> -main.cpp:577)enter_qemu_main_loop(int, char**)
> 0x00000001062b63a9(libQt5Core.5.dylib
> -qthread_unix.cpp:344)QThreadPrivate::start(void*)
> 0x00007fff65118660
> 0x00007fff6511850c
> 0x00007fff65117bf8
> 0x00000001062b623f(libQt5Core.5.dylib+ 0x0002623f)
>
> Thanks,
>
> Frank
>