All of lore.kernel.org
 help / color / mirror / Atom feed
From: 08005325@163.com
To: kwolf@redhat.com, mreitz@redhat.com, jsnow@redhat.com
Cc: Michael Qiu <qiudayu@huayun.com>,
	qemu-devel@nongnu.org, qemu-block@nongnu.org
Subject: [PATCH v2] Fix crash with IOthread when block commit after snapshot
Date: Tue, 26 Jan 2021 11:11:44 +0800	[thread overview]
Message-ID: <20210126031144.13121-1-08005325@163.com> (raw)
In-Reply-To: <20210125120727.7799-1-08005325@163.com>

From: Michael Qiu <qiudayu@huayun.com>

v2: modify the coredump backtrace within commit log with the newest
    qemu with master branch

Currently, if guest has workloads, IO thread will acquire aio_context
lock before do io_submit, it leads to segmentfault when do block commit
after snapshot. Just like below:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f7c7d91f700 (LWP 99907)]
0x00005576d0f65aab in bdrv_mirror_top_pwritev at ../block/mirror.c:1437
1437    ../block/mirror.c: No such file or directory.
(gdb) p s->job
$17 = (MirrorBlockJob *) 0x0
(gdb) p s->stop
$18 = false

(gdb) bt
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f7c7d91f700 (LWP 99907)]
0x00005576d0f65aab in bdrv_mirror_top_pwritev at ../block/mirror.c:1437
1437    ../block/mirror.c: No such file or directory.

(gdb) bt
#0  0x00005576d0f65aab in bdrv_mirror_top_pwritev at ../block/mirror.c:1437
#1  0x00005576d0f7f3ab in bdrv_driver_pwritev at ../block/io.c:1174
#2  0x00005576d0f8139d in bdrv_aligned_pwritev at ../block/io.c:1988
#3  0x00005576d0f81b65 in bdrv_co_pwritev_part at ../block/io.c:2156
#4  0x00005576d0f8e6b7 in blk_do_pwritev_part at ../block/block-backend.c:1260
#5  0x00005576d0f8e84d in blk_aio_write_entry at ../block/block-backend.c:1476
#6  0x00005576d1060ddb in coroutine_trampoline at ../util/coroutine-ucontext.c:173
#7  0x00007f7c8d3be0d0 in __start_context at /lib/../lib64/libc.so.6
#8  0x00007f7b52beb1e0 in  ()
#9  0x0000000000000000 in  ()

Switch to qemu main thread:
#0  0x00007f903be704ed in __lll_lock_wait at
/lib/../lib64/libpthread.so.0
#1  0x00007f903be6bde6 in _L_lock_941 at /lib/../lib64/libpthread.so.0
#2  0x00007f903be6bcdf in pthread_mutex_lock at
/lib/../lib64/libpthread.so.0
#3  0x0000564b21456889 in qemu_mutex_lock_impl at
../util/qemu-thread-posix.c:79
#4  0x0000564b213af8a5 in block_job_add_bdrv at ../blockjob.c:224
#5  0x0000564b213b00ad in block_job_create at ../blockjob.c:440
#6  0x0000564b21357c0a in mirror_start_job at ../block/mirror.c:1622
#7  0x0000564b2135a9af in commit_active_start at ../block/mirror.c:1867
#8  0x0000564b2133d132 in qmp_block_commit at ../blockdev.c:2768
#9  0x0000564b2141fef3 in qmp_marshal_block_commit at
qapi/qapi-commands-block-core.c:346
#10 0x0000564b214503c9 in do_qmp_dispatch_bh at
../qapi/qmp-dispatch.c:110
#11 0x0000564b21451996 in aio_bh_poll at ../util/async.c:164
#12 0x0000564b2146018e in aio_dispatch at ../util/aio-posix.c:381
#13 0x0000564b2145187e in aio_ctx_dispatch at ../util/async.c:306
#14 0x00007f9040239049 in g_main_context_dispatch at
/lib/../lib64/libglib-2.0.so.0
#15 0x0000564b21447368 in main_loop_wait at ../util/main-loop.c:232
#16 0x0000564b21447368 in main_loop_wait at ../util/main-loop.c:255
#17 0x0000564b21447368 in main_loop_wait at ../util/main-loop.c:531
#18 0x0000564b212304e1 in qemu_main_loop at ../softmmu/runstate.c:721
#19 0x0000564b20f7975e in main at ../softmmu/main.c:50

in IO thread when do bdrv_mirror_top_pwritev, the job is NULL, and stop field
is false, this means the s object has not been initialized, and this object
is initialized by block_job_create(), but the initialize process stuck in
acquire the lock.

The rootcause is that qemu do release/acquire when hold the lock,
at the same time, IO thread get the lock after release stage, and the crash
occured.

Actually, in this situation, job->job.aio_context will not equal to
qemu_get_aio_context(), and will be the same as bs->aio_context,
thus, no need to release the lock, becasue bdrv_root_attach_child()
will not change the context.

This patch fix this issue.

Signed-off-by: Michael Qiu <qiudayu@huayun.com>
---
 blockjob.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/blockjob.c b/blockjob.c
index c6e20e2f..e1d41db9 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -214,12 +214,14 @@ int block_job_add_bdrv(BlockJob *job, const char *name, BlockDriverState *bs,
     BdrvChild *c;
 
     bdrv_ref(bs);
-    if (job->job.aio_context != qemu_get_aio_context()) {
+    if (bdrv_get_aio_context(bs) != job->job.aio_context &&
+        job->job.aio_context != qemu_get_aio_context()) {
         aio_context_release(job->job.aio_context);
     }
     c = bdrv_root_attach_child(bs, name, &child_job, job->job.aio_context,
                                perm, shared_perm, job, errp);
-    if (job->job.aio_context != qemu_get_aio_context()) {
+    if (bdrv_get_aio_context(bs) != job->job.aio_context &&
+        job->job.aio_context != qemu_get_aio_context()) {
         aio_context_acquire(job->job.aio_context);
     }
     if (c == NULL) {
-- 
2.22.0




  reply	other threads:[~2021-01-26  3:13 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-01-25 12:07 [PATCH] Fix crash with IOthread when block commit after snapshot 08005325
2021-01-26  3:11 ` 08005325 [this message]
2021-01-26  3:25   ` [PATCH v3] " 08005325
2021-01-28  1:30     ` [PATCH v4] blockjob: " 08005325
2021-01-28  5:16       ` 仇大玉
2021-02-01  2:40         ` 仇大玉
2021-02-01 10:27       ` Vladimir Sementsov-Ogievskiy
2021-02-01 11:26         ` 仇大玉
2021-02-01 12:07           ` Peng Liang
2021-02-01 12:44             ` Vladimir Sementsov-Ogievskiy
2021-02-01 13:09               ` Michael Qiu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210126031144.13121-1-08005325@163.com \
    --to=08005325@163.com \
    --cc=jsnow@redhat.com \
    --cc=kwolf@redhat.com \
    --cc=mreitz@redhat.com \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    --cc=qiudayu@huayun.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.