From: Stefan Reiter <s.reiter@proxmox.com>
To: Markus Armbruster <armbru@redhat.com>,
Kevin Wolf <kwolf@redhat.com>,
Paolo Bonzini <pbonzini@redhat.com>,
Wolfgang Bumiller <w.bumiller@proxmox.com>,
Thomas Lamprecht <t.lamprecht@proxmox.com>
Cc: qemu-devel@nongnu.org
Subject: [PATCH v2] monitor/qmp: fix race on CHR_EVENT_CLOSED without OOB
Date: Mon, 22 Mar 2021 16:40:24 +0100 [thread overview]
Message-ID: <20210322154024.15011-1-s.reiter@proxmox.com> (raw)
The QMP dispatcher coroutine holds the qmp_queue_lock over a yield
point, where it expects to be rescheduled from the main context. If a
CHR_EVENT_CLOSED event is received just then, it can race and block the
main thread on the mutex in monitor_qmp_cleanup_queue_and_resume.
monitor_resume does not need to be called from main context, so we can
call it immediately after popping a request from the queue, which allows
us to drop the qmp_queue_lock mutex before yielding.
Suggested-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
---
v2:
* different approach: move everything that needs the qmp_queue_lock mutex before
the yield point, instead of moving the event handling to a different context
monitor/qmp.c | 40 ++++++++++++++++++++++------------------
1 file changed, 22 insertions(+), 18 deletions(-)
diff --git a/monitor/qmp.c b/monitor/qmp.c
index 2b0308f933..092c527b6f 100644
--- a/monitor/qmp.c
+++ b/monitor/qmp.c
@@ -257,24 +257,6 @@ void coroutine_fn monitor_qmp_dispatcher_co(void *data)
trace_monitor_qmp_in_band_dequeue(req_obj,
req_obj->mon->qmp_requests->length);
- if (qatomic_xchg(&qmp_dispatcher_co_busy, true) == true) {
- /*
- * Someone rescheduled us (probably because a new requests
- * came in), but we didn't actually yield. Do that now,
- * only to be immediately reentered and removed from the
- * list of scheduled coroutines.
- */
- qemu_coroutine_yield();
- }
-
- /*
- * Move the coroutine from iohandler_ctx to qemu_aio_context for
- * executing the command handler so that it can make progress if it
- * involves an AIO_WAIT_WHILE().
- */
- aio_co_schedule(qemu_get_aio_context(), qmp_dispatcher_co);
- qemu_coroutine_yield();
-
/*
* @req_obj has a request, we hold req_obj->mon->qmp_queue_lock
*/
@@ -298,8 +280,30 @@ void coroutine_fn monitor_qmp_dispatcher_co(void *data)
monitor_resume(&mon->common);
}
+ /*
+ * Drop the queue mutex now, before yielding, otherwise we might
+ * deadlock if the main thread tries to lock it.
+ */
qemu_mutex_unlock(&mon->qmp_queue_lock);
+ if (qatomic_xchg(&qmp_dispatcher_co_busy, true) == true) {
+ /*
+ * Someone rescheduled us (probably because a new requests
+ * came in), but we didn't actually yield. Do that now,
+ * only to be immediately reentered and removed from the
+ * list of scheduled coroutines.
+ */
+ qemu_coroutine_yield();
+ }
+
+ /*
+ * Move the coroutine from iohandler_ctx to qemu_aio_context for
+ * executing the command handler so that it can make progress if it
+ * involves an AIO_WAIT_WHILE().
+ */
+ aio_co_schedule(qemu_get_aio_context(), qmp_dispatcher_co);
+ qemu_coroutine_yield();
+
/* Process request */
if (req_obj->req) {
if (trace_event_get_state(TRACE_MONITOR_QMP_CMD_IN_BAND)) {
--
2.20.1
next reply other threads:[~2021-03-22 15:43 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-22 15:40 Stefan Reiter [this message]
2021-04-07 13:19 ` [PATCH v2] monitor/qmp: fix race on CHR_EVENT_CLOSED without OOB Kevin Wolf
2021-04-08 9:21 ` Markus Armbruster
2021-04-08 10:27 ` Kevin Wolf
2021-04-08 12:49 ` Markus Armbruster
2021-04-08 13:27 ` Thomas Lamprecht
2021-04-08 14:10 ` Markus Armbruster
2021-04-09 15:30 ` Markus Armbruster
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210322154024.15011-1-s.reiter@proxmox.com \
--to=s.reiter@proxmox.com \
--cc=armbru@redhat.com \
--cc=kwolf@redhat.com \
--cc=pbonzini@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=t.lamprecht@proxmox.com \
--cc=w.bumiller@proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).